Recommendation Reversals in Gastroenterology Clinical Practice Guidelines

Abstract Background Recommendations in clinical practice guidelines (CPGs) may be reversed when evidence emerges to show they are futile or unsafe. In this study, we identified and characterized recommendation reversals in gastroenterology CPGs. Methods We searched CPGs published by 20 gastroenterology societies from January 1990 to December 2019. We included guidelines which had at least two iterations of the same topic. We defined reversals as when (a) the more recent iteration of a CPG recommends against a specific practice that was previously recommend in an earlier iteration of a CPG from the same body, and (b) the recommendation in the previous iteration of the CPG is not replaced by a new diagnostic or therapeutic recommendation in the more recent iteration of the CPG. The primary outcome was the number of recommendation reversals. Secondary outcomes included the strength of recommendations and quality of evidence cited for reversals. Results Twenty societies published 1022 CPGs from 1990 to 2019. Our sample for analysis included 129 unique CPGs. There were 11 recommendation reversals from 10 guidelines. New evidence was presented for 10 recommendation reversals. Meta-analyses were cited for two reversals, and randomized controlled trials (RCTs) for seven reversals. Recommendations were stronger after the reversal for three cases, weaker in two cases, and of similar strength in three cases. We were unable to compare recommendation strengths for three reversals. Conclusion Recommendation reversals in gastroenterology CPGs are uncommon but highlight low value or harmful practices.


Background
Recommendations in clinical practice guidelines (CPGs) should be updated as new evidence emerges (1)(2)(3). While this evidence often leads to the inclusion of new therapies in guidelines, recommendations may also be reversed when they are shown to be futile or harmful (4,5). Identifying these cases of recommendation reversal is of key importance and should purge practices that jeopardize patient safety and healthcare resource management.
Practices in clinical gastroenterology change over time. For example, thiopurine therapy was believed to be potentially beneficial in early Crohn's disease (6). The Azathioprine for Treatment of Early Crohn's Disease in Adults (AZTEC) trial (7), however, found it to have no effect on clinical remission and put patients at increased risk of adverse events when compared with placebo. When novel medical evidence leads to the removal of a prior recommendation, we refer to the phenomenon as a recommendation reversal. In this instance, despite the small sample size and wide confidence interval for the primary outcome, the AZTEC trial led the European Crohn's and Colitis Organization (ECCO) to recommend against early introduction with thiopurines in newly diagnosed Crohn's disease (8).
Many recommendations in gastroenterology society guidelines are based on weak evidence (9)(10)(11). Across medical specialties, practices that are promoted based on poor quality data or pathophysiological rationales can be subsequently found to be futile or harmful (12,13). Well-known examples include stenting for stable coronary artery disease and routine pulmonary artery catheterization for critically ill patients (14)(15)(16). Prior work has examined specific medical practices across fields, but the frequency of recommendation reversals in clinical practice guidelines is unknown. For this reason, we identified and characterized recommendation reversals among major gastroenterology society CPGs.

METHODS
We systematically searched all clinical practice guidelines published by select gastroenterology societies from January 1990 to December 2019 and identified recommendation reversals as defined below.

Clinical Practice Guideline Identification
We included North American, European, Asian, Australian, and international societies that produced guidelines in English primarily for topics in gastroenterology, including endoscopy, hepatology, inflammatory bowel disease, nutrition, and gastrointestinal surgery. We did not include societies that publish guidelines on topics that include illnesses that affect the gastrointestinal system such as oncology or primary care. Supplementary Table 1 lists the societies included in this study.
Two authors (R.G. and A.R.) hand-searched, independently and in duplicate, all CPGs published by these societies during the period of January 1990 to December 2019. The CPGs were collected from the societies' respective websites. As we were interested in recommendations that changes over time, we only included guidelines that had two or more iterations on the same topic. We included guidelines that graded the strength of their recommendations and/or evidence. We excluded position statements and clinical pathway documents that were not based on evidence.

Data Extraction
The two authors used a standardized data collection form to record the following information: name of society producing the guideline, guideline topic, iteration, year of publication, and type of evidence grading system. Explicit recommendations were extracted and categorized based on topic, and strength of recommendation and level of evidence as rated by the guideline authors/committee members. We compared recommendations addressing the same clinical practice in different guideline iterations from the same society. We then identified all instances of recommendation reversals. Evidence cited in both the original and reversed recommendations were recorded. Discrepancies with respect to data collection and recommendation reversal inclusion were resolved by a third author (R.K.).
We defined a recommendation reversal using two criteria, both of which were required for inclusion: a. The most recent iteration of a CPG recommended against a specific practice, in contradiction of a previous iteration of the CPG from the same body that recommended the opposite.
b. The recommendation in the previous iteration of the CPG is not replaced by a new diagnostic or therapeutic recommendation in the more recent iteration of the CPG.
Two other authors (N.G., S.C.G.) reviewed all identified reversals to ensure that the above two criteria were met. For each reversal that was identified, we also noted both the strength of recommendations and the quality of the underlying evidence for the newer recommendation.

Outcomes and Data Analysis
The primary outcome was the number of recommendation reversals. Secondary outcomes included new evidence provided in CPGs for recommendation reversals, and changes in recommendation strength after a recommendation was reversed. All data retrieved from the CPGs were managed using Microsoft Excel (2019).

RESULTS
Twenty societies published a total of 1022 CPGs from 1990 to 2019. There were 129 unique guideline topics for which there was more than one iteration. Our final sample consisted of 129 final iterations of CPGs, and 292 total CPGs when considering each iteration as an individual guideline ( Figure 1). Guidelines were from a range of gastroenterology societies from North America, Europe, and Asia (Table 1).

Recommendation Reversals
There were 11 recommendation reversals ( Japan Gastroenterological Endoscopy Society ( JGES). Seven reversals were for medical therapies, two for procedures (e.g., endoscopy), one for lifestyle modifications, and one for a diagnostic modality. When considering subspecialties within gastrointestinal disease, four reversals were related to luminal gastroenterology, four related to liver disease, and one each for gastrointestinal oncology, inflammatory bowel disease, and pancreatic disease respectively ( Table 2).

Recommendation Strength and Quality of Evidence
New evidence presented in guidelines for 10 recommendation reversals, with 1 reversal lacking new data (20,21). Meta-analyses were cited for two reversals, and randomized controlled trials (RCTs) for seven reversals ( Table 3). The other evidence cited for reversals included single-arm trials, observational studies, narrative reviews, and other clinical guidelines.
Recommendations were stronger after the reversal for three cases, weaker in two cases, and of similar strength in three cases (Table 4). We were unable to compare recommendation strengths for three reversals because one iteration did not provide a recommendation strength.

Discussion
In this study of gastroenterology CPGs, we found 11 recommendation reversals from 1990 to 2019. These reversals were from major gastroenterology society guidelines and most commonly related to luminal gastroenterology and liver disease. Most reversals were based on evidence that identified harm or a lack of efficacy for established interventions. To our knowledge, this is the first study to evaluate reversals among gastroenterology CPG recommendations. We propose to categorize the reversals in this study post-hoc into three groups. While we recognize that these categorizations are not mutually exclusive, they can provide a conceptual framework of how reversals can arise. In the first group, initial recommendations based on low quality evidence, such as expert opinion, were reversed in subsequent guidelines after higher quality studies were published. For example, the 2010 ECCO guideline on the management of Crohn's disease suggested that patients who have poor clinical prognostic factors may be suitable for early thiopurine therapy based on expert opinion and physiological considerations (6). When a large randomized study was conducted however, patients receiving azathioprine experienced more harm compared to those who received placebo (7). As a result, the recommendation was reversed in 2016 (8). A similar pattern can be found for reversals related to radiation doses higher than 59Gy for anal squamous neoplasms (23,24), endoscopic shock wave lithotripsy combined with endoscopic retrograde cholangiopancreatography for pancreatic stones (30,31), and bridging anticoagulation for patients on warfarin before endoscopy (32,33). For two of the above cases, guideline authors assigned a higher recommendation strength (e.g., strong recommendation) after the reversal when supported by higher-quality data.
In the second group, recommendations based on high quality studies, such as randomized trials, were reversed after subsequent studies found conflicting results. For example, the 2010 ACG (34) and 2012 EASL (28) guidelines on alcoholic hepatitis recommended pentoxifylline in certain situations based on an RCT showing a 30-day mortality benefit with pentoxifylline compared to placebo (37). There were however, nine further RCTs which showed that pentoxifylline did not improve short-term survival compared to placebo (40)(41)(42)(43), compared to corticosteroids (44,45), as an adjunct to corticosteroids (46,47), and among patients with no response to corticosteroids (48). As a result, the 2018 ACG (5) and EASL (29) guidelines recommended against using pentoxifylline in alcoholic hepatitis. Similarly, a recommendation for isosorbide mononitrate as primary prophylaxis of variceal hemorrhage (27), based on an RCT demonstrating similar efficacy to nadolol (57), was reversed (4) after a larger trial showed its lack of efficacy compared to placebo (59). These cases highlight the importance of replication even for seminal trials that guide practice. Estimates of efficacy of medical practices from single, small RCTs may be erroneous in further testing (72). Additionally, changes in the treatment and standards of care for specific diseases may alter the efficacy of earlier interventions (73).
In the third group, recommendations were reversed based on changes to practice in other medical specialties. A 2003 ASGE guideline recommended antibiotic prophylaxis before endoscopy for patients at high risk of developing endocarditis or with synthetic vascular grafts less than 1 year old (25). The 2008 ASGE guideline recommended against antibiotic therapy before endoscopy for both of the above circumstances (26), based American Heart Association documents (53,56). Similarly, a recommendation for pulmonary artery catheterization for assessment of shock in acute liver failure in a 2005 AASLD guideline (20) was reversed in a 2011 AASLD (21) guideline. This reversal likely reflected a practice change in critical care, where pulmonary artery catheterization was shown to have little benefit and substantial morbidity (14)(15)(16).
This study has several limitations. First, we may have missed reversals in guidelines published by societies that we did not include. Second, our small sample size of reversals precluded the ability to perform quantitative analysis. Third, there are not enough data in this study to comment on whether weak recommendations based on lower quality evidence are more likely to be reversed compared to recommendations supported by higher quality evidence. Fourth, while we stated whether recommendation reversals were made based on stronger or weaker evidence compared to the original recommendation, we did not analyze whether the initial recommendation was made prematurely. Fifth, we did not evaluate the quality of independent trials independently, but rather relied on the level and strength of evidence as judged by the guidelines included in this study. Finally, as there is no previously established definition of what constitutes a recommendation reversal, applying different definitions may lead other investigators to identifying other reversals.
Our findings reveal that recommendation reversals are infrequent in gastroenterology CPGs, though they may have important clinical consequences. Although prior research has found that 40% of studies in high-impact general medical journals find existing practices to be inferior to a prior standard (12,13,74), this result occurs among the subset of trials that rigorously test standard of care. For example, bispectral index monitoring, used in half of operating rooms in the United States by 2007 to detect anaesthesia awareness, was found to be no different to standardized sedation monitoring when tested in a large randomized study in 2008. Reversals in guidelines are much less frequent for several possible reasons. First, most gastroenterology guideline recommendations are based on low-quality evidence (9-11) such as expert opinion or physiologic principles. Testing such recommendations with robust, well controlled studies may occur rarely. Additionally, recommendations that are shown to be futile or harmful may persist in guidelines because subspecialty societies do not readily accept newer evidence contradicting their existing practices. A recent article found that compared to practices recommended by editorials in peer review journals, CPGs written by specialist societies were more likely to continue recommending those same practices despite new evidence (75). In gastroenterology, well controlled studies are needed to test practices rooted in poor evidence, and identify those that should be abandoned.

CONCLUSION
In this large analysis of CPGs in gastroenterology, we identify only 11 recommendations that were subsequently reversed. Guideline reversals are rare, but pertain to important clinical questions and decisions. Future trials should seek to answer longstanding questions in gastroenterology and guideline committees should review post-hoc when new evidence comes to light. In order to deliver the best medical care to our patients, revising recommendations in CPGs based on new evidence is paramount. The 2018 EASL guideline on alcoholic hepatitis stated that pentoxifylline can no longer be recommended due to very weak evidence. The original recommendation was based on three RCTs. One trial demonstrated that pentoxifylline reduced the incidence of hepatorenal syndrome without significant changes in liver function (37). Two other RCTs showed that pentoxifylline had a preventive effect on hepatorenal syndrome (40,45).

RCTs
This reversal was based on four RCTs which showed that pentoxifylline did not improve short-term survival compared to placebo (40), compared to corticosteroids (44), as an adjunct to corticosteroids (46), and among patients with no response to corticosteroids (48

RCTs
This reversal was based on one randomized trial in which patients did not have significantly different clinical remission rates but higher adverse event rates with azathioprine maintenance therapy compared to placebo (7). Table 3.