Antibiotic Prescribing Patterns for Respiratory Tract Illnesses Following the Conclusion of an Education and Feedback Intervention in Primary Care

Abstract Background A study previously conducted in primary care practices found that implementation of an educational session and peer comparison feedback was associated with reduced antibiotic prescribing for respiratory tract diagnoses (RTDs). Here, we assess the long-term effects of this intervention on antibiotic prescribing following cessation of feedback. Methods RTD encounters were grouped into tiers based on antibiotic prescribing appropriateness: tier 1, almost always indicated; tier 2, possibly indicated; and tier 3, rarely indicated. A χ2 test was used to compare prescribing between 3 time periods: pre-intervention, intervention, and post-intervention (14 months following cessation of feedback). A mixed-effects multivariable logistic regression analysis was performed to assess the association between period and prescribing. Results We analyzed 260 900 RTD encounters from 29 practices. Antibiotic prescribing was more frequent in the post-intervention period than in the intervention period (28.9% vs 23.0%, P < .001) but remained lower than the 35.2% pre-intervention rate (P < .001). In multivariable analysis, the odds of prescribing were higher in the post-intervention period than the intervention period for tier 2 (odds ratio [OR], 1.19; 95% confidence interval [CI]: 1.10–1.30; P < .05) and tier 3 (OR, 1.20; 95% CI: 1.12–1.30) indications but was lower compared to the pre-intervention period for each tier (OR, 0.66; 95% CI: 0.59–0.73 tier 2; OR, 0.68; 95% CI: 0.61–0.75 tier 3). Conclusions The intervention effects appeared to last beyond the intervention period. However, without ongoing provider feedback, there was a trend toward increased prescribing. Future studies are needed to determine optimal strategies to sustain intervention effects.

Inappropriate ambulatory antibiotic prescribing represents an ongoing challenge for antibiotic stewardship programs.Factors that contribute to this include heterogeneous patient settings, lack of rapid diagnostics for some common infections, and ingrained practices and attitudes toward antibiotic prescribing [1][2][3].It is estimated that 59.1% of antibiotic expenditures occur in nonhospital settings.In 2021, there were approximately 636 outpatient antibiotic courses for every 1000 persons in the United States [4,5].
Excessive antibiotic use carries increased financial costs to patients, side effects including Clostridioides difficile infection, and risks of subsequent infections with drug-resistant organisms [6,7].Respiratory tract diagnoses (RTDs) represent a significant portion of outpatient antibiotic prescribing, particularly in primary care, and antibiotic prescribing for RTDs is frequently unnecessary [8,9].While outpatient stewardship interventions have been associated with decreased rates of antibiotic prescribing for RTDs, there is limited evidence on the long-term post-completion effects of such interventions [3,[10][11][12][13][14], which has implications for the generalizability and maintenance of outpatient stewardship programs.In this study, we assess long-term durability after the conclusion of a successful time-limited, RTD-focused education and peer-comparator feedback intervention on the rates of antibiotic prescribing for RTDs in primary care [15].

Study Design and Population
We previously conducted a stepped-wedge cluster randomized trial in 31 primary care (PC) clinics in the University of Pennsylvania Health System (UPHS), the results of which have

Clinical Infectious Diseases
M A J O R A R T I C L E been described [15].In this previous study, practices were randomized into 6 clusters.For the first 6 months of the intervention (1 October 2017 through 31 March 2018), on a rolling basis, clinics in each cluster received a 1-time educational session along with clinic resources focused on appropriate antibiotic prescribing for common RTDs.Starting the month after this educational session, prescribers in each clinic were given monthly electronic feedback on their rates of prescribing with peer comparison through 31 October 2018, at which point funding for the intervention concluded and feedback was ceased.
For the study described here, we included the same PC clinics and prescribers that were included in the initial study.Resident physicians as well as other prescribers who joined the participating practices after the start of the intervention (and thus were not included in the initial analysis) were excluded.All encounters from 1 July 2016 through 31 December 2019 were included.We defined the post-intervention period (the period following cessation of peer comparison feedback reports) as encounters that occurred between 1 November 2018 and 31 December 2019, concluding prior to the onset of the coronavirus disease 2019 (COVID-19) pandemic.We included the same preintervention and intervention periods (the comparison periods) of the initial trial, which consisted of a pre-intervention period starting 1 July 2016 up until the month each cluster was given the education intervention, and an intervention period beginning the month after each cluster received the education intervention continuing through 31 October 2018.

Data Collection
Patient demographic data, antibiotic prescribing data, and International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM), codes were abstracted from a data warehouse that extracts all data from the electronic health record.Charlson comorbidity indices for each encounter were calculated using ICD-10-CM codes from the index visit and previous 6 months using a method previously described [16].Prescriber and practice demographic data from the original intervention study data were used [15].

Outcome Assessment and Definitions
The primary outcome was the presence of an antibiotic prescription during an in-person RTD visit.Analyses were performed at the visit level so that patients with multiple visits could contribute each visit to the study period.Antibiotics that were included were limited to oral agents that would plausibly be prescribed for RTDs; antibiotics administered via alternative routes (eg, topical) or exclusively used for diagnoses not related to the respiratory tract (eg, fosfomycin) were excluded, as were antiviral and antifungal agents.
To determine the appropriateness of an antibiotic prescription, we adapted a tiered appropriateness assessment strategy, originally outlined by Fleming-Dutra et al [8] and validated in the UPHS PC practices by Degnan et al 9].In this approach, encounters were assigned to 1 of 3 tiers based on how frequently antibiotic therapy would be justified for a particular diagnosis, as assessed by ICD-10-CM codes assigned at the encounter.Tier 1 included diagnoses for which antibiotic prescribing is almost always appropriate (eg, bacterial pneumonia); tier 2 included diagnoses for which antibiotics are sometimes appropriate (eg, sinusitis); and tier 3 included diagnoses for which antibiotics are rarely indicated (eg, asthma, viral upper respiratory tract infection).For encounters in which multiple ICD-10-CM codes fell into multiple tiers of appropriateness, the lowest (ie, most appropriate) tier was selected.Visits were also grouped by the presence of ICD-10-CM codes for specific RTDs and respiratory signs and symptoms.

Statistical Analyses
At the encounter level, patient-specific variables were compared during all 3 intervention time periods using χ 2 or Wilcoxon rank sum testing as appropriate.A multivariable mixed-effects logistic regression model was conducted on the visit level to compare the odds of antibiotic prescribing in the post-intervention period to the prior 2 periods.We considered variables with an odds ratio (OR) having a P value <.25 on the univariable analysis for inclusion in the final model, adding those variables of interest one at a time and including those that were significant (P < .05) in the final model.To account for the interaction between the intervention and diagnosis tier seen in the initial study analysis, the model with the final set of covariates, adding a tier-intervention interaction term was assessed.Month and year were included in the model to account for seasonal variations and secular trends in antibiotic prescribing.By performing nested clustering on patient and prescriber, we accounted for the possibility of patients and prescribers contributing multiple visits during each intervention period.
Rates of antibiotic prescribing were compared using the χ 2 test to assess rates of overall prescribing, prescribing rates among tiers of appropriateness, and prescribing rates for specific RTDs.A 2-tailed P value of .05 was used as the cutoff for statistical significance.All calculations were performed using Stata, version 17.0 (StataCorp, College Station, TX), with visualizations produced using R version 4.2.2 (R Foundation for Statistical Computing, Vienna, Austria) [17].

RESULTS
There were 260 900 unique RTD encounters included in the final analysis.These encounters represented approximately 24.9% of all in-person visits during the pre-intervention period, 22.9% during the intervention period, and 23.9% during the post-intervention period.The post-intervention period comprised 75 145 of these encounters, representing 49 847 unique patients, 29 unique clinics, and 165 unique prescribers, with a median of 1 encounter per patient (interquartile range, 1-2).The initial analysis included 31 clinics; however, 2 clinics dissolved during the study period, leaving 29 PC clinics in our final analysis.Of the 167 prescribers who participated in the educational session (and were included in the initial analysis), 151 (90.4%) had at least 1 encounter in the final month of the analysis.Patient demographics are outlined in Table 1, with prescriber and clinic characteristics provided in Table 2.
When compared with the pre-intervention period, the overall proportion of RTD visits with an antibiotic prescription was lower in the post-intervention period, with 28.9% of visits resulting in an antibiotic prescription (post-intervention) compared with 35.2% (pre-intervention); however, this rate was higher than the intervention period prescribing rate of 23.0% (P < .001).In tier 3 (rarely) visits, 14.4% of encounters resulted in an antibiotic prescription in the post-intervention time period compared with 19.2% (pre-intervention) and 11.3% (intervention period; P < .001).The change in tier 3 (rarely) prescribing between the intervention and post-intervention periods was not associated with the amount of time a given cohort of providers was exposed to the initial intervention.Rates of overall and tier 3 (rarely) encounter antibiotic prescribing are displayed in Figure 1.As RTD visits can be associated with multiple ICD-10-CM codes, multiple RTD groups may be accounted for in a single encounter, as shown in Table 3.Most notably, there was an increase in prescribing for otitis media in the post-intervention period compared with both the intervention and pre-intervention periods (55.6% vs 44.6% intervention and 50.3% pre-intervention, P < .001).Prescribing rates of specific antibiotics (as a proportion of total antibiotic prescriptions for each period), outlined in Table 4, showed a relative decrease in fluoroquinolone and increase in doxycycline prescriptions between the intervention and postintervention periods.
The results of the univariable and multivariable analyses are contained in Tables 5 and 6.ORs comparing the odds of prescribing during each time period and for each tier are presented separately, given that the interaction between prescribing tier and time period was found to be significant.Comparing the post-intervention period to the pre-intervention period, the odds of antibiotic prescribing were lower for both tier 3 (OR, 0.68; 95% confidence interval [CI]: .61-.75) and tier 2 (OR, 0.66; 95% CI: .59-.73), with increased odds of prescribing for tier 1 visits (OR, 1.55; 95% CI: 1.31-1.82).When comparing the post-intervention period to the intervention period, odds of antibiotic prescribing were higher for all 3 tiers of appropriateness (tier 3 OR, 1.2; 95% CI: 1.12-1.30;tier 2 OR, 1.19; 95% CI: 1.10-1.30;tier 1 OR, 1.57; 95% CI: 1.32-1.86).

DISCUSSION
Our analysis showed a moderately sustained reduction in the rates of both overall and tier 3 (rarely) antibiotic prescribing for RTDs after cessation of peer comparison feedback.This study adds to the growing literature surrounding efforts in outpatient antibiotic stewardship, showing that targeted interventions can have long-term effects on prescribing, though sustainability remains a challenge.While a retrospective analysis is limited in its ability to draw causal conclusions, there are a few possible attributes of the intervention that may have contributed to durability.The initial intervention was focused on RTDs rather than all outpatient antibiotics and provided evidenced-based resources that could be easily used during encounters when peer comparison feedback ceased.The provided feedback was also specific, focusing on 2 easily interpreted metrics rather than an extensive review of prescribing practices.The simplicity of feedback reporting coupled with the relatively high frequency of feedback (eg, monthly during the intervention) may have helped reinforce appropriate prescribing decisions once feedback concluded.Additionally, increasing national awareness of  excessive outpatient antibiotic use may have increased the efficacy of the intervention, making prescribers more receptive to stewardship efforts [18].However, despite the lower rates of antibiotic prescribing compared with the pre-intervention period, the increase compared with the intervention period suggests some reversion to less appropriate prescribing and may demonstrate the need for maintenance of this type of stewardship intervention to fully sustain the effects.Gerber and colleagues reported on the long-term durability of an education and feedback program on the rates of broadspectrum antibiotic prescribing for pediatric outpatient acute respiratory tract infections [19,20].While their initial intervention demonstrated a 50% reduction in broad-spectrum antibiotic prescribing, their 18-month difference-in-differences analysis showed no difference between the intervention and control groups by the end of the 12-month feedback-free period.However, a Cochrane review conducted by Arnold and Straus in 2005 of 39 trials found that long-term durability, evaluated in 3 studies, did show a sustained effect in the following 1 to 4 years [19].In contrast to the study published by Gerber, the feedback provided in our initial study was simpler, focusing on 2 specific metrics (overall and inappropriate antibiotic prescribing), provided monthly (rather than quarterly) to prescribers.We suspect that the more targeted and repetitive nature of our intervention may have helped shift prescribers' practice patterns.Additionally, while a longer-term retrospective analysis beyond 14 months may have shown further increases in prescribing with time, the onset of the COVID-19 pandemic limited our ability to study this without confounding.Overall, the mixed results of previously published studies suggest that outpatient stewardship programs need continual monitoring for success rather than adopting a "one-size-fits-all" approach.
During the post-intervention period, there were some notable changes in prescribing trends.We found a relative increase in doxycycline prescribing with decreased levofloxacin prescribing when compared with the pre-intervention and intervention periods.This may be reflective of national trends in fluroquinolone prescribing or could be an unintended consequence of  Other (%) 5951 (12.9) 1657 (11.9) 1953 (8.9) <.001 a n denotes visits with antibiotics prescribed, not total visits.
the original intervention that was intended to reduce inappropriate antibiotic prescriptions but not necessarily to impact antibiotic choice [20,21].It is also notable that rates of azithromycin prescribing remained lower during the post-intervention period compared with earlier periods, contrasting national trends of increased macrolide prescribing [22].Azithromycin has been associated with high rates of inappropriate prescribing, so the decrease seen in our study could suggest a sustained change in practice patterns [23,24].Rates of antibiotic prescribing for specific RTD groups (eg, sinusitis) generally followed a similar pattern of a mild increase during the post-intervention period that was lower than in the pre-intervention period.One notable exception was antibiotic prescribing for otitis media, which was higher in the post-intervention period than in the preceding periods.In exploring otitis media by ICD-10-CM code, there was an increase in relative prescribing for nonsuppurative otitis media (a tier 3 indication) in the post-intervention period (29.6% compared with 12.9% intervention and 19.8% pre-intervention), which may reflect a future educational opportunity.However, seasonal variation limits the interpretation of these results, as different intervention periods varied in the number of winter vs non-winter months.
While the focus of this analysis was on the 2 metrics associated with the feedback intervention (overall and tier 3 [rarely] prescribing), we did note some trends in tier 1 (always) diagnoses and antibiotic prescribing.There was a slight increase in tier 1 diagnoses in the post-intervention period compared with prior periods (3.2% in the post-intervention period vs 2.6% in the intervention period and 2.3% in the pre-intervention period).Notably, there was a difference in the seasonal composition of the 3 intervention time periods, with a higher proportion of fall/winter months in the post-intervention period, which influences the rates of both RTDs and antibiotic prescribing.In analyzing the breakdown of ICD-10-CM codes, we noted that a majority of tier 1 (always) visits were for pneumonia diagnoses across all 3 periods.However, there was an increase in streptococcal pharyngitis during the post-intervention period (n = 591) compared with the intervention (n = 1) and preintervention (n = 15) periods.The increase was not limited to a single clinic or cohort of providers.The lack of provider feedback during this period makes diagnostic shifting (eg, selecting a higher tier code to justify antibiotic prescribing) unlikely.While the Centers for Disease Control and Prevention does not monitor trends in streptococcal pharyngitis, there was no increase in invasive group A Streptococcus infections during this time period [25,26].We suspect that this may be due to broader adoption of rapid antigen testing and plan to monitor this as part of ongoing outpatient stewardship efforts.While low prescribing rates for pneumonia (a tier 1 diagnosis) were noted in all 3 intervention periods, many patients with an RTD visit for pneumonia are seen in follow-up after a hospital or emergency department encounter, which was confirmed by random chart review [15].
The main strength of this study is that it builds on a wealth of validated prescribing data included in the initial trial.By using data from a large cohort of PC clinics and prescribers for whom we have already assessed the efficacy of education and peercomparison feedback, we were able to study the postintervention effects and decay over time when the ongoing feedback part of the intervention ended.The time studied also enhances the strength of the study, allowing us to account for seasonal variations in prescribing over the course of a year yet still concluding the analysis prior to the onset of the COVID-19 pandemic.By including both month and year in our multivariable analysis, we accounted for seasonality and secular trends that influence antibiotic prescribing.
However, as the entirety of this study was performed before the rapid expansion of telemedicine with the COVID-19 pandemic, we were unable to study associations between the intervention and rates of antibiotic prescribing for telemedicine encounters.In a meta-analysis, Suzuki and colleagues found increased rates of prescribing for certain RTDs in telemedicine encounters, which, if seen in our population, would lead to a further reduction of the impact of our intervention [27].Additionally, our study was limited to prescribers who participated in the initial education and feedback intervention and continued to practice in the UPHS health system.This means that the cohort of providers decreased with time due to job changes and retirements and that providers who joined the health system later and were not part of the initial intervention were excluded from this analysis, a trend likely amplified during the COVID-19 pandemic.While providers not included in the initial intervention had higher rates of overall antibiotic prescribing in the post-intervention period (31.1% vs 28.9%, P < .001),we were limited in our ability to draw meaningful conclusions from this data.All UPHS PC providers were included in the initial intervention, meaning there was no control group, and there was minimal turnover during the intervention period, and thus data are only available for the postintervention period.Continuing to study these trends in prescribing over time may help inform the generalizability of this type of intervention, whether it creates a "cultural shift" within a practice or is limited to only prescribers who receive an intervention.This will be an important direction for future research, as prior studies have shown that prescriber attitudes and beliefs play a significant role in antibiotic use [1,14].
This study builds on the results of the initial trial, demonstrating that despite increased antibiotic prescribing in the absence of feedback, rates of both overall and inappropriate antibiotic prescribing remained lower early in the postintervention period.These results support the need for ongoing interventions as part of an active outpatient antibiotic stewardship program to educate and monitor appropriate antibiotic prescribing.Additional studies are needed to determine the ideal balance between education and feedback as well as the optimal frequency of feedback (eg, monthly, quarterly, ondemand), not only to allocate stewardship resources but also

Figure 1 .
Figure 1.Monthly rates of antibiotic prescribing across intervention periods.Vertical dashed lines represent stepped-wise transition.

Table 1 . Patient Demographics and Encounter Characteristics
Abbreviations: COPD, chronic obstructive pulmonary disease; RTD, respiratory tract diagnoses.a An individual encounter may be associated with multiple International Classification of Diseases, Tenth Revision, Clinical Modification, codes and multiple RTD categories.b Includes diagnoses such as tonsilitis, laryngitis, tracheitis, and unspecified acute upper respiratory tract infections.

Table 3 . Rates of Antibiotic Prescribing for Specific Respiratory Tract Diagnosis Groups by Study Period
An individual patient encounter may be associated with multiple International Classification of Diseases, Tenth Revision, Clinical Modification, codes and multiple RTD categories.
Abbreviation: COPD, chronic obstructive pulmonary disease.aP value for χ 2 comparison of prescribing frequency between 3 intervention periods.bIncludes diagnoses such as tonsilitis, laryngitis, tracheitis, and unspecified acute upper respiratory tract infection.

Table 6 . Multivariable Analysis of Factors Associated With Antibiotic Prescribing
Additional variables included in multivariable model: practice type (teaching vs nonteaching), practice specialty (family medicine vs internal medicine), provider type (medical doctor/doctor of osteopathic medicine vs nurse practitioner/physician assistant), encounter month, encounter year, Charlson comorbidity score, patient age, patient race, provider gender.