Evaluation of Multisite Programmatic Bundle to Reduce Unnecessary Antibiotic Prescribing for Respiratory Infections: A Retrospective Cohort Study

Abstract Background The aim of this study was to evaluate the frequency of unnecessary antibiotic prescribing for Tier 3 upper respiratory infection (URI) syndromes across the Mayo Clinic Enterprise before and after a multifaceted antimicrobial stewardship intervention, and to determine ongoing factors associated with antibiotic prescribing and repeat respiratory healthcare contact in the postintervention period. Methods This was a quasi-experimental, pre/post, retrospective cohort study from 1 January 2019 through 31 December 2022, with 12-month washout during implementation from 1 July 2020 through 30 June 2021. All outpatient encounters, adult and pediatric, from primary care, urgent care, and emergency medicine specialties with a Tier 3 URI diagnosis were included. The intervention was a multifaceted outpatient antibiotic stewardship bundle. The primary outcome was the rate of antibiotic prescribing in Tier 3 encounters. Secondary outcomes included 14-day repeat healthcare contact for respiratory indications and factors associated with persistent unnecessary prescribing. Results A total of 165 658 Tier 3 encounters, 96 125 in the preintervention and 69 533 in the postintervention period, were included. Following intervention, the prescribing rate for Tier 3 encounters decreased from 21.7% to 11.2% (P < .001). Repeat 14-day respiratory healthcare contact in the no antibiotic group was lower postintervention (9.9.% vs 9.4%; P = .004). Multivariable models indicated that increasing patient age, Charlson comorbidity index, and primary diagnosis selected were the most important factors associated with persistent unnecessary antibiotic prescribing. Conclusions Outpatient antibiotic stewardship initiatives can reduce unnecessary antibiotic prescribing for Tier 3 URIs without increasing repeat respiratory healthcare contact. Advancing age and number of comorbidities remain risk factors for persistent unnecessary antibiotic prescribing.

Approximately 80%-90% of all antimicrobial consumption occurs in outpatient settings [1]; however, up to 50% of these prescriptions may be inappropriate, with roughly 1 in 3 completely unnecessary [1][2][3][4].Indiscriminate antibiotic prescribing is associated with harm to both individual patients (eg, adverse effects, such as diarrhea, rash, and photosensitivity) [5] and the broader population (eg, antimicrobial resistance and increased healthcare costs) [6].In 2016, the Centers for Disease Control and Prevention published the Core Elements of Outpatient Antibiotic Stewardship, which encourages outpatient antimicrobial stewardship programs (ASP) to identify 1 or more high-priority targets [7].
Upper respiratory infections (URIs) represent the most common indication for outpatient antibiotic prescribing; however, a majority of URIs are viral in etiology [8].For example, despite guidelines recommending against antibiotic therapy for most patients with acute uncomplicated bronchitis, studies indicate that antibiotics may be prescribed in as many as 70%-85% of outpatient bronchitis encounters [9,10].Even for viral URIs with a known etiology, antibiotic prescribing remains an issue, with 1 study noting a prescribing rate of 29% in patients positive for influenza without findings of pneumonia [11].
Numerous antimicrobial stewardship interventions have been effective in reducing unnecessary URI-related antibiotic prescribing [12].Examples of interventions include peer comparison reports, provider education, patient education, order preference list modifications, antibiotic commitment posters, viral prescription pads, and electronic order sets [13][14][15][16].Although the literature supports outpatient antibiotic stewardship practices, few studies evaluate (1) the durability of intervention(s), (2) the impact of intervention(s) across various Outpatient ASP Bundle • OFID • 1 Open Forum Infectious Diseases M A J O R A R T I C L E geographic sites, (3) control outcomes (eg, rates of repeat healthcare contact for URI with and without antibiotic prescriptions), and (4) factors associated with ongoing, unnecessary/inappropriate antibiotic prescribing following initial intervention.
We aim to evaluate the frequency of unnecessary antibiotic prescribing for URI and the rate of repeat respiratory-related healthcare contact before and after a multifaceted, health system-wide antimicrobial stewardship initiative.Furthermore, we aim to determine patient, provider, and encounter-level factors associated with continued unnecessary antibiotic prescribing in the postimplementation period.

Patient Consent Statement
Patient consent was not required as this retrospective cohort study was deemed exempt by the Mayo Clinic institutional review board (IRB number 23-001010) and was reported following the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guideline for cohort studies.

Study Design, Setting, and Interventions
This quasi-experimental, pre/post retrospective cohort study spanned 1 January 2019 to 31 December 2022.Starting 1 January 2020, Mayo Clinic implemented a comprehensive, Enterprise-wide outpatient ASP aimed at reducing unnecessary antibiotic prescribing for URIs not expected to benefit from antibiotic therapy.The Mayo Clinic Enterprise consists of 3 major destination medical centers in Rochester (Minnesota), Arizona, and Florida, along with the Mayo Clinic Health System, a network of hospitals and clinics throughout 4 regions in Minnesota and Wisconsin (southwest Wisconsin, northwest Wisconsin, southeast Minnesota, and southwest Minnesota).The initiative targeted 3 ambulatory care specialties, including primary care (ie, family medicine, community-internal medicine, women's health internal medicine, and community pediatrics and adolescent medicine), urgent care, and emergency medicine.
A multifaceted bundle of programmatic ASP interventions was implemented across the Enterprise in a stepwise fashion beginning 1 July 2020.Specific interventions included standardized provider education, dissemination of patient handouts promoting symptomatic management (ie, Viral Rx pad [17]), development of a syndrome-based, prepopulated ambulatory order panel (ie, clinical decision support tool, see Supplementary Materials), a patient-facing antibiotic commitment poster, peer comparison reporting, and a provider-facing data dashboard to facilitate self-auditing of cases where prescribing was flagged as unnecessary [17,18].Regional efforts were administered by local ASPs, consisting of at least 1 pharmacist and/or 1 physician trained in infectious diseases and antimicrobial stewardship.

Participants, Data Collection, and Outcomes
Respiratory International Classification of Diseases, 10th Revision (ICD-10) diagnosis codes were grouped into tiers according to whether antibiotics are always indicated (Tier 1), sometimes indicated (Tier 2), or never indicated (Tier 3) as previously described (see Supplementary Materials) [8,17].Given lack of standardization across the full dataset and concerns for skewing of the data, coronavirus disease 2019 (COVID-19)-related diagnosis codes were excluded.All ambulatory encounters, including both in-person and virtual, with Tier 3 primary diagnosis codes from primary care, emergency medicine, and urgent care departments were eligible for inclusion.Minnesota patients required documentation of the Minnesota Research Authorization.Encounters with secondary diagnoses that included Tier 1 or Tier 2 URI codes were excluded.For patients with multiple encounters, each unique encounter was included if inclusion criteria were met.
Data were extracted from an electronic health record database and an institutional data warehouse.Data elements included encounter (eg, region, encounter type, encounter date, and ICD-10 billing code), patient (eg, age, sex, race/ethnicity, antibiotic prescription, Charlson comorbidity index [CCI] [19], respiratory comorbidities), and provider features (eg, provider type, department), as well as encounter season.Patient demographic data were self-identified and included to ascertain the influence these characteristics on prescribing patterns and overall healthcare utilization.Monthly encounter volumes were determined in the postimplementation cohort to assess the impact of overall encounter volumes (ie, clinic workload) on unnecessary antibiotic prescribing.Monthly encounter volumes for all clinic encounters (including nonrespiratory encounters) were averaged for each specific clinic during the postimplementation period to determine a baseline or "expected" number of total monthly encounters.Monthly clinic encounter volumes were then compared to this baseline and converted to ordinal values of low (>1 standard deviation [SD] below mean), mild (between 0 and 1 SD below mean), moderate (between 0 and 1 SD above mean), and high (>1 SD above mean) volume for each month.Thus, high clinic volume months indicate busier months, whereas low clinic volume months indicate less busy months.
The primary outcome was the percentage of Tier 3 encounters that resulted in an antibiotic prescription.Secondary outcomes included the rate of repeat respiratory-related healthcare contact (ie, hospitalization, clinic visits, and emergency encounters) within 14 days of the index visit and patient, provider, and encounter-level factors associated with unnecessary prescribing or repeat respiratory-related healthcare contact after implementation of the intervention bundle.

Statistical Analysis
Study periods were defined as preimplementation (1 January 2019-30 June 2020), implementation/washout (1 July 2020- Means and standard errors are presented for continuous variables and frequencies, and percentages are presented for categorical variables.Pearson χ 2 test was used to compare pre-and postintervention periods for antibiotic prescription and 14-day repeat respiratory-related healthcare contact.These comparisons were done for the overall sample and within a priori-specified patient, provider, and encounter-level subgroups. Univariate and multivariable logistic regression models were fit in the postimplementation cohort to identify patient, provider, and encounter-level factors associated with both antibiotic prescribing and with 14-day repeat healthcare contact for respiratory indications.Gradient boosting machine models were used to estimate the relative influence of individual variables to identify smaller models that retained most of the predictive ability.A complete-case analysis was done in the case of missing data.R statistical software version 4.1 was utilized for all analysis (R Foundation for Statistical Computing, Vienna, Austria).Statistical significance was defined as P < .05.

RESULTS
There was a total of 511 196 respiratory encounters across the Mayo Clinic Enterprise between 1 January 2019 and 31 December 2022.Of these, 184 417 Tier 3 encounters were eligible for inclusion, including 96 125, 18 759, and 69 533 in the preintervention, implementation/washout, and postintervention periods, respectively (Figure 1).
Baseline characteristics were similar between pre-and postintervention periods (Table 1).Approximately 45% of encounters were patients aged ≤18 years, with 20% of overall encounters among patients aged between 0 and 2 years.Most patients (89.4%) identified as White, while just over half were female (56.1%).Overall, 15.6% of all encounters included patients with at least 1 significant preexisting respiratory condition (ie, asthma, cystic fibrosis, pulmonary fibrosis, and bronchiectasis).Most encounters (68.8%) occurred between October and March.Primary care settings accounted for most visits (45.8%), while urgent care and emergency medicine accounted for 29.4% and 24.8% of overall encounters, respectively.Telehealth visits were more frequent in the post-compared to the preintervention period at 8.1% versus 3.5%, respectively.

Antibiotic Prescribing
Antibiotic prescribing for Tier 3 indications decreased from 21.7% in the preintervention period to 11.2% in the postintervention period (P < .001;Table 2, Figure 1).Significant reductions in prescribing were observed among all geographic regions and departments (P < .001for all comparisons), with the largest improvement noted among urgent care clinics (51.8% relative reduction, from 24.5% to 11.8%).Rates of Outpatient ASP Bundle • OFID • 3 specific antibiotics prescribed in each period are supplied in Supplementary Table 1.Overall, the relative frequency of specific antibiotic prescriptions shifted following the intervention, with azithromycin accounting for a lower percentage of all antibiotic prescriptions in the postimplementation period (47.1% preimplementation vs 30.4% postimplementation), whereas β-lactam prescription frequency increased.

Repeat Respiratory-Related Healthcare Contact
Repeat healthcare contact for respiratory conditions within 14 days of index encounter was less common when an antibiotic was prescribed in the overall cohort (6.9% antibiotics vs 9.7% no antibiotic, P < .001;Supplementary Table 2).This finding was consistent in both the pre-and postintervention cohorts.The rate of repeat respiratory-related healthcare contact in those receiving antibiotic prescriptions was numerically higher but not significantly different in the postintervention compared to the preintervention cohort (6.7% pre vs 7.3% post, P = .116).Repeat respiratory-related healthcare contact was less common in the postimplementation cohort among encounters where an antibiotic was not prescribed (9.9% preintervention vs 9.4% postintervention, P = .004).
In the multivariable model, advancing age, increasing CCI, and serous acute otitis media (AOM)/ear disorders were associated with antibiotic prescribing, whereas high total monthly clinic encounter volume was associated with less antibiotic prescribing (Table 3).No difference was noted among provider type.Afternoon encounters were marginally associated with antibiotic prescribing (OR, 1.06 [95% CI, 1.01-1.12];P = .027).Males were more likely to receive an antibiotic prescription compared to females in the overall model (OR, 1.08 [95% CI, 1.02-1.14];P = .007).Patients identifying as Asian, Black or African American, or other race were slightly less likely to receive antibiotic prescriptions relative to White patients.The top 3 variables influencing antibiotic prescribing included the primary diagnosis (ie, syndrome) from index encounter, patient age, and CCI, which accounted for >95% of relative influence (Supplementary Figure 1), suggesting that persistent prescribing can be largely predicted by these 3 factors.This is also supported by the closeness of the area under the receiver operating characteristic (ROC) curves from the 2-multivariable models (0.799 for the full model vs 0.795 for the 3-variable model).

Predictors of 14-Day Repeat Respiratory-Related Healthcare Contact in the Postimplementation Cohort
In the univariate model for 14-day repeat respiratory-related healthcare contact, relative to encounters of patients aged 0-2 years, all other age groups were less likely to return for a respiratory indication within 14 days (Supplementary Table 4).Additionally, urgent care, emergency medicine, telehealth, high monthly total clinic encounter volumes, patients with concomitant asthma, and CCI scores ≥5 were more likely to result in repeat healthcare contact.Receipt of an antibiotic prescription was associated with a lower rate of repeat contact (OR, 0.74 [95% CI, .67-.81]; P < .001),but was not one of the most important variables according to relative influence (Supplementary Figure 2).No differences were noted among provider type, patient sex, or patient race.
Three different multivariable models were fit for repeat respiratory-related healthcare contact comprising all, top 8, and top 5 variables by relative influence (Table 4, Supplementary Figure 2).When limiting the model to only the top 5 contributing variables, telehealth visit, department specialty, primary diagnosis, CCI, and age group had the most influence on repeat respiratory-related 14-day healthcare contact.When comparing the multivariable models on area under the ROC curve, they are similar (full model = 0.644, 8-variable model = 0.640, 5-variable model = 0.636), indicating that a reduced set of predictors is sufficient.

DISCUSSION
Our study adds to mounting evidence that targeted outpatient antibiotic stewardship programs are effective at reducing unnecessary or inappropriate antibiotic prescribing for URIs [12,13,15,16,[20][21][22][23][24].We observed a 48.4% relative reduction   in prescriptions for Tier 3 URI following comprehensive outpatient antimicrobial stewardship program implementation, amounting to approximately 7300 unnecessary antibiotic prescriptions avoided.This finding was consistent among all specialties, regions, providers, and patient groups, thereby increasing external validity.Encounters where an antibiotic was prescribed were less likely to result in repeat respiratory-related healthcare contact for a respiratory indication within 14 days of index visit in the overall, pre-, and postintervention cohorts; however, the rate of repeat contact in patients who did not receive antibiotic therapy slightly decreased following implementation of the multifaceted intervention.
The decision of whether to prescribe or withhold an antibiotic is complicated, encompassing not only clinical, but also mutually dependent nonclinical factors, such as patient, provider, and healthcare system-related factors [25].Even time of day has been correlated with the decision to prescribe antibiotics, with later appointments more likely to result in antibiotic prescription than earlier appointments, a phenomenon likely attributable to decision fatigue [26].Socioeconomic patient factors not assessed in this report, including access to clinics and/or clinic location, may additionally impact prescribing decisions.Given the complexity of these influential factors, determining the relative magnitude of each is important to provide clarity to outpatient ASPs, particularly in the context of persistent inappropriate prescribing following intervention.
Using univariate and multivariable regression models in the postimplementation cohort, we were able to identify multiple factors impacting the decision to prescribe antibiotics.Consistent with prior research, we too found that afternoon appointments were more likely to result in unnecessary antibiotic prescriptions, even when controlling for other variables in the multivariate model.Interestingly, busier clinic months were associated with lower rates of unnecessary prescribing.One explanation for this might be the seasonality of cold and flu season, when encounter volumes peak and providers simultaneously feel more confident in attributing symptoms to circulating viruses.However, while encounter-level factors were influential in the overall model, we found that the most significant factors impacting the decision to prescribe antibiotics by relative influence are specific diagnosis, patient age, and patient comorbidities.Serous AOM/ear disorders were the diagnoses most strongly associated with antibiotic prescribing (OR, 2.27 [95% CI, 2.10-2.46]),which could indicate diagnostic uncertainty, a knowledge deficit, or diagnostic coding issues, all of which can be addressed with targeted follow-up interventions.The influence of advancing age and higher CCI on the propensity to prescribe antibiotics is not surprising in the context of the well-known influence of fear in prescribing decisions.In this patient population, fear of complications secondary to "missing" a bacterial diagnosis may be clouding the complete risk/benefit picture, wherein antibiotic-related complications, microbiota disruptions, and the development of antibiotic resistance are viewed as less common, less severe, and/or more abstract [27,28].
Of particular importance is the association between antibiotic prescriptions and repeat respiratory-related 14-day respiratory healthcare contact, which was consistent in the overall, pre-, and postintervention cohorts.Previous studies have documented that antibiotic prescriptions are associated with higher patient satisfaction scores [29,30], particularly if patients enter encounters expecting an antibiotic prescription [31].It is possible that lack of antibiotic prescription drove certain patients (ie, the inconvincible patients) [32] to seek prescriptions elsewhere within the health system, thereby driving repeat contact.Such behaviors might be mitigated through patient education on diagnosis (eg, "you have a viral illness for which antibiotics have no proven benefit but may cause harm"), expectations (eg, "you may experience a lingering cough for up to 3 weeks"), and symptomatic management (eg, "take pseudoephedrine 60 mg by mouth every 4-6 hours as needed for congestion").We provided a tool to providers (ie, Viral Rx pad) with patient education and preselected recommendations for over-the-counter medications to manage specific symptoms.This was available in multiple languages, and accessible in both in print and prepopulated progress note formats, with links directly to the resource from the prepopulated ambulatory order panel/clinical decision support tool.
The COVID-19 pandemic impacted the healthcare system in dramatic ways, including reduced clinic encounter volumes, hyperawareness of viral causes of URI, and a shift to telemedicine [33,34].In our study, we chose to exclude all encounters with COVID-19-related diagnosis codes.Even though antibiotics are not indicated for COVID-19, including these encounters in the overall denominators resulted in a very dramatic and clearly artificial drop in the prescribing rate, which coincided with fluctuations of COVID-19 encounters [35].We also found increased use of telemedicine in the postimplementation (post-July 2021) cohort.Importantly, telehealth visits were associated with an increased rate of repeat respiratory-related healthcare contact, representing the fourth most important factor by relative influence behind patient age, department specialty, and primary diagnosis; however, we are unable to fully elucidate whether these repeat healthcare contact events represent planned in-person follow-ups or unplanned encounters for ongoing respiratory syndromes.

Limitations
Our study has several limitations, the most prevalent being a retrospective design and lack of randomization, which limits the ability to attribute causality of the intervention.Importantly, this study and the intervention period coincided with the COVID-19 pandemic, which could have influenced the study results in unmeasurable ways; however, as mentioned previously, COVID-19-related diagnoses were excluded from the analysis to limit the impact of an inflated denominator on Tier 3 prescribing rates.Utilizing encounter ICD-10 diagnosis codes may have resulted in some misclassification bias, including with use of unspecified codes, which comprised a large portion of our cohort (56% overall).However, diagnosis codes represent the most utilized method to classify encounters in this type of retrospective work, and methods similar to ours have been used in prior studies [1,8,23].We additionally were unable to track delayed prescriptions (ie, prescriptions intended to only be filled if symptoms failed to improve with time), which may explain why we observed a higher rate of prescribing with serous AOM.Furthermore, we only tracked prescriptions written, rather than filled, which limits our ability to account for patient adherence as a confounding factor.Our primary balancing measure of 14-day repeat respiratory-related healthcare contact was broad and lacked the specificity to determine if an antibiotic prescription might have made a difference, nor did it account for planned follow-up compared to unplanned recontact with the care system.Last, we were unable to detect the utilization of certain interventions, such as patient-level counseling strategies, which might be important in preventing repeat respiratory-related healthcare contact for persistent symptoms that are within the expected course of illness for viral infections.

CONCLUSIONS
Implementation of a multifaceted outpatient antimicrobial stewardship initiative successfully reduced unnecessary antibiotic prescribing for URIs without increasing the rate of 14-day repeat respiratory-related healthcare contact.Persistent antibiotic prescribing after bundle implementation was noted among encounters for patients with advanced age and increasing comorbidities.Additional awareness about the risks and benefits of antibiotics in older, medically complex individuals may be helpful in further reducing unnecessary antibiotic prescriptions.

Figure 1 .
Figure 1.A, Total number of Tier 3 (never prescribe) ambulatory encounters over time by quarter across intervention periods.B, Tier 3 encounter antibiotic prescribing rate over time by quarter across intervention periods.