The Standardized Antimicrobial Administration Ratio: A New Metric for Measuring and Comparing Antibiotic Use

Abstract

Background

To provide a standardized, risk-adjusted method for summarizing antibiotic use (AU), enable hospitals to track their AU over time and compare their AU data to national benchmarks, the Centers for Disease Control and Prevention developed the Standardized Antimicrobial Administration Ratio (SAAR).

Methods

Hospitals reporting to the National Healthcare Safety Network (NHSN) AU Option collect and submit aggregated AU data electronically as antimicrobial days of therapy per patient days present. SAARs were developed for specific NHSN adult and pediatric patient care locations and cover five antimicrobial agent categories: (1) broad-spectrum agents predominantly used for hospital-onset/multi-drug resistant bacteria; (2) broad-spectrum agents predominantly used for community-acquired infections; (3) anti–methicillin-resistant Staphylococcus aureus agents; (4) agents predominantly used for surgical site infection prophylaxis; and (5) all antibiotic agents. The SAAR is an observed-to-predicted use ratio where predicted use is estimated from a statistical model; a SAAR of 1 indicates that observed use and predicted use are equal.

Results

Most location-level SAARs were statistically significantly different than 1: adult locations up to 52% lower than 1 and up to 41% higher than 1. Median SAARs in adult and pediatric ICUs had a range of 0.667–1.119. SAAR distributions serve as an external comparison to national SAARs.

Conclusions

This is the first aggregate AU metric that uses point-of-care, antimicrobial administration data electronically reported to a national surveillance system to enable risk-adjusted, AU comparisons across multiple hospitals. Endorsed by the National Quality Forum, SAARs provide AU benchmarks that stewardship programs can use to help drive improvements.

CDC, NHSN, antibiotic use, benchmark, risk adjustment

Concerns about the closely linked hazards of antibiotic overuse and drug-resistant pathogens, along with calls for clinical quality measures that focus on improvement and accountability, are prompting new attention to measuring antibiotic use (AU) and providing data that are actionable for stewardship [1–3].

Changing the way that antibiotics are used is paramount to improving patient safety and slowing the development and spread of antibiotic resistance, and national efforts are under way to strengthen antibiotic stewardship programs (ASPs) in inpatient, outpatient, and long-term care settings [4]. These efforts can reduce other negative consequences of excessive AU as well, including avoidable adverse drug events and unnecessary healthcare costs [5].

Evidence-based guidelines for ASPs include the recommendation that every program use a standardized method for measuring AU [6]. The Centers for Disease Control and Prevention (CDC) National Healthcare Safety Network (NHSN) enables hospitals to report AU data via the Antimicrobial Use and Resistance (AUR) Module, analyze data hospital-wide and by patient care location, and compare data to national benchmarks [7].

ASPs in hospitals and health systems can use the NHSN application to calculate unadjusted AU rates for specific drugs, for which the reported antimicrobial days of therapy are the numerator and the reported patient-days present are the denominator. Although these rates can be useful, in the absence of risk adjustment, rates cannot be used to make comparisons within and across hospitals. Interhospital comparisons of risk-adjusted AU use (ie, benchmarking) are recommended by the Infectious Diseases Society of America and Society for Healthcare Epidemiology of America 2007 guidelines [3].

To provide a standardized, risk-adjusted benchmark of antibiotic use, CDC developed the standardized antimicrobial administration ratio (SAAR). The SAAR is an observed-to-predicted ratio, in which reported antimicrobial days are the numerator and predicted, statistically modeled antimicrobial days are the denominator; SAAR values are nonnegative and typically >0. A SAAR statistically significantly above 1 indicates more antimicrobial use than predicted; a SAAR statistically significantly below 1 indicates less antimicrobial use than predicted. ASPs can use the NHSN application to generate SAARs. The National Quality Forum (NQF) endorsed the SAAR metric in December 2015 for performance improvement and public health surveillance [8]. In this report, we outline the methodological development and testing of the SAAR, report SAAR values for calendar year 2014, and discuss the uses, limitations, and future directions of the SAAR.

METHODS

Data Source

To participate in the AU Option of NHSN’s AUR Module, hospitals must be enrolled in NHSN and indicate their intent to submit AU data in their NHSN monthly reporting plan. Data submission to the AU Option is voluntary. The AUR Module protocol encourages hospitals to submit AU data from all inpatient locations and 3 select outpatient locations (eg, emergency department, pediatric emergency department, and 24-hour observation area) from which both the numerator (antimicrobial days) and denominator (days present) can be accurately electronically captured [6]. All data elements must be electronically submitted using the standardized format provided in the Health Level 7 Clinical Document Architecture Implementation Guide [9].

Hospitals submit antimicrobial-days and days present data that have been extracted and aggregated from either the electronic medication administration record (eMAR) or bar code medication record (BCMA) systems and patient location tracking systems such as Admission Discharge Transfer. Data can be submitted via an AU reporting system developed and implemented by an information technology vendor or via a system developed and implemented by a hospital or health system on its own. As more vendors offer AU reporting, hospitals are increasingly opting to use vendor systems to submit AU data.

Total antimicrobial-days (also known as days of therapy) for each patient care location are reported for 89 antimicrobials (including antibacterials, antifungals, and anti-influenza agents) administered via the following routes: intravenous, intramuscular, digestive, and respiratory. Antimicrobial days are enumerated by identifying any amount of a specific antimicrobial agent administered in a calendar day to a particular patient, as documented in the eMAR or BCMA system [7, 10–12]. Days present are enumerated by counting the number of admitted patients spending any amount of time in a specific patient care location each day during a calendar month. Days present are not the same as patient-days used in other parts of NHSN as patients can contribute days present in multiple locations on a calendar day, but only 1 patient-day. Both antimicrobial days and days present counts are aggregated to the location level prior to monthly NHSN submission; NHSN does not receive patient level data as part of the AU Option.

The CDC encourages participating hospitals to conduct systematic validation of their processes for ascertaining and aggregating AU numerator and denominator data prior to data submission and to assess the accuracy and completeness of data submitted. The NHSN AU Option validation protocol provides guidance for 3 validation steps: (1) confirming that eMAR/BCMA data are transferred accurately and completely from clinical databases to an AU reporting system; (2) confirming the accuracy and completeness of antimicrobial-days and days present by patient care location; and (3) checking AU data submitted into NHSN using the NHSN analysis features [13]. Of the 77 hospitals that provided AU data used for SAAR modeling, 55 completed 1 or more steps in the validation protocol.

Combining Patient Care Locations and Antimicrobial Categories to Produce SAARs

The CDC worked with infectious disease physicians and pharmacists in several ASPs to develop groupings of antibiotics for analytic and reporting purposes. The guiding principles were to categorize individual antibiotics into mutually exclusive sets of agents according to the most common clinical uses of each agent, to provide comprehensive groupings that were actionable for stewardship efforts.

The CDC developed SAARs for the following NHSN defined patient care locations: adult medical intensive care units (ICUs), adult medical wards, adult medical/surgical ICUs, adult medical/surgical wards, adult surgical ICUs, adult surgical wards, pediatric medical ICUs, pediatric medical wards, pediatric medical/surgical ICUs, pediatric medical/surgical wards, pediatric surgical ICUs, and pediatric surgical wards. These locations are identified by facilities in accordance with NHSN patient care location definitions and NHSN mapping algorithms [14].

SAARs cover 5 different antimicrobial agent categories: (1) broad-spectrum agents predominantly used for hospital-onset/multidrug-resistant bacteria; (2) broad-spectrum agents predominantly used for community-acquired infections; (3) anti–methicillin-resistant Staphylococcus aureus (MRSA) agents; (4) agents predominantly used for surgical site infection (SSI) prophylaxis; and (5) all antibiotic agents. Agent groups 1–4 are mutually exclusive and are referred to as high-value targets. The fifth group, all antibiotic agents, is a “high level indicator”; it includes all antibiotic agents in the NHSN AUR protocol, including those in agent groups 1–4 (antivirals and antifungals are excluded). Together, these combinations of antimicrobial agents and patient care locations amount to 16 discrete SAARs. SAARs can be aggregated to summarize use across all ICU and wards for both pediatric and adult locations. More information on SAAR groupings can be found in the Supplementary Data, as well as Appendix D of the AUR Module protocol [7].

Developing Predictive Models

Data used to develop SAARs were reported by hospitals for calendar year 2014 and submitted to NHSN by March 2015. To be included, hospitals must have reported 12 months of data for the location(s) considered: adult and pediatric medical, medical/surgical, and surgical ICUs and wards (as defined by NHSN location definitions [14]). There were 88 hospitals enrolled and reporting for 760 locations at this time. However, only 77 hospitals were reporting from the NHSN patient care locations for which SAARs are available: 350 adult medical, medical/surgical, and surgical ICUs and wards and 33 pediatric medical, medical/surgical, and surgical ICUs and wards. Locations reported not used for SAARs were primarily other adult/pediatric wards, labor and delivery, and neonatal units.

AU in each SAAR agent category was modeled separately using forward stage-wise negative binomial regression with available hospital and location-level factors: hospital bedsize, hospital number of ICU beds, medical school affiliation, location bedsize, and location type (ICU/ward, medical/medical surgical/surgical, adult/pediatric). Hospital and location factors are self-reported using NHSN guidelines. Patient level factors were not available for developing predictive models, as the NHSN AU Option collects data aggregated monthly to the patient care location level. Negative binomial regression was chosen because it reflects both the numerator and denominator of the crude incidence rate, taking into account AU as well as person-time; additionally, it is appropriate for under dispersion present in our models.

Models were evaluated using Akaike information criteria and likelihood ratio tests; model performance was assessed using a dispersion-based pseudo-adjusted R² method to assess fit [15]. All models were validated using a bootstrap resampling method, which evaluated regression coefficients to determine if the empirical 95% confidence intervals included zero [16]. All statistical analyses were conducted using SAS version 9.3 software (SAS Institute, Cary, North Carolina).

National Benchmarks

The SAAR metric is an indirectly standardized metric, which allows for comparison to the 2014 national baseline. Indirect standardization allows data to be collapsed to create facility, state, and/or national level SAARs. SAARs themselves serve as a comparison, SAARs different than 1 indicate more or less AU than the national benchmark predicts. Distributions of SAARs are used to identify where a facility’s SAAR lies compared to other facilities, and these distributions serve as a national comparison and benchmark.

RESULTS

SAAR models were constructed using 2014 NHSN AU Option data from 77 hospitals with 350 adult and 33 pediatric locations (377 other non-SAAR locations were reported). The number of locations that met SAAR criteria (eg, time period and patient care location) ranged from 1 to 13 per hospital. The mean number of locations was 4.97 (median, 4). The 77 hospitals came from 25 states; there were 39 general acute care hospitals, 30 Veterans Administration hospitals, 6 critical access hospitals, and 2 children’s hospitals.

The models estimate the number of predicted antimicrobial days for a given location and SAAR agent category. Nonsignificant risk factors that were assessed but were not in the final models include hospital bedsize, hospital number of ICU beds, and location bed size. Final models for each of the 5 agent categories adjusted for different statistically significant risk factors. A common factor used in risk adjustment across all models was an indicator variable for ICU locations: these locations generally have higher AU and were risk-adjusted accordingly; additional model specifications can be found in Table 1. The dispersion-based pseudo-adjusted R² was 0.52 for broad-spectrum agents predominantly used for hospital-onset/multidrug-resistant bacteria, 0.14 for broad-spectrum agents predominantly used for community-acquired infections, 0.48 for anti-MRSA agents, 0.19 for agents predominantly used for SSI prophylaxis, and 0.40 for all antibiotic agents predominantly used for community-acquired infections, 0.48 for anti-MRSA agents, 0.19 for agents predominantly used for SSI prophylaxis, and 0.40 for all antibiotic agents.

Table 1.

Standardized Antimicrobial Administration Ratio Risk Model Parameter Estimates, by Antibiotic Use Category

Parameter	Estimate	Standard Error	Wald 95% Confidence Limits		Wald χ2	χ2P Value
Model A: Broad-spectrum agents predominantly used for hospital-onset/multidrug-resistant infections
Intercept	–2.669	0.081	–2.827	–2.511	1092.18	<.0001
ICU locationa	0.971	0.052	0.868	1.074	343.77	<.0001
Location type
Medical unit	0.522	0.088	0.349	0.695	34.98	<.0001
Medical/surgical unit	0.444	0.090	0.266	0.621	24.05	<.0001
Surgical unit	0.406	0.098	0.213	0.598	17.02	<.0001
Pediatric unit	Ref	…	…	…	…	…
Model B: Broad-spectrum agents predominantly used for community-acquired infections
Intercept	–1.759	0.051	–1.859	–1.659	1187.70	<.0001
Teaching status	–0.376	0.055	–0.483	–0.268	46.97	<.0001
ICU locationa	0.122	0.049	0.026	0.219	6.17	.013
Pediatric location	–0.202	0.079	–0.356	–0.047	6.54	.011
Model C: Anti-MRSA agents
Intercept	–3.506	0.097	–3.697	–3.316	1297.25	<.0001
ICU locationa	1.432	0.208	1.023	1.840	47.22	<.0001
Location type
Medical unit	1.052	0.107	0.842	1.262	96.47	<.0001
Medical/surgical unit	0.892	0.110	0.676	1.108	65.49	<.0001
Surgical unit	1.095	0.123	0.853	1.337	78.72	<.0001
Pediatric unit	Ref	…	…	…	…	…
Interaction of ICU and location type
Medical unit	–0.521	0.227	–0.965	–0.077	5.3	.021
Medical/surgical unit	–0.542	0.230	–0.993	–0.092	5.56	.018
Surgical unit	–0.839	0.242	–1.313	–0.364	11.99	.001
Pediatric unit	Ref	…	…	…	…	…
Model D: Agents predominantly used for surgical site infection prophylaxis
Intercept	–3.288	0.055	–3.397	–3.180	3524.5	<.0001
ICU locationa	0.343	0.099	0.148	0.537	11.95	.001
Surgical location	0.967	0.115	0.741	1.193	70.21	<.0001
Model E: All antibiotic agents
Intercept	–0.786	0.053	–0.890	–0.683	221.9	<.0001
ICU locationa	0.501	0.034	0.433	0.568	212.62	<.0001
Location type
Medical unit	0.166	0.058	0.053	0.279	8.33	.004
Medical/surgical unit	0.178	0.059	0.063	0.294	9.13	.003
Surgical unit	0.140	0.064	0.014	0.266	4.72	.030
Pediatric unit	Ref	…	…	…	…	…

Abbreviations: ICU, intensive care unit; MRSA, methicillin-resistant Staphylococcus aureus.

^aICU location is an indicator variable (1/0) for a unit that is designated as a critical care location.

SAARs were calculated for each agent category stratified by adult ICUs, adult wards, pediatric ICUs, and pediatric wards, with the exception of 2 groups: SSI agents and all antibacterial agents, which were stratified only by adult and pediatric locations. Figure 1 shows the percentage of SAARs statistically significantly above and below 1 for all 16 SAAR agent/location groups. This figure serves as an external comparison to the national SAAR. A SAAR of 1 indicates that observed AU is equivalent to predicted AU, and the majority of SAARs for 2014 are statistically significantly different than 1, indicating they have more or less use than predicted. In adult locations, 43%–52% of SAARs showed less AU than predicted, while 22%–41% of locations had more AU than predicted; these differed by SAAR agent grouping and location type. In pediatric locations there was more variation, 14%–63% of SAARs showed less AU than predicted, whereas 14%–53% showed more than predicted.

Figure 1.

Percentage of standardized antimicrobial administration ratios (SAARs) statistically significantly above and below 1, by agent grouping and location, 2014.

Abbreviations for locations and SAAR groupings: Anti-MRSA, anti–methicillin-resistant Staphylococcus aureus agents; CO, broad-spectrum agents predominantly used for community-acquired infections; HO/MDRO, broad-spectrum agents predominantly used for hospital-onset/multidrug-resistant organisms; ICU, intensive care unit; PED., pediatric; SSI prophylaxis, agents predominantly used for surgical site infection prophylaxis.

In Figure 2, distributions for ICU locations are shown: these 10 agent/location groupings had median SAARs that ranged from 0.667 for anti-MRSA agents in pediatric locations to 1.119 for broad-spectrum agents predominantly used for community- acquired infections in pediatric ICUs. Adult locations generally had smaller interdecile ranges than their pediatric counterparts due to greater precision. These distributions highlight the value of location groupings (adult vs pediatric) and SAAR groupings, as important meaningful differences in SAARs by location may be missed when looking at facility-level data.

Figure 2.

Median standardized antimicrobial administration ratios (SAARs) and interdecile range in adult and pediatric intensive care units, by SAAR type, 2014.

Abbreviations for locations and SAAR groupings: Anti-MRSA, anti–methicillin-resistant Staphylococcus aureus agents; CO, broad-spectrum agents predominantly used for community-acquired infections; HO/MDRO, broad-spectrum agents predominantly used for hospital-onset/multidrug-resistant organisms; ICU, intensive care unit; Ped., pediatric; SSI prophylaxis, agents predominantly used for surgical site infection prophylaxis.

DISCUSSION

The SAAR is a NQF-endorsed clinical quality measure that combines (1) a novel method for grouping individual antibiotics and patient care locations into broader categories for analytic purposes, and (2) well-established methods for reporting observed AU counts (ie, antimicrobial-days of therapy). SAAR calculations for each combination of patient care location and antibiotics provide a measure of the reported-to-predicted number of antimicrobial-days, where the predicted number is estimated using predictive models that take into account differences in patient care location and hospital characteristics, and indirect standardization is applied to compare reported to predicted counts. The 16 SAARs are intended to meet the needs of hospital and health system ASPs for monitoring AU, comparing local AU data to national AU data, and initiating further evaluation and possible remediation, particularly when observed AU is significantly higher than predicted. While there have been other comparisons of AU across hospital settings with a variety of data sources and statistical methods [3, 11, 12, 17, 18], this is the first aggregate AU metric that allows for comparison of risk-adjusted AU across multiple hospitals and systems using different electronic medical record systems. Additionally, AU in NHSN is collected from the eMAR or BCMA systems, which provide more accurate depictions of bedside antibiotic administrations than other data sources such as pharmacy orders. However, the SAAR metric does not provide a definitive measure of AU appropriateness or judiciousness, and in its initial iteration the measure should not be used for public reporting or other external accountability purposes. Still, the current version of the measure provides a set of quantitative signals for which ASP follow-up and interventions may be warranted and substantial benefits for patients may be achieved. To assist with the evaluation of SAAR values, CDC partnered with the Pew Charitable Trusts and other stewardship experts to develop a SAAR assessment tool [19].

The SAAR enables detection of statistically significant AU data (SAAR different than 1) when observed-to-predicted AU ratios are calculated for categorical combinations of antibiotics and patient care locations (Figure 2). Individual location-specific SAARs provide actionable AU data at the location level, allowing ASPs to see meaningful differences in SAARs that may be missed if facilities focused solely on facility-level AU. Still, the individual antibiotic/patient care location categories reflect priorities for many, but not all, hospital and health system ASPs, and further work is needed to better understand the linkage between the process of care measured by the SAAR and adverse healthcare outcomes associated with use or overuse of antibiotics. Additional work also may be needed on antibiotic–patient care location groupings; mutually exclusive categorization of antibiotics according to their common indications for use does not account for the diverse uses of these agents, which is likely why model performance (as assessed by the pseudo R²) was better in some models. Furthermore, the patient care locations included in the initial iteration of the SAAR are limited to adult and pediatric inpatient locations.

Model performance and predictive power were better in adult locations compared to pediatric locations, due largely to the higher number of adult locations reporting (n = 350) compared to pediatric locations (n = 33). Pediatric SAARs serve as a first look at pediatric AU reported to NHSN, and providing both pediatric and adult SAARs allows for a more comprehensive view of inpatient AU in acute care hospitals. The initial pediatric and adult SAARs can serve as targets, to be refined as AU reporting increases. NHSN AU reporting has increased steadily since its inception in 2012; as of June 2017, 300 facilities have reported AU data, a marked increase from the 77 facilities used to create the SAAR in 2014. The SAAR itself serves as a benchmark to the 2014 national AU data in NHSN, and distributions included in this publication allow for further comparison to the 2014 national AU data. Future updates to SAAR distributions will be instrumental in external comparisons of AU to current national AU use, as reported to NHSN. Additional data collection and analysis will be needed to develop SAARs for neonatal ICUs, emergency departments, and patient care locations in nonhospital settings.

Optimizing AU and minimizing adverse events attributable to overuse of antibiotics call for clinical quality measures that are practical, reliable, and valid, as well as measurement data that are timely and used in strategic ways for improvement and accountability purposes. As with other efforts to close performance gaps in healthcare, knowing measure results is a necessary step but not sufficient to making meaningful progress; effective change strategies and their systematic use provide the connection between measurement and improvement [20]. The NQF-endorsed SAAR is designed primarily for hospital and health system ASPs to use in tracking and comparing their AU data to a national aggregate [8]. The SAAR has already proved to be a useful tool to assess ASPs at the Roudebush Veterans Affairs Medical Center where several stewardship efforts implemented in 2014 were retrospectively assessed by looking at reductions in SAARs in 2013–2015 [21].

While AU measurement data produced using the SAAR metric can provide an impetus for internal quality improvement, additional experience and methodologic enhancements of the SAAR likely will be needed to avoid unintended consequences before the measurement data can be used for public reporting and other accountability purposes. A high priority for CDC is exploring ways to take indications for AU, such as measures of infectious disease burden or comorbidity indices, into account in AU predictive models. A balance should be struck between the burden of increased reporting requirements and the benefits of these data for methodological enhancement of a quality measure. Although initial start-up costs are of concern, use of electronic data sources to the fullest extent possible for reporting purposes is important in ongoing efforts to meet demands for quality measurement data [2]. CDC is committed to working collaboratively with ASPs, researchers, professional organizations, and other partners to further develop the SAAR and extend its use in ways that improve performance and accountability.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the author to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the author, so questions or comments should be addressed to the author.

Note

Potential conflicts of interest. All authors: No reported conflicts of interest. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.