Automated identification of antibiotic overdoses and adverse drug events via analysis of prescribing alerts and medication administration records

Objectives: Electronic trigger detection tools hold promise to reduce Adverse drug event (ADEs) through efficiencies of scale and real-time reporting. We hypothesized that such a tool could automatically detect medication dosing errors as well as manage and evaluate dosing rule modifications.

Materials and Methods: We created an order and alert analysis system that identified antibiotic medication orders and evaluated user response to dosing alerts. Orders associated with overridden alerts were examined for evidence of administration and the delivered dose was compared to pharmacy-derived dosing rules to confirm true overdoses. True overdose cases were reviewed for association with known ADEs.

Results: Of 55 546 orders reviewed, 539 were true overdose orders, which lead to 1965 known overdose administrations. Documentation of loose stools and diarrhea was significantly increased following drug administration in the overdose group. Dosing rule thresholds were altered to reflect clinically accurate dosing. These rule changes decreased overall alert burden and improved the salience of alerts.

Discussion: Electronic algorithm-based detection systems can identify antibiotic overdoses that are clinically relevant and are associated with known ADEs. The system also serves as a platform for evaluating the effects of modifying electronic dosing rules. These modifications lead to decreased alert burden and improvements in response to decision support alerts.

Conclusion: The success of this test case suggests that gains are possible in reducing medication errors and improving patient safety with automated algorithm-based detection systems. Follow-up studies will determine if the positive effects of the system persist and if these changes lead to improved safety outcomes.

electronic health record, electronic medical record, medical order entry system, CPOE, decision support systems, clinical, adverse drug event, patient safety, risk management

BACKGROUND AND SIGNIFICANCE

Adverse drug events (ADEs) are a common occurrence, prevalent in both adult and pediatric populations. As the model of health care has moved towards prevention, national efforts have been put forth to curtail this estimated problem, which is estimated to cost a 700 bed hospital $5.6 million dollars per year.¹ Government efforts range from the Food & Drug Administration’s MedWatch program to report severe adverse drug events to the Agency of Healthcare Research and Quality’s Centers for Education and Research on Therapeutics (CERTs) program, which empowers academic centers across the country to improve the quality of healthcare by reducing adverse drug events.^2–5 Despite large-scale interventions, ADEs continue to burden the healthcare system by increasing length of stay by an average of 1.7 days and by increasing the risk of mortality in patients who experience ADE by 1.9 times over patients who do not.⁶ The pediatric population is particularly at risk for adverse drug reactions, especially those with more complicated medical problems requiring a greater number of medications.⁷ In this cohort, opiates and antibiotics account for more adverse drug events than any other drug classes.⁸

To identify, quantify, and mitigate adverse events, systems must be put in place to detect ADEs. The detection of ADEs can be accomplished via several methodologies such as voluntary reporting systems, manual chart review, and the use of manual trigger programs. Many healthcare organizations employ a combination of these. Each approach has advantages and disadvantages but most are resource-intensive, requiring extensive human review of the patient record. Additionally, the utility of these methods is frequently limited by retrospective review of sampled data.

Broad adoption of electronic health records (EHRs) and Computerized Provider Order Entry (CPOE) has increased the availability of prescribing and other clinical data. More readily available data means ADE detection can be automated through electronic trigger tools which provide efficiencies of scale by surveying the entire health system and real-time reporting. Successful use of electronic trigger tools to facilitate detection of ADEs has been previously demonstrated.⁹

Errors in medication dosing are the most common type of medical error in pediatrics. This is due to several factors including weight-based dosing, the use of different formulations in different age and developmental groups, etc. The increased complexity of dosing in children introduces increased risk in an already vulnerable population. Clinical Decision Support (CDS) has been shown to decrease medical errors. The most common type of CDS tool used to prevent prescribing errors is alerting. In the pediatric population, alerts are most commonly seen in patients <1 month of age, and less likely to be seen in patients aged 15–18 years of age.¹⁰ Alerts are directed to all stages of the medication process, including prescribing, ordering, dispensing, and administration.

Reviews of a single year of pediatric antimicrobial stewardship intervention at the Alfred I. DuPont Hospital for Children found a rate of 7 antimicrobial interventions per 1000 patient-days among 13 targeted antimicrobials. Of these, 61% of interventions were made to correct inappropriate antimicrobial dosing.¹¹ This is not unexpected, as antibiotic dosing can be complex, due to multiple components in formulations (e.g., amoxicillin-clavulanate), dosing guidelines based on daily doses that the provider has to convert into per dose instructions (amoxicillin, 80–90 mg/kg/day, divided into 2 doses), and different dosing based upon clinical indication. Due to the dose-dependent nature of many antibiotic adverse events (such as diarrhea or loose stools), one could expect to see harm associated with overdoses of these medications.

The use of CDS alerts to detect prescribed overdoses has been previously demonstrated.¹² The hypothesis of this study was that an enhanced CDS-based identification methodology could be developed to automatically detect overdoses and reduce the noise associated with medication alerts.

The objectives of this study were to (1) develop EHR-based algorithm that automate the detection of overdoses that reached patients, (2) demonstrate viability of the system with a use case, (3) identify safety outcomes associated with identified overdoses, and (4) enact operational changes that would increase the performance of the CDS related to antibiotics.

MATERIALS AND METHODS

Setting

Cincinnati Children's Hospital Medical Center (CCHMC) is a 587-bed, free-standing children’s hospital with 15 000 employees and over 800 faculty members. It has over 30 000 admissions, 33 000 surgeries, 900 000 ambulatory encounters, and 125 000 emergency department (ED) visits annually. All ambulatory clinics operate as part of the hospital. The institution brought the first ambulatory clinics live on an integrated EHR (EpicCare ®, Verona, WI, USA) in late 2007. It brought the remainder of the system live in stages, including inpatient and perioperative areas in January 2010, and the final of 38 ambulatory departments in January 2012. The project team for this study consisted of clinical informaticists, the Director of Research Informatics Technology Services, pharmacists, clinicians from the Division of Hospital Medicine, and a project manager.

Dosing decision support configuration

The EHR is configured to use a combination of the Medi-Span (Wolters Kluwer Health, Philadelphia, PA, USA) drug dosing decision support rules and supplemented with pediatric-specific, custom dosing rules created and maintained by the CCHMC Pharmacy. The rules contain a 10% dosing variance allowance to account for rounding and do not fire within this variance. Additional details on the decision support rules can be found in a previous publication.¹²

Methodological approach

To achieve the objectives of the project, the subsequent steps were followed. First, using the EHR, we identified candidate overdose orders by analyzing inpatient-only medication alert data in a pre–post study design from 2011 to 2014. ED, ambulatory and peri-operative orders were excluded from this study. We evaluated user response to dosing alerts to find overdose orders that were not intercepted (or modified) by the CPOE system’s CDS tools. Orders with an overdose alert that was overridden were examined for evidence of user modification that occurred after the alert was ignored. Medication Administration Record (MAR) data was evaluated to identify possible overdoses that reached the patient and had the potential to cause harm. Next, we validated the delivered dose was a true clinical overdose using pharmacy-derived rules independent of the electronic dosing rules. We then screened patients who received overdoses for associated adverse events (diarrhea and loose stool). C. Difficile toxin testing was examined due to common consideration in the setting of antibiotic administration and diarrhea. Finally, we modified the dosing rules to improve traditionally high-noise medication alerts through improved clinical accuracy. We then measured the resulting effects on alert burden and salience rates of the altered rules. The steps are described in detail below.

Development of EHR-based algorithms to detect overdose orders

EHR-based algorithms were created to detect true medication overdoses. These algorithms expanded upon our previous work and used an analytic database of medication orders and alerts.¹² MAR data were added so that orders and alerts that actually impacted patient care and affected clinical outcomes could be analyzed. Algorithmic queries were performed in SQL Developer (Oracle Corporation, Redwood Shores, CA, USA) to determine which candidate orders may represent overdoses. Some data points were extracted by simple text parsing routines from the stored text descriptions of the alerts as discrete, structured data equivalents were not available. Given that the presence of an overdose alert was used to identify a potential overdose and prior research has shown that vendor dosing rules can be highly inaccurate, a clinical overdose screening algorithm (Table 1) was implemented to validate clinical overdoses. The clinical overdose screening algorithm incorporated formulation-specific dosing for amoxicillin-clavulanate, as well as defining clinically-relevant weight-based and total-daily dosing.¹³

Table 1.

Clinical dosing rules for amoxicillin, amoxicillin-clavulanate, clindamycin

Antibiotic dose limits
Agent and formulation	Dosage form	Weight- based maximum daily dose (mg/kg)	Absolute maximum daily dose (mg)
Amoxicillin	Capsules, suspension, or tablets	90	4000
Amoxicillin/Clavulanate 125/31.25/5 mL	Suspension	40	1500
Amoxicillin/Clavulanate 200/28.5/5 mL	Suspension	45	1750
Amoxicillin/Clavulanate 250/62.5/5 mL	Suspension	40	1500
Amoxicillin/Clavulanate 400/57/5 mL	Suspension	45	1750
Amoxicillin/Clavulanate 600/42.9/5 mL	Suspension	90	3600
Amoxicillin/Clavulanate 500/125	Tablets	40	1500
Amoxicillin/Clavulanate 875/125	Tablets	45	1750
Amoxicillin/Clavulanate 1000/62.5	Tablets	90	4000
Clindamycin	Capsules or suspension	30	1800
Clindamycin	Injection	40	3600

Justification for oral antibiotics as use case

A use case was selected to demonstrate the overdose algorithms’ performance capabilities. The project team selected the following candidate medication criteria: (1) medication overdose must be associated with a detectable adverse event; (2) must be fairly common medication; and (3) and has dosing complexity that lends itself to errors. Our project team selected the common oral antibiotics amoxicillin, amoxicillin-clavulanate, and clindamycin because overdoses of these medications are associated with known adverse events that are easily detectable (diarrhea and loose stools),¹⁴ are one of the most commonly prescribed antibiotics in pediatrics, and are often advised to be dosed on a mg/kg/day regimen (e.g., amoxicillin; 80–90 mg/kg/day, divided every 12 h for acute otitis media).¹³ In addition, formulation-dose mismatches of amoxicillin-clavulanate were relatively common from anecdotal experience, whereby the incorrect ratio of amoxicillin-clavulanate components is selected by the prescriber leading to an appropriate dose of amoxicillin, but an excessive dose of clavulanate. The potential exposure to excessive clavulanic acid due to inappropriate prescribing was predicted to be associated with increased incidence of adverse effects. Medication order and alert data from 2011 to 2014 were used in this study.

Identifying adverse events from the algorithm output

To identify association between adverse events and overdosed medications we first used the automated algorithm to identify patients who received overdoses. We then compared the output to lab data queries to identify patients who were tested for C. difficile infection, postulating that a clinician ordering this test was encountering diarrhea in the context of antibiotic administration. Finally, a manual chart review of overdosed patients and controls was performed to gather additional information. Timing of the detected overdose was confirmed through examination of the MAR. Clinical notes and nursing flowsheets were then reviewed to look for newly documented diarrhea or loose stool following the administered overdose.

All C. difficile orders during the study period were included for patient matching. Presence of a toxin assay order and the result (positive or negative) were aligned with included patients. Order data were included if patients had a toxin assay order within 4 weeks after an overdose administration. Repeat orders within 3 months were counted as the original assay. If the C. difficile result was before the antibiotic order, it was not included in the analysis.

Optimization and changes to the medication alert rules

Electronic medication dosing rules (dosing eRules) are typically more conservative than true clinical practice. Based upon clinical assessment of prior alerting data, alerts for the study medications were reviewed for alignment with actually clinical practice and adjusted by the project team pharmacist to decrease in appropriate alerting while still encouraging safe prescribing.

These changes included (1) the adjustment of the amoxicillin single dose maximum from 875 to 2000 mg; (2) the addition of an amoxicillin daily dose limit of 4000 mg; (3) the adjustment of the amoxicillin-clavulanate dosing rules to account for the clavulanate portion of the drug, which limits both the frequency of administration as well as the amount of amoxicillin that can be prescribed; and (4) the adjustment of the clindamycin single dose maximum from 450 to 900 mg.

Objectives and outcome measures

The primary objective of our study was to develop EHR-based algorithms that automate the detection of overdoses that reached patients. Secondary objectives included utilizing a use-case to demonstrate viability of the system, safety outcomes related to overdoses, and implementation of operational changes to increase the performance of the CDS related to antibiotics. Measures to achieve our objectives, specifically, safety outcomes were loose stool recorded in the medical chart following administration of an overdose and C. diff toxin assay orders and results. Our hypothesis for this objective was that the selected overdose cases had a higher rate of adverse events than a randomly selected sample of control patients receiving the same medications.

Data types and analytic measures

The data types collected in this study included the number of orders placed, single dose overdose (SDOD) orders placed, and the dosing alerts including user response (override, cancel, modify) generated by the EHR. We also collected the number of SDOD orders remaining after alerting, SDOD orders not modified, SDOD orders documented as being given to patients (MAR). From the EHR, we extracted the total true overdose orders after clinical validation and the total true overdoses given to patients. We calculated aggregate demographic values of patients receiving overdoses including age, sex, length of stay; distribution of sex and age of patients who received overdoses; and distribution of overdoses by magnitude patients received.

Medication order data was obtained from CCHMC’s clinical relational database for analysis. Additional data obtained through chart review included provider notes, intake and output flow sheets, nursing notes, and ED encounter notes. All clinical notes for patients detected to have an antibiotic overdose or were selected as the controls were evaluated for evidence of diarrhea or loose stool within 72 h of a documented overdosed (antibiotic) administration. Relevant observations included: stool consistency, color, and frequency. Patients with stool consistency mentioned in their record were included. Repeat visits were counted as new encounters. A random sample of patients each year with a medication administration but without a clinical overdose was included as controls.

The effects of the eRule changes were analyzed in the pre- and post-modification study periods. Measures of both the absolute counts and rates of the alerts pertaining to the three study antibiotics, as well as salience rates are reported. Salience rate, described in earlier work, is the degree to which a user reacted (in a corrective manner) to an overdose alert and is defined by the following¹²:

$$\text{Salience rate}=\frac{\text{Count of alerts heeded}}{\text{Count of alerts heeded}+\text{Count of alerts overridden}}$$

Statistical analysis

Each variable of interest was reported using median (interquartile range) or n (%) as appropriate. Patients who received overdoses were compared to controls with respect to continuous outcomes using the Wilcoxon Mann-Whitney test; Fisher’s exact test or Chi-square test, as appropriate, were used to examine differences with respect to categorical outcomes. Frequencies combined across year were compared between groups (SDOD and non-SDOD) using the chi-square test for independence. Given the relatively low frequency of events within group and year, the Cochran-Mantel-Haenszel test was used to compare annual safety outcome rates between groups. Groups were compared for each year using Fisher’s exact test, and Bonferroni-Holm adjustment was performed to account for multiple comparisons.¹⁵ Results with P-values less than the significance level (i.e., P < .05) were considered statistically significant. Analyses were implemented using R software for statistical computing.¹⁶

RESULTS

Order, overdose alert, and MAR data from the study period (2011–2014) for the selected study antibiotics were processed as per the study methodology. The pathways of the order and alerts, as well as the order end states, were mapped into a flow diagram (Figure 1). During the analysis, the determination of the definitive end state of a small proportion of the orders was beyond the scope of the study due to effort required to accomplish it. Such situations involved tracking down MAR status related to “Patient/Family Administration,” “Given by Other Clinician,” for example. For a variety of reasons, a small number of orders were unavailable for analysis and we chose to leave those orders as undetermined but quantified (see nodes in the Figure 1 labeled as “undetermined,” “possibly delivered,” and “no weight” nodes). Of the 55 770 orders, 18 162 (33%) were for amoxicillin, 6371 (11%) were for amoxicillin-clavulanate, and 31 237 (56%) were for clindamycin. About 1% of the total orders placed were actual overdoses that reached the patient. Five hundred and thirty-nine orders were true overdose orders. Of these 539 orders, 276 have 1 documented MAR delivery. The max number of MAR deliveries documented from one order was 44. In all, the 539 orders lead to a total of 1965 known overdose administrations.

Figure 1.

Antibiotic Order Lifecycle Flowchart. Antibiotic medication orders and associated dosing alerts as they progressed from the order to administration phases. A prescriber must initially select formulation and then dose either via manual entry or pre-set radio button. Order modification can be performed by either the provider or pharmacy depending on system response. MAR status reflects whether or not an order had an associated administration documented in the EHR. The overdose validation stage represents the application of our clinical overdose algorithms.

The demographics of the patients who received overdoses and the controls are shown in Table 2. Patients who received overdoses had longer lengths of stay.

Table 2.

Patient demographics and characteristics

Characteristic		Patients who received true overdoses			CCHMC patient population(during trial time period)
	Average (%)	Median	95% CI	Average (%)	Median	95% CI
Average LOS (days)*	9.0	2.6	(7.0-11.0)	5.0	1.8	(4.9-5.0%)
% LOS > 30*			(5.73-5.77%)	2.3		(2.3-2.3%)
Average LOS when LOS > 30 days	90.8	70.7	(74.5-107.1)	77.1	54.2	(74.8-79.4%)
Patient age (years)	7.7	6.5	(7.1-8.2)	8.76	7.8	(8.7-8.8%)
Gender (% male)	56.6		(56.5-56.6%)	52.9		(52.9-52.9%)
Race:
American Indian/AlaskaNative	0		(0-0%)	0.1		(0.1-0.1%)
Native Hawaiian/PacificIslander	0		(0-0%)	0.1		(0.1-0.1%)
Asian	0.6		(0.6-0.6%)	1.1		(1.1-1.1%)
Other/Refused/Unknown	6.7		(6.7-6.7%)	6.6		(6.5-6.6%)
Black/African American	18.7		(18.7-18.8%)	18.4		(18.4-18.4%)
Multiple	2.2		(2.2-2.2%)	2.7		(2.7-2.7%)
White or Caucasian	71.8		(71.8-71.8%)	71.1		(71.1-71.1%)
Ethnicity:
Hispanic	4.3		(4.3-4.3%)	4.0		(4.0-4.0%)
Non-Hispanic	94.8		(94.8-94.8%)	95.0		(95.0-95.0%)
Unknown/Refused	0.93		(0.93-0.93%)	1.01		(1.01-1.01%)

Asterisks denotes statistically significant values.

The age and sex distribution of the patients who received at least one antibiotic overdose are shown in Figure 2, categorized by both number of orders and number of total doses received. Patients 18 years of age and older were grouped together to ease the analysis.

Figure 2.

Age and sex distribution of patients who received oral antibiotic medication orders and overdoses.

Figure 3 demonstrates the number of delivered antibiotic overdoses by the order of overdose magnitude. The count of overdoses in the <10% overdose bin is minimal because our dose-range checking alert threshold is configured to allow up to a 10% rounding allowance under normal circumstances. Users can conjure dose-range checking however, which accounts for the data in that bin. The majority of overdoses were in the 20–39% range. A small but concerning number of overdoses (∼50 overdoses) were more than 3 times the single-dose alert upper threshold.

Figure 3.

Number of doses delivered by overdose magnitude. Number of overdoses delivered by order of magnitude. Our system allows for 10% rounding errors (does not alert for these very minor overdoses). Since the starting point for identifying potential overdoses begins with alerts, this group has very few instances as depicted in the bar chart.

The safety outcomes for the study are shown in Table 3. The overdose cohort had a greater percentage of loose stool recorded in chart than controls (P < .0001). Post-hoc analysis of individual years showed that the overdose cohort had significantly higher percentages of loose stool records, compared to controls. Although annual trends differed between the 2 cohorts with respect to orders for C. difficile tests (P = .04), post-hoc comparisons by year were not statistically significant after adjustment for multiple testing. Orders for C. difficile tests were significantly different overall between overdose cohort and controls. The numbers of patients with loose stool recorded in the chart and C. difficile toxin assay orders did not significantly differ across years for either group (P = .4 and .6, respectively). Figure 4 shows the effects of the dosing alert rule modifications. There are clear trends of an increase in alert salience as the total number of alerts presented to users decreased after modifications of the parameters in the eRules themselves.

Figure 4.

Effects of antibiotic rule changes. Time series chart of total alerts and salience rates for antibiotic rules, pre- and post-modification. After modification of the corresponding electronic rules, the monthly count of total alerts decreased while the salience of the alerts increased, indicating the prescribers were heeding the overdose alerts more frequently.

Table 3.

Safety outcomes

Outcome Measure	2011	2012	2013	2014	Total
SDOD Cases	110	128	159	142	539
Loose stool recorded in chart (%)	18 (16.3)**	16 (12.5)	20 (12.6)*	25 (17.6)	79 (17.1)††
Clostridium difficile toxin order	7 (6.4)	6 (4.7)	10 (6.3)	4 (2.8)	27 (5.0)†
Clostridium difficile positive toxina	0 (0)	0 (0)	1 (0.6)	0 (0)	1 (0.2)
Non-SDOD Cases	100	100	110	104	414
Loose stool recorded in chart	2 (2.0)	4 (4.0)	4 (3.6)	10 (9.6)	20 (4.8)
Clostridium difficile toxin orders	12 (12.0)	11 (11.0)	8 (7.3)	5 (4.8)	36 (8.7)
Clostridium difficile positive toxina	1 (1.0)	1 (1.0)	1 (0.9)	0 (0)	3 (0.7)

SDOD: Single dose overdose; ^†P < .01 and ^†† 0.05 > P ≥ 0.01 for overall group differences using Chi-square test; *0.05 > P ≥ 0.01 and **P < 0.01 for annual group comparison using Fisher’s exact test. ^aStatistical testing not performed due to low number of orders.

DISCUSSION

In this project we were able to detect, through an automated electronic algorithm, clinically relevant overdoses of several antibiotics. This was accomplished using combinations of retrospective ordering, alerting, and MAR data elements. We have augmented our previous overdose detection work by three main modifications: (1) the addition of MAR data to determine which medication overdose orders actually reached patients; (2) overlaying clinical overdose algorithms on top of the orders our CPOE/CDS system thought were overdoses based on electronic dosing rules; and (3) the addition of a highly-targeted manual chart review to detect association with known ADEs. This approach allowed us to simultaneously examine the performance of the antibiotic dosing rules employed by our CPOE/CDS system and to link detected overdoses to patient safety outcomes, thereby achieving the project’s 4 objectives. Our analytic database now allows us to follow medication orders through the entirety of the patient-medication lifecycle, beginning with ordering of a medication and ending with the administration and monitoring phases, and determining the outcomes of each order including whether or not CDS was effective in changing inappropriate prescribing behaviors. This modified overdose detection tool can be used for safety research, quality improvement, as well as advanced and agile review of EHR records to determine the clinical circumstances around the lifecycle of medication orders.

The specific use case of detecting antibiotic overdoses demonstrated several interesting clinical and informatics findings. First, only about 1% of all orders for our three candidate antibiotics were true overdoses (539 of the original 55 770 orders). Seventy-seven percent of the overdose-related alerts were overridden, a rate similar to other studies examining alert override rates among physicians.^17–19 This may be due to the oft-reported phenomenon of alert fatigue, whereby high false positive rate of alerts leads to inappropriate alerting and decreased mindfulness of physicians using the system.¹⁸ After the initial alerts presented to the prescribers were overridden, it was uncommon for overdose orders to be altered by any other safety net process. In fact, the most effective mitigation factor for these orders was if a medication dose was never delivered to the patient, which occurred about 1/3 of the time. Only 38% of orders that were alerted as overdoses and delivered to the patient were determined to be true overdoses when a pharmacist-created overdose algorithm was applied. This indicates a substantial gap between the CPOE dosing rule parameters and clinically acceptable dosing practice. Prior work by this group has shown that this gap often exists ≥50% of the time. The prevalence of dosing rule/practice gap may contribute even further to alert fatigue, especially in pediatrics where these gaps are often wider and more common.²⁰ Furthermore, this gap, and its likely contribution to alert fatigue, reinforces the need to present only clinically relevant overdoses from alerting systems to end-users and event detection systems.

Our algorithm was successful in detecting an increased association between overdoses and loose stools in the overdose cohort when compared to controls. Our observations in the SDOD group are classified as possible ADEs (score = 3) according to Naranjo et al.²¹ Attribution of causality was limited by the retrospective nature of our chart review methods. This system could be employed through addition of a causality algorithm to real-time or near real-time chart reviews to detect clinically relevant overdoses. Reduced time to detection of drug overdoses means actions such as real-time monitoring for relevant adverse events and even preventative or therapeutic actions could be initiated. Capitalization on the temporal advantages of an operational overdose detection system could dramatically improve the safety of patient care considering the prevalence of medication errors. From an organizational standpoint, accelerated detection and action would not only lead to safer patient care, but could also help lead to decision support and quality improvement efforts to mitigate risks associated with ADEs much more quickly than traditional methods of reporting and manual chart review. As noted above, we altered the dosing rules for the medications involved in this study to further optimize our decision support. Considering the potential improvements in medication safety, patient care, and risk mitigation, this work has the potential to reduce the burdens associated with adverse drug reactions. The success of this use case is encouraging for the future development of more accurate means to detect clinically meaningful medication errors, identify harm where warranted, and improve alerting and CDS to prevent these errors from reaching the patient.

Identification and correction of the electronic medication dosing rules (eRules) is necessary to optimize the currently employed rules-based CDS systems. We were able to demonstrate that, at least in the short term, we were able to decrease the alert burden for prescribers while simultaneously increasing the clinical specificity and salience of the eRules. It is yet to be determined if this increase in CDS performance for this small corpus of rules will translate to improved clinical outcomes. At a minimum, we have achieved greater efficiency in the antibiotic ordering process by eliminating noisy and useless alerts from the system and saving users time that can otherwise be spent delivering care to patients. The effect of this multiplies when considering that the decrease of alerts occurs at several levels (prescribers, pharmacists, nursing). Future work in this area should incorporate more reliable and real-time ADE detection into alert monitoring to ensure that modifications of dosing rules do not lead to detrimental changes in prescribing practices or ADEs.

It should also be noted that the ultimate success of this project relied on an engaged team of clinicians, pharmacists and informaticians to draw actionable conclusions from alerting data, perform chart reviews, and judiciously alter existing drug alerting rules. While parts of this process (notably the processing of alerting data) can be automated, applying the lessons learned here to other medications will require careful resource and effort allocation. For this reason, the process of identifying candidate medications should include frequency of ordering and ADEs for the medication, current alert salience and frequency, and possible methods of ADE detection.

Limitations

This study has several limitations, most notably its dependence on clinical documentation for identification of clinically significant overdose events. If a provider failed to document details of a patient’s diarrhea in appropriate and timely manner our review is likely to have missed a clinically significant overdose event. However, this reliance on documentation is true for both our overdose and control cohorts and would not explain the statistical significance between the 2 groups. It should be noted that the limitations of clinical documentation as a vehicle for identifying overdoses associated with diarrhea is also affected by length of stay in this inpatient study. Children ordered clinically significant overdoses who were transitioned to oral antibiotics just prior to discharge would be difficult to identify using this methodology. Our review also utilized alerts to detect overdose events. Our dependence on alerts to detect these events could lead to incomplete event capture in the case of faulty alert rules or alerting systems. Since this test case focused on a small number of medications, our group was able to verify the accuracy and function of dosing rules in place, leading us to believe all overdose events were captured. Since our work focused on oral antibiotics, we decided to look for the most common GI-related side effects. Our narrow focus could have missed other adverse drug events for these oral antibiotics. Lastly, our group has not yet studied the clinical outcomes of the modifications to our eRules.

Future directions

Future directions of this work include alternate methods of overdose detection which do not suffer from the shortcomings of our alert-based identification, further refinement in overdose detection, the addition of an causality algorithm for ADEs associated with overdoses, and better characterization of the clinical outcomes that result from our rule modifications. While the success of this test case is encouraging, spread of this method to other medications will require significant collaborative work to build a body of clinically significant medication dosing rules. More tests cases, focusing on medications that offer other dosing challenges are needed before operationalization can be considered.

CONCLUSION

The success of this test case suggests that gains are possible in reducing medication errors and improving patient safety with automated algorithm-based detection systems. Follow-up studies will determine if the positive effects of the system persist and if these changes lead to improved safety outcomes.

FUNDING

This project did not utilize any external funding.

COMPETING INTERESTS

None.

ACKNOWLEDGMENTS

The authors wish to acknowledge and thank Cecilia (Monifa) Mahdi for her data analysis assistance on this project.