## Abstract

Objective: Computerized monitors can effectively detect and potentially prevent adverse drug events (ADEs). Most monitors have been developed in large academic hospitals and are not readily usable in other settings. We assessed the ability of a commercial program to identify and prevent ADEs in a community hospital.

Design and Measurement: We prospectively evaluated the commercial application in a community-based hospital. We examined the frequency and types of alerts produced, how often they were associated with ADEs and potential ADEs, and the potential financial impact of monitoring for ADEs.

Results: Among 2,407 patients screened, the application generated 516 high priority alerts. We were able to review 266 alerts at the time they were generated and among these, 30 (11.3%) were considered substantially important to warrant contacting the physician caring for the patient. These 30 alerts were associated with 4 ADEs and 11 potential ADEs. In all 15 cases, the responsible physician was unaware of the event, leading to a change in clinical care in 14 cases. Overall, 23% of high priority alerts were associated with an ADE (95% confidence interval [CI] 12% to 34%) and another 15% were associated with a potential ADE (95% CI 6% to 24%). Active surveillance used approximately 1.5 hours of pharmacist time daily.

Conclusions: A commercially available, computer-based ADE detection tool was effective at identifying ADEs. When used as part of an active surveillance program, it can have an impact on preventing or ameliorating ADEs.

## Introduction

Readily deployable adverse drug event (ADE) monitors that effectively track and help prevent ADEs would be very helpful to hospitals without sophisticated electronic health records (EHRs). However, prior evaluations have focused on home-grown systems that are usually tied to sophisticated EHRs.1,2 It is unknown whether a commercially-available system that might be deployable by a large number of hospitals can be used to identify and ameliorate ADEs in community-based hospitals.

The Vigilanz Corporation has developed the Dynamic Pharmacovigilance (DPV), which monitors laboratory and pharmacy data among hospitalized patients and uses preset rules to determine whether an adverse event might be taking place.3 Given that commercial systems like this could be widely used by hospitals across the nation, as all that are required are demographics and computerized laboratory and drug data which many hospitals already have, we prospectively evaluated the ability of this application to identify ADEs, avert ADEs and examine the burden they might place on resources of a community-based hospital.

## Background

Adverse events represent a major cause of morbidity and mortality for hospitalized patients and medications are the single biggest cause of this type of medical injury.4,5 Each year in the United States, adverse drug events (ADEs) may contribute to the deaths of up to 140,000 Americans and have substantial financial costs to both patients and the healthcare system.6 Despite their heavy burden, it has been difficult to monitor and prevent ADEs. Most hospitals still use voluntary reporting by clinicians as their primary source of surveillance data, although numerous studies have shown that this approach identifies less than 1% of all ADEs.7 A more comprehensive approach, the use of chart review by nurses or pharmacists, is prohibitively expensive and therefore impractical for routine use, although sampling and use of trigger criteria can be helpful8 but still expensive when applied to all patients. Computer-based monitors represent a sensible compromise: although less sensitive than chart review, they are far more effective at identifying ADEs than self-report,9 and can detect a substantial proportion of ADEs.1,2,10 These monitors were developed nearly 30 years ago,9 and successive evaluations have shown that they can effectively identify nearly as many ADEs as manual chart review.1 A recent Institute of Medicine report suggested that this likely represents the future of ADE detection.11

A major challenge, however, has been implementing such monitors in routine care, much of which is delivered in medium-sized and small community hospitals that do not have sophisticated information systems.12 Most of the data on computer-based monitoring of ADEs come from large academic centers that have internally-developed electronic health records (EHRs), which concomitantly built computerized monitoring tools.1,2 Although most small hospitals do have an electronic patient tracking system (often known as an Admission/Discharge/Transfer system or ADT), as well as laboratory and pharmacy information in coded electronic format,13,14 these systems are rarely linked.15 Having an effective ADE surveillance system that can readily be installed in hospitals without EHRs but with these three databases (ADT, laboratory, and pharmacy) could make computerized monitoring much more accessible to a broad range of hospitals. Therefore, we sought to determine whether a commercially-available computer-based monitoring system that can be easily installed in most hospitals could help identify and prevent ADEs.

## Methods

### Setting

We used data from a 150-bed nonprofit, community-based teaching hospital located in Boston, MA. In fiscal year 2005, it had 8,200 discharges and cared primarily for general medical and surgical patients. The hospital used information systems sold by MEDITECH (Westwood, MA) that primarily had demographics, laboratory and radiology results, and discharge summaries available on most of the inpatient floors at the time of the study. The hospital did not have an EHR but was in the early stages of implementing a computerized physician order entry (CPOE) system. To install the DPV, the main requirement was that the hospital have three databases: one that contains laboratory results, another with pharmacy data, and a third with patient demographics such as an admission/discharge/transfer database. Prior surveys have found that most hospitals in the U.S. have these three sources of data available in electronic format.13 The installation time for the vendor system is typically less than 90 days.

### Patients

The patients in this study included all adults admitted to five inpatient units during a 4 month period from July to August and from November to December 2004. The units included two medical units, two general surgical units, and a medical intensive care unit. During the study period, a total of 2,407 patients were admitted to these units and screened by the DPV system.

### Outcomes

The outcomes measured were presence of an adverse drug event (ADE) or a potential adverse drug event (PADE). An ADE was defined as an injury resulting from an intervention related to a drug.4 An injury could be a sign (e.g., bleeding), a symptom (e.g., vomiting) or a dangerously abnormal laboratory value (e.g., hemoglobin of 6 gm/L). A potential ADE was defined as a medication error with the potential to cause injury but in which no actual injury occurred.4 A medication error was defined as an error in the prescribing, dispensing, transcription, administration, or monitoring of a drug.16 Actual ADEs and potential ADEs were considered to be preventable if the injury (or potential injury) was due to an identifiable medical error that could be prevented given today's technology and knowledge-base.

### The DPV System and Rule Base

The rules were developed using combinations of medications, laboratory data and patient demographic data that met certain conditions. For example, a patient on gentamicin with a rising serum creatinine level would trigger an alert. Medications were identified by a pharmacy order (as opposed to a medication administration), and the rules allowed for linking of multiple medications and laboratory tests. An example of this would be a potential bleed in the setting of an order for heparin in conjunction with a recent abciximab order. The use of these medications together would activate the second part of the rule to monitor the lab results of the hematocrit and associated red blood cell count or hemoglobin level.

The rule set designates the alerts with a low, medium, high or critical priority, and a baseline level is established for the key lab within the rule. In the previous gentamicin example, a serum creatinine level greater than 1.3 mg/dL but less than or equal to 2.5 mg/dL would be a low priority alert. A serum creatinine level greater than 2.5 mg/dL but less than or equal to 3.5 mg/dL would constitute a medium priority alert, while a high priority alert would require a serum creatinine level greater than 3.5 mg/dL but less than or equal to 3.9 mg/dL. Finally, any serum creatinine level greater than 3.9 mg/dL would be classified as a critical alert. The system is designed with the flexibility to allow individual institutions to adjust these thresholds based on local practice and priorities.

A rule was considered to be activated if all of the rule's criteria were met (i.e., patient is on gentamicin and patient has an elevated creatinine). The activated rule could change to an alert if a designated amount of time, called the Good Medical Practice (GMP) interval, had passed and the medication had not been stopped, a new test had not been ordered, or the medication dose of interest had been decreased. Depending on the urgency of the alert, the GMP interval varied from 2 hours (e.g., critical serum potassium) to 1 day (e.g., colchicine and serum creatinine greater then 2.0 g/dL in last 1 day) before another alert was generated.

### Case Identification

All high-priority alerts generated by the DPV system were evaluated each morning (excluding weekends, holidays, and other days when a reviewer was not available) by either a pharmacist or a physician. The alert report contained patient identifiers, demographics, and details regarding the alert. The reviewer evaluated the alert based on the clinical data available in the hospital computer system and standard references, such as Thomson's Micromedex,17 and expert guidelines. If the reviewer thought the patient was being treated appropriately or if there was evidence that the clinicians knew about the unfolding event, no further action was taken. However, if the reviewer considered this to be a potentially important clinical event where a call to the physician might be valuable, the review would contact that provider and convey the information. The reviewer kept track of the time required for evaluating alerts and contacting the physician caring for the patient.

All patients with alerts that required contact with the physician underwent a manual chart review. For patients with alerts where the reviewer had decided that no MD contact was required, we reviewed 10% of the charts for further evaluation. Finally, because many of the alerts were never reviewed (because they occurred on days when the reviewer was not available), we also sampled 10% of all patients with such alerts to determine how many ADEs were missed due to the lack of daily evaluation of the alerts.

### Analysis

We had three primary sets of results. In the first analysis, we examined whether the VigiLanz monitoring system helped identify ADEs and potential ADEs. Second, we determined whether the surveillance process (generation of alerts, review by a physician or pharmacist, and subsequent calls to responsible clinician) would avert ADEs or potential ADEs. Third, we tracked the efficiency of the VigiLanz system by examining the amount of pharmacist time used during the surveillance process.

### Funding

The VigiLanz Corporation provided limited financial support for this project. The hospital paid a discounted price (compared to the retail price) for the use of the software. The company provided no resources for the evaluation of its product and had no input into the analytic plan. The lead author (AKJ) did not receive any financial compensation, although one co-author (DWB) did receive an honorarium for a talk he gave on ADE monitoring at the annual meeting of the American Society for Health Systems Pharmacy. This talk was given prior to the initiation of any planning for the study. Finally, the company had no editorial input into the writing of the manuscript and was not shown any drafts of the manuscript before submission.

## Results

During the study period, the application screened 2,407 general medical and surgical patients admitted to the wards or the intensive care unit of an academic community hospital. The average age of the patients was 73.7 years (standard deviation 14.8 years) and 52.8% of them were female. Overall, 88.4% of the patients were admitted to a ward and 11.6% were admitted to the ICU.

Among the 2,407 screened patients, the application generated 516 high priority alerts among 352 (15% of all) patients, in which both a set of clinical criteria had been met and the GMP timeframe had expired. The most common trigger for a high priority alert was a patient on enoxaparin with a rapidly rising or very high serum creatinine, accounting for nearly 15% of all alerts. Most of these cases were due to inappropriately high enoxaparin doses in the setting of rising creatinine (acute renal failure) or in patients with severe renal insufficiency (chronic renal failure). Other drugs and their use in the setting of renal failure were also responsible for triggering a large number of high priority alerts: levofloxacin inappropriately dosed in the setting of renal failure was the second most common cause of alerts (nearly 14% of all high priority alerts) and use of angiotensin-converting enzyme inhibitors (ACE-I, captopril and lisinopril) represented the 4th and 5th most common triggers of high priority alerts. Critically elevated serum potassium levels constituted 43 (8.3%) of all high priority alerts (Table 1).

Table 1

Frequency of Alert Drug and Lab for All Alerts Generated by VigiLanz (N=516)*

 Alert Drug Alert Lab Number of Alerts % of Alerts Cumulative % of Alerts Enoxaparin serum Creatinine 77 14.9% 14.9% Levofloxacin serum Creatinine 71 13.8% 28.7% None Critical serum Potassium 43 8.3% 37.0% Captopril serum Creatinine 38 7.4% 44.4% Lisinopril serum Creatinine 35 6.8% 51.2% Enoxaparin Low Platelet Count 18 3.5% 54.7% Ciprofloxacin serum Creatinine 18 3.5% 58.1% Vancomycin serum Creatinine 18 3.5% 61.6% Atenolol serum Creatinine 16 3.1% 64.7% Allopurinol serum Creatinine 15 2.9% 67.6% Furoseamide serum Creatinine 15 2.9% 70.5% Enoxaparin Hematocrit 14 2.7% 73.3% Digoxin serum Creatinine 14 2.7% 76.0% Digoxin Serum Potassium 13 2.5% 78.5% Famotidine serum Creatinine 12 2.3% 80.8% Heparin Low Platelet Count 10 1.9% 82.8% Aspirin, NSAIDs serum Creatinine 9 1.7% 84.5% Heparin PTT 9 1.7% 86.2% Famotidine Low Platelet Count 8 1.6% 87.8% Amiodarone & Digoxin None 5 1.0% 88.8% Gentamicin serum Creatinine 5 1.0% 89.7% Ibuprofen serum Creatinine 5 1.0% 90.7% Refecoxib serum Creatinine 4 0.8% 91.5% Clopidogrel serum Creatinine 3 0.6% 92.1% Acyclovir serum Creatinine 3 0.6% 92.6% Metformin serum Creatinine 3 0.6% 93.2% Ranitidine serum Creatinine 3 0.6% 93.8% Glyburide serum Creatinine 2 0.4% 94.2% Others Others 29 5.8% 100% Total 516 100% 100%
 Alert Drug Alert Lab Number of Alerts % of Alerts Cumulative % of Alerts Enoxaparin serum Creatinine 77 14.9% 14.9% Levofloxacin serum Creatinine 71 13.8% 28.7% None Critical serum Potassium 43 8.3% 37.0% Captopril serum Creatinine 38 7.4% 44.4% Lisinopril serum Creatinine 35 6.8% 51.2% Enoxaparin Low Platelet Count 18 3.5% 54.7% Ciprofloxacin serum Creatinine 18 3.5% 58.1% Vancomycin serum Creatinine 18 3.5% 61.6% Atenolol serum Creatinine 16 3.1% 64.7% Allopurinol serum Creatinine 15 2.9% 67.6% Furoseamide serum Creatinine 15 2.9% 70.5% Enoxaparin Hematocrit 14 2.7% 73.3% Digoxin serum Creatinine 14 2.7% 76.0% Digoxin Serum Potassium 13 2.5% 78.5% Famotidine serum Creatinine 12 2.3% 80.8% Heparin Low Platelet Count 10 1.9% 82.8% Aspirin, NSAIDs serum Creatinine 9 1.7% 84.5% Heparin PTT 9 1.7% 86.2% Famotidine Low Platelet Count 8 1.6% 87.8% Amiodarone & Digoxin None 5 1.0% 88.8% Gentamicin serum Creatinine 5 1.0% 89.7% Ibuprofen serum Creatinine 5 1.0% 90.7% Refecoxib serum Creatinine 4 0.8% 91.5% Clopidogrel serum Creatinine 3 0.6% 92.1% Acyclovir serum Creatinine 3 0.6% 92.6% Metformin serum Creatinine 3 0.6% 93.2% Ranitidine serum Creatinine 3 0.6% 93.8% Glyburide serum Creatinine 2 0.4% 94.2% Others Others 29 5.8% 100% Total 516 100% 100%
*

352 patients generated 516 alerts. For patients with multiple alerts of the same type, only the first alert was counted.

NSAIDS = nonsteroidal antiinflammatory drugs; PTT = Partial Thromboplastin Time.

### Alerts Reviewed and Their Outcomes

Of these alerts, 266 (52%) occurred on days when a reviewer was available to examine the alerts and make a determination as to the clinical relevance of the alert while 250 occurred on either weekends, holidays, or on days where a reviewer was not otherwise available. The alerts were typically reviewed between 12 and 24 hours after they were triggered by the system. Of the 266 alerts reviewed prospectively, 30 (11.3%) alerts (among 26 patients) were considered substantially important to warrant contacting the physician caring for the patient (Table 2).

Table 2

Summary of Alert Characteristic Generated by VigiLanz during Study Period

*

Based on 10% sample of patients.

When we reviewed cases that triggered reviewer calls, we found that there were 4 ADEs and 11 potential ADEs among the 30 alerts (positive predictive value of 15/30 or 50%) among these 26 patients. All 4 ADEs were serious and all 4 were also considered preventable. Of the 11 potential ADEs, 1 was classified as life-threatening, 8 as serious, and 2 as significant (Table 2). In all 15 cases, the resident physician caring for the patient was not aware of the ADE or PADE. In 14 of the 15 cases, the resident physician made a change in clinical care (changing the dose of the medication or stopping the offending medication).

Two ADEs and 8 potential ADEs in this group were related either to renal failure from a drug or to inappropriate dosing in the setting of renal failure (Table 3). One of these cases, sulindac and increasing creatinine, was a situation in which a patient with congestive heart failure and baseline renal insufficiency was given sulindac with resultant renal failure. The resident physicians were not aware that the sulindac was a likely culprit in this instance and after receiving the call from the reviewer, the medication was stopped. One ADE and one PADE were due to the use of anti-thrombotic agents in the setting of low platelet count leading either to bleeding (ADE) or to severe thrombocytopenia (PADE).

Table 3

Frequency of Alert Drug and Lab for all ADEs and Potential ADEs Detected in Alerts with MD contact (N=15)

 Alert Drug Alert Lab Number of Alerts Adverse Event ADEs Sulindac Serum Creatinine 1 Renal Failure Famotidine Low Platelet Count 1 Bleeding Digoxin Serum Creatinine 1 Arrhythmia Carbamazepine Pancytopenia 1 Worsening infection Total 4 Potential ADEs Vancomycin Serum Creatinine 3 Markedly elevated vancomycin levels Levofloxacin Serum Creatinine 3 Inappropriately high levofloxacin dose Enoxaparin Low Platelet Count 2 Potential bleeding Enoxaparin Serum Creatinine 1 Potential bleeding Clopidogrel, aspirin Low Platelet Count 1 Potential bleeding NSAIDs, Aspirin Serum Creatinine 1 Potential bleeding Total 11
 Alert Drug Alert Lab Number of Alerts Adverse Event ADEs Sulindac Serum Creatinine 1 Renal Failure Famotidine Low Platelet Count 1 Bleeding Digoxin Serum Creatinine 1 Arrhythmia Carbamazepine Pancytopenia 1 Worsening infection Total 4 Potential ADEs Vancomycin Serum Creatinine 3 Markedly elevated vancomycin levels Levofloxacin Serum Creatinine 3 Inappropriately high levofloxacin dose Enoxaparin Low Platelet Count 2 Potential bleeding Enoxaparin Serum Creatinine 1 Potential bleeding Clopidogrel, aspirin Low Platelet Count 1 Potential bleeding NSAIDs, Aspirin Serum Creatinine 1 Potential bleeding Total 11

The DPV system helped identify several patients who had potential ADEs due to very high vancomycin levels. The triggering rule was vancomycin and elevated creatinine but on further examination, the reviewer found that vancomycin levels were nearly three times the upper limit of therapeutic range. When the reviewer contacted the medical teams, each was aware of the elevated levels but the clinician neither knew the reasons for the elevated levels nor the fact that there were other patients (cared for by other clinicians) who had similarly elevated levels during the same time period. The reviewer saw the multiple alerts and was able to determine that the underlying issue was dual administration of vancomycin in patients on hemodialysis. Three patients had received vancomycin on the wards where a CPOE system had been partly implemented, while the attending physician in the dialysis unit, where CPOE had not yet been installed, was using paper records to administer the dose in the dialysis unit. Neither the medical team nor the dialysis physician was aware that the other was giving the patient vancomycin. Catching this double-dosing allowed the hospital to create a new program in the pharmacy that would check for these duplicate orders.

When we examined a sample of the 236 alerts that were examined by the reviewer but no action was taken, we found that 12% of these alerts were associated with adverse drug events and another 18% were associated with potential adverse drug events. These typically involved abnormal potassium levels or rising creatinine in the setting of nephrotoxic drugs—but in each instance, the physician had already begun to address the issue by the time the reviewer had examined the alert.

### Alerts Not Reviewed and Their Outcomes

When we combined all of our results from both those reviewed and those not reviewed and examined overall ADE and PADE rates among all high priority alerts, we found that 23% of alerts (95% confidence interval 12% to 34%) were associated with an ADE and an additional 15% of alerts (95% CI 6% to 24%) were associated with a PADE for a combined rate of 38% (95% CI 25% to 51%).

### Comparison of Cost of Monitoring with Costs of ADEs

There were important costs of monitoring that must be considered. The reviewer typically spent 1.5 hours per day during the 70 days of the study period when he or she was able to review the alerts. If the hourly wage of the reviewer, who was typically a pharmacist, was $75 per hour (including benefits), the estimated costs for using the pharmacist were approximately$7,900 during the study period, annualized to approximately $40,000 (if we assume daily monitoring for 365 days per year). Further, for a 150-bed community-based hospital, the retail price for installation is approximately$25,000 with annual licensing fees of approximately $35,000. A hospital that chose to have daily monitoring by a pharmacist might spend up to$80,000 annually in pharmacist time, licensing cost, and installation (amortized over five years) cost. Given that each preventable adverse event costs well over \$5,000 in 2007 dollars,20 one would have to prevent approximately 16 ADEs to recoup the costs of the system and the time it takes a pharmacist to review the alerts and contact physicians when appropriate. Of course, whether hospitals recoup the costs of prevented ADEs is dependent heavily on payer-mix and how hospitals are reimbursed.

## Discussion

We prospectively evaluated a commercially-available adverse drug event monitoring system in a community hospital and found that it was effective at identifying ADEs and potential ADEs. When used in conjunction with a pharmacist reviewer who examined the charts daily, the system identified a large number of ADEs and potential ADEs that were unknown to the physician caring for the patient in nearly every instance and lead to a change in clinician decision making. The system was efficient, requiring less than 90 minutes of pharmacist time to evaluate patients on 4 large hospital wards and an ICU. Only about half of signals were reviewed for logistical reasons; the sample of signals that occurred on off hours revealed an even higher rate of adverse events and near misses, which might be expected, as staffing is generally lower during off-peak times. Our study offers evidence that a commercially-available system was easily deployed in a community-based setting and was effective at detecting and preventing ADEs.

Others have evaluated electronic monitoring systems before,1,2,9,21–23 but they have usually been examined in large academic centers using homegrown systems. In this study, we evaluated a commercially available program that can be implemented in community-based hospitals even if they do not have pre-existing advanced information systems. While others have examined ADE surveillance in community hospitals,21,24 one recent study is particularly relevant. Kilbridge and colleagues recently used a combination of automated surveillance and voluntary reporting to compare rates and characteristics of ADEs in an academic and a community hospital in the Duke University Health System.25 They found that the ADE rates in the community hospital were substantially higher than those in the academic medical center. While the Kilbridge evaluation was innovative in examining the effectiveness of computerized ADE monitoring in community hospitals, they used a home-grown system that worked well because the community hospital shared common structural components and IT infrastructure with the academic center.

Although few hospitals have adopted computerized surveillance systems to address ADEs, some community-based hospitals have implemented other measures. Cohen et al. examined efforts by Missouri Baptist Medical Center to decrease ADEs by attempting to change the safety culture of the institution, hiring a patient safety specialist and offering rewards to staff for safety ideas.26 An automated surveillance system might have helped to determine if their efforts were successful.

For hospitals looking for tools to screen patients and track ADEs, our approach may be reasonable. While prior studies have found that approximately 4% to 6% of hospitalized patients suffer an ADE, the rate of ADEs among patients with alerts generated by the DPV system was substantially higher, suggesting that this tool likely identifies a group of high risk patients. Therefore, the DPV can even be used to identify a pool of high-risk patients. The ability to customize would allow individual hospitals to choose or modify rules to meet their clinical needs and to have the capacity to follow up signals. Furthermore, it is important to note that this approach is still beneficial even if CPOE is in place, because even though CPOE reduces medication errors,18 many errors and ADEs still occur in hospitals with CPOE.27 In addition, CPOE can create new errors, which computerized monitoring can help identify.

An important use of this program was the live-time alerts which, when used in conjunction with a reviewer, seemed effective at identifying ADEs and potential ADEs early and helping to change clinical management. We were only able to have a covering reviewer for approximately half the days that the system was running and therefore, reviewed high priority alerts on 152 patients. However, the reviewer intervened on 1 in 6 of these patients (30 phone calls on 26 patients) leading to a change in practice in 15 cases. Therefore, among all patients with a high priority alert, a reviewer was able to dismiss alerts on 80% of patients as being not significant without having to make a call. The phone calls, which occurred on 20% of high priority alerts, led to a change in clinical care in 50% of the cases.

There are two other benefits of this program. First, its ability to highlight the mini-epidemic of elevated vancomycin levels was extremely useful in allowing the hospital to design a program to prevent duplicate orders. Over time, systems such as DPV will likely help highlight other defects and errors that occur in hospitals' medication ordering and administration systems. A second benefit is that the DPV system highlighted important areas for future education to improve medication prescribing. For example, we found that a large number of alerts were due to inappropriate dosing in the setting of renal dysfunction. This could easily become a target for continuing medical education for the physicians who cared for patients in a particular hospital. Alternatively, an organization might decide to prioritize implementation of renal dosing decision support, which has been found to be effective.28 By engaging in ongoing monitoring, hospitals have the ability to identify where important deficiencies might lie and target interventions to rectify those deficiencies.

Our study has important limitations. First, while we evaluated one particular system, there are other commercial systems now becoming available that have very similar features. It is possible, even likely, that they can be effective at identifying and possibly preventing ADEs. Secondly, the DPV system was examined with a limited set of rules and did not include data from other key areas such as microbiology, because coded microbiology data were not available. Prior studies have shown that a major cause of ADEs (and other adverse events) in community hospitals is antibiotic-associated colitis or other hospital-acquired infections.21 A modified system that included data from these or other types of results (such as pathology or cardiology) would allow more robust rules to be created to improve quality and safety in these hospitals. Another limitation of the study is that although the alerts were generally evaluated in the morning (based on activations that had occurred up to 24 hours earlier), it is possible that physicians would have identified those issues themselves even without reviewer intervention.

Another important limitation of our study is that we only sampled 10% of the alerts that did not lead to phone calls by the reviewer for further chart review. Therefore, although we were able to make estimates about the rates of ADEs and potential ADEs in those and other groups of patients, given the small sample sizes, our confidence intervals for most of our estimates were wide. Although we were able to identify a high rate of ADEs using the DPV system, it clearly missed many types of ADEs, including those that lead to symptoms not identifiable by this screening tool (i.e., vomiting due to a medication). Finally, two important questions remain: are commercially-available systems truly widely deployable, and will other hospitals that adopt such systems see comparable benefits in detecting ADEs?

In conclusion, we examined the DPV surveillance tool in a community hospital and found that it readily identified a group of patients who were at high risk for an adverse drug event or potential adverse drug event. Further, we found that when a reviewer was available to spend less than 90 minutes a day to review the alerts and make phone calls, the DPV system allowed the reviewer to intervene and change clinical practice, likely averting a substantial number of ADEs. Although the system could benefit from additional refinement, its ability to be deployed in hospitals with rudimentary electronic systems makes it one effective option for hospitals dedicated to monitoring and reducing ADEs.

## References

1
Jha
AK
Kuperman
GJ
Teich
JM
et al
.
Identifying adverse drug events: development of a computer-based monitor and comparison with chart review and stimulated voluntary report
.
J Am Med Inform Assoc

May–Jun
1998
;
5
(
3
):
305
314
.
2
Classen
DC
Pestotnik
SL
Evans
RS
Burke
JP
.
Computerized surveillance of adverse drug events in hospital patients
.
JAMA

Nov
27
1991
;
266
(
20
):
2847
2851
.
3
Gandhi
TK
Seger
DL
Bates
DW
.
Identifying drug safety issues: from research to practice
.
Int J Qual Health Care

Feb
2000
;
12
(
1
):
69
76
.
4
Bates
DW
Cullen
DJ
Laird
N
et al
.
ADE Prevention Study Group Incidence of adverse drug events and potential adverse drug events. Implications for prevention
.
JAMA

Jul
5
1995
;
274
(
1
):
29
34
.
5
Leape
LL
.
Reporting of adverse events
.
N Engl J Med

Nov
14
2002
;
347
(
20
):
1633
1638
.
6
Lazarou
J
Pomeranz
BH
Corey
PN
.
Incidence of adverse drug reactions in hospitalized patients: a meta-analysis of prospective studies
.
JAMA

Apr
15
1998
;
279
(
15
):
1200
1205
.
7
Cullen
DJ
Bates
DW
Small
SD
Cooper
JB
Nemeskal
AR
Leape
LL
.
The incident reporting system does not detect adverse drug events: a problem for quality improvement
.
Jt Comm J Qual Improv

Oct
1995
;
21
(
10
):
541
548
.
8
Resar
RK
Rozich
JD
Simmonds
T
CR
.
A trigger tool to identify adverse events in the intensive care unit
.
Jt Comm J Qual Patient Saf

Oct
2006
;
32
(
10
):
585
590
.
9
Hulse
RK
Clark
SJ
Jackson
JC
Warner
HR
Gardner
RM
.
Computerized medication monitoring system
.
Am J Hosp Pharm

1976
;
33
(
10
):
1061
1064
.
10
Murff
HJ
Forster
AJ
Peterson
JF
Fiskio
JM
Heiman
HL
Bates
DW
.
Electronically screening discharge summaries for adverse medical events
.
J Am Med Inform Assoc

Jul–Aug
2003
;
10
(
4
):
339
350
.
11
Aspden
P
Wolcott
J
Bootman
JL
Cronenwett
L
.
Preventing Medication Errors: Quality Chasm Series
.
Washington DC
:
Institute of Medicine
;
2006
.
12
Jha
A
Ferris
T
Donelan
K
et al
.
How Common Are Electronic Health Records in the United States?. A Summary of the Evidence
.
Health Affairs

2006
;
25
(
6
):
496
507
.
13
Laschober
M
Maxfield
M
Lee
M
Kovac
M
Potter
F
Felt-Lisk
S
.
Hospital Responses to Reporting of Quality Data to CMS: 2005 Survey of Hospitals
.
Washington, D.C.
:
Mathematica Policy Research, Inc.
;
October
12
,
2005
.
14
HIMSS Analytics Essentials of the US Hospital IT Market
.
Chicago, IL
:
HIMSS Analytics
;
2007
.
15
Schiff
GD
Klass
D
Peterson
J
Shah
G
Bates
DW
.
Linking laboratory and pharmacy: opportunities for reducing errors and improving care
.
Arch Intern Med

Apr
28
2003
;
163
(
8
):
893
900
.
16
Bates
DW
Teich
JM
Lee
J
et al
.
The impact of computerized physician order entry on medication error prevention
.
J Am Med Inform Assoc

Jul–Aug
1999
;
6
(
4
):
313
321
.
17
Micromedex [computer program]. Version: Thomson Healthcare
. Available at: http://www.micromedex.com. Accessed June 15, 2008.
18
Bates
DW
.
Drugs and adverse drug reactions: how worried should we be?
JAMA

Apr
15
1998
;
279
(
15
):
1216
1217
.
19
Naranjo
CA
Busto
U
Sellers
EM
et al
.
A method for estimating the probability of adverse drug reactions
.
Clin Pharmacol Ther

Aug
1981
;
30
(
2
):
239
245
.
20
Bates
DW
Spell
N
Cullen
DJ
et al
.
The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group
.
JAMA

Jan
22–29
1997
;
277
(
4
):
307
311
.
21
Raschke
RA
Gollihare
B
Wunderlich
TA
et al
.
A computer alert system to prevent injury from adverse drug events: development and evaluation in a community teaching hospital
.
JAMA

Oct
21
1998
;
280
(
15
):
1317
1320
.
22
Evans
RS
Pestotnik
SL
Classen
DC
Horn
SD
Bass
SB
Burke
JP
.
Preventing adverse drug events in hospitalized patients
.
Ann Pharmacother

Apr
1994
;
28
(
4
):
523
527
.
23
Levy
M
Azaz-Livshits
T
B
Shalit
M
Geisslinger
G
Brune
K
.
Computerized surveillance of adverse drug reactions in hospital: implementation
.
Eur J Clin Pharmacol

Jan
1999
;
54
(
11
):
887
892
.
24
Grasela
TH
Walawander
CA
Kennedy
DL
Jolson
HM
.
Capability of hospital computer systems in performing drug-use evaluations and adverse drug event monitoring
.
Am J Hosp Pharm

Sep
1993
;
50
(
9
):
1889
1895
.
25
Kilbridge
PM
Campbell
UC
Cozart
HB
MG
.
Automated surveillance for adverse drug events at a community hospital and an academic medical center
.
J Am Med Inform Assoc

Jul–Aug
2006
;
13
(
4
):
372
377
.
26
Cohen
MM
Kimmel
NL
Benage
MK
et al
.
Medication safety program reduces adverse drug events in a community hospital
.
Qual Saf Health Care

Jun
2005
;
14
(
3
):
169
174
.
27
Koppel
R
Metlay
JP
Cohen
A
et al
.
Role of computerized physician order entry systems in facilitating medication errors
.
JAMA

Mar
9
2005
;
293
(
10
):
1197
1203
.
28
Chertow
GM
Lee
J
Kuperman
GJ
et al
.
Guided medication dosing for inpatients with renal insufficiency
.
JAMA

Dec
12
2001
;
286
(
22
):
2839
2844
.
Dr. Jha was supported by the Physician Faculty Scholar program of the Robert Wood Johnson Foundation.
The VigiLanz Corporation provided limited financial support for this project. The hospital paid a discounted price (compared to the retail price) for the use of the software. The company provided no resources for the evaluation of its product and had no input into the analytic plan. The lead author (AKJ) did not receive any financial compensation, although one co-author (DWB) did receive an honorarium from VigiLanz for a talk he gave on ADE monitoring at the annual meeting of the American Society for Health Systems Pharmacy. This talk was given prior to the initiation of any planning for the study. Finally, the company had no editorial input into the writing of the manuscript and was not shown any drafts of the manuscript before submission. All data were collected and analyzed independently of the company.