## Abstract

Alerts and prompts represent promising types of decision support in electronic prescribing to tackle inadequacies in prescribing. A systematic review was conducted to evaluate the efficacy of computerized drug alerts and prompts searching EMBASE, CINHAL, MEDLINE, and PsychINFO up to May 2007. Studies assessing the impact of electronic alerts and prompts on clinicians' prescribing behavior were selected and categorized by decision support type. Most alerts and prompts (23 out of 27) demonstrated benefit in improving prescribing behavior and/or reducing error rates. The impact appeared to vary based on the type of decision support. Some of these alerts (n = 5) reported a positive impact on clinical and health service management outcomes. For many categories of reminders, the number of studies was very small and few data were available from the outpatient setting. None of the studies evaluated features that might make alerts and prompts more effective. Details of an updated search run in Jan 2009 are included in the supplement section of this review.

## Introduction

Prescribing medication represents an opportunity for providing appropriate and effective treatment, and yet medication errors are responsible for considerable levels of morbidity and mortality. Many errors which result in harm can potentially be prevented by the use of appropriately designed computerized clinical decision support (CDS), in particular computerized alerts and prompts.

## Background

Medication errors are common and cause substantial harm; the injuries they cause are called preventable adverse drug events (ADEs). Studies have estimated that 2.4–3.6% of hospital admissions are caused by ADEs of which up to 69% would have been preventable.1 The incidence rate of ADEs in hospitalized patients is thought to be between 0.7 and 6.5% of which the preventable proportion reported has ranged from 17 to 56%.2 Since the main error in most preventable ADEs occurs at the stage of drug ordering3 increasingly sophisticated clinical computer systems have been seen as a major opportunity to prevent inappropriate prescribing.

Clinical provider order entry (CPOE) systems are computer-assisted information systems that enable providers to write all orders, including prescribing medications. Computerized decision support systems (CDSS) are linked with CPOE to provide support for decision-making in patient care by integrating clinical and patient information. They range from simple to more complex decision support systems,4 with all the more complex using patient-specific data and some of the most complex involving detailed clinical protocols and patient specific data. The category of decision support used and its timing within the clinical workflow are important questions to be addressed when CDSS are implemented.5 Reminding and alerting represent an important category of providing decision support to the clinician at the point of care. In the area of prescribing, drug reminder or drug alerting systems generate information that warn healthcare staff against potential prescribing errors or support them in their prescribing decisions.

Several systematic and narrative reviews of aspects of CDSS in healthcare have shown general benefits, such as improved practitioner performance and reduced error rates.2,6–14 A Cochrane review on computerized dosing advice even showed moderate but significant reduction in hospital stay.15 However, none of the reviews have specifically evaluated the impact of different alerting and reminding systems in computerized medication prescribing.

Therefore, we performed a systematic review to gather evidence of the types of alerting and reminding mechanisms that are in use. Our overall objective was to evaluate the impact of these systems on clinicians' prescribing behavior. In the literature, there are no clear distinctions made between “reminders”, “alerts”, and “prompts”. These terms will therefore be used interchangeably throughout the review.

## Methods

### Literature Search

We searched CINHAL (1982 to May 2007), EMBASE (1980 to May 2007), Inspec (1969 to May 2007), MEDLINE (1950 to May 2007) and PsychINFO (1967 to May 2007) for empiric studies of computerized prescribing alerts and prompts. Our search strategy combined several keywords around electronic prescribing, reminder systems and decision support systems. Articles from all languages were included. The bibliographies of obtained articles were used to identify further relevant studies following guidelines about study selection from the Cochrane Handbook of Systemic Reviews.16 Previous literature reviews, in particular, were searched in full and any relevant references were added to the electronic selection process if considered appropriate. We rerun the search in Jan 2009 and found two recent studies that would qualify for inclusion in this review. Details concerning these studies, a summary of results and the updated search process can be found in the online appendix of this review (supplements 1–4).

### Study Selection

Reference Manager (RefMan Version 10) was used to manage data and eliminate duplicate references. The identified references were distributed among four reviewers (AS, VP, CM, AA) who screened titles and abstracts for relevant studies. Relevant publications were obtained as full text articles and all were assessed by the four reviewers independently applying the following predetermined inclusion criteria:

• Empiric trials

Studies were included if they were randomized controlled trials (RCTs), time series analysis and before-after studies.

• Health care professionals prescribing drug treatment:

Studies had to target clinicians such as doctors, nurses and other health care professionals, who were actively involved in the prescribing and administration of medications for patients of any age receiving drug therapy for the treatment of a disease or for prevention of a medical condition.

• Prompts at point of prescribing

Interventions of interest were computerized alerting or reminding systems consisting in electronic alerts, reminders or prompts related to electronically prescribed medications of a particular drug or class of drug, appearing on the computer screen at the point of prescribing.

• Altered prescribing

Outcome measures were any changes in prescribing related to clinically relevant outcomes such as reduction in medication errors, reduction of risk or harm to patients, decrease of non-recommended medication, increase of recommended medication and increase/decrease of laboratory abnormalities with potential of adverse drug events.

An article was included if it met all the inclusion criteria listed above. Uncertainties or disagreements at any stage were resolved by discussion among reviewers.

### Data Extraction

Study descriptions and outcomes were extracted using predefined collection tables that had been developed for the review. Data were extracted by four reviewers (AA, VP, AS, CM). Extracted data were verified by two reviewers independently (AS, CM).

We applied a classification system based on a modified version previously proposed by Kuperman, et al.4 This allowed us to group different types of prompted CDS according to their characteristics and to compare effectiveness between the groups. There were three main categories of alerts and prompts: Basic drug alerts consisting of drug allergy warnings, drug–drug interaction warnings, duplication warnings and basic medication order guidance; advanced drug alerts including drug-laboratory alerts, drug-condition alerts, drug-formulary alerts and dosing guidelines; complex alert systems representing a set of CDSSs containing features of both basic and advanced alerts (see Fig 1 for more details).

Figure 1

Figure 1

### Data Analysis

Alerts and prompts applied in the studies were grouped in tables and results were compared independently. On the basis of study designs, characteristics of population involved, medication prescribed, types of alerts used and outcome measures reported it was decided whether pooling of outcome data would be appropriate.

### Quality Assessment

Studies were classed according to the appropriateness and quality of their design. Criteria such as study duration, similar before/after outcome assessment and before/after group comparison were applied to evaluate the quality of before-after studies. Randomized controlled trials were assessed considering their main sources of bias including selection bias, performance bias, attrition bias and detection bias. Please see tables in supplement 4 for more detail.

## Results

### Description of Studies

The search identified 14,137 publications of which 14,036 did not fulfill our inclusion criteria when assessing title and abstract (not an experimental trial, not concerned with alerts, not related to e-prescribing of a particular drug/class of drug and not concerned with change in clinician behavior). The 101 remaining full text articles were retrieved and 81 studies were excluded. The 20 remaining studies evaluating 27 different types of alerts addressed our objectives and were included in the review (see flow chart in Fig 2).

Figure 2

Flow chart detailing process of identification and selection of relevant papers.

Figure 2

Flow chart detailing process of identification and selection of relevant papers.

The geographic setting of most studies was the United States (18 studies). One study came from France and another from The Netherlands.17,18 Five of the studies took place in a primary care or outpatient setting. The remaining 15 took place in a hospital/inpatient setting; hence most of the providers were hospital doctors. In some studies hospital nurses and nurse practitioners contributed to the pool of providers.18–22

The medications prescribed in the studies varied widely. Some studies focused only on one class of drugs, such as sedatives, lipid lowering drugs,18,19 asthma treatment,18 or antibiotic prescribing.18,20,23 Five studies evaluated more complex alert systems involving a wide range of prescribing situations.9,24–28

Four out of 20 studies evaluated clinical outcomes such as drug-induced renal impairment,29 serious adverse drug events,25 prescribing related falls30 and length of hospital stay.31 Cost savings through modified prescribing were reported by two studies21,32 (see supplements 2 and 4).

### Heterogeneity

Studies showed considerable variation in characteristics of providers, patients, and health care settings, types of alert interventions and time of observation. More importantly, there was a lack of consistency in the reporting of outcome measures. Studies looked at optimal, suboptimal, dangerous and harmful prescribing using outcome measures such as serious or nonserious medication errors, actual or potential adverse drug reactions, prescribing patterns as per guidelines, increased prescribing of recommended drugs, and decreased prescribing of nonrecommended drugs. As a consequence, pooling of the data or direct comparison between studies was inappropriate. Instead, we classified studies in groups to allow independent comparison.

### Methodological Quality

An RCT represents the most robust method to evaluate the effects of an alert system. In these RCTs, some of the prescribing units are generally randomized to the newly implemented alerting system while the remainders are on the prealert system. Clustered randomized trials in particular have the ability to overcome potential for contamination but require large sample sizes. However, at times when a new and complex computer alert system is introduced in a health care setting, conducting an RCT is often not feasible. This may explain why only four out of 20 studies used RCT designs.18,19,22,23 These varied in quality with no trials meeting all the criteria to avoid bias (see supplement 3). One study did not provide the appropriate data to evaluate the entire effect of computer prompts.19

Other methods to compare pre- and post alert prescribing behavior are more open to bias. Quasi experimental designs used in the studies of this review were uncontrolled before-and-after-studies, time series analyses and interrupted time series analyses. Interrupted time series analysis (ITS) in particular is a practical and valuable tool to determine the impact of a complex intervention, both immediately and over time. Multiple measurements are taken before and after an intervention and are tested for their serial correlation. If sequential prescribing data confirm an immediate and long-lasting effect following the introduction of a new alert system, the prompt system is likely to be responsible for the change.

To account for temporal changes in prescribing patterns, such as seasonal prescribing changes, it is desirable to have long sequential baseline and follow up periods. Other important validity factors are similarity of control and intervention group and continuity of pre- and post alert outcome assessment. Four studies used interrupted time series analyses,21,26,33,34 six studies used time series analyses17,25,28–31 and six used before-after designs.20,24,26,27,32,35 Four authors refined the time series analysis approach by alternating control periods and intervention periods to control for other influences.17,29–31 Important validity criteria were fulfilled by most studies (see supplement 3).

### Change in Prescribing Outcomes

Some of the 20 studies included in this review evaluated more than one type of alert and were therefore classified in more than one category.24,28,31 In total, 27 types of alerts and prompts were assessed in terms of their ability to improve prescribing. Twenty-three of the 27 alerts resulted in statistically significant improved prescribing behavior and/or reductions in medication errors. Most of the changes noted were clinically relevant. The remaining four showed statistically nonsignificant improvements in prescribing18,24,33 or inappropriate study design19 (see Table 1 and supplement 2).

Table 1

Category of Alerts and Types of Alerts Showing a Beneficial Effect on Prescribing and/or Clinical Outcome

Alert types showing beneficial effect on prescribing (second column), no effect on prescribing (third column) and positive effect on clinical outcomes (fourth column) compared to total number of alerts in a certain alert category (first column).

### Change in Clinical Outcomes

Of the four alert types that examined clinical outcomes, three had a positive and statistically significant impact leading to decreased prescribing-related renal impairment,29 fewer falls in elderly people30 and a reduced length of hospital stay.31 One alert achieved a reduction in serious and life-threatening adverse drug events, but there was no p value reported.25 Financial implications of two alerts were evaluated and both achieved annual cost savings of up to US$36,50032 and US$6,400.21

### Types of Alerts and Changes in Outcomes

Applying the Kuperman framework4 we classified 27 types of alerts and prompts used across the 20 studies according to their categories and subcategories of alerts as outlined above.

Two papers investigated the effects of four types of basic alerts of which three reported statistically significant beneficial effects on prescribing (table, Supplement 2):

• Drug allergy warnings decreased allergy error events by 56% (p = 0.009).24

• Providing default dosing via basic medication order guidance alerts resulted in reduced dose errors in two studies of 23% (p = 0.02)24 and 71% (p = 0.0013).28

• The 40% reduction in error events achieved by drug-drug interaction warnings did not reach statistical significance (p = 0.89).24

All the 20 papers evaluated more advanced alert types and statistically significant effects were shown in 21 out of 23 types across five categories:

• Four drug-laboratory alerts increased appropriate prescribing from 30 to 53% (p < 0.001),31 enhanced prescribing of electrolyte supplements from 6 to 35% and 5–49% (p < 0.01),26 reduced the use of contraindicated medication in renal insufficiency from 89 to 47% (p < 0.0001)36 and prevented hazardous prescribing by 50% (p = 0.03).27 One study achieved a positive clinical outcome by an alert resulting in a small reduction in length of hospital stay from 4.5 to 4.3 days (p < 0.009).31

• Five out of eight types of alerts investigating the effects in relation to patients' medical conditions or age showed beneficial effects that were statistically significant. Drug-condition alerts increased prescribing for venous thromboembolism prophylaxis in patients at risk from 83 to 95% (p < 0.001)17 and 24–47% (p < 0.001).28 Three out of four drug-age alerts for older people showed a reduction of nonrecommended drugs by 18% (95% CI 24–13%),35 22% (p = 0.004)34 and 30% (p < 0.001).30 In addition, a reduction in falls from 6.4 to 2.8 per 1,000 patients was achieved by one alert type (p < 0.001).30 Prescribing of lipid lowering medication was increased in a study using computer prompts in combination with a complex intervention package, but the study design did not allow evaluation of the alert effect on its own.19 Prescribing benefit did not reach the significance level in two studies. One was underpowered18 and the other was poorly designed comparing a refined age specific reminding system with a very similar, less sophisticated application.33

• The investigation of drug-formulary alerts was mostly based on cost implications. All four alert types had the intended effect on prescribing21,23,28,32 and reductions in costs were reported by two.21,28 Each alert achieved changes in specific formulary items such as reductions in methylprednisolone prescriptions by 55% (p < 0.0001);32 increases of oral rather than intravenous quinolone orders by 13% (p < 0.001);21 reductions in unnecessary vancomycin orders by 29% (p = 0.03)23 and an over five times increase in the use of a predefined H2 blocker (p < 0.001).28

• Three studies investigated computerized drug-dosing prompts. Two alert types showed benefits for dosing guidance in renal impairment in terms of more appropriate dose prescribing from 54 to 67% (p < 0.001)31 and quicker response to high creatinine levels from 97.5 to 75.9 hours (p < 0.0001).29 Another claimed a reduction to an error rate of zero in pediatric weight-based dosing for gentamicin, though proof of significance was not reported.20 In addition to prescribing outcomes, a reduction in prescribing related renal impairment by 55% b (95% CI 6–78%) was achieved by the alert system used in one of the studies.29

• All the four alerting systems that combined basic and advanced alerts in a complex system of alerts and prompts demonstrated statistically significant benefits with respect to prescribing. Bates (1998) demonstrated a reduction in serious medication errors by 55% (p = 0.01) with the rate of potential ADEs falling by 84% (p = 0.002).24 The preventable ADE rate fell by 17% (statistically not significant) and the non-preventable adverse drug event rate was unchanged. Another system applied by Bates (1999) demonstrated an 81% reduction in all medication errors (p < 0.0001) and an 86% reduction in serious medication errors after introduction of a system with multiple types of alert (p = 0.0003).25 Teich et al demonstrated several marked improvements in the prescription of preferred agents from 16 to 81% (p < 0.001), in correct frequency prescribing from 6 to 75% (p < 0.001) and in reduced overdose prescribing from 2.1 to 0.6% (p < 0.001).28 A modest impact with borderline statistical significance was demonstrated by an alert system aiming at more appropriate prescribing in 13 different areas improving overall prescribing from 28 to 31% (relative risk 1.11 with 95% CI 1.00–1.22).22

For many alert categories, only a small number of alert types had been performed, and for some categories we found no studies (Table 1). Types with few or no published data included drug-allergy warnings (n = 1), drug–drug interaction alerts (n = 1), duplicate medication alerts (n = 0), drug-disease contraindication alerts (n = 0) and dosing guidelines based on age (n = 0), pregnancy/child bearing age (n = 0), pediatric patients (n = 1) and indications (n = 0).

## Discussion

### Limitations

Our study employed rigorous inclusion criteria for the identification of studies relevant to the objectives of the review. In particular, we used a wide variety of potentially relevant literature databases and we carefully selected relevant search terms and adapted these for each database. In addition, we carefully searched the bibliographies of studies and reviews that we obtained to identify other potentially relevant articles. Given the limitations of time and resources we split up the searches of publication titles and abstracts between three reviewers. We did however check three validation samples of 1,000 references and this showed no discrepancies in the selection of main outcome studies. Throughout the process of conducting the review any issues arising, including discrepancies, were resolved through discussion. The data concerning the main outcomes in Tables 1 and 2 were checked by two reviewers independently (AS and CM).

This study had several limitations. The area of computerized decision support systems has become a wide field. In this review, we were specifically interested in studies evaluating point-of-care decision support systems that confront the clinician during the prescribing process. Studies were only included if CDSS were concerned with e-prescribing of a particular drug or class of drugs, not considering systems concerned with improved screening or better responses to abnormal laboratory results. Some might view this intentional narrow focus as an important limitation of the current study.

It is possible that some citations were missed, although the search strategy was comprehensive. The diversity in study characteristics and outcome measures did not allow pooling of the data and even made comparison between categories difficult. The wide variation of alerts and prompts requires a framework to categorize different alert types when reviewing the literature. In this review we applied a classification system based on a version by Kuperman, et al, which is a complex framework of alerts and prompts.4 Due to the limited number of studies, it was not possible to reveal superiority of any category of alerts. Other categorizations of alerts and prompts according to their functionality, purpose, timing and workflow interruptions have been developed and represent an alternative way of categorization.5

### Implementation for Practice and Further Research

We identified 20 studies and 27 types of alerts and prompts of which 23 achieved improved prescribing in terms of changes in prescribing behavior and/or reductions in medication errors. In many of the studies, the changes noted were clinically relevant. Positive effects were noted for a wide range of alerts and prompts. Three of the alert types with lacking benefit showed weaknesses in their methodology18 or design.19,33 These findings provide support for the continued development and use of computerized alerts and prompts for prescribing. The alert types and categories presented in this review represent relevant information for developers of computerized prescribing systems when setting up of new systems or updating older systems.

The limited number of studies conducted did not cover all alert categories, and it will be useful to target these alerts and prompts in future research. Drug–drug interaction alerts, drug-disease contraindication alerts and dosing guidelines based on age are prescribing categories with great potential for affecting prescribing. However, the lack of study data are surprising. These types of alerts, in particular, need to be further evaluated in future experimental trials using appropriate study designs. Randomized controlled trials are the most valid and robust method to evaluate the effect of an alert systems and should be used whenever possible.

The main obstacle in comparing effects on prescribing was the lack of standardized outcome measures for measuring medication errors. The most common outcomes reported across the studies of this review were medication error (serious and nonserious) and ADEs (actual and potential; preventable and non preventable). A variety of efforts are underway now to build consensus around these and other safety measures.37 Other outcomes were the impact of alerts on prescribing patterns as per guidelines, prescribing fewer nonrecommended drugs and prescribing more recommended drugs. To improve validity of future studies, an effort has to be made to use more uniform definitions of outcome measures around suboptimal prescribing.

Also, few studies have assessed the impact of computerized alerts and prompts on clinical or health service management outcomes. Such studies need to have a large enough sample size to have sufficient power to assess the effects of these interventions. Evidence from these types of studies is likely to have the greatest impact on the decision making processes of clinical information system implementers.

## Conclusions

We conclude that most empiric studies evaluating the effects of computerized prompts and alerts on prescribing behavior show positive, and often substantial, effects. However, while a wide range of types of alerts and prompts have been evaluated, additional studies should be done to determine the design features that are most strongly associated with improved impact on prescribing and clinical outcomes.

## References

1
EE
Gilbert
AL
Primrose
JG
Sansom
LN
.
Drug-related hospital admissions: A review of Australian studies
.
Med J Aust

1998
;
168
(
8
):
405
408
.
2
Von Laue
NC
Schwappach
DLB
Koeck
CM
.
The epidemiology of preventable adverse drug events: A review of the literature
.
Wien Klin Wochenschr

2003
;
115
(
12
):
407
415
.
3
Bates
DW
Cullen
DJ
Laird
N
et al
.
Incidence of adverse drug events and potential adverse drug events. Implications for prevention. ADE Prevention Study Group
.
J Am Med Assoc

1995
;
274
:
29
34
.
4
Kuperman
GJ
Bobb
A
Payne
TH
et al
.
Medication-related clinical decision support in computerized provider order entry systems: A review
.
J Am Med Inform Assoc

2007
;
14
(
1
):
29
40
.
5
Miller
RA
Waitman
LR
Chen
S
Rosenbloom
ST
.
The anatomy of decision support during inpatient care provider order entry (CPOE): Empirical observations from a decade of CPOE experience at Vanderbilt
.
J Biomed Inform

2005
;
38
(
6
):
469
485
.
6
Aranow
M
.
What works: Clinical information systems. Order entry rules. Healthcare enterprise achieves physician acceptance, reduced medication errors and improved patient outcomes through CIS and CPOE technology
.
Health Manag Technol

2002
;
23
:
34
38
.
7
Bennett
JW
Glasziou
PP
.
Computerised reminders and feedback in medication management: A systematic review of randomised controlled trials
.
Med J Aust

2003
;
5
:
217
222
.
8
Bobb
A
Gleason
K
Husch
M
et al
.
The epidemiology of prescribing errors: The potential impact of computerized prescriber order entry
.
Arch Intern Med

2004
;
164
(
7
):
785
792
.
9
Garg
AX
NKJ
McDonald
H
et al
.
Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: A systematic review
.
JAMA. J Am Med Assoc

2005
;
293
(
10
):
1223
1238
.
10
Hodgkinson
B
Koch
S
Nay
R
Nichols
K
.
Strategies to reduce medication errors with reference to older adults
.
Int J Evid Based Healthc

2006
;
4
:
2
41
.
11
Kaushal
R
Shojania
KG
Bates
DW
.
Effects of computerized physician order entry and clinical decision support systems on medication safety: A systematic review
.
Arch Intern Med

2003
;
163
:
1409
1416
.
12
Oren
E
Shaffer
ER
Guglielmo
BJ
.
Impact of emerging technologies on medication errors and adverse drug events
.
Am J Health Syst Pharm

2003
;
60
(
14
):
1447
1458
.
13
Rabol
LI
Anhoj
J
Pedersen
A
Pedersen
BL
Hellebek
AH
.
Clinical decision support: Is the number of medication errors reduced?
.
Ugeskr Laeger

2006
;
168
(
48
):
4179
4184
.
14
Van der Sijs
H
Aarts
J
Vulto
A
Berg
M
.
Overriding of drug safety alerts in computerized physician order entry
.
J Am Med Inform Assoc

2006
;
13
(
2
):
138
147
.
15
Walton
RT
Harvey
E
Dovey
S
Freemantle
N
.
Computerised advice on drug dosage to improve prescribing practice
.
Cochrane Database Syst Rev

2001
:
CD002894
.
16
Higgins
JPT
Green
S
.
Formulating the problem
.
Cochrane Handbook for Systematic Reviews of Interventions 4.2.6. Section 5, 2008
Available at: http://www.cochrane.org/resources/handbook/hbook.htm. Accessed: May 5, 2008.
17
Durieux
P
Nizard
R
Ravaud
P
Mounier
N
Lepage
E
.
Clinical decision support system for prevention of venous thromboembolism: Effect on physician behavior
.
J Am Med Assoc

2000
;
283
(
21
):
2816
2821
.
18
Martens
JD
Van der
WT
Severens
JL
et al
.
The effect of computer reminders on GPs' prescribing behaviour: A cluster-randomised trial
.
Int J Med Inform

2007
.
Suppl 3
:
S403
S416
.
19
Bloomfield
HE
Nelson
DB
van Ryn
M
et al
.
A trial of education, prompts, and opinion leaders to improve prescription of lipid modifying therapy by primary care physicians for patients with ischemic heart disease
.
Qual Saf Health Care

2005
;
14
(
4
):
258
263
.
20
Cordero
L
Kuehn
L
Kumar
RR
Mekhjian
HS
.
Impact of computerized physician order entry on clinical practice in a newborn intensive care unit
.
J Perinatol

2004
;
24
(
2
):
88
93
.
21
Hulgan
T
Rosenbloom
ST
Hargrove
F
et al
.
Oral quinolones in hospitalized patients: An evaluation of a computerized decision support intervention
.
J Intern Med

2004
;
256
(
4
):
349
357
.
22
Judge
J
Field
TS
DeFlorio
M
et al
.
Prescribers' responses to alerts during medication ordering in the long term care setting
.
J Am Med Inform Assoc

2006
;
13
(
4
):
385
390
.
23
Shojania
KG
Yokoe
D
Platt
R
et al
.
Reducing vancomycin use utilizing a computer guideline: Results of a randomized controlled trial
.
J Am Med Inform Assoc

1998
;
5
(
6
):
554
556
.
24
Bates
DW
Leape
LL
Cullen
DJ
et al
.
Effect of computerized physician order entry and a team intervention on prevention of serious medication errors
.
J Am Med Assoc

1998
;
280
(
15
):
1311
1316
.
25
Bates
DW
Teich
JM
Lee
J
et al
.
The impact of computerized physician order entry on medication error prevention
.
J Am Med Inform Assoc

1999
;
6
(
4
):
313
321
.
26
Galanter
WL
Polikaitis
A
Didomenico
RJ
.
A trial of automated safety alerts for inpatient digoxin use with computerized physician order entry
.
J Am Med Inform Assoc

2004
;
11
(
4
):
270
277
.
27
Steele
AW
Eisert
S
Witter
J
et al
.
The effect of automated alerts on provider ordering behavior in an outpatient setting
.
PLoS Med

2005
;
2
(
9
):
255
.
28
Teich
JM
Merchia
PR
Schmiz
JL
et al
.
Effects of computerized physician order entry on prescribing practices
.
Arch Intern Med

2000
;
160
:
2741
2747
.
29
Rind
DM
Safran
C
Phillips
RS
et al
.
Effect of computer-based alerts on the treatment and outcomes of hospitalized patients
.
Arch Intern Med

1994
;
154
(
13
):
1511
1517
.
30
Peterson
JF
Kuperman
GJ
Shek
C
et al
.
Guided prescription of psychotropic medications for geriatric inpatients
.
Arch Intern Med

2005
;
165
(
7
):
802
807
.
31
Chertow
GM
Lee
J
Kuperman
GJ
et al
.
Guided medication dosing for inpatients with renal insufficiency
.
J Am Med Assoc

2001
;
286
(
22
):
2839
2844
.
32
Clinical Informatics Outcomes Research Group
Bogucki
B
Jacobs
BR
Hingle
J
.
Computerized reminders reduce the use of medications during shortages
.
J Am Med Inform Assoc

2004
;
11
(
4
):
278
280
.
33
Simon
SR
Smith
DH
Feldstein
AC
et al
.
Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people
.
J Am Geriatr Soc

2006
;
54
(
6
):
963
968
.
34
Smith
DH
Perrin
N
Feldstein
A
et al
.
The impact of prescribing safety alerts for elderly persons in an electronic medical record: An interrupted time series evaluation
.
Arch Intern Med

2006
;
166
(
10
):
1098
1104
.
35
Agostini
JV
Zhang
Y
Inouye
SK
.
Use of a computer-based reminder to improve sedative-hypnotic prescribing in older hospitalized patients
.
J Am Geriatr Soc

2007
;
55
:
43
48
.
36
Galanter
WL
Didomenico
RJ
Polikaitis
A
.
A trial of automated decision support alerts for contraindicated medications using computerized physician order entry
.
J Am Med Inform Assoc

2005
;
12
(
3
):
269
274
.
37
WHO
.
World Alliance for Patient Safety
. Available at: http://www.who.int/patientsafety/en/. Accessed: May 5, 2008.
38
Shah
NR
Seger
AC
Seger
DL
et al
.
Improving acceptance of computerized prescribing alerts in ambulatory care
.
J Am Med Inform Assoc

2006
;
13
(
1
):
5
11
.
39
Avery
AJ
Savelyich
BSP
Sheikh
A
et al
.
Identifying and establishing consensus on the most important safety features of GP computer systems: E-Delphi study
.
Inform Prim Care

2005
;
13
(
1
):
3
11
.
40
Feldstein
A
Simon
SR
Schneider
J
et al
.
How to design computerized alerts to safe prescribing practices
.
Jt Comm J Qual Saf

2004
;
30
(
11
):
602
613
.
41
Krall
MA
Sittig
DF
.
Subjective assessment of usefulness and appropriate presentation mode of alerts and reminders in the outpatient setting
.
Proceedings AMIA

2001
,
Annual Symposium
:
334
338
.
42
Glassman
PA
Simon
B
Belperio
P
Lanto
A
.
Improving recognition of drug interactions: Benefits and barriers to using automated drug alerts
.
Med Care

2002
;
40
(
12
):
1161
1171
.
43
Magnus
D
Rodgers
S
Avery
AJ
.
GPs. Views on computerized drug interaction alerts: Questionnaire survey
.
J Clin Pharm Ther

2002
;
27
(
5
):
377
382
.
44
Kawamoto
K
Houlihan
CA
Balas
EA
Lobach
DF
.
Improving clinical practice using clinical decision support systems: A systematic review of trials to identify features critical to success
.
BMJ

2005
;
330
(
7494
):
765
.
45
Hellier
E
Edworthy
J
Derbyshire
N
Costello
A
.
Considering the impact of medicine label design characteristics on patient safety
.
Ergonomics

2006
;
49
(
5–6
):
617
630
.
46
Center for Information Technology. Leadership (CITL)
.
Report on Ambulatory Computerized Provider Order Entry, ACPOE
. Available at: http://www.citl.org/research/ACPOE.htm. Accessed: Oct 07, 2008.
47
Bates
DW
Kuperman
GJ
Wang
S
et al
.
Ten commandments for effective clinical decision support: Making the practice of evidence-based medicine a reality
.
J Am Med Inform Assoc

2003
;
10
(
6
):
523
530
.