## Abstract

Objective: Although computerized physician order entry (CPOE) may decrease errors and improve quality, hospital adoption has been slow. The high costs and limited data on financial benefits of CPOE systems are a major barrier to adoption. The authors assessed the costs and financial benefits of the CPOE system at Brigham and Women's Hospital over ten years.

Design: Cost and benefit estimates of a hospital CPOE system at Brigham and Women's Hospital (BWH), a 720-adult bed, tertiary care, academic hospital in Boston.

Measurements: Institutional experts provided data about the costs of the CPOE system. Benefits were determined from published studies of the BWH CPOE system, interviews with hospital experts, and relevant internal documents. Net overall savings to the institution and operating budget savings were determined. All data are presented as value figures represented in 2002 dollars.

Results: Between 1993 and 2002, the BWH spent $11.8 million to develop, implement, and operate CPOE. Over ten years, the system saved BWH$28.5 million for cumulative net savings of $16.7 million and net operating budget savings of$9.5 million given the institutional 80% prospective reimbursement rate. The CPOE system elements that resulted in the greatest cumulative savings were renal dosing guidance, nursing time utilization, specific drug guidance, and adverse drug event prevention. The CPOE system at BWH has resulted in substantial savings, including operating budget savings, to the institution over ten years.

Conclusion: Other hospitals may be able to save money and improve patient safety by investing in CPOE systems.

Between 44,000 to 98,000 Americans die each year due to medical errors, and about 1 million people are injured.1 Although there is controversy regarding the accuracy of the mortality estimates, general agreement exists that iatrogenic injuries are common, costly, and often preventable.2–8

Medications represent the single most common cause of iatrogenic injuries, accounting for nearly 20% of all such events.9 Reducing medical errors requires a multifaceted approach. Computerized physician order entry (CPOE) with clinical decision support systems (CDSS) is a promising intervention to improving medication safety.10 In one study, CPOE reduced the serious medication error rate by 55%.11 Other studies have also demonstrated reductions in medication errors.12–14 Most CPOE systems allow physicians to enter medication orders as well as laboratory, admission, radiology, and transfusion orders electronically. When combined with clinical decision support, CPOE improves medication safety, clinical laboratory and radiology testing, medication costs, adoption of critical pathways, and the efficiency of hospital workflow.13–17

Despite growing evidence and mandates to implement CPOE, adoption has been slow, with only an estimated 15% of hospitals having even partially implemented CPOE.18,19 High costs are one important barrier to faster adoption.20 Hospitals must make a large up-front capital investment without clear data on return on investment or confidence in physician use of implemented systems. These costs include the time required of hospital staff, particularly physicians, for training and use of CPOE systems. Further, while hospitals must shoulder the costs of CPOE, benefits accrue to multiple parties including payers, employers, and patients.21

Given the tension between the clinical benefits of CPOE and the high up-front costs, hospitals deciding whether to implement CPOE need to better understand how and when financial benefits of such systems accrue. Brigham and Women's Hospital (BWH), a 720-adult bed, academic, tertiary care medical center in Boston, MA, implemented a home-grown CPOE system in 1993. We assessed the costs and benefits associated with the implementation of the BWH CPOE system over ten years.

## Methods

BWH internally developed and implemented a CPOE system in 1993 with CDSS introduced incrementally over the ensuing years as the system was enhanced. We began by calculating the costs and all financial benefits of the BWH CPOE system. Since many CPOE savings are not demonstrable in an organization's operating budget, we then calculated the benefits that are actually reflected in the organization's operating budget such as decreased drug costs.

### Cost Data

For BWH, based on internal documents and interviews with the developers of the CPOE system, we determined the capital costs and assigned 60% of the costs to 1992, 20% to 1993, and 20% to 1994. We began to count operational costs starting on January 1, 1993, and running through December 31, 2002. We included costs such as hardware including workstations and printers, software, network, leadership, and training costs. However, we did not include costs such as those for a pharmacy system, medication administration system, or clinical data repository.

### Benefit Data

To estimate the benefits from CPOE, we identified each intervention and calculated the associated cost savings. We obtained actual BWH benefit data from relevant published literature, institutional key informants, and internal documents. We depict the method and amount of cost savings for the most financially profitable clinical decision elements in Table 1. We reference relevant literature in Table 1 as well, although much internal data were also used.

Table 1

Cumulative Benefits for CDSS Elements at Brigham and Women's Hospital

CDSS = clinical decision support system; ADE = adverse drug event; KUB = kidney, ureter, and bladder.

This table depicts the cumulative benefits (in 2002 millions of dollars) from 1992 to 2002 for each element of CDSS at Brigham and Women's Hospital given an 80% prospective reimbursement rate.

The most rigorously studied aspect of the BWH CPOE system is CDSS to reduce adverse drug events.3,15,22,23 Consequently, we calculated many benefits based on the number of averted adverse drug events (ADEs). For example, the BWH system has drug–drug interaction alerts that check and warn for hazardous interactions between drugs.3 We determined cost savings from these alerts by multiplying the number of averted ADEs by the average cost of an ADE ($4,685 in 1997 dollars).24 We estimated other benefits based on decreased drug costs generally through decreased use. For example, a clinical decision support tool to decrease the frequency of ceftriaxone from twice a day to once a day was introduced in 1999, resulting in 80% of orders being switched to daily dosing.25 We calculated cost savings by multiplying the number of saved doses by the cost of each dose. Another important intervention leading to decreased drug costs was early conversion of intravenous to oral medications for patients who were already taking either other oral medications or an oral diet.26 In this case, we estimated the amount of savings by subtracting the costs of the oral medications from the costs of the intravenous medications. Some interventions saved money by decreasing laboratory test usage such as an alert that warned physicians when redundant laboratory tests were ordered.27 For laboratory savings, we only had access to laboratory charge savings rather than cost savings due to the internal structuring and recording of hospital data. Although the ratio of charges to costs varied for individual laboratory tests, we used a 0.2 charge-to-cost ratio (i.e., a charge of$1 is associated with a cost of $0.20) on the advice of institutional laboratory experts. Other interventions addressed radiology test overuse and misuse. For example, one intervention improved appropriate ordering of radiology tests.28 We calculated the savings from this intervention by subtracting the costs of performed tests from the costs of canceled inappropriate tests. Improved workflow and efficiency led to additional cost savings. For example, improved nursing time utilization resulted in staff and resource savings. Similarly there were savings in physician time utilization. For some interventions, particularly those that did not show up in the operating budget, we were unable to calculate benefits, and while benefits likely exist for many, we excluded them from the analysis. These interventions include advanced chemotherapy decision support that prevented incorrect dose calculations, lack of rescue medications, and inappropriate administration of medications to the wrong patient. We did not include guided dose algorithms that prevent either over- or underdose of medications requiring calculations such as heparin and digoxin. Nor did we include several radiological interventions such as online scheduling, patient preparatory instructions, and allergy tracking. We also did not include pathway order sets, automated cosign and documentation requirements, or automated sign outs. Finally, some interventions such as transfusion guidance were implemented too recently to include in these analyses. ### Live Date and End Date CPOE CDSS interventions at BWH were phased in during the period of analysis. To account for the costs and benefits of each intervention, we used a number of conventions. When an intervention became active (live date) in the middle of a month, we assumed benefits would not start to accrue until the first day of the following month. When we only had data on the year in which an intervention went live, we assumed that benefits would start to accrue at the midpoint of the year (July 1). Many interventions were initially introduced and then significantly enhanced. In these cases, we reported multiple start dates in Table 1. ### Operating Budget Analysis We also performed an analysis to assess benefits that would be included in the hospital's operating budget. Many savings from CPOE are not realized in the operating budget. We analyzed each clinical decision support element to determine whether the resulting savings were observable in the operating budget. For example, cost savings from lower drug costs directly improve the hospital's operating income. Savings attributable to workflow improvements, on the other hand, may not directly reduce operating costs since improvement in efficiency does not necessarily translate into full-time equivalent reduction. ### Discounting, Annualization, Constant Dollars We discounted all costs and benefits at a 7% annual percentage rate in accordance with the recommendations of the U.S. Office of Management and Budget for economic analyses performed for the federal government.29 This represents a societal discount rate rather than a hospital-specific rate. We discounted all costs and benefits on a monthly basis with costs discounted using a “beginning of period” convention, while benefits discounted using an “end-of-period” convention. In addition to discounted values, we calculated annualized values. Annualization converts the entire stream of discounted costs and benefits into a series of equal annual payments analogous to mortgage payments on a house. All current dollar values for costs and benefits were converted to a constant dollar basis to adjust for inflation. We used the Bureau of Labor Statistics' Producer Price Index time series for General Medical and Surgical Hospitals to deflate values to a constant 2002 base year. ### Prospective Reimbursement Prospective reimbursement rates affect the amount of hospital savings from CPOE. If a patient's care is not prospectively reimbursed, then savings do not necessarily accrue to the hospital from an avoided ADE or an unnecessary test since the hospital may be reimbursed by the insurance company regardless of whether the utilization was avoidable. For this period of time at BWH, the average prospective reimbursement rate was 80%. ## Results ### The Brigham and Women's Hospital Experience All results are present value figures reported in constant 2002 dollars. In 1992, BWH spent approximately$3.7 million in capital costs and between $600,000 to$1.1 million per ensuing year from 1993 to 2002 in operational costs for total costs of $11.8 million to develop, implement, and operate CPOE (Fig. 1). During these 11 years, the CPOE system saved a total of$28.5 million given the 80% prospective reimbursement rate at BWH. This resulted in a net benefit of $16.7 million ($2.2 million annualized). The operating budget benefits totaled $21.3 million for a net cumulative present value of$9.5 million ($1.3 million annualized). As the project length increased, benefits increased since most of the costs were incurred early in the implementation. It took over five years for BWH to realize a net benefit and over seven years for BWH to realize an operating budget benefit from CPOE. Figure 1 The net cumulative present value of computerized physician order entry (CPOE) at Brigham and Women's Hospital (BWH) from 1992 to 2002 given an 80% prospective reimbursement rate. Six years after the implementation of CPOE, in 1998, BWH began to make a net profit from the CPOE system. This net profit continues to grow steeply. In 1999, 7.5 years after the implementation of CPOE, a financial benefit accrued in the operating budget. Figure 1 The net cumulative present value of computerized physician order entry (CPOE) at Brigham and Women's Hospital (BWH) from 1992 to 2002 given an 80% prospective reimbursement rate. Six years after the implementation of CPOE, in 1998, BWH began to make a net profit from the CPOE system. This net profit continues to grow steeply. In 1999, 7.5 years after the implementation of CPOE, a financial benefit accrued in the operating budget. ### Cumulative Computerized Physician Order Entry Clinical Decision Support System Benefits for Brigham and Women's Hospital The largest cumulative savings were from renal dosing guidance ($6.3 million), improved nursing time utilization ($6.0 million), specific or expensive drug guidance (human growth hormone, vancomycin, ceftriaxone, ondansetron, and histamine receptor antagonists) ($4.9 million), ADE prevention ($3.7 million), laboratory charge display and redundant laboratory warnings ($1.9 million), and panic laboratory alerting ($1.8 million)3,15,22–25,27,30–33 (Table 1). Because many of the clinical decision support elements were not implemented until the latter part of the evaluated time period, the savings were much greater during the latter years. Renal dosing guidance saved the institution the most money. In this intervention, the system recommends drug-dosing adjustments based on the patient's renal function. In a large study of this intervention, the number of appropriate orders increased from 30% to 51% (p < 0.001) and mean adjusted length of stay decreased from 4.5 days in the control group to 4.3 days in the intervention group (p = 0.009).30 The annual cost savings were most heavily driven by the decreased length of stay. The BWH CPOE system improved the efficiency of nursing time utilization. The largest efficiency was attributed to the automation of the medication administration record eliminating the need for a handwritten medication administration record. Other cost savings were due to decreased rework of problematic orders to avoid medication errors or ADEs since the CPOE system was performing many of these checks at the time of order creation. Specific or expensive drug guidance saved the institution$4.9 million during the ten-year study period.15,25,31 An intervention that suggested appropriate indications for initiation and continuation of vancomycin through explicit indications and prompts to discontinue the drug after three days resulted in 32% fewer vancomycin orders. The net effect was $20,042 in annual cost savings ($80,166 in vancomycin savings and $60,124 in replacement drug costs).31 Prior to another intervention, 80% of ceftriaxone orders were written for twice-daily dosing. Providers were prompted to dose daily instead with an 80% switch to once-daily dosing and a cost savings of$175,094 annually.25 An intervention to reduce the use of human growth hormone by suggesting appropriate indications resulted in $320,000 in annual cost savings. Another intervention increased the number of ondansetron orders with a three-times daily frequency instead of four times daily, from 5.9% to 93.5% of orders. This medication was ordered an average of 975 times each year. After adjusting for the average duration of each order, the annual savings for this intervention was demonstrated to be$249,218 in 2002 dollars given an 80% prospective reimbursement rate.15 A final intervention recommended nizatidine and ranitidine as histamine-2 receptor antagonist of choice in compliance with the BWH formulary at the time. In a time-series analysis, nizatidine and ranitidine use increased from 11.7% and 0%, respectively, to 97.9% and 97%, respectively.15

A number of interventions aimed at improving patient safety were demonstrated to decrease preventable ADEs from 4.69 to 3.88 per 1,000 patient days in one study.11 Annual cost savings associated with these interventions were determined by multiplying the reduction in ADEs by the cost of ADEs. These interventions include, among others, default drug dose, frequencies, and routes as well as drug–drug and drug–allergy checks.

Another important intervention of the BWH CPOE system allows critical laboratory results to be automatically detected and the responsible provider paged. This system reduces time to appropriate treatment thereby averting ADEs.33

Three important interventions were introduced too recently to be included in the analyses; however, some data are available on cost savings associated with them. A pilot study has indicated that a transfusion guidance system saved the hospital $1.3 million in 2002 dollars. An intervention that suggests appropriate ordering of Clostridium difficile cytotoxin assays is estimated to save$12,000 per year, while an intervention to improve ordering of digoxin levels is estimated to save $74,000 per year. These three interventions were not included in the study. ### Annual Computerized Physician Order Entry Clinical Decision Support System Benefits for Brigham and Women's Hospital Since the cumulative benefits are dependent on the length of intervention, we also summarized the annual savings in 2002 dollars for each CDSS intervention (Table 2). Again renal dosing guidance led the interventions (2.24 million), followed by ADE prevention, nurse time utilization, and specific or expensive drug guidance. Table 2 Annual Benefits for CDSS Elements at Brigham and Women's Hospital  CDSS Element Total Benefits Renal dosing guidance 2.24 ADE prevention 1.05 Nurse time utilization 0.96 Specific or expensive drug guidance (human growth hormone, vancomycin, ceftriaxone, ondansetron, histamine-2 receptor blockers) 0.88 ADE monitor 0.76 Intravenous to oral guidance 0.74 Laboratory charge display and redundant laboratory warnings 0.34 Panic laboratory alerting 0.34 Radiology indications, rule-out, and assistant 0.15 Automated medication summary at hospital discharge 0.10 Physician time utilization 0.10 Elderly dosing guidance 0.05 Specific drug level guidance (antiepileptics, rheumatologic tests) 0.02  CDSS Element Total Benefits Renal dosing guidance 2.24 ADE prevention 1.05 Nurse time utilization 0.96 Specific or expensive drug guidance (human growth hormone, vancomycin, ceftriaxone, ondansetron, histamine-2 receptor blockers) 0.88 ADE monitor 0.76 Intravenous to oral guidance 0.74 Laboratory charge display and redundant laboratory warnings 0.34 Panic laboratory alerting 0.34 Radiology indications, rule-out, and assistant 0.15 Automated medication summary at hospital discharge 0.10 Physician time utilization 0.10 Elderly dosing guidance 0.05 Specific drug level guidance (antiepileptics, rheumatologic tests) 0.02 This table depicts the annual benefits (in 2002 millions of dollars) for each element of CDSS at Brigham and Women's Hospital given an 80% prospective reimbursement rate. ## Discussion A CPOE system at a large academic hospital that was implemented about 10 years ago saved the hospital about$2.2 million annually with current savings of $16.7 million per year. The operating budget savings were$9.5 million (\$1.3 million annualized). It took over five years for the BWH system to begin accruing a net benefit and over seven years to begin accruing an operating budget benefit.

The level and type of decision support were directly related to the amount of savings the hospital achieved. Renal drug dosing, ADE prevention, and expensive or specific drug guidance were the most financially beneficial interventions. It is important to note that the majority of savings accrued from a relatively small number of interventions. These results suggest that hospitals should consider focusing on these CDSS interventions to increase the chance of financial profitability from their CPOE systems. In addition, hospitals should pay careful attention to the method of workflow integration to save nursing and physician time. Expensive drug guidance is an increasingly important type of decision support given the rapid emergence of new drugs. In this model, we included cost containment of human growth hormone and ondansetron but not other expensive medications.

Of note, we included only BWH CDSS elements for which there were good estimates of cost savings in the model. We were unable to include several BWH CDSS elements in the study due to either their timing of implementation or the lack of reliable cost savings data. Even though many interventions have been implemented at BWH, this CPOE system lacks numerous other highly effective interventions such as LDS Hospital's antibiotic assistant.34,35 Further studies of the potential benefits of specific elements of CDSS are necessary for hospitals to accurately understand the value of CPOE.

In performing these analyses, we assumed that costs and benefits would be equally affected by inflation over time. If the price of medical services is growing more rapidly than general inflation, as is likely, then the discount rate is actually declining over time in real terms. If the discount is overestimated in this way, the model would tend to underestimate benefits rather than costs as we front-loaded costs and back-loaded benefits in our analysis.

To achieve the types of benefits modeled based on BWH data, a hospital must have nearly 100% physician use, well-designed CDSS elements, and effective interfaces among CPOE, pharmacy, laboratory, and medication administration record systems. Some hospitals have overcome large financial barriers to implement CPOE, only to fail to achieve widespread use due to physician resistance.36,37 In addition, the automated knowledge necessary for CDSS elements must be represented in ways that allow it to be readily interchanged between different computer systems. Benefits may vary substantially among different vendor applications based on factors like these. The benefits of increased workflow efficiency are perhaps the most difficult to achieve as they require all these factors along with a quick system, although they are very important given the national shortages of nurses and pharmacists.

This study has several limitations. Ideally, all benefit data would have been collected prospectively rather than retrospectively. Active data collection would have decreased the number of estimates by institutional experts. In addition, we did not include less direct benefits of CPOE such as averted malpractice litigation from fewer ADEs.38 We excluded several decision support elements of the BWH system for which we could not calculate benefits. Our model is purely cost avoidance and does not directly address increased revenue. CPOE systems often result in improved billing, but these savings were not incorporated in our benefit estimates. Nor did we include increased efficiencies for personnel such as pharmacists since reliable institutional estimates were not available.

Of note, we did not include all the costs of knowledge engineering by clinicians and engineers to create and encode clinical information with CDSSs. Information technology staff time is included, but not the time devoted by clinicians and researchers in developing clinical rules. These hidden costs were piecemeal over many years and probably represented a relatively small part of the entire costs. Finally, it is essential to note that the vast majority of implemented CPOE systems are vendor based rather than home grown such as the BWH system. Clearly the benefits of CDSS from vendor systems may be different than those from the BWH system limiting generalizability.

In conclusion, the BWH saved significant money by implementing the CPOE system. Other hospitals may realize even greater benefits, particularly if they have high levels of clinical decision support and rates of prospective reimbursement. While many hospitals need assistance accessing capital to purchase a CPOE system, the financial benefits may help justify the expense. Furthermore, hospitals may not require long-term ongoing financial support, although it will be important for institutions to take a long-term perspective. Patient safety is a critical part of the health care mission and the implementation of CPOE can make health care safer and save money.

## References

1
Institute of Medicine
.
To err is human. Building a safer health system.

Washington, DC
:
;
1999
.
2
Barker
KN
Mikeal
RL
Pearson
RE
Illig
NA
Morse
ML
.
Medication errors in nursing homes and small hospitals
.
Am J Hosp Pharm

1982
;
39
:
987
91
.
3
Bates
DW
Cullen
DJ
Laird
N
Petersen
LA
Small
S
Servi
D
et al
.
Incidence of adverse drug events and potential adverse drug events: implications for prevention. ADE Prevention Study Group
.
JAMA

1995
;
274
:
29
34
.
4
Dean
BS
Allan
EL
Barber
ND
Barker
KN
.
Comparison of medication errors in an American and a British hospital
.
Am J Health Syst Pharm

1995
;
52
:
2543
9
.
5
Classen
DC
Pestotnik
SL
Evans
RS
Lloyd
JF
Burke
JP
.
Adverse drug events in hospitalized patients. Excess length of stay, extra costs, and attributable mortality
.
JAMA

1997
;
277
:
301
6
.
6
Cullen
DJ
Sweitzer
BJ
Bates
DW
Burdick
E
Edmondson
A
Leape
LL
.
Preventable adverse drug events in hospitalized patients: a comparative study of intensive care and general care units
.
Crit Care Med

1997
;
25
:
1289
97
.
7
Lesar
TS
Lomaestro
BM
Pohl
H
.
Medication-prescribing errors in a teaching hospital. A 9-year experience
.
Arch Intern Med

1997
;
157
:
1569
76
.
8
Kaushal
R
Bates
DW
Landrigan
C
McKenna
KJ
Clapp
MD
Federico
F
et al
.
Medication errors and adverse drug events in pediatric inpatients
.
JAMA

2001
;
285
:
2114
20
.
9
Brennan
TA
Leape
LL
Laird
NM
Hebert
L
Localio
AR
Lawthers
AG
et al
.
Incidence of adverse events and negligence in hospitalized patients: results of the Harvard Medical Practice Study I
.
N Engl J Med

1991
;
324
:
370
6
.
10
Kaushal
R
Shojania
KG
Bates
DW
.
Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review
.
Arch Intern Med

2003
;
163
:
1409
16
.
11
Bates
DW
Leape
LL
Cullen
DJ
Laird
N
Petersen
LA
Teich
JM
et al
.
Effect of computerized physician order entry and a team intervention on prevention of serious medication errors
.
JAMA

1998
;
280
:
1311
6
.
12
Bates
DW
Teich
JM
Lee
J
et al
.
The impact of computerized physician order entry on medication error prevention
.
J Am Med Inform Assoc

1999
;
6
:
313
21
.
13
Teich
JM
Glaser
JP
Beckley
RF
et al
.
The Brigham integrated computing system (BICS): advanced clinical systems in an academic hospital environment
.
Int J Med Inform

1999
;
54
:
197
208
.
14
Overhage
JM
Tierney
WM
Zhou
XH
McDonald
CJ
.
A randomized trial of “corollary orders” to prevent errors of omission
.
J Am Med Inform Assoc

1997
;
4
:
364
75
.
15
Teich
JM
Merchia
PR
Schmiz
JL
Kuperman
GJ
Spurr
CD
Bates
DW
.
Effects of computerized physician order entry on prescribing practices
.
Arch Intern Med

2000
;
160
:
2741
7
.
16
Taylor
R
Manzo
J
Sinnett
M
.
Quantifying value for physician order-entry systems: a balance of cost and quality
.
Healthc Financ Manage

2002
;
56
:
44
8
.
17
Dexter
PR
Perkins
S
Overhage
JM
Maharry
K
Kohler
RB
McDonald
CJ
.
A computerized reminder system to increase the use of preventive care for hospitalized patients
.
N Engl J Med

2001
;
345
:
965
70
.
18
Doolan
DF
Bates
DW
.
Computerized physician order entry systems in hospitals: mandates and incentives
.
Health Aff (Millwood)

2002
;
21
:
180
8
.
19
Ash
JS
Gorman
PN
V
Hersh
WR
.
Computerized physician order entry in U.S. hospitals: results of a 2002 survey
.
J Am Med Inform Assoc

2004
;
11
:
95
9
.
20
Poon
EG
Blumenthal
D
Jaggi
T
Honour
MM
Bates
DW
Kaushal
R
.
Overcoming barriers to adopting and implementing computerized physician order entry systems in U.S. hospitals
.
Health Aff (Millwood)

2004
;
23
:
184
90
.
21
Kaushal
R
Landrigan
C
Bates
DW
.
Improving Patient Safety in Massachusetts
. In:
Moore
RT
(ed).
Massachusetts Memos to the Governor
.
Bloomington
,
Indiana
;
2003
, pp.
98
113
.
22
Abookire
SA
Teich
JM
Sandige
H
et al
Improving allergy alerting in a computerized physician order entry system
.
Proc AMIA Symp
.
2000
:
2
6
.
23
Teich
JM
Schmiz
JL
O'Connell
EM
Fanikos
J
Marks
PW
Shulman
LN
.
An information system to improve the safety and efficiency of chemotherapy ordering
.
Proc AMIA Annu Fall Symp
.
1996
:
498
502
.
24
Bates
DW
Spell
N
Cullen
DJ
et al
.
The costs of adverse drug events in hospitalized patients
.
Adverse Drug Events Prevention Study Group

1997
;
277
:
307
11
.
25
Bates
DW
.
Using information systems to improve practice
.
Schweiz Med Wochenschr

1999
;
129
:
1913
9
.
26
Teich
JM
Petronzio
AM
Gerner
JR
Seger
DL
Shek
C
Fanikos
J
.
An information system to promote intravenous-to-oral medication conversion
.
Proc AMIA Symp
.
1999
415
9
.
27
Bates
DW
Kuperman
GJ
Rittenberg
E
et al
.
A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests
.
Am J Med

1999
;
106
:
144
50
.
28
Harpole
LH
Khorasani
R
Fiskio
J
Kuperman
GJ
Bates
DW
.
Automated evidence-based critiquing of orders for abdominal radiographs: impact on utilization and appropriateness
.
J Am Med Inform Assoc

1997
;
4
:
511
21
.
29
Guidelines and discount rates for benefit-cost analysis of federal programs
.
October
29
,
1992
; http://www.whitehouse.gov/omb/circulars/a094/a094.html%238. Accessed: March 20, 2006.
30
Chertow
GM
Lee
J
Kuperman
GJ
et al
.
Guided medication dosing for inpatients with renal insufficiency
.
JAMA

2001
;
286
:
2839
44
.
31
Shojania
KG
Yokoe
D
Platt
R
Fiskio
J
Ma'luf
N
Bates
DW
.
Reducing vancomycin use utilizing a computer guideline: results of a randomized controlled trial
.
J Am Med Inform Assoc

1998
;
5
:
554
62
.
32
Bates
DW
Kuperman
GJ
Jha
A
et al
.
Does the computerized display of charges affect inpatient ancillary test utilization?
.
Arch Intern Med

1997
;
157
:
2501
8
.
33
Kuperman
GJ
Teich
JM
Tanasijevic
MJ
et al
.
Improving response to critical laboratory results with automation: results of a randomized controlled trial
.
J Am Med Inform Assoc

1999
;
6
:
512
22
.
34
Evans
RS
Classen
DC
Pestotnik
SL
Lundsgaarde
HP
Burke
JP
.
Improving empiric antibiotic selection using computer decision support
.
Arch Intern Med

1994
;
154
:
878
84
.
35
Evans
RS
Pestotnik
SL
Classen
DC
et al
.
A computer-assisted management program for antibiotics and other antiinfective agents
.
N Engl J Med

1998
;
338
:
232
8
.
36
Langberg
M
.
Challenges to implementing CPOE: a case study of a work in progree at Cedars-Sinai
.
Mod Physician

2003
;
7
:
21
2
.
37
38
Rothschild
JM
Federico
FA
Gandhi
TK
Kaushal
R
Williams
DH
Bates
DW
.
Analysis of medication-related malpractice claims: causes, preventability, and costs. Arch Intern Med
.
Nov 25

2002
;
162
:
2414
20
.
39
Chen
P
Tanasijevic
MJ
Schoenenberger
RA
Fiskio
J
Kuperman
GJ
Bates
DW
.
A computer-based intervention for improving the appropriateness of antiepileptic drug level monitoring
.
Am J Clin Pathol

2003
;
119
:
432
8
.
40
Solomon
DH
Shmerling
RH
Schur
PH
Lew
R
Fiskio
J
Bates
DW
.
A computer based intervention to reduce unnecessary serologic testing
.
J Rheumatol

1999
;
26
:
2578
84
.
Most of the individual studies that were aggregated in this report were supported by the Agency for Healthcare Research and Quality, Rockville, MD.
Members of the Brigham and Women's Hospital CPOE Working Group include Melissa Christino, William Churchill, Petr Jarolim, Joshua Peterson, Eric Poon, Jeffrey Rothschild, Cynthia Spurr, Carl Stapinski, Jonathan M. Teich, Debra Thomas, and Samuel Wang.