Abstract

Objective To examine the financial impact of electronic health record (EHR) implementation on ambulatory practices.

Methods We tracked the practice productivity (ie, number of patient visits) and reimbursement of 30 ambulatory practices for 2 years after EHR implementation and compared each practice to their pre-EHR implementation baseline.

Results Reimbursements significantly increased after EHR implementation even though practice productivity (ie, the number of patient visits) decreased over the 2-year observation period. We saw no evidence of upcoding or increased reimbursement rates to explain the increased revenues. Instead, they were associated with an increase in ancillary office procedures (eg, drawing blood, immunizations, wound care, ultrasounds).

Discussion The bottom line result—that EHR implementation is associated with increased revenues—is reassuring and offers a basis for further EHR investment. While the productivity losses are consistent with field reports, they also reflect a type of efficiency—the practices are receiving more reimbursement for fewer seeing patients. For the practices still seeing fewer patients after 2 years, the solution likely involves advancing their EHR functionality to include analytics. Although they may still see fewer patients, with EHR analytics, they can focus on seeing the right patients.

Conclusions Practice reimbursements increased after EHR implementation, but there was a long-term decrease in the number of patient visits seen in this ambulatory practice context.

Background and significance

Ambulatory physician practices have been slow to utilize electronic health records (EHRs), despite the anticipated benefits to patients1,2 and billions of dollars in incentive payments.3 A major reason behind this reluctance is concern that EHRs will have a negative impact on practice productivity and reimbursement.4–6 EHRs reportedly slow patient care because of ‘…the considerable work and time needed to accommodate these disruptive technologies into day-to-day care’.7 Practices worry that taking more time to see each patient will lead to fewer patient visits and lower reimbursements, threatening their revenue stream.

Is EHR implementation really a threat to practices’ revenue stream? EHR proponents argue, based on the experience of early adopters, that EHR implementation temporarily decreases practice productivity by 30–50%, but patient volumes and reimbursement return to baseline within about 3 months.8 Research to confirm this has yielded mixed results, with some showing an increase,9 but others confirming a decrease in practice productivity.10 This research has been limited by three issues. First, studies have substituted proxy indicators of productivity (eg, length of patient visits or relative value units (RVUs)) instead of tracking the actual number of patient visits.9–11 Second, studies have looked at the impact of EHR across an entire health system, making it difficult to infer the financial dynamics of individual practices.2,12 Third, studies have reported high-level financial measures such as return on investment or profits that do not illustrate the financial dynamics of ambulatory practices—the number of patients seen and the reimbursements from those visits.13,14 As a result, we still have limited knowledge of whether EHR implementation threatens a practice's revenue stream.

Objective

In this study, we adopt a perspective that is commonly used in the business literature.15 To examine the financial impact of EHR implementation, we measure the number of patients seen by each practice and the reimbursements received as a result of those visits for 2 years after EHR implementation and compare it to a pre-implementation baseline.

Materials and methods

In order to assess the effects of EHR implementation on practice productivity and reimbursements, we used a quasi-experimental design with repeated measures for 1 year before and 2 years after each practice implemented an EHR. All data management was conducted using SAS V.9.3. In order to facilitate pre/post comparisons, the data were structured as difference scores, with the post-implementation score subtracted from the pre-implementation baseline for that practice. The data were, in other words, group-centered on the baseline.16

The data were extracted from the billing program of 30 practices (six medical primary care practices, 12 medical specialty practices, five obstetrics & gynecology practices, six surgical practices, and a sleep center) within the faculty practice plan of an academic medical center. These practices represent about 90% of the practice plan. A practice plan is the administrative organization that manages the faculty practices at a medical school. A single EHR system was sequentially implemented from February 2007 to April 2009. The order of EHR implementation was determined by the informatics leadership. All of the EHRs functioned at stage 4 of the Healthcare Information and Management Systems Society (HIMSS) Ambulatory Adoption Model, which includes problem lists, medication lists, order entry, review of labs and x-rays, prescribing, and progress notes.17 None of the practices used an automatic coding function, so all of the billing codes were generated by the physician both before and after EHR implementation.

Why did we conduct at the practice level instead of the individual physician level? First, as is common in this type of analysis, we found high levels of incomplete data for individual physicians across the 3-year observation period. In this academic medical center context, there is more provider turnover than is typical in a private ambulatory practice, even though we are only considering attending physicians and not residents or fellows. As a result, analyzing the data at the practice level allowed us to maximize the information in the data and standardize comparisons across the observation period. We also thought the practice level-of-analysis better because we found evidence of patient-shifting between physicians within practices. For example, we saw that when a new physician came into a practice, the patient volume for established physicians decreased as the new physician became more active. This decline in productivity could spuriously appear to be related to EHR implementation if we only considered the individual physician level-of-analysis. Finally, analyzing the data at the practice level made sense from the business perspective because the EHRs were implemented and utilized at the practice level.

In examining the practice productivity/reimbursement differences scores, we found that the large practices had disproportionate impact. The primary care practices not only had more providers in each practice compared to the specialty practices but they also saw more patients per hour. As a result, changes of a few hundred patients per quarter would be within the seasonal variation for a large primary care practice but equal the quarterly output for a specialty practice. To adjust for these practice size effects, we divided the number of patients seen by each practice by the number of physicians in the practice that quarter (range 1–19). As a result, even though we are measuring practice productivity, our units of change are on a per-physician basis.

The final data management decision was to measure change at quarterly (ie, 3-month) intervals. Although previous similar research used biannual intervals,18 we found that quarterly intervals offered the optimal balance of illustrating the patterns of change in the data while reducing noise. The data were synchronized so that the month that EHR implementation began occurred at time 1 (T1). EHR implementation is defined as occurring when the practices begin to use the software in patient care. The quarters were calculated as 3 months beginning from the first day of the month that EHR implementation began. As a result, we were not using traditional fiscal quarters, but the calculation of the quarterly intervals was counted as 3 months from when that individual practice started using the EHR.

A multilevel model for change was used to assess the statistical significance of changes in the variables over time; we used the analytical procedures recommended by Singer and Willet,16,19–21 using IBM SPSS V.20 with the MIXED procedure. A model without any predictors (ie, an unconditional means model) was first estimated before adding the isolated effects of time on the dependent variable as a baseline (ie, an unconditional growth model). Covariates (ie, a series of conditional growth models) were then added that might influence the pattern of change within each of the practices over time (ie, the within-practice or level-1 effects). As a final step, covariates were added that might cause the practices to react differently to EHR implementation (ie, the between-practice or level-2 effects). We emphasize that in developing and assessing these models, our primary interest is the effect of EHR implementation. That is, the time variable is our primary focus. We include the covariates only as control variables to reduce unexplained variance in the model and optimize the estimate of the time variable.16 Details of the analytical procedures are available in an online supplementary technical appendix. We will only report the results of the final models here.

Results

Table 1 provides an overview of the distribution of the patient visits and providers across the practices. As seen in table 1, practice size (in terms of the number of providers) and specialty were highly correlated: the surgical specialties had relatively few physicians (mean=2.6) while the medical specialties (mean=5.4), obstetrics & gynecology (mean=7.0), and primary care (mean=9.1) had progressively more physicians.

Table 1:

The distribution of patients and providers across the practices

 Patients/practice/quarter Patients/provider/quarter Providers/practice 
All practices 943 (882.7) 164 (125.1) 5.7 (4.1) 
Primary care 1945 (451.0) 238 (95.3) 9.1 (3.3) 
Medical specialties 840 (885.4) 147 (140.5) 5.4 (2.9) 
Obstetrics & gynecology 691 (731.4) 74 (52.1) 7.0 (5.5) 
Surgery 460 (561.2) 186 (114.2) 2.6 (1.9) 
 Patients/practice/quarter Patients/provider/quarter Providers/practice 
All practices 943 (882.7) 164 (125.1) 5.7 (4.1) 
Primary care 1945 (451.0) 238 (95.3) 9.1 (3.3) 
Medical specialties 840 (885.4) 147 (140.5) 5.4 (2.9) 
Obstetrics & gynecology 691 (731.4) 74 (52.1) 7.0 (5.5) 
Surgery 460 (561.2) 186 (114.2) 2.6 (1.9) 

SDs are in parentheses.

Practice productivity

The first research question involves the impact of EHR implementation on practice productivity, defined as the number of patient visits during that quarter. The number of patients seen in a practice was determined by summing the evaluation and management (E/M) Current Procedural Terminology codes (99201–99205 and 99211–99215). We divided the total number of patients seen in the practice by the unique number of physicians that saw patients in the practice during that quarter. This approach to counting patient visits allows patients to be counted more than once. The same patient may visit his or her primary care doctor more than once in a quarter or see both a primary care doctor and a specialist. Since these are change scores, the resulting number reflects changes in practice productivity per physician per quarter. If a practice with five physicians, for example, saw 10 fewer patients/physician/quarter, then the practice had 50 fewer patient visits in that quarter.

In the quarter prior to EHR implementation, practices saw seven fewer patients/physician (SD=38.8), so productivity declined before implementing the software. This productivity decline continued after the EHR was implemented. Across all post-implementation quarters, practices saw an average of 15 fewer patients/physician/quarter compared to the baseline (SD=53.1). As can be seen in figure 1, there was a general trend that practice productivity declined in each quarter after EHR implementation. The exception is a brief recovery in the T2 and T3 periods, about 3–6 months after EHR implementation. We also broke out the productivity scatterplot by specialties, available in the online supplementary technical appendix, which showed that all the specialties had similar patterns of change.

Figure 1:

Changes in practice productivity around electronic health record (EHR) implementation. Changes are measured as the group-centered average of patients/provider/quarter (ppq) compared to the baseline (time −3 to time −1). First and third IQRs are in parentheses. The arrow at time 1 denotes EHR implementation.

Figure 1:

Changes in practice productivity around electronic health record (EHR) implementation. Changes are measured as the group-centered average of patients/provider/quarter (ppq) compared to the baseline (time −3 to time −1). First and third IQRs are in parentheses. The arrow at time 1 denotes EHR implementation.

To assess the statistical significance of these changes, we analyzed a multilevel model for change. The time variable captures the effect of EHR implementation because the data were synchronized so that EHR implementation occurred at T1. Three level-l covariates were added to model how individual practices change over time—secular growth, practice effects, and a variable designed to capture the recovery in T2/T3. The secular growth variable captures the general demand for patient services and seasonal variation. Practice effects capture the degree to which the implementation team improved with experience and thus accelerated the implementation process. The T2/3 recovery variable captures the discontinuity in the slope seen in figure 1.

Level-2 variables capture difference between practices in how they adapt to EHR implementation. The number of physicians in a practice or the specialty of the practice, for example, could influence how practices adapt to EHR. Larger practices would be more likely to have economies of scale that could synergistically boost productivity beyond the additive effects of simply having more physicians. Specialty practices have different types of care processes and so would be expected to have differing responses to EHR implementation. As described earlier, we found that practice size and specialty were correlated (ie, the primary care practices were larger and the specialty practices were smaller). Given that specialty is a nominal variable and the sample has only 30 practices, we decided to measure both of these factors together by using the average number of physicians in each practice in a given quarter as the indicator for both practice size and specialty.

EHR implementation (coeff=−18.0, t=−2.44, p=0.02) had a significant and negative impact on practice productivity across the observation period. Neither secular growth (coeff=0.16, t=0.37, p=0.71) nor practice effects (coeff=−0.29, t=−0.34, p=0.73) had a significant effect on practice productivity. The discontinuity in the slope around T2 and T3 (coeff=12.1, t=1.65, p=0.10) was marginally significant with an increase of 12 patients around T2 and T3. For the level-2 covariate, practice specialty specialty/size (coeff= −0.58, t=−0.65, p=0.52) did not have either a significant or a substantial effect on how practices reacted to EHR implementation.

In summary, we found that EHR implementation had a significant and negative impact on practice productivity by 18 patients/physician/quarter. Given that the average practice had about six physicians per practice, the average loss in practice productivity was about 108 patients per quarter. This is not to say that all practices in the sample had productivity losses—some of the practices recovered after EHR implementation and their productivity returned to baseline. But enough of the sample (six practices across multiple specialties and practice types) were still having productivity losses at the end of the observation, to drag the slope down for the entire organization.

Post-hoc analysis 1: are the productivity losses due to the recession?

One possible explanation for the decline in practice productivity is that the number of patients decreased because the entire organization was seeing fewer patients. There was an economic slowdown during the observation period, so perhaps there were simply fewer patients visiting their doctor, which happened to coincide with this EHR implementation? We conducted two analyses to assess this possibility. We first examined the total number of visits across all of the practice plan. As can be seen in the solid line of figure 2, however, the organization was seeing greater numbers of patients across the observation period. How is it possible that the practices had major productivity losses after EHR implementation, but the practice plan was still seeing more patients overall? The practice plan was adding more physicians, as seen in the dotted line of figure 2, which was associated with more overall visits for the practice plan. The number of providers trend is illustrated with the dotted line on the secondary vertical axis of figure 2.

Figure 2:

The total number of visits and providers across the organization during the study observation period.

Figure 2:

The total number of visits and providers across the organization during the study observation period.

Did the practice plan hire too many physicians in response to the increasing demand, causing an excess of physicians and contributing to the decreased number of patient visits per provider that we saw after EHR implementation? Across the implementation time period, from 2006 through 2010, physicians were actually more busy—going from 190 visits/physician in the first quarter of 2006 to around 230 visits/physician at the end of 2010. As a result, we do not believe that the practice plan hired too many physicians and diluted demand for patient services. The graph is available in the online supplementary technical appendix.

As a confirmation on the demand issue, we also compared the productivity of two similar practices—one of the first to implement the EHR in April 2007 and one of the last practices to implement the EHR in March 2009. Both were very similar large, primary care practices. Patients were not able to easily shift between practices. We examined the productivity of the late practice while the early practice was implementing the EHR. The implementation times for each of the practices are shown by arrows in figure 3. As can be seen, the number of patients seen in the late-implementing practice continued to increase after the early practice implemented the EHR. Both of these analyses suggest that there was increasing demand for patient visits across the observation times in which the EHR was being implemented.

Figure 3:

A comparison of the number of patient visits (ie, demand) for two similar primary care practices, one of which implemented electronic health record (EHR) early and the other implemented EHR late.

Figure 3:

A comparison of the number of patient visits (ie, demand) for two similar primary care practices, one of which implemented electronic health record (EHR) early and the other implemented EHR late.

Results for reimbursement

Our next research question focuses on how EHR implementation affects practice reimbursement. We define reimbursement as the revenues that result from the provision of patient care services in the ambulatory office where the EHR was implemented. We measure the change in practice reimbursements after EHR implementation compared to a pre-implementation baseline. Reimbursements varied widely, as is common with revenue analyses, by specialty and quarter. To offset the undue influence of larger practices, we again adjusted for the number of physicians seeing patients and group centered on the mean for the individual practices. Our treatment of this reimbursement dependent variable, therefore, is identical to that of the productivity analysis. The practice reimbursement trend is illustrated in figure 4 with a scatterplot using a Loess fit line. As can be seen, reimbursement/physician/quarter increases in the first year after EHR implementation but flattens after time 4 (or in year 2).

Figure 4:

Scatterplot of change in practice reimbursement from the pre-implementation baseline. Below, the scatterplot for the log-transformed variable used in the statistical analysis.

Figure 4:

Scatterplot of change in practice reimbursement from the pre-implementation baseline. Below, the scatterplot for the log-transformed variable used in the statistical analysis.

Is this upward trend in practice reimbursements after EHR implementation significant? Prior to conducting the multilevel model for growth analysis, we found that the change in practice reimbursement measure was highly skewed (−6.5) and kurtotic (45.6). Log transformation created a suitably normal (skewness=−0.38 and kurtosis=0.41) variable for statistical analysis. Practice effects and secular growth covariates were again included at level-1 and size/specialty at level-2.

The log of reimbursements/physician/quarter (coeff=0.22, t=4.12, p<0.001) increased significantly after EHR implementation. The model fit was optimized with the unconditional growth model, so we only report the time coefficient.

In summary, our analysis to this point has yielded the counterintuitive result that EHR implementation reduces practice productivity but increases practice reimbursements. How is this happening?

Post-hoc analysis 2: how does revenue increase while productivity decreases?

Logically, the practices must be either charging more per visit—that is, upcoding,14 receiving more reimbursement for the same level of charges, or billing more ancillary procedures at each visit. To assess upcoding, we first checked to see if physicians were claiming more RVUs per visit. RVUs are the basis for reimbursement, so RVU analysis provides a sensitive measure of upcoding. As can be seen in the scatterplot of RVUs per E/M in figure 5A, there was no evidence of upcoding around EHR implementation. A second possibility is that insurers became more generous in their reimbursements for the charges submitted by the practices. As can be seen in figure 5B the reimbursement/charge ratio remained flat throughout the observation period.

Figure 5:

Exploring how revenues increased with decreased productivity. (A) The ratio of RVUs billed per E/M visit. (B) The ratio of reimbursement to charges.

Figure 5:

Exploring how revenues increased with decreased productivity. (A) The ratio of RVUs billed per E/M visit. (B) The ratio of reimbursement to charges.

A final alternative is that the practices were billing more ancillary procedure codes. Physicians are not limited to just billing an E/M code for a patient visit. They can also bill for ancillary procedures, which are patient-care activities that complement the E/M codes. Ancillary procedures are often, but not always performed by a physician. At times a nurse, technician, or medical assistant can provide an ancillary procedure (eg, venepuncture, ultrasound scanning, or application of a wound dressing). The most common ancillary procedures varied by specialty, as shown in table 2.

Table 2:

Typical examples of ancillary procedures performed in each specialty

Specialty Procedures 
Medicine (primary care) Drawing blood 
Immunization 
Electrocardiogram 
Medical specialties Pulmonary function testing 
Injecting a joint 
Medication infusions 
Obstetrics & gynecology Microscopic exam 
Venepuncture 
Fetal ultrasound 
Surgery Wound debridement 
Abscess drainage 
Bandaging wounds 
Specialty Procedures 
Medicine (primary care) Drawing blood 
Immunization 
Electrocardiogram 
Medical specialties Pulmonary function testing 
Injecting a joint 
Medication infusions 
Obstetrics & gynecology Microscopic exam 
Venepuncture 
Fetal ultrasound 
Surgery Wound debridement 
Abscess drainage 
Bandaging wounds 

We examined the total number of ancillary procedures over time. Since providers other than physicians can perform ancillary procedures, we did not adjust for the number of physicians in the practice but simply compared the total number of ancillary procedures across the practice to a pre-implementation baseline. The result is a total-practice difference score. Change in the numbers of ancillary procedures per practice are shown in a scatterplot  (figure 6). Details of the means and SDs are available in the online supplementary technical appendix. As can be seen, the plot of ancillary procedures per practice trends upward after EHR implementation.

Figure 6:

Scatterplot of ancillary procedures around electronic health record (EHR) implementation. The dependent variable is the total practice difference score, calculated by the total number of ancillary procedures after EHR implementation in the practice that quarter divided by the baseline number of ancillary procedures.

Figure 6:

Scatterplot of ancillary procedures around electronic health record (EHR) implementation. The dependent variable is the total practice difference score, calculated by the total number of ancillary procedures after EHR implementation in the practice that quarter divided by the baseline number of ancillary procedures.

To assess the statistical significance of these changes, we again developed a multilevel model for change with ancillary procedures as the dependent variable. Ancillary procedures were calculated by counting the number of non-E/M procedures/practice/quarter after EHR implementation and subtracting the average number of procedures per quarter in the baseline. We included practice effects in level-1 and size/specialty in level-2 of the multilevel model for change.

EHR implementation had a significant effect on the total number of ancillary procedures (coeff=94.1, t=2.80, p=0.01) compared to the pre-implementation baseline. Practices were billing an additional 94 procedures per quarter after EHR implementation. We did not find any significant practice effects (coeff=4.2, t=0.49, p=0.62), but the size/specialty (coeff=17.1, t=1.81, p=0.08) was marginally significant. In looking at the sub-group plots, the primary care medical practices and obstetrics & gynecology practices (available in the online supplementary technical appendix) were billing the most additional procedures. There were modest changes in the medical specialties and no change in the surgical areas.

Discussion

A major reason for the slow uptake of EHRs—and delayed access to the benefits of EHR—has been financial concerns about the impact of EHR implementation on practice productivity and reimbursements. Our ability to continue to make EHR updates depends on generating revenues to pay for these investments. The purpose of this study was to study the financial drivers of EHR implementation—practice productivity and the reimbursements that result from these patient visits. The bottom line news is good: practice revenues increased during EHR implementation despite persistent productivity losses.

EHR implementation in this study increased reimbursements but reduced long-term practice productivity across all specialties. While the productivity losses can be seen in a negative light, these findings also suggest a type of efficiency in which the practices are getting paid more for seeing fewer patients. It is also possible that these practices were taking better care of fewer patients. We have no data on this issue, but this perspective would be consistent with the literature in which EHRs are associated with greater patient safety and higher scores on some measures of quality.22,23 Our data are silent on the quality of care patients received, but we emphasize that the ‘better care for fewer patients’ approach depends on advanced EHR analytic functions.

The brief practice productivity recovery seen around T2/T3 may offer insights into the mechanisms of the EHR productivity problem. This brief recovery followed by decline may be related to how the practices managed patient bookings around EHR implementation. As can be seen in figure 1, the practices appear to intentionally decrease the number of patients scheduled to be seen even before EHR implementation. This particular slowdown is likely due to software training and other preparations for EHR. After implementation (ie, T1 and T2), practice managers were increasing the scheduled patients, leading to the recovery bump. What happened after T3 (6 months)? One possibility is that patient flow became congested as providers could not cope with the increasing bookings after the initial EHR implementation—leading to overcrowding, stress, and a growing backlog of patients leading to persistent productivity problems. As a result, practices may benefit from a different approach toward increasing patient volume after EHR implementation.

This scenario is also consistent with physician dissatisfaction associated with EHRs. Many physicians have found the software non-intuitive and difficult to use, creating dissatisfaction and intruding into the patient care process.24 Scribes have been found to be a suitable workaround to the issues.25 We would point out, however, that hiring a scribe would take up much of revenue gains of about $3000/provider/quarter shown in the top panel of figure 4. Instead of reinvesting to enhance EHR functionality, this would lead to investing in a work-around.

A second implication of the productivity losses is patient access. We now have an influx of patients into the system from implementation of the Affordable Care Act. To the degree that EHRs reduce provider productivity, they reduce the capacity of the healthcare system and potentially threaten patient access to care.

What is the ultimate solution to these productivity challenges after EHR implementation? In discussing our findings with a variety of experts with differing perspectives, they all converge on a similar recommendation. They believe that we should focus our efforts on developing analytical capabilities that will lead to population health management approaches to caring for their patient panels. In this way, they could focus their limited patient encounter capabilities on those patients in which they could have the greatest benefits. This approach would also position the practices to take advantage of value-based reimbursement approaches that are becoming increasingly popular. Even under current fee-for-service reimbursement approaches, however, this ‘jump to analytics’ approach would also maximize the revenue benefits the organization is seeing from their ancillary procedures. The key is to not get bogged down in the provider productivity issue, but focus on developing EHR analytics capabilities.

We recognize that measuring the number of patients seen by physicians is ultimately not the ideal method of assessing the value of an EHR. The goal of EHR investments should be to improve the health of the practice's patients. It is a reality, however, that most practices still get paid by the number of patients seen, so we cannot discard our concern about the number of patients seen. In the future, practices may get paid by how much health and wellness they deliver to their patients. In this world, an EHR should greatly enhance physician effectiveness even if fewer patients are seen by the physician. In the meantime, this study offers some reassurance that the reimbursement stream can be protected despite productivity declines.

Limitations

Although the generalizability of this study is enhanced with the multi-specialty design, the study took place at an academic medical center. While many of the physicians in this setting may be younger and more technologically oriented, it may also be that physicians had less time to devote to EHR implementation because of teaching or research duties. This EHR implementation occurred before the HITECH incentives, so this organization is an early adopter. Late-majority adopters, with less technological orientation, could be at risk for greater productivity losses.

Any quasi-experimental design is prone to bias from lack of random assignment.26 While we accounted for the variance due to secular growth, seasonal variation, and practice effects, there may be other unrecognized factors that influence our results.

Conclusion

Implementation of an EHR was associated with increased revenues but also sustained losses of productivity in ambulatory practices for at least 2 years. Practices were able to maintain reimbursements by billing more ancillary procedures such as immunizations, venepuncture, and EKGs in the primary care practices.

Contributors

EC and NH provided the data and the operational expertise on EHR implementation. PD led the discussion. MH drafted the manuscript and led the statistical analysis. All co-authors edited the final manuscript.

Funding

This work was supported by a grant from the Drexel University Provost's Office through the Health Informatics Jumpstart program.

Competing interests

None.

Ethics approval

Drexel University IRB.

Provenance and peer review

Not commissioned; externally peer reviewed.

SUPPLEMENTARY MATERIAL

Supplementary material is available online at http://jamia. oxfordjournals.org/.

REFERENCES

1
Chaudhry
B
Wang
J
Wu
S
et al
.
Systematic review: impact of health information technology on quality, efficiency, and costs of medical care
.
Ann Intern Med
 .
2006
;
144
:
742
752
.
2
Hillestad
R
Bigelow
J
Bower
A
et al
.
Can electronic medical record systems transform health care? Potential health benefits, savings, and costs
.
Health Aff
 .
2005
;
24
:
1103
1117
.
3
Government Accountability Office
.
Electronic Health Records: Number and Characteristics of Providers Awarded Medicaid Incentive Payments for 2011. 13 December 2012. http://www.gao.gov/products/GAO-13-146R. Accessed February 12, 2013
.
4
Medical Group Management Association (MGMA)
.
Electronic Health Records: Status, Needs and Lessons
.
2011
. .
5
DesRoches
CM
Campbell
EG
Rao
SR
et al
.
Electronic health records in ambulatory care—a national survey of physicians
.
N Engl J Med
 .
2008
;
359
:
50
60
.
6
Kellermann
AL
Jones
SS
.
What it will take to achieve the as-yet-unfulfilled promises of health information technology
.
Health Aff (Millwood)
 .
2013
;
32
:
63
68
.
7
Cresswell
K
.
Electronic health record technology
.
JAMA
 .
2012
;
307
:
2255
2256
.
8
Wang
SJ
Middleton
B
Prosser
LA
et al
.
A cost-benefit analysis of electronic medical records in primary care
.
Am J Med
 .
2003
;
114
:
397
403
.
9
Brotzman
GL
Guse
CE
Fay
DL
et al
.
Implementing an electronic medical record at a residency site: physicians’ perceived effects on quality of care, documentation, and productivity
.
WMJ
 .
2009
;
108
:
99
103
.
10
Samaan
ZM
Klein
MD
Mansour
ME
et al
.
The impact of the electronic health record on an academic pediatric primary care center
.
J Ambul Care Manage
 .
2009
;
32
:
180
187
.
11
Adler-Milstein
J
Huckman
RS
.
The impact of electronic health record use on physician productivity
.
Am J Manag Care
 .
2013
;
19
:
SP345
SP352
.
12
Adler-Milstein
J
Salzberg
C
Franz
C
et al
.
Effect of electronic health records on health care costs: longitudinal comparative evidence from community practices
.
Ann Intern Med
 .
2013
;
159
:
97
104
.
13
Adler-Milstein
J
Green
CE
Bates
DW
.
A survey analysis suggests that electronic health records will yield revenue gains for some practices and losses for many
.
Health Aff
 .
2013
;
32
:
562
570
.
14
Miller
RH
West
C
Brown
TM
et al
.
The value of electronic health records in solo or small group practices
.
Health Aff
 .
2005
;
24
:
1127
1137
.
15
Gupta
S
Hanssens
D
Hardie
B
et al
.
Modeling customer lifetime value
.
J Serv Res
 .
2006
;
9
:
139
55
.
16
Singer
JD
Willett
JB
.
Applied longitudinal data analysis
 .
Oxford
:
Oxford University Press
,
2003
.
17
Healthcare Information and Management Systems Society (HIMSS)
.
Ambulatory EMR Adoption Model
. .
18
Fleming
NS
.
Impact of health IT on primary care workflow and financial measures
.
Health Serv Res
 .
2014
;
49
(1 Pt 2)
:
 405
420
.
19
Heck
RH
Thomas
SL
Tabata
LN
.
Multilevel and longitudinal modeling with IBM SPSS
 .
New York, NY
:
Routledge
,
2010
.
20
Peugh
JL
Enders
C
.
Using the SPSS mixed procedure to fit cross-sectional and longitudinal multilevel models
.
Ed Psych Meas
 .
2005
;
65
:
717
741
.
21
Singer
JD
.
Using SAS proc mixed to fit multilevel models, hierarchical models, and individual growth models
.
J Ed Beh Stat
 .
1998
;
24
:
323
355
.
22
Kazley
AS
Ozcan
YA
.
Do hospitals with Electronic Medical Records (EMRs) provide higher quality care? An examination of three clinical conditions
.
Med Care Res Rev
 .
2008
;
65
:
496
513
.
23
Parente
ST
McCullough
JS
.
Health information technology and patient safety: evidence from panel data
.
Health Aff
 .
2009
;
28
:
357
360
.
24
Verdon
DR
.
EHRs: the real story
.
Med Econ
 .
2014
;
91
:
18–27
.
25
Hafner
K
.
A Busy Doctor's Right Hand, Ever Ready to Type. NY Times. Accessed Jan 12, 2014
.
26
Shadish
WR
Cook
TD
Campbell
DT
.
Experimental and quasi-experimental designs for generalized causal inference
 .
Boston, MA
:
Houghton Mifflin Company
,
2002
.

Supplementary data

Comments

0 Comments