Abstract

In response to mounting evidence that use of electronic medical record systems may cause unintended consequences, and even patient harm, the AMIA Board of Directors convened a Task Force on Usability to examine evidence from the literature and make recommendations. This task force was composed of representatives from both academic settings and vendors of electronic health record (EHR) systems. After a careful review of the literature and of vendor experiences with EHR design and implementation, the task force developed 10 recommendations in four areas: (1) human factors health information technology (IT) research, (2) health IT policy, (3) industry recommendations, and (4) recommendations for the clinician end-user of EHR software. These AMIA recommendations are intended to stimulate informed debate, provide a plan to increase understanding of the impact of usability on the effective use of health IT, and lead to safer and higher quality care with the adoption of useful and usable EHR systems.

Introduction

US healthcare delivery is in the midst of a profound transformation which results, at least in part, from Federal public policy efforts to encourage the adoption and use of health information technology (health IT). For example, HITECH regulations1 within the American Recovery and Reinvestment Act2 incentivize health IT use,3,4 and are changing the practice of medicine and clinical care delivery in both beneficial3,5,6 and untoward ways.7 Increased adoption of electronic health record (EHR) systems has been accompanied by heightened recognition of issues related to ‘goodness of fit’ in the user-friendliness of EHR systems.8 Some EHR users lament that health IT seems designed more for clinical transactions than for clinical care, and may not be easy to use in some care settings.9,10 In addition, many EHR systems require extensive training and lack standard user interfaces so that clinicians who work in multiple care settings using disparate technologies may struggle with the differences in interface design, with adverse impact on patient safety.11 User interface design can influence provider productivity: well-designed interfaces speed work, while poorly designed interfaces steal minutes from busy schedules. The Institute of Medicine (IOM) report, Health IT and Patient Safety: Building Safer Systems for Better Care identified means by which health IT can lead to safer care, as well as introduce new safety risks. A critical component of safe and effective use of health IT is usability—‘the effectiveness, efficiency, and satisfaction with which the intended users can achieve their tasks in the intended context of product use.’12 The IOM recommended that ‘[t]he Secretary of HHS [Health and Human Services] should specify the quality and risk management process requirements that health IT vendors must adopt, with a particular focus on human factors, safety culture, and usability’ (recommendation 6, p 99).

Purpose of this AMIA statement

Given the anticipated adoption of health IT, and the potential for increased health IT-related harm or potential error, the AMIA Board of Directors convened a task force of members drawn from academia, clinical practice, and industry to produce a set of AMIA recommendations on enhancing patient safety and the quality of care with improved usability of EHR systems. These AMIA recommendations are intended to stimulate informed debate, form the basis of a plan to increase understanding of the impact of usability on the effective use of health IT, and lead to safer and higher quality care with the adoption of useful and usable EHR systems.

To address this issue, the task force convened for over a year. Subcommittees reviewed the literature on usability in health IT, current related activities underway at various US Federal agencies, lessons learned regarding usability and human factors in other industries, and current federally funded research activities. The key principles and recommendations described below are based on these reviews.

Relationship of usability to optimal healthcare practice

To frame this discussion, the AMIA Task Force on Usability considered the following issues related to health IT: (1) safe and effective use of EHR, (2) EHR usability, and (3) EHR usability-associated medical errors. Recent reports describe the safe and effective use of EHR as a property resulting from the careful integration of multiple factors in a broad socio-technical framework,13 including coordination and consideration across requirements assessment, application design, usability and human factors engineering, implementation, training, monitoring, and feedback to application developers.1,14–16 Following best practices for EHR implementation is essential to safe and effective use.2,17 Analyses of facilitators and barriers to physicians' use of EHR systems suggest that usability is a major theme among system attributes, along with functionality, speed, support for hardware and software, required learning time, typing proficiency, understanding of the EHR system, motivation and personal initiative, and user-developed strategies and workarounds.3,4,18 Further analysis suggests that understanding user behavioral models is important to achieving effective use.3,5,6,19,20 Overcoming each of these issues is essential for improving the usability of EHR systems in clinical practice.

User error may result in untoward outcomes and unintended negative consequences. These may also occur as a result of poor usability,7,21–24 and may also be an emergent property only demonstrated after system implementation or widespread use.8,9 User errors may occur without adverse events,9,10,24–26 and some may not even be apparent to the user, analyzed by hospital or clinic review boards, or reported to the vendor.11 While not causing undue harm, these errors are still problematic as they represent a mismatch between the user's model of the task and expected outcome,12,25 and the application's functionality and the resulting action or event.9,13,27 They may also represent a potential health IT-related error yet to happen.13,28 In some cases, clinicians use more than one commercial EHR system, with differences between the model of the task and the software functionality to execute the task, as well as differences in the terminologies used in the systems.29,30 Anecdotal reports suggest that these application differences result in an increased training burden for EHR users. Excessive alert fatigue can undermine the efficacy of clinical decision support in computer-based provider order entry (CPOE)31,32 and in other IT functions,11,33 and can result in very high override rates.34–36 Some suggest that the expected gains sought with the adoption of EHR are not yet realized.37–39

Actual adverse events or medical errors resulting from application design and usability, however, have also been described.9,10,14,15,40–42 Walker et al define an EHR-related system flaw as ‘Any characteristic of an EHR or of its interactions with other healthcare systems that has the potential to worsen care quality or patient outcomes. Other healthcare systems include individuals, care teams, facilities, policies, care processes, and healthcare organizations. Flaws may be introduced during the specification, design, configuration, or continuous-improvement phases of the EHR lifecycle’ (p 273).10 Sittig and Singh define EHR-related errors as occurring ‘anytime health IT is unavailable for use, malfunctions during use, is used incorrectly by someone, or when health IT interacts with another system component incorrectly, resulting in data being lost or incorrectly entered, displayed, or transmitted’ (p 1281).43 The design of software applications requires both technical expertise and the ability to completely understand the user's goal, the user's workflow, and the socio-technical context of the intent.5,6,13,15,16,18,44 The design of clinical information systems is evolving from transaction-oriented systems toward a design focusing on patient-centered care, and on the needs of patients and healthcare teams.14,15,19,20,45 To achieve current US policy objectives,1 transform our healthcare delivery system,9 and create a learning healthcare system,46–49 clinicians need to use usable, efficient health IT that enhances patient safety and the quality of care. Some experts suggest that improving the usability of EHR may be critical to the continued successful diffusion of the technology.50 Taken together, these factors suggest a need for renewed attention and focus on improving the usability of health IT to enhance patient safety and the quality of care.

AMIA's guiding principles

Framing usability assessment in health IT

This work focuses on the usability of EHR, and leaves aside medical devices, mobile devices, personal health records, and other related health information technologies. In particular, this work focuses on usability as it pertains to EHR systems and EHR module components.51 We touch upon usability principles, use cases, and interface guidelines52 as essential building blocks for effective user-centered design (UCD) and system implementation. For this work, we reference the Healthcare Information Management and Systems Society (HIMSS) usability definition, which includes nine attributes: simplicity, naturalness, consistency, forgiveness and feedback, effective use of language, efficient interactions, effective information presentation, preservation of context, and minimization of cognitive load.53 A more comprehensive and evidence-based perspective on usability is provided by Zhang and Walji,54 where usability refers to how useful, usable, and satisfying a system is for the intended users to accomplish goals by performing certain sequences of tasks.

The research team at the National Center for Cognitive Informatics and Decision Making in Healthcare, based upon an evidence review, proposed 14 usability principles which may guide the design and implementation of EHR.54 These principles are listed in table 1.

Table 1

Fourteen usability principles for the design of electronic medical records

1. Consistency—Design consistency and standards utilization 8. Message—Useful error messages 
2. Visibility—System state visibility 9. Error—Use error prevention 
3. Match—System and world match 10. Closure—Clear closure 
4. Minimalism—Minimalist design 11. Reversibility—Reversible actions 
5. Memory—Memory load minimization 12. Language—User language utilization 
6. Feedback—Informative feedback 13. Control—User control 
7. Flexibility—Flexible and customizable system 14. Documentation—Help and documentation 
1. Consistency—Design consistency and standards utilization 8. Message—Useful error messages 
2. Visibility—System state visibility 9. Error—Use error prevention 
3. Match—System and world match 10. Closure—Clear closure 
4. Minimalism—Minimalist design 11. Reversibility—Reversible actions 
5. Memory—Memory load minimization 12. Language—User language utilization 
6. Feedback—Informative feedback 13. Control—User control 
7. Flexibility—Flexible and customizable system 14. Documentation—Help and documentation 

Adapted from Zhang and Walji.54

Attributes of usability may be assessed with a wide array of qualitative and quantitative methods. Usability assessment methodologies have a rich scientific literature and these techniques are commonly used in software intensive industries55 including healthcare.56–63 These methods may be applied in simulation experiments, in laboratory-based evaluations, and in real-world use settings. They may combine subjective assessments made by an end-user (or group),64 as well as quantitative assessments, such as time to task completion.59 Often a ‘use case’ is defined to assess the ability of a user to use the software application to execute a particular task in a timely manner without error.65 No one method has been seen as dominant over others, and typically a combination of methods may be used throughout the software development lifecycle.9,14–16,64 One area that has received less attention, however, is monitoring of use in practice or real-world settings to continually assess the user's usage patterns and monitor for unintended consequences, adverse effects, or medical error.66

Use cases are particularly important as standardized instruments that may be used in usability evaluation and certification. The Office of the National Coordinator for Health Information Technology (ONC) published the Test Procedure for §170.314(g)(3) Safety-enhanced design, which was added to the 2014 edition of Health Information Technology: Standards, Implementation Specifications, and Certification Criteria for Electronic Health Record Technology (Meaningful Use 2014). Test Procedure §170.314(g)(3) was developed to emphasize the importance of UCD, and as a first step to improving EHR usability. Test Procedure §170.314(g)(3) includes eight use cases wherein an EHR vendor would need to document their UCD approach (and if a UCD approach did not previously exist one must be adopted) using the National Institute of Standards and Technology (NIST) Customized Common Industry Format Template for EHR Usability Testing (NISTIR7742).67 The eight use cases are CPOE, drug and allergy interaction checks, medication list, medication allergy list, clinical decision support, electronic medication administration record (inpatient setting only), electronic prescribing (medication order entry), and clinical information reconciliation (patient problems, medications, and allergies). The ONC selects these use cases because of the associated risk for patient harm with poor usability, and the greatest immediate opportunity for error prevention and user experience improvement.

Finally, another important component for EHR design and effective use is the application of standard user interface guidelines (‘style guides’) that provide guidance on color, controls, screen layout, and application flow to developers and users who are customizing an application.52 Little research to date has focused on the similarities and differences in the application style guides for various EHR products. While standardized use cases define functional characteristics and prototypical workflow, they do not define more broadly the ‘look and feel’ of an application. For example, screen layout, controls such as buttons, dialog boxes, entry modules, and other interface artifacts should be designed to have consistent visual and functional attributes across all component modules of a complex system. A notable initiative involving the UK National Health Service (NHS) and Microsoft Corporation has focused on a ‘Common User Interface’ for health applications used in the NHS.68 Since 1994, this effort has examined and developed recommendations for clinical documentation and standard user interface terminologies, consistent navigation, user interaction for medication ordering and management, patient identification, and other miscellaneous controls.

EHR industry perspectives

While some EHR vendors have adopted user-centered design when developing health information technologies, the practice is not universal69 and may be difficult to apply to legacy systems. Current practice suggests that EHR vendors use multiple techniques with variable efficacy. In an Agency for Healthcare Research and Quality (AHRQ) workshop on usability in health IT in July 2010, all eight participating vendors agreed that usability was important and many suggested it was a competitive differentiator, although some considered that usability was in the eye of the beholder and that the discipline of usability evaluation was an imperfect science, with results that were not useful.69 Nevertheless, the workshop participants agreed upon recommendations in three key areas: standards in design and development, usability testing and evaluation, and post-deployment monitoring and patient safety. They suggested that usability as a component of certification had to be viewed carefully and only when truly valid measures were available.69 Clearly, the vendors surveyed by the AHRQ workshop said they were motivated to build and implement high quality software systems that would lead to improved patient safety and quality of care, yet current software practices appear to be highly variable. Some believe it is difficult or impossible to reliably compare one product with another on the basis of usability given the challenges in assessment of products as implemented.

Challenges with usability assessment

Challenges to assessing EHR usability include the complexity of the EHR interaction with the full socio-technical context in which it is used,24,26,57 the professional roles of the intended users,11,70 the peculiarities of the clinical collaboration patterns,25,71 and the difficulty in measuring the influence that systems have on downstream processes.27,72 Usability in complex socio-technical systems such as healthcare are particularly challenging since a software product is designed to meet the needs of multiple different user types who have varying requirements, work across geographic, temporal, organizational, and cultural boundaries, and who may in fact be participating in the product design.28 The ability to perform meaningful, reproducible, objective usability metrics for EHR systems is limited by the socio-technical nature of the systems. They are typically highly configurable and different providers will implement them in different ways, with markedly different processes. Many effects of health IT can be considered to be ‘emergent’ or only discovered after monitoring a system in use.25,31,73,74 Typical metrics identified for usability are completion rate (or error rate), time on task, and subjective user satisfaction.33,44 These metrics require multiple measures or a composite score75,76 to be useful to developers and users.

Lessons learned from aviation

The field of human factors engineering and usability assessment has a rich set of scientific methods, a strong evidence base, and is widely applied effectively in other industries.77–79 There are many process standards (HCI/GUI design, evaluation, and verification), software user interface standards and guidelines (human computer interface/graphical user interface appearance and behavior),52 standards for quality management in software design, and a standard EHR usability evaluation protocol which has been proposed by NIST.80 The potential value of employing these methods in the design and effective use of EHR technologies is well described in the literature.9,10,17,20,60,61,74,77,78,81–87

The experience in aviation—with similarities in the expertise required for decision making, significant customer safety risks, high utilization of technology, strong professional culture issues, and significant regulatory oversight—is particularly relevant.88,89 The aircraft industry developed industry groups where topics are discussed and conventions sanctioned by a large number of airframe manufacturers and aviation software and hardware vendors. These cooperative efforts have occurred despite the intensely competitive nature of the aviation sector as competitors see value in working together in this arena. These changes have occurred over the century of human powered flight, facing the same resistance change has evoked in other areas of activity. Nevertheless, each new generation of aircraft has resulted in safer and more efficient flight. No single measure alone accounts for this success, but the mixture of measurement, reporting, and regulatory incentives has resulted in an admirable level of safety and continuous improvement from which other industries—including healthcare IT—can learn. In the USA, the Federal Aviation Administration regulates aircraft airworthiness, issues aircraft safety alerts, operates the Aviation Safety Reporting System, and in many other ways regulates aviation. Some have suggested that a structure analogous to the Aviation Safety Reporting System should exist in healthcare.90–92

AMIA recommendations

Given the potential impact of EHR technology to improve healthcare delivery and increase inadvertent patient harm, AMIA believes it is now critical to coordinate and accelerate the numerous efforts underway focusing on the issue of EHR usability. Vendors and users of health IT both seek to improve the quality of care delivered with EHR, but current evidence suggests that some health IT may facilitate certain types of adverse events and medical errors, and that these problems may be related to usability issues. The recommendations below do not address all aspects of the safe and effective use of EHR, but they help focus attention on critical usability issues that adversely affect patient safety and the quality of care.

1. Usability and human factors research agenda in health IT

Despite the wide variety of methods used in usability assessment and human factors engineering, additional experience is required in assessing their use in EHR applications. AMIA proposes accelerating the research agenda in three critical areas to support broad adoption of improved usability practices among EHR developers and users.

a. Prioritize standardized use cases

Standardized use cases should be established and maintained for selected EHR functionalities. Patient-safety sensitive EHR functions include patient selection (correct patient), clinical documentation (correct record, appropriate documentation quality), medication ordering and management (right drug, dose, route, etc), allergy documentation (correct allergen and reaction), results review (correct data and appropriate view), and advance directive documentation (correct status), among others suggested by the ONC. Standardizing and prioritizing a set of use cases for patient-safety sensitive EHR functions would provide the foundation for a process by which different EHR may be measured and certified. This work should be sponsored by the ONC.

b. Develop a core set of measures for adverse events related to health IT use

Use cases will facilitate the development and validation of standardized performance measures for assessing the incidence of adverse events and medical errors. These measures should be developed with the participation of experts and representatives drawn from the measure development community, clinical informatics, end-users, and the vendor community.

Such a collaborative effort would facilitate the development of a practical approach to the measurement of EHR use related to adverse events or medical error. This effort could also define methods to prioritize the importance of IT-related adverse events or medical errors with the goal of defining and distinguishing ‘EHR never events’ from events of lesser importance. This will be critical for establishing guidelines for IT-related adverse event reporting (see recommendation 4b) and developing a representative experience base across diverse products and various care settings. Ultimately, this work may inform development of a certification criterion or set of criteria and potential rule making regarding the usability of EHR. This work should be sponsored by the AHRQ.

c. Research and promote best practices for safe implementation of EHR

It is also critical that best practices are assessed and defined for the safe implementation and ongoing effective use of EHR. This includes identifying implementation best practices that address the broader socio-technical context of safe and effective use of EHR beyond just the usability component. They must address effective initial and ongoing training requirements, assessment of application configuration(s), technology (hardware/software) infrastructure and support, systems integration, workflow process(es), organizational culture and policies, and externalities which may confound the safe and effective use of EHR. This work should be sponsored by the AHRQ.

2. Policy recommendations

AMIA offers several recommendations for Federal policy initiatives that could help provide a sound evidence base to enhance the quality of care and patient safety using EHR.

a. Standardization and interoperability across EHR systems should take account of usability concerns

Because many patient-safety sensitive EHR functionalities may be related to competing or non-standard use of interface terminologies, it is critical that the ONC's Standards and Interoperability Framework includes usability considerations for patient-sensitive functions related to controlled medical terminologies, and application functions, expressed ‘at the glass’ for the end-user. This work should be sponsored by the ONC.

b. Establish an adverse event reporting system for health IT and voluntary health IT event reporting

A voluntary reporting process could leverage the AHRQ patient safety organizations (PSO), and would investigate and report on adverse events and medical errors related to usability. PSO could assume responsibility and accountability for establishing an IT-related voluntary error measurement and public reporting system. They should follow the NIST Common Industry Format.67 Reports should be captured locally and reviewed by end-users (facilitated by application functionalities designed for this purpose), and summary reports should be sent to the application vendor and to PSO. PSO governance bodies can convene relevant stakeholders to determine best practices for end-user and vendor product anonymity, appropriate levels of data aggregation, report details and frequency, and what summary data are made public. Use of the AHRQ Health IT Hazard Manager is a potential application for this purpose.66 This work should be sponsored by the AHRQ.

c. Develop and disseminate an educational campaign on the safe and effective use of EHR

An educational campaign is needed, with the goal of bringing increased attention to issues of patient-safety sensitive functions of EHR or EHR modules, and to usability in general. This industry is still relatively early in its evolution, with far less than 50% of users using EHR technologies, and many new companies entering the field, with different types of novel technologies. Thus, there is still an opportunity to provide meaningful guidance to the industry in this formative stage to facilitate achievement of the core policy goals of EHR adoption, interoperability, and meaningful use, and improved design of EHR for the benefit of vendors, end-users, and patients. This work should be sponsored by the ONC, utilizing the infrastructure of the Health IT Regional Extension Centers.

3. Industry recommendations

Vendors play a critical role in addressing challenges with EHR usability. AMIA makes two recommendations for the EHR industry.

a. Develop a common user interface style guide for select EHR functionalities

AMIA recommends that an industry coalition address the issue of developing a common style guide for patient-safety sensitive functions of EHR. This does not imply a common style guide for all aspects of the human–computer interaction in EHR, nor does it suggest there are not still ways in which vendors may innovate and compete with their product design and implementations. It does imply, however, that there should be a minimum set of design patterns shared among vendors that improve the usability for patient-safety sensitive functions within and across EHR, for the benefit of the patient and end-users. Much like the common features of information sources within automobiles (speedometer, odometer, fuel reserve, drive gear, etc), this set of common elements will lessen the training burden on end-users, and lead toward standardization of user interaction with the system in critical, patient-safety sensitive functions. Industry bodies such as the HIMSS EHR Association could spearhead this work, in collaboration with professional societies such as AMIA, and leading investigators in the field of human factors in the use of technology.

b. Perform formal usability assessments on patient-safety sensitive EHR functionalities

AMIA recommends that the EHR industry utilize usability assessments of a select set of patient-safety sensitive EHR functionalities that assess agreed upon attributes as defined in recommendations 1.a and 3.a above—for example the eight use cases from the ONC Meaningful Use Final Rule (2014 Edition), with more to come as patient-safety sensitive EHR functionalities are better understood through measurement and voluntary reporting (recommendations 1b and 2b above). If successful, the industry will improve usability of these patient-safety sensitive EHR functionalities over time. This work should be sponsored by the ONC and conducted in collaboration with NIST.

4. Clinical end-user recommendations

While the academic and research informatics community and the vendor community have critical roles to pursue as outlined in the recommendations above, AMIA believes that the end-user of EHR technologies also has a critically important role to play in enhancing the usability of EHR applications.

a. Adopt best practices for EHR system implementation and ongoing management

End-users of EHR systems are ultimately accountable for their safe and effective use, like any tool in clinical care. With the growing installed base of EHR, experience is being gathered on best practices for EHR implementation. Clinicians using EHR should exploit the vendor's experience base with implementations in similar settings, of course, but should also ‘take ownership’ for leading the configuration of the system for their environment to meet their clinical and business goals. With increasing experience, end-users can adopt best practices based upon the evidence to guide implementation of the EHR system and ensure safe and effective use.

b. Monitor how IT systems are used and report IT-related adverse events

The end-user should monitor how health IT systems are used. Measurement of patient-safety sensitive adverse events, and medical errors, related to IT use should occur regularly and be reviewed internally much like a clinicopathologic case review. Such reviews could be shared like the AHRQ M&M (morbidity and mortality) conferences on the web.93 This monitoring can also be reported in a standard fashion to the vendor to seek the vendor's guidance on remediation, and to a PSO to help gather the evidence base on IT-related adverse events or medical error.

Next steps

These AMIA recommendations on patient safety and the usability of EHR aim to stimulate ongoing and informed discussions and bring about increased understanding of the impact of usability on the safe and effective use of EHR systems. The adoption of useful and usable EHR will lead to safer and higher quality care, and a better return on investment for institutions that adopt them. The recommendations are necessarily wide ranging, and the Task Force on Usability recognizes and appreciates the attention that these issues are already receiving from end-users, vendors, and several federal agencies such as the ONC. Nevertheless, the essential recommendation and next step is that these diverse activities already underway become more coordinated to collectively address the AMIA recommendations described in this article.

Funding

None.

Competing interests

None.

Provenance and peer review

Not commissioned; peer reviewed by AMIA Board of Directors.

References

1
Blumenthal
D
.
Launching HITECH
.
N Engl J Med
 
2010
;
362
:
382
5
.
2
Steinbrook
R
.
Health care and the American Recovery and Reinvestment Act
.
N Engl J Med
 
2009
;
360
:
1057
60
.
3
Blumenthal
D
.
Stimulating the adoption of health information technology
.
N Engl J Med
 
2009
;
360
:
1477
9
.
4
Hersh
W
.
Health care information technology: progress and barriers
.
JAMA
 
2004
;
292
:
2273
4
.
5
Goldzweig
CL
Towfigh
A
Maglione
M
et al
.
Costs and benefits of health information technology: new trends from the literature
.
Health Aff
 
2009
:
28
:
w282
93
.
6
Buntin
MB
Burke
MF
Hoaglin
MC
et al
.
The benefits of health information technology: a review of the recent literature shows predominantly positive results
.
Health Affairs
 
2011
;
30
:
464
71
.
7
Metzger
J
Welebob
E
Bates
DW
et al
.
Mixed results in the safety performance of computerized physician order entry
.
Health Aff
 
2010
;
29
:
655
63
.
8
Stead
WW
Lin
HS
.
Computational technology for effective health care: immediate steps and strategic directions
 .
Washington, DC
:
The National Academies Press
,
2009
.
9
IOM (Institute of Medicine)
.
Health IT and patient safety: building safer systems for better care
 .
Washington DC
:
The National Academies Press
,
2012
.
10
Walker
JM
Carayon
P
Leveson
N
et al
.
EHR safety: the way forward to safe and effective systems
.
JAMIA
 
2008
;
15
:
272
7
.
11
Shneiderman
B
.
Tragic errors: usability and electronic health records
.
Interactions
 
2011
;
18
:
60
3
.
12
Bevan
N
.
International standards for HCI and usability
.
Int J Hum-Comput Stud
 
2001
;
55
:
533
52
.
13
Ancker
JS
Kern
LM
Abramson
E
et al
.
The Triangle Model for evaluating the effect of health information technology on healthcare quality and safety
.
JAMIA
 
2012
;
19
:
61
5
.
14
Kushniruk
A
Beuscart-Zéphir
M-C
Grzes
A
et al
.
Increasing the safety of healthcare information systems through improved procurement: toward a framework for selection of safe healthcare systems
.
Healthc Q
 
2010
;
13
Spec No
:
53
8
.
15
Karsh
BT
.
Beyond usability: designing effective technology implementation systems to promote patient safety
.
Qual Saf Health Care
 
2004
;
13
:
388
94
.
16
Lorenzi
NM
Kouroubali
A
Detmer
DE
et al
.
How to successfully select and implement electronic health records (EHR) in small ambulatory practice settings
.
BMC Med Inform Decis Mak
 
2009
;
9
:
15
.
17
Keshavjee
K
Bosomworth
J
Copen
J
et al
.
Best practices in EMR implementation: a systematic review
.
AMIA Ann Symp Proc
 
2006
;
982
.
18
Holden
RJ
.
What stands in the way of technology-mediated patient safety improvements?: a study of facilitators and barriers to physicians' use of electronic health records
.
J Patient Saf
 
2011
;
7
:
193
203
.
19
Holden
RJ
Karsh
B-T
.
A theoretical model of health information technology usage behaviour with implications for patient safety
.
Behav Inf Technol
 
2009
;
28
:
21
38
.
20
Carayon
P
.
Human factors in patient safety as an innovation
.
Appl Ergon
 
2010
;
41
:
657
65
.
21
Johnson
CW
.
Why did that happen? Exploring the proliferation of barely usable software in healthcare systems
.
Qual Saf Health Care
 
2006
;
15
(
Suppl 1
):
i76
81
.
22
Karsh
B-T
Weinger
MB
Abbott
PA
et al
.
Health information technology: fallacies and sober realities
.
JAMIA
 
2010
;
17
:
617
23
.
23
Sittig
DF
Singh
H
.
Eight rights of safe electronic health record use
.
JAMA
 
2009
;
303
:
1111
13
.
24
Koppel
R
Metlay
JP
Cohen
A
et al
.
Role of computerized physician order entry systems in facilitating medication errors
.
JAMA
 
2005
;
293
:
1197
203
.
25
Computational technology for effective health care: immediate steps and strategic directions. Washington, DC: The National Academies Press, 2009:1–120.
26
Ash
J
Berg
M
.
Some unintended consequences of information technology in health care: the nature of patient care information system-related errors
.
JAMIA
 
2004
;
11
:
104
12
.
27
Harrison
MI
Koppel
R
Bar-Lev
S
.
Unintended consequences of information technologies in health care an interactive sociotechnical analysis
.
JAMIA
 
2007
;
14
:
542
9
.
28
Ash
JS
Sittig
DF
Poon
EG
et al
.
The extent and importance of unintended consequences related to computerized provider order entry
.
JAMIA
 
2007
;
14
:
415
23
.
29
Smelcer
J
Miller-Jacobs
H
.
Usability of electronic medical records
.
J Usability Stud
 
2009
;
4
:
70
84
.
30
Rosenbloom
ST
Miller
RA
Johnson
KB
et al
.
Interface terminologies: facilitating direct entry of clinical data into electronic health record systems
.
JAMIA
 
2006
;
13
:
277
88
.
31
Strom
BL
Schinnar
R
Aberra
F
et al
.
Unintended effects of a computerized physician order entry nearly hard-stop alert to prevent a drug interaction: a randomized controlled trial
.
Arch Intern Med
 
2010
;
170
:
1578
83
.
32
Isaac
T
Weissman
J
Davis
R
et al
.
Overrides of medication alerts in ambulatory care
.
Arch Intern Med
 
2009
;
169
:
305
.
33
Chused
A
Kuperman
GJ
Stetson
PD
.
Alert override reasons: a failure to communicate
.
AMIA Annu Symp Proc
 
2008
:
111
15
.
34
van der Sijs
H
Aarts
J
Vulto
A
et al
.
Overriding of drug safety alerts in computerized physician order entry
.
JAMIA
 
2006
;
13
:
138
47
.
35
Weingart
SN
Toth
M
Sands
DZ
et al
.
Physicians' decisions to override computerized drug alerts in primary care
.
Arch Intern Med
 
2003
;
163
:
2625
31
.
36
Shah
NR
Seger
AC
Seger
DL
et al
.
Improving acceptance of computerized prescribing alerts in ambulatory care
.
JAMIA
 
2006
;
13
:
5
11
.
37
Leape
LL
Berwick
DM
.
Five years after To Err Is Human: what have we learned?
JAMA
 
2005
;
293
:
2384
90
.
38
Altman
DE
Clancy
C
Blendon
RJ
.
Improving patient safety—five years after the IOM report
.
N Engl J Med
 
2004
;
351
:
2041
3
.
39
Baron
RJ
Fabens
EL
Schiffman
M
et al
.
Electronic health records: just around the corner? Or over the cliff?
Ann Intern Med
 
2005
;
143
:
222
.
40
Sittig
DF
.
A socio-technical model of health information technology-related e-iatrogenesis
.
AMIA Annu Symp Proc
 
2008
:
1209
10
.
41
Quality AFHRA. Chapter 47. Patient Safety and Health Information Technology: Role of the Electronic Health Record. 2008:1–43
42
Weiner
JP
Kfuri
T
Chan
K
et al
. ‘
e-Iatrogenesis’: the most critical unintended consequence of CPOE and other HIT
.
JAMIA
 
2007
;
14
:
387
8
.
discussion 389
.
43
Sittig
DF
Singh
H
.
Defining health information technology-related errors: new developments since to err is human
.
Arch Intern Med
 
2011
;
171
:
1281
4
.
44
Redish
J
.
Usability in Health IT: Technical Strategy, Research, and Implementation Summary of Workshop
.
National Institute for Standards and Technology; NISTIR 7743
,
2010
:
1
93
.
45
Walker
JM
Carayon
P
.
From tasks to processes: the case for changing health information technology to improve health care
.
Health Aff
 
2009
;
28
:
467
77
.
46
Jamal
A
McKenzie
K
Clark
M
.
The impact of health information technology on the quality of medical and health care: a systematic review
.
HIM J
 
2009
;
38
:
26
37
.
47
Friedman
CP
Wong
AK
Blumenthal
D
.
Achieving a nationwide learning health system
.
Sci Transl Med
 
2010
;
2
:
57cm29
.
48
Chaudhry
B
Wang
J
Wu
S
et al
.
Systematic review: impact of health information technology on quality, efficiency, and costs of medical care
.
Ann Intern Med
 
2006
;
144
:
742
52
.
49
Etheredge
L
.
A rapid-learning health system
.
Health Aff
 
2007
;
26
:
w107
18
.
50
DesRoches
CM
Campbell
EG
Rao
SR
et al
.
Electronic health records in ambulatory care—a national survey of physicians
.
N Engl J Med
 
2008
;
359
:
50
60
.
51
Blumenthal
D
Tavenner
M
.
The “meaningful use” regulation for electronic health records
.
N Engl J Med
 
2010
;
363
:
501
4
.
52
Reed
P
Holdaway
K
Isensee
S
et al
.
User interface guidelines and standards: progress, issues, and prospects
.
Interacting Comput
 
1999
;
12
:
119
42
.
53
Defining and Testing EMR Usability. Chicago, Illinois: Healthcare Information Management and Systems Society, 2009. http://blackbookrankings.com/pdf/Defining-and-Testing-EMR-Usability.pdf
(accessed 11 Feb 2012).
54
Zhang
J
Walji
MF
.
TURF: toward a unified framework of EHR usability
.
J Biomed Inform
 
2011
;
44
:
1056
67
.
55
Helmreich
RL
.
On error management: lessons from aviation
.
BMJ
 
2000
;
320
:
781
5
.
56
Ammenwerth
E
de Keizer
N
.
An inventory of evaluation studies of information technology in health care trends in evaluation research 1982–2002
.
Methods Inf Med
 
2005
;
44
:
44
56
.
57
Ammenwerth
E
Gräber
S
Herrmann
G
.
Evaluation of health information systems—problems and challenges
.
Int J Med Inf
 
2003
;
74
:
125
35
.
58
Rahimi
B
Vimarlund
V
.
Methods to evaluate health information systems in healthcare settings: a literature review
.
J Med Syst
 
2007
;
31
:
397
432
.
59
Yen
P-Y
Bakken
S
.
Review of health information technology usability study methodologies
.
JAMIA
 
2012
;
19
:
413
22
.
60
Phansalkar
S
Edworthy
J
Hellier
E
et al
.
A review of human factors principles for the design and implementation of medication safety alerts in clinical information systems
.
JAMIA
 
2010
;
17
:
493
501
.
61
Staggers
N
.
Human factors: imperative concepts for information systems in critical care
.
AACN Clin Issues
 
2003
;
14
:
310
19
;
quiz 397–8
.
62
Karsh
BT
Holden
RJ
Alper
SJ
et al
.
A human factors engineering paradigm for patient safety: designing to support the performance of the healthcare professional
.
Qual Saf Health Care
 
2006
;
15
(
Suppl 1
):
i59
65
.
63
Gawron
VJ
Drury
CG
Fairbanks
RJ
et al
.
Medical error and human factors engineering: where are we now?
Am J Med Qual
 
2006
;
21
:
57
67
.
64
Rose
AF
Schnipper
JL
Park
ER
et al
.
Using qualitative studies to improve the usability of an EMR
.
J Biomed Inform
 
2005
;
38
:
51
60
.
65
van der Peijl
J
Klein
J
Grass
C
et al
.
Design for risk control: the role of usability engineering in the management of use-related risks
.
J Biomed Inform
 
2012
;
45
:
795
812
.
66
Walker
JM
Hassol
A
Bradshaw
B
et al
.
Health IT Hazard Manager Beta-Test: Final Report. AHRQ Publication No. 12-0058-EF. Rockville, MD:
Agency for Healthcare Research and Quality, May
2012
:
1
77
.
67
Schumacher
RM
Lowry
SZ
.
Customized common industry format template for electronic health record usability testing
. (accessed 12 Feb 2012).
68
Microsoft Corporation, UK National Health Service
.
Common User Interface—Guidance Overview. http://www.mscui.net/DesignGuide/DesignGuide.aspx
(accessed 3 Sept 2012).
69
McDonnell
C
Werner
K
,
Wendel L. Electronic Health Record Usability: Vendor Practices and Perspectives. AHRQ Publication No. 09(10)-0091-3-EF. Rockville, MD: Agency for Healthcare Research and Quality
,
May
2010
.
70
Succi
MJ
Walter
ZD
.
Theory of user acceptance of information technologies: an examination of health care professionals
.
System Sciences, 1999. HICSS-32. Proceedings of the 32nd Annual Hawaii International Conference on
;
7pp
.
1999
. doi:
71
Safran
C
Jones
PC
Rind
D
et al
.
Electronic communication and collaboration in a health care practice
.
Artif Intell Med
 
1998
;
12
:
137
51
.
72
Kaplan
B
Shaw
N
.
People, organizational, and social issues: evaluation as an exemplar
 .
Stuttgart
:
Shattauer Yearbook of Medical Informatics
,
2002
:71–8. .
73
Patel
VL
Kushniruk
AW
Yang
S
et al
.
Impact of a computer-based patient record system on data collection, knowledge organization, and reasoning
.
JAMIA
 
2000
;
7
:
569
85
.
74
Kushniruk
AW
Patel
VL
.
Cognitive and usability engineering methods for the evaluation of clinical information systems
.
J Biomed Inform
 
2004
;
37
:
56
76
.
75
Sauro
J
.
A method to standardize usability metrics into a single score
.
Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
;
2005
:
401
9
.
76
Sauro
J
Kindlund
E
.
Making sense of usability metrics: Usability and six sigma
.
Proceedings of UPA, 1–10
,
2005
.
77
Perry
SJ
.
An overlooked alliance: using human factors engineering to reduce patient harm
.
Jt Comm J Qual Saf
 
2004
;
30
:
455
9
.
78
Gosbee
JW
.
Conclusion: You need human factors engineering expertise to see design hazards that are hiding in “plain sight!”
.
Jt Comm J Qual Saf
 
2004
;
30
:
696
700
.
79
Despont-Gros
C
Mueller
H
Lovis
C
.
Evaluating user interactions with clinical information systems: a model based on human-computer interaction models
.
J Biomed Inform
 
2005
;
38
:
244
55
.
80
Schumacher
R
Patterson
ES
North
R
et al
.
Technical evaluation, testing and validation of the usability of electronic health records
.
NISTIR 7804
;
2011
:
1
108
.
81
Beuscart-Zéphir
M-C
Pelayo
S
Bernonville
S
.
Example of a human factors engineering approach to a medication administration work system: potential impact on patient safety
.
Int J Med Inform
 
2010
;
79
:
e43
57
.
82
Weinger
MB
.
Human factors research in anesthesia patient safety: techniques to elucidate factors affecting clinical task performance and decision making
.
JAMIA
 
2002
;
9
:
58S
63
.
83
Samaras
GM
Horst
RL
.
A systems engineering perspective on the human-centered design of health information systems
.
J Biomed Inform
 
2005
;
38
:
61
74
.
84
Magrabi
F
Ong
M-S
Runciman
W
et al
.
An analysis of computer-related patient safety incidents to inform the development of a classification
.
JAMIA
 
2010
;
17
:
663
70
.
85
Fairbanks
RJ
Caplan
S
.
Poor interface design and lack of usability testing facilitate medical error
.
Jt Comm J Qual Saf
 
2004
;
30
:
579
84
.
86
Horsky
J
McColgan
K
Pang
JE
et al
.
Complementary methods of system usability evaluation: surveys and observations during software design and development cycles
.
J Biomed Inform
 
2010
;
43
:
782
90
.
87
Anderson
J
Fleak
F
Garrity
K
et al
.
Integrating usability techniques into software development
.
IEEE Softw
 
2001
;
18
:
46
53
.
88
Golemboski
K
.
Improving patient safety: lessons from other disciplines
.
Clin Lab Sci
 
2011
;
24
:
114
19
.
89
Kilbridge
P
.
The informatics opportunities at the intersection of patient safety and clinical informatics
.
JAMIA
 
2008
;
15
:
397
407
.
90
Sittig
D
Classen
DD
.
Safe electronic health record use requires a comprehensive monitoring and evaluation framework
.
JAMA
 
2010
;
303
:
450
1
.
91
Singh
H
Classen
DC
Sittig
DF
.
Creating an oversight infrastructure for electronic health record-related patient safety hazards
.
J Patient Saf
 
2011
;
7
:
169
74
.
92
Goodman
KW
Berner
ES
Dente
MA
et al
.
Challenges in ethics, safety, best practices, and oversight regarding HIT vendors, their customers, and patients: a report of an AMIA special task force
.
JAMIA
 
2011
;
18
:
77
81
.
93
Wachter
RM
Shojania
KG
Minichiello
T
et al
.
AHRQ WebM&M—online medical error reporting and analysis
.
Adv Patient Saf
 
2005
;
4
:
211
21
.

Comments

0 Comments