Abstract

Background

Hospital incident reporting systems are usually evaluated on their theoretical benefit to the hospital or increase in reporting rates alone.

Objective

To evaluate a workflow-based response system on staff incident reporting rates.

Design, Setting and Participants

A prospective cohort study of incident reports made by staff members before (2006–2007) and after (2008–2009) the system was implemented on 1 January 2008 at a medical center in southern Taiwan. Pre-system and post-system data were based on 713 129 and 730 176 inpatient days and 160 692 and 168 850 emergency department visits.

Intervention

The addition of a workflow-based response system to a reporting system processing incident reports and intra-hospital responses.

Main Outcome Measures

Voluntary incident reporting rates and distribution of incident severities.

Results

Inpatient reports [9.9 vs. 28.8 per 10 000 patient days; rate ratio (RR): 2.9, 95% confidence interval (CI): 2.7–3.2, P < 0.001] and emergency department reports (5.9 vs. 19.2 per 10 000 visits, RR: 3.3, 95% CI: 2.6–4.1, P < 0.001) increased significantly, particularly in doctors in inpatient areas (RR: 2.7, 95% CI: 1.8–4.1, P < 0.001), emergency department nurses (RR: 9.4, 95% CI: 6.1–14.4, P < 0.001) and allied health professionals in inpatient areas (RR: 2.2, 95% CI:1.8–2.6, P < 0.001). Post-system reported incidents were more evenly distributed over five severity levels than pre-sytem incidents, moving more toward the very severe level (RR: 17.6, 95% CI: 8.4–37.0, P < 0.001) and no harm level (RR: 6.2, 95% CI: 4.5–8.7, P < 0.001).

Conclusion

The addition of the workflow-based response system to the hospital incident reporting system significantly increased hospital-wide voluntary incident report rates at all incident injury levels.

Introduction

One of the most fundamental ways for a hospital to improve patient safety is to have a well-designed internal voluntary incident reporting system, one that helps identify causes and prevents future errors [1, 2]. The underreporting of incidents, however, is a common problem [3–5]. Hoping to facilitate the process, some hospitals have replaced or supplemented traditional paper-based reporting systems using new forms of information technology including Web, PDA and call centers [6–10], while others have abandoned the use of information technology and returned to paper [11–13].

A hospital's response system may be just as important as its reporting system, as incident reports normally require some actions [2, 14]. Therefore, in addition to incident reporting, there is a need for timely responses and effective feedback, which would theoretically increase vigilance, speed-up response and better ensure patient safety. One qualitative study based on case studies and semi-structured interviews has suggested that effective feedback increases incident reporting [15], but, to our best knowledge, no empirical study has explored the effect of a response system on incident reporting rates.

A workflow-based response system, achieved through automatic digital messaging within an intra-hospital communication system, should improve coordination among people and workflow efficiency. Ideally, it would ensure that the right person gets the right information about what needs to be done at what time in what order [16]. The ideal system would reduce the time spent in processing and delivering, improve workflow control and keep track of progress. This improved response should theoretically increase incident report rates. To find out, this study investigated the effect of adding a hospital workflow-based response system to a web-based intra-hospital voluntary incident reporting system on incident reporting rates and severities of incidents reported.

Methods

In this prospective cohort study, we collected patient safety incident report information submitted by front-line staff members during the 2-year period (1 January 2006–31 December 2007) before and 2-year period (1 January 2008–31 December 2009) after a workflow-based response system was implemented on 1 January 2008 at a 1330-bed government affiliated medical center located in southern Taiwan.

Before the new system was implemented, an investigating officer of the patient safety committee would investigate an incident after it was reported by the web-based reporting system. If they needed further help, they would use a paper-based form to ask the unit chiefs to provide more information or recommendations. After the investigation, the report along with the suggestions was then sent to a decision-maker (the vice-chair or chair of the patient safety committee). Once the report was returned, the investigating officer logged on to Taiwan's Patient Safety Reporting System to register the incident, and filed the report in the paper form at the hospital.

Workflow-based response system intervention

To implement the workflow-based response system, we established a dedicated team, including a director and two officers from the medical quality department and an engineer from the information center to redesign the response process, communicate the benefits of the program to all departments and develop the workflow-based information system. Once developed, educational sessions were held for selected department members to introduce the new system, and workshops were held for all response team members to teach them how to use it. A variety of statistical charts and diagrams for patient safety incident reports were posted monthly on the web-based billboard.

There were no other significant hospital policy changes after the system was implemented, though it was decided that, for every incident reported, the reporting staff member would receive an additional NT$50 (US$1.67) to his or her monthly salary. This insignificant amount would do little to encourage the staff report incidents, but it could draw some attentions about the hospital's increased interest in incident reporting.

New response system

The response system was re-engineered to confirm the facts related to the incident, ensure the completeness of the report, uncover the underlying systems failures and focus more on organizational factors and propose ameliorating actions. To do this, a multidisciplinary report response team was created using previous employed staff members to respond to all incident reports. The team consisted of three sub-teams: patient safety team, investigation team and support team. The patient safety team had risk assessor, decision-maker and dispatcher to review and handle the initial incident reports. The investigation team consisted of experienced medical practitioners, including head nurses, medical unit chiefs, administrative unit chiefs and officers of other committees. The members of this sub-team were further divided into investigators and verifiers, both carefully reviewing and confirming the incidents. The support team consisted of social workers who coordinated communications among patients, their family members and healthcare pratitioners, and crisis management officers who kept track of how crisis situations were managed.

Workflow automation

As is depicted in the activity diagram in Fig. 1, the automatic response workflow process was triggered by front-line staff members reporting a patient safety incident through the web-based reporting system. The dispatcher in the patient safety team confirmed the accuracy and completeness of the report and delivered the initial incident report to at least one pairs of investigators and verifiers based on the incident type, incident characteristics, patient outcomes and organizational outcomes. The paired investigation team members first collected the details of the incident and made recommendations on how to handle it. That report was then referred to other members of the investigation team to verify the facts of the case and suggest actions. If further support was needed, the incident was reported to social workers. Each step of this process was kept transparent, documented and easily observed by the staff member making the initial report.

Figure 1

Adding a workflow-baed response systems to a web-based intra-hospital voluntary incident reporting system.

Figure 1

Adding a workflow-baed response systems to a web-based intra-hospital voluntary incident reporting system.

Once verified, the suggested actions were sent by the investigation team to the patient safety team's risk assessor who assessed the risk of possible solutions and assigned a safety assessment code [17]. The safety assessment code was based on two factors: actual or potential severity and likelihood of recurrence. For example, if an incident was very severe and occurred most weeks, it was assigned an safety assessment code (SAC) value of 1. If an incident was defined as moderate and occurred most weeks, it was assigned an SAC value of 2. If an incident was defined as minor and occurred every one or two years, it was assigned an SAC value of 3. The lowest SAC value was 4. SACs were generally handled differently. SAC 1 incident reports were forwarded to chairs to determine whether there was a need for root cause analysis. SAC 2 reports were forwarded to vice-chairs and the department head, who would start a practice improvement program and monitor these incidents continuously. SAC 3 reports were sent to the department head alone who would initiate a practice improvement activity and monitor such incidents continuously. SAC 4 reports were managed routinely. If the incident was deemed a crisis, a crisis management officer in the support team was automatically notified. The crisis monitor would then oversee the changes closely and keep a log of all responses until the crisis was resolved. At the end, reports were automatically generated and forwarded in a XML format to the Taiwan's Patient Safety Reporting System. Meanwhile, all stakeholders were automatically notified of these steps by web-based messaging.

The transparency of this system is unique. The initial front-line reporters were notified upon the completion of each step, and were authorized to follow the progress of the case.

Study outcomes and variables

The primary study outcomes were incident reporting rates per 10 000 patient days (PDs) or per 10 000 ED visits. A report was defined as an incident submitted by a front-line staff member through our intra-hospital web-based reporting system. The rates were further classified by department (general inpatient ward, ICU or ED), reporter level (physican, nurse or allied health professional), anonymous status and level of severity. The severity of every incident was assessed using actual or potential outcome. ‘Very severe’ was defined as death or permanent loss of function, as was seen, for example, when a terminally ill patient was not responding to chemotherapy and his family members were ignoring him. This patient became emotionally unstable and committed suicide in the hospital. ‘Severe’ and ‘moderate’ were defined as increases in length of stay or level of care, respectively. These were seen, for example, in a dermatology outpatient needing in-hospital debridement and antibiotic therapy after bacterial infection caused by an injection and in a mentally ill patient needing wound care and constraints after he jabbed himself with a pencil, respectively. ‘Minor’ was defined as no need for increased stay or increased care. This was seen, for instance, in a patient who fell and bruised his arm. ‘No harm’ was defined in incidents that caused no injury. For example, a patient might have been given a wrong medication, but the problem was resolved before any injury was incurred.

Statistical analysis

Reporting rates were calculated based on monthly admission in PDs and ED in visits. Rate ratio (RR) and 95% CIs were used to present the changes in the reporting rate after intervention. All statistical operations were performed using SPSS version 17.

Results

As can be seen in Table 1, a summary of the hospitalized patients treated before and after the implementation of the systems, there was almost no significant change in the number of inpatient encounters, number of medicine encounters, number of surgical encounters, number of other encounters, number of patient days, number of ICU patients admitted, number of ICU days, number of ED encounters, median age of all inpatient encounters and number of deaths.

Table 1

Summary of the patients treated before and after the workflow-based response system implementation

 Pre-intervention Post-intervention 
Number of inpatient encounters 82 801 81 218 
Number of medicine encounters 29 429 29 761 
Number of surgical encounters 21 979 21 559 
Number of other encounters 31 393 29 898 
Number of patient days 713 129 730 176 
Number of ICU admits 9241 8650 
Number of ICU days 62 556 61 819 
Number of ED encounters 160 692 168 850 
Median age of all inpatient encounters 56 56 
Number of deaths 1954 2122 
 Pre-intervention Post-intervention 
Number of inpatient encounters 82 801 81 218 
Number of medicine encounters 29 429 29 761 
Number of surgical encounters 21 979 21 559 
Number of other encounters 31 393 29 898 
Number of patient days 713 129 730 176 
Number of ICU admits 9241 8650 
Number of ICU days 62 556 61 819 
Number of ED encounters 160 692 168 850 
Median age of all inpatient encounters 56 56 
Number of deaths 1954 2122 

From 2006 to 2009, covering both periods of study, inpatient PDs increased 2% and ED visits 5%. Comparing pre- and post-system incident reports, we found the total number of inpatient and ED incident reports to increase by 208 and 173% (704 vs. 2106 and 94 vs. 324) (Table 2). Inpatient incident rates increased from 9.9 to 28.8 per 10 000 PDs (RR: 2.9, 95% CI: 2.7–3.2, P < 0.001). The inpatient ward and ICU accounted for most of the increase (Table 2), especially the ICUs, whose reporting rates increased 10 times (4.3 vs. 46.3 per 10 000 PDs; RR: 10.8, 95% CI: 7.25–15.97, P < 0.001). ED reporting rates increased from 5.9 to 19.2 per 10 000 visits (RR: 3.29, 95% CI: 2.61–4.13, P < 0.001). Anonymous reporting rates increased significantly (1.8 vs. 2.3 inpatient PDs; ED visits per 10 000; RR: 1.3, 95% CI: 1.0–1.6, P = 0.02). Psysician and nurse reporting rates increased in both inpatient wards and EDs, particularly for nurses in ED (RR: 9.4, 95% CI: 6.1–14.4, P < 0.001), as it did for allied health professionals in inpatient wards (RR: 2.2, 95% CI: 1.8–2.6), P < 0.001).

Table 2

Comparisons of reporting rates before (2006–2007) and after (2008–2009) the workflow-based response system implementation by incident location, anonymous/confidential and profession designation

 Pre-intervention % (n1/n2) Post-intervention % (n1/n2) Rate ratio (95% CI) P-value 
Hospital wide 9.1(798/873 821) 27.0 (2430/899 026) 3.0 (2.7–3.2) <0.001 
Incident locationa 
 Inpatient area 9.9 (704/713 129) 28.8 (2106/730 176) 2.9 (2.7–3.2) <0.001 
 Non-ICU 10.4 (677/650 573) 27.2 (1820/668 357) 2.6 (2.4–2.9) <0.001 
 ICU 4.3 (27/62 556) 46.3 (286/61 819) 10.8 (7.3–16.0) <0.001 
 ED 5.9 (94/160 692) 19.2 (324/168 850) 3.3 (2.6–4.1) <0.001 
Anonymous or confidential reportb 
 Anonymous 1.8 (156/873 831) 2.3 (205/898 821) 1.3 (1.0–1.6) 0.02 
 Confidential 7.4 (642/873 821) 24.8 (2225/899 026) 3.4 (3.1–3.7) <0.001 
Profession designation 
Inpatient areac 
 Psysicians 0.4 (29/713 129) 1.1 (80/730 176) 2.7 (1.8–4.1) <0.001 
 Nurses 6.9 (493/713 129) 22.3 (1625/730 176) 3.2 (2.9–3.6) <0.001 
 Allied health 2.6 (182/713 129) 5.5 (324/730 176) 2.2 (1.8–2.6) <0.001 
EDd 
 Doctors 0.8 (13/160 692) 2.0 (34/168 850) 2.5 (1.3–4.7) 0.004 
 Nurses 1.4 (23/160 692) 13.4 (227/168 850) 9.4 (6.1–14.4) <0.001 
 Allied health 3.6 (58/160 692) 3.7 (63/168 850) 1.0 (0.7–1.5) 0.86 
 Pre-intervention % (n1/n2) Post-intervention % (n1/n2) Rate ratio (95% CI) P-value 
Hospital wide 9.1(798/873 821) 27.0 (2430/899 026) 3.0 (2.7–3.2) <0.001 
Incident locationa 
 Inpatient area 9.9 (704/713 129) 28.8 (2106/730 176) 2.9 (2.7–3.2) <0.001 
 Non-ICU 10.4 (677/650 573) 27.2 (1820/668 357) 2.6 (2.4–2.9) <0.001 
 ICU 4.3 (27/62 556) 46.3 (286/61 819) 10.8 (7.3–16.0) <0.001 
 ED 5.9 (94/160 692) 19.2 (324/168 850) 3.3 (2.6–4.1) <0.001 
Anonymous or confidential reportb 
 Anonymous 1.8 (156/873 831) 2.3 (205/898 821) 1.3 (1.0–1.6) 0.02 
 Confidential 7.4 (642/873 821) 24.8 (2225/899 026) 3.4 (3.1–3.7) <0.001 
Profession designation 
Inpatient areac 
 Psysicians 0.4 (29/713 129) 1.1 (80/730 176) 2.7 (1.8–4.1) <0.001 
 Nurses 6.9 (493/713 129) 22.3 (1625/730 176) 3.2 (2.9–3.6) <0.001 
 Allied health 2.6 (182/713 129) 5.5 (324/730 176) 2.2 (1.8–2.6) <0.001 
EDd 
 Doctors 0.8 (13/160 692) 2.0 (34/168 850) 2.5 (1.3–4.7) 0.004 
 Nurses 1.4 (23/160 692) 13.4 (227/168 850) 9.4 (6.1–14.4) <0.001 
 Allied health 3.6 (58/160 692) 3.7 (63/168 850) 1.0 (0.7–1.5) 0.86 

aIncident location (reporting rates per 10 000 PDs in inpatient area or 10 000 attendences in ED).

bAnonymous or confidential report [reporting rates per 10 000 PDs and 10 000 ED attendances (combined)].

cInpatient area (reporting rates per 10 000 PDs).

dED (reporting rates per 10 000 ED attendances).

As can be seen in Fig. 2, a summary of monthly incident reports per 10 000 PDs and 10 000 ED visits combined. Although monthly rates varied without specific trend in both periods, there was a dramatic increase in the number of incident reports right after the system was implemented in January 2008, with the mean reporting rates increasing from 9.1 per 10 000 inpatient PDs and ED visits combined before the system was implemented to 27 per 10 000 inpatient PDs and ED visits combined afterwards.

Figure 2

Reporting rates per 10 000 patient days and 10 000 emergency department visits by month. The pre-intervention period was from 1 January 2006 to 31 December 2007 and the post-intervention period from 1 January 2008 to 31 December 2009. UCL indicates upper control limit; LCL indicates lower control limit. The control limits are set at mean ± 3SD (standard deviation).

Figure 2

Reporting rates per 10 000 patient days and 10 000 emergency department visits by month. The pre-intervention period was from 1 January 2006 to 31 December 2007 and the post-intervention period from 1 January 2008 to 31 December 2009. UCL indicates upper control limit; LCL indicates lower control limit. The control limits are set at mean ± 3SD (standard deviation).

Reported incidents were categorized as very severe, severe, moderate, minor and no harm (Table 3). Before the workflow-based response system was implemented, 72.3% of the reported incidents were minor and 0.9% very severe. Afterwards, they were more evenly spread over very severe, severe, moderate, minor and no harm (15.4, 12.4, 16.7, 29.0 and 26.5%, respectively), with most significant increases found in very severe, severe and no harm categories and most significant decreases in reports of minor incidents.

Table 3

Comparison of reporting rates before (2006–2007) and after (2008–2009) the workflow-based response system implementation by actual or potential severity grade

 Pre-intervention % of reports (nPost-intervention % of reports (nRelative risk (95% CI) P-value 
Very severe 0.9 (7) 15.4 (375) 17.6 (8.4–37.0) <0.001 
Severe 2.9 (23) 12.4 (301) 4.3 (2.8–6.5) <0.001 
Moderate 19.7 (157) 16.7 (406) 0.8 (0.7–1.0) 0.06 
Minor 72.3 (577) 29.0 (704) 0.4 (0.4–0.4) <0.001 
No harm 4.3 (34) 26.5 (644) 6.2 (4.5–8.7) <0.001 
 Pre-intervention % of reports (nPost-intervention % of reports (nRelative risk (95% CI) P-value 
Very severe 0.9 (7) 15.4 (375) 17.6 (8.4–37.0) <0.001 
Severe 2.9 (23) 12.4 (301) 4.3 (2.8–6.5) <0.001 
Moderate 19.7 (157) 16.7 (406) 0.8 (0.7–1.0) 0.06 
Minor 72.3 (577) 29.0 (704) 0.4 (0.4–0.4) <0.001 
No harm 4.3 (34) 26.5 (644) 6.2 (4.5–8.7) <0.001 

n is the number of reports.

Discussion

This study, to our best knowledge, the first empirical study to investigate the effect of a workflow-based response system on a incident reporting system, found that the newly implemented system significantly increased the hospital-wide reporting rates (Table 2 and Fig. 2) in both inpatient ward and EDs (Table 2) by both physicians and nurses.

Although the reporting system inplemented by our hospital was unique compared with other previously evaluated systems with its focus on the response system, our findings are consistant with other studies, which have reported increased incident reporting rates following changes in the implementation systems [7, 8, 11–13]. One previous study investigating the effect of an electronic reporting system that had replaced a paper-based reporting system at a 750-bed teaching hospital in the USA found that it had efficiently improved reporting rates [18]. They found a steady linear rise following its implementation, suggesting their study results may be confounded by other factors such as history or learning curve effects, or both. Our study observed a sharp increase in monthly reporting rates immediately after the system implementation period (Fig. 2), indicating that the changes we measured were directly related to the changes in the system. Another evaluative study conducted in four major cities and two regional hospitals in south Australia indicated that reporting rates had increased after a package of interventions that included education, reducing the existing paper-based forms from three pages to one page, offering a new reporting channel (call center), changes in report management and feedback enhancement [8]. That study reported a significant increase in the reporting rate in both inpatient wards and EDs, though the number of reports submitted via their call centers were only one quarter that submitted in paper (264 vs. 1005). In that study, the system was redesigned to include e-mail notification to all stakeholders. That study did not report a change in report-processing time. We believe the increase in reporting rates, however, may be a result of quicker processing times that would likely result from the use of e-mail messaged versus previously paper-only reports. According to one study, the inclusion of other information technologies besides simple e-mail notification might strengthen reporting process further improving rates [19].

A recent series of studies conducted at one [11], two [13] and three [12] ICUs that chose to return to the original paper-based reporting programs also found a significant increase in reporting rates. The reason for this surprising finding could be that their previous web-based systems were less than user friendly or simply because any change may bring about improved performance, at least temporarily. Regardless, paper-based systems are limited in their accessibility for long-term tracking.

The current study found not only an increase in reporting rates but also a more even distribution of incidents of varying severity after our system was implemented (Fig. 3). The increase in rates and the greater variety of severity may have resulted from shorten report processing, though we did not have access to pre-system response times. During the post-system period, inspecting, delgating, investigating, confirming, assessing, decision-making and forwarding to Taiwan's Patient Safety Reporting System (Fig. 1) was on average were 1.3, 4.2, 6.6, 8.1, 8.4, 13.9 and 14.8 days. Thirty-seven percent (897/2430) of the reports were delegated within 1 day, and 17% (408/2430) of the reports were completely investigated and all reported completed within 1 day of the initial incident report by front-staffer. Most cases of reported incidents were closed and forwarded to Taiwan's Patient Safety Reporting System within 15 days. Meanwhile, all stakeholders were notified of case status step by step throughout the process, a mechanism thought to be an important characteristic of successful timely reporting system [20]. An average of 8.8 professionals contributed to the handling of each reported incident, and initial front-staff reporters had transparent access to the status and recommendations of these different teams. Theoretically, it is believed that if a front-line staff member sees his/her opinions receiving immediate and close attention, he/she would be motivated to be more responsible in incident reporting. In addition, our newly implemented system could be considered relatively easy to use, as the reporting system was already web based. We believe the new system contributed most to the reporting rate increases, and the real reward quite insignificantly (US$1.67).

Figure 3

Number of reports by actual or potential severity grade. The pre-intervention period was from 1 January 2006 to 31 December 2007 and the post-intervention period from 1 January 2008 to 31 December 2009.

Figure 3

Number of reports by actual or potential severity grade. The pre-intervention period was from 1 January 2006 to 31 December 2007 and the post-intervention period from 1 January 2008 to 31 December 2009.

This study has some limitations. It was conducted in a single medical center. Caution should be made when generalizing the results to other hospitals. Ours was a modification of a web-based reporting system already in place. Some hospitals may not have such systems in place, so moving from a paper-based system to an advance web-based system might meet with greater resistance. Secondly, this study was a cohort study without a historical control group. We may not have adjusted for secular trends that may have influenced study outcomes after adding workflow automation to the reporting system. Because we used a quasi-experimental design, we cannot totally attribute the increased reporting rates to the implementation of system. Thirdly, any consideration of implementing such a program should consider its costs. We did not estimate the exact costs in this study because this is an established policy in the study hospital and part of the program was already in place. Further research is needed to address this issue. Finally, we did not survey provider acceptance and patient safety culture in this study. Future studies of the effect of implementing new systems may need to administer appropriate questionnaires to collect needed information.

In conclusion, this study represents the first of its kind to not only study pre- and post-system incident reporting rates but also other more specific variables of a web based modified to include a workflow-based response system in a effort to improve incident reporting rates. The workflow-basd response system almost immediately increased the incident reporting rates of varying severity by physicians, nurses and allied health professionals. This improvement was probably a result of increased transpareny and shortened repsonse times, though further study is needed to confirm this hypothesis.

Funding

This work was partially supported by the National Science Council of Taiwan [NSC 99-2410-H-110-008-MY2 to YCLI].

References

1
Linad
T
Kohn
JC
Molla
S
Donaldson. To Err is Human: Building a Safer Health System
 , 
1999
Washington
The National Academies Press
2
Safety WAfP
WHO draft guides for adverse event reporting and learning systems: from information to action
2005
Geneva
World Health Organization
3
Hutchinson
A
Young
TA
Cooper
KL
, et al.  . 
Trends in healthcare incident reporting and relationship to safety and quality data in acute hospitals: results from the National Reporting and Learning System
Qual Saf Health Care
 , 
2009
, vol. 
18
 (pg. 
5
-
10
)
4
Barach
P
Small
SD
Reporting and preventing medical mishaps: lessons from non-medical near miss reporting systems
BMJ
 , 
2000
, vol. 
320
 (pg. 
759
-
63
)
5
Stanhope
N
Crowley-Murphy
M
Vincent
C
, et al.  . 
An evaluation of adverse incident reporting
J Eval Clin Pract
 , 
1999
, vol. 
5
 (pg. 
5
-
12
)
6
Holzmueller
CG
Pronovost
PJ
Dickman
F
, et al.  . 
Creating the web-based intensive care unit safety reporting system
J Am Med Inf Assoc
 , 
2005
, vol. 
12
 (pg. 
130
-
9
)
7
Nakajima
K
Kurata
Y
Takeda
H
A web-based incident reporting system and multidisciplinary collaborative projects for patient safety in a Japanese hospital
Qual Saf Health Care
 , 
2005
, vol. 
14
 (pg. 
123
-
9
)
8
Evans
SM
Smith
BJ
Esterman
A
, et al.  . 
Evaluation of an intervention aimed at improving voluntary incident reporting in hospitals
Qual Saf Health Care
 , 
2007
, vol. 
16
 (pg. 
169
-
75
)
9
Hoffmann
B
Beyer
M
Rohe
J
, et al.  . 
‘Every error counts’: a web-based incident reporting and learning system for general practice
Qual Saf Health Care
 , 
2008
, vol. 
17
 (pg. 
307
-
12
)
10
Freestone
L
Bolsin
SN
Colson
M
, et al.  . 
Voluntary incident reporting by anaesthetic trainees in an Australian hospital
Int J Qual Health Care
 , 
2006
, vol. 
18
 (pg. 
452
-
7
)
11
Schuerer
DJE
Nast
PA
Harris
CB
, et al.  . 
A new safety event reporting system improves physician reporting in the surgical intensive care unit
J Am Coll Surg
 , 
2006
, vol. 
202
 (pg. 
881
-
7
)
12
Harris
CB
Krauss
MJ
Coopersmith
CM
, et al.  . 
Patient safety event reporting in critical care: a study of three intensive care units
Crit Care Med
 , 
2007
, vol. 
35
 (pg. 
1068
-
76
)
13
Ilan
R
Squires
M
Panopoulos
C
, et al.  . 
Increasing patient safety event reporting in 2 intensive care units: a prospective interventional study
J Crit Care
 , 
2011
, vol. 
26
 (pg. 
431.e11
-
e18
)
14
McDonald
TB
Helmchen
LA
Smith
KM
, et al.  . 
Responding to patient safety incidents: the ‘seven pillars
Qual Saf Health Care
 , 
2010
, vol. 
19
 (pg. 
1
-
4
)
15
Benn
J
Koutantji
M
Wallace
L
, et al.  . 
Feedback from incident reporting: information and action to improve patient safety
Qual Saf Health Care
 , 
2009
, vol. 
18
 (pg. 
11
-
21
)
16
Khoshafian SaMb
Introduction to Groupware, Workflow, and Workgroup Computing
 , 
1995
New York
John Wiley & Sons Inc
17
Bagian
JP
Lee
CL
Gosbee
J
, et al.  . 
Developing and deploying a patient safety program in a large health care delivery system: you can't fix what you don't know about
Jt Comm J Qual Improv
 , 
2001
, vol. 
27
 (pg. 
522
-
32
)
18
Tuttle
D
Holloway
R
Baird
T
, et al.  . 
Electronic reporting to improve patient safety
Qual Saf Health Care
 , 
2004
, vol. 
13
 (pg. 
281
-
6
)
19
Bates
DW
Gawande
AA
Improving safety with information technology
New Engl J Med
 , 
2003
, vol. 
348
 (pg. 
2526
-
34
)
20
Leape
LL
Reporting adverse event
New Engl J Med
 , 
2002
, vol. 
347
 (pg. 
1633
-
8
)

Supplementary data