Using High-Technology to Enforce Low-Technology Safety Measures: The Use of Third-party Remote Video Auditing and Real-time Feedback in Healthcare

Abstract

(See the Editorial Commentary by Palmore and Henderson, on pages 8–9.)

Background. Hand hygiene is a key measure in preventing infections. We evaluated healthcare worker (HCW) hand hygiene with the use of remote video auditing with and without feedback.

Methods. The study was conducted in an 17-bed intensive care unit from June 2008 through June 2010. We placed cameras with views of every sink and hand sanitizer dispenser to record hand hygiene of HCWs. Sensors in doorways identified when an individual(s) entered/exited. When video auditors observed a HCW performing hand hygiene upon entering/exiting, they assigned a pass; if not, a fail was assigned. Hand hygiene was measured during a 16-week period of remote video auditing without feedback and a 91-week period with feedback of data. Performance feedback was continuously displayed on electronic boards mounted within the hallways, and summary reports were delivered to supervisors by electronic mail.

Results. During the 16-week prefeedback period, hand hygiene rates were less than 10% (3933/60 542) and in the 16-week postfeedback period it was 81.6% (59 627/73 080). The increase was maintained through 75 weeks at 87.9% (262 826/298 860).

Conclusions. The data suggest that remote video auditing combined with feedback produced a significant and sustained improvement in hand hygiene.

Hand hygiene is an important factor in protecting patients from healthcare-associated infections (HAIs) [1, 2], yet reported compliance with hand hygiene averages about 38.7% with mean baseline rates ranging from 5% to 89% [1]. Intensive care units (ICUs) report lower rates of hand hygiene compliance (30%–40%) compared with other healthcare settings (50%–60%). The majority of studies on hand hygiene are based on direct measurements, such as intermittent human observation, healthcare worker (HCW) self-reporting, and indirect measurement of hand hygiene product usage [3–6].

A variation of direct observation is the use of video cameras to monitor human behavior. Traffic light cameras that automatically photograph vehicles have been used to increase drivers’ perceptions of the risk of being caught for running a red light and has reduced violations [7–9]. Oakley, et al. [10] used video monitoring to identify pediatric resuscitation errors listed in the medical record compared with the analyzed video, which recorded 54 errors compared with the 10 errors documented in the medical record. Video monitoring has also been used to identify poor technique and a lack of hand hygiene prior to food handling in 40 home kitchens in Melbourne, Australia [11].

The use of video cameras to observe hand hygiene practices in healthcare settings has been limited. Nishimura et al. [12] used concealed video cameras to observe hand hygiene practices of all people who entered the ICU for a 7-day period. Results showed a 71% compliance rate for ICU HCWs, 74% of non-ICU HCWs, and 95% of patient visitors. Brown et al. [13] placed a video camera in a neonatal ICU to record hand hygiene compliance data on random nursing shifts at all times of the day for 2 months. The results of the 267 contacts reported the following observed compliance rates: nurses, 24.7%; physicians, 31.8%; respiratory therapists, 20.5%; and parents, 6.6%.

Despite the knowledge that hand hygiene is universally accepted as an intervention to reduce HAIs, the quality of the interventional studies to increase and sustain compliance among HCWs has been poor [14]. The present study evaluated 432 482 hand hygiene observations over 107 weeks (∼25 months) with the use of video monitoring without and with feedback. The aim was to evaluate the use of third-party remote video auditing without and with near-real-time feedback on hand hygiene compliance.

METHODS

Study Population

An 17-bed medical intensive care unit (MICU) at a tertiary hospital in the northeast United States was selected for monitoring hand hygiene with remote video auditing. The unit, with a 95% occupancy rate, had 2 four-bedded rooms and 9 single rooms. All HCWs with direct patient contact (physicians, nurses, aides, and house staff) were included in the study. Patients, visitors, and staff such as environmental services and dieticians were excluded.

Remote Video Auditing

Remote video auditing consists of remote third-party human auditors utilizing a Web-based workflow software program to visualize and assess hand hygiene compliance by clinical staff with the use of video cameras and doorway motion sensors. As performance data were collected, feedback metrics were automatically tabulated by a central server database and delivered back to the hospital staff through electronic light-emitting diode (LED) boards, electronic mail summaries (intrashift and end-of-shift electronic mail summaries) and comprehensive weekly performance reports.

Study Implementation

In March 2008, 21 video cameras were installed at a cost of about $50 000 in the hallways and patient rooms so that hand hygiene events could be observed and quantified. The cameras only had views of the hand-washing sinks and sanitizer dispensers to protect patient privacy. Motion sensors were installed in the doorway of each patient room, enabling detection of all entrances and exits. When activated, the doorway motion sensor sent a signal with a time stamp to the digital video recorder. Refer to Figure 1 to view a diagram of the unit for the placement of the cameras plus door sensors. The third-party auditing and technology company, Arrowsight, Inc., connected remotely and uploaded the video from the digital video recorder. These independent auditors reviewed 20 second clips of video data surrounding each sensor-detected event to document hand hygiene compliance. The auditors rated events as being a pass, a fail, or not evaluable. All the data were automatically loaded into an electronic database that organized data chronologically by room and type of HCW (eg, “other healthcare professionals” or “attending physician”) on a 24-hour/7-days-a-week (24/7) basis for all entrances and exits. Nurses, aides, house staff, and other clinicians wearing any type of scrub or uniform were classified into the category of other healthcare professional, and staff wearing a laboratory coat and not any type of scrub were classified as an attending physician. Nonclinical workers wearing department-specific uniforms and visitors were excluded.

Figure 1.

Diagram of the placement of cameras and door sensors within the study unit.

Arrowsight’s Network Operations Center remote observers were stationed in India and staffed with 20 remote observers to cover continuous remote video auditing. The network operations center in India employed 3 observers and the Huntsville, AL. Network Operations Center employed 5 Quality Assurance personnel to perform quality audits on 5% of the observed events as outlined in this study. Auditors scoring below 97% were immediately removed from the auditing program and placed in remedial training. Upon successful completion of the training program, the auditor was allowed to return to auditing under direct supervision of a Quality Auditor Manager. If the auditor fell below 97% twice during a 30-day period, they were permanently removed from performing specific audits. During the study period, 1 auditor fell below 97%, was retrained, fell below again, and was reassigned.

Leadership in the unit was informed of the study and supported the initiative. The HCWs within the unit were aware that hand hygiene is a condition of employment, signed an annual hand hygiene contract, and were informed that cameras would be installed to monitor hand hygiene, which generated aggregate data only. The collected data did not include any patient or employee identifying information, and as a result, an exempt status was issued by the Feinstein Institute’s Institutional Review Board (IRB).

A “passing” hand hygiene event was assessed if a HCW remained within a patient room for 60 seconds or more and performed hand hygiene at the entrance or inside the room, or inside or outside an adjacent room, within 10 seconds before or after entering or exiting a patient room. A “failing” hand hygiene event was categorized when hand hygiene was not performed as described. Discarded events were those in which there were entries/exits by nonclinical staff or visitors, multiple staff entering simultaneously, and when a HCW was in the room for fewer than 60 seconds. The auditors did not monitor actual touch of the patient or the quality of hand hygiene.

The cameras visualized hand hygiene for 12 weeks beginning 3 March 2008 without computation of compliance rates or staff feedback. This period was used to validate that hand hygiene opportunities were being appropriately captured by the cameras and the auditors. Official auditing was initiated between June through September 2008 when baseline hand hygiene compliance assessment and rate computation began, but the results of the audits were not provided to staff or managers within the unit. This period was shorter than the feedback period because the data were remarkably consistent. Beginning 6 October 2008 through June 2010, real-time hand hygiene compliance data captured via remote video auditing was provided to the staff. The third-party auditors scored the hand hygiene (pass/fail) practices via remote video auditing, which were aggregated. The aggregate data were updated every 10 minutes with compliance rates by room, shift, and provider type. As these scores were produced, they were displayed to the staff using 2 LED boards in the hallway of the unit for ongoing viewing by the HCWs. The dataset had no unique identifiers of staff and only included the date and hour of the observation, the number of opportunities to perform hand hygiene during the audit interval, and the number of times hand hygiene was performed during the same audit interval. Refer to Figure 2 to view examples of hand hygiene compliance messages posted on the LED boards within the unit.

Figure 2.

Examples of hand hygiene messages posted on the light-emitting diode boards and a sample report sent by electronic mail to the study unit’s leadership twice per shift.

Detailed analyses were sent to nurse managers and other relevant healthcare professionals via electronic mail 3 hours after the start of each 12-hour shift and at the conclusion of each shift, and a detailed weekly summary was sent at the end of each week. These analyses included comparisons among rooms and trend data examining current performance as compared with past performance, as well as compliance rates on an hour-by-hour, day-of-week, and shift bases. Refer to Figure 2 for a sample report sent by electronic mail to unit leadership twice per shift.

Statistical Analysis

Data were obtained from Arrowsight representing weekly summaries of observations from 16 June 2008 through 4 July 2010. The weekly measurements consisted of the number of observations made during that week and the number that were in compliance with hand hygiene protocols. The first 16 weeks of observations (before 5 October 2008) were the prefeedback period. To evaluate the impact of the introduction of feedback, a time period of equivalent length after this date was identified (6 October 2008 through 24 January 2009). The effect of the introduction of feedback was tested as an interrupted time series with a segmented regression model adjusted for autocorrelation. The time series was divided into 2 segments at the week beginning 6 October 2008. In order to evaluate whether the effect was maintained, weekly data were obtained through 4 July 2010. The maintenance period was defined as beginning with the week of 25 January 2009 and ending on 4 July 2010. Statistical significance was evaluated at the P < .05 level. Analyses were conducted using Statistical Analysis Software version 9.2.

RESULTS

During the 16 prefeedback weeks, there were 60 542 hand hygiene events observed and 3933 events were categorized as passing, for an overall hand hygiene compliance rate of 6.5%, ranging from a weekly low of 3.5% to a high of 9.8%. For the 16 postfeedback weeks, there were 73 080 observations, with 59 627 categorized as passing, for an overall compliance rate of 81.6%, with rates ranging from 30.8% to 91.2%. During the 75-week maintenance period, 298 860 observations were made, with 262 826 in compliance (87.9%). Weekly rates ranged from 83.5% to 91.6%.

A visual examination of the pre- and postfeedback periods indicated a low prefeedback rate with little variation, with a sharp increase following the introduction of feedback. Refer to Figure 3. The segmented regression model supports this observation. The prefeedback slope was not significant and this term was excluded from the model. In the week immediately following the intervention, the model estimated an increase in compliance of 17.5% (P = .0057) with an additional 4% increase for each following week (P = .0019).

Figure 3.

Hand hygiene compliance by week during impact period following feedback.

The observations for the maintenance period are displayed in Figure 4, which presents a rate that appears consistent over time. The model for the maintenance period estimated a baseline value of 89% (P < .001) and very small but significant decreasing weekly trend of −0.04%, P = .0265.

Figure 4.

Hand hygiene compliance by week during maintenance period.

DISCUSSION

We used a definition of compliance that required the HCW to perform hand hygiene both before and after all patient room entries/exits to achieve compliance with hand hygiene before and after patient care, after touching objects in the patient’s environment, and after removing gloves [2]. Although the hand hygiene criteria can be changed, the major point of the study was to demonstrate that the technology can monitor hand hygiene by the HCW and can reinforce behavior for a prolonged period of time.

The healthcare literature does not frequently cite the use of ongoing remote video auditing to improve quality. Currently, there is no recognized standard for monitoring hand hygiene compliance; therefore, hand hygiene performance in different hospitals cannot be compared. This technology allows one to capture and analyze large volumes of performance data with detailed stratification over a prolonged period of time. By observing the intermittent fluctuations in performance throughout the study period and the subsequent action taken by the unit leaders to motivate staff to reclaim their high levels of performance, we believe ongoing monitoring is required to sustain high rates of hand hygiene compliance.

While this study was not designed to compare the difference between the prestudy hospital hand hygiene compliance rates and the hand hygiene compliance rates compiled during the prefeedback period with the remote video auditing program, we were surprised by the variance in scores between the 2 auditing methodologies. The prestudy internal hospital-based hand hygiene rates, which were collected by direct observation methods, were approximately 60%, far higher than the <10% hand hygiene rates recorded in the prefeedback period of this study (unpublished data). The 24/7 monitoring of hand hygiene reduced selection bias related to intermittent audits performed by independent human observers that falsely increased rates due to the Hawthorne effect [5]. Despite ongoing training, many clinicians mistakenly think that hand washing or using an alcohol-based hand sanitizer either prior to or after having patient contact is sufficient, while others think the use of disposable gloves can take the place of hand hygiene. This aspect of hand hygiene, plus the “passing” and “failing” definitions were communicated to HCWs when the feedback was introduced. Once feedback was introduced, adherences to the outlined practices were reinforced with the posted compliance rates on the LED boards and electronic reports. The low baseline rates could be explained by the lack of knowledge of hand hygiene guidelines, remote video auditing monitoring rules, and information regarding compliance with hand hygiene.

The most interesting observation is that feedback to staff improved hand hygiene and not just the presence of the camera technology. During the baseline period, the staff was aware of the cameras and understood their purpose. However, it was not until the hand hygiene performance metrics were posted to the LED boards that the hand hygiene rates began to significantly improve. We believe that the feedback was necessary for a sustained effect and that ongoing monitoring and feedback is required to sustain high rates of hand hygiene compliance [15–18]. Compliance rates have remained relatively unchanged in the period following completion of the study with ongoing monitoring and feedback with third-party remote video auditing.

Although the purpose of this study was not to investigate cost effectiveness of using this technology to reduce infection rates, we believe it can be a cost-effective method to accomplish this goal. This will depend on whether more prolonged studies demonstrate a change in the infection rates. The cost may also be justified in areas in which patients are at highest risk for infection. As healthcare reimbursement transitions into value-based purchasing, technology such as this may prove to be cost effective. It is also likely that the frequency of the audits could be reduced without compromising the result to reduce the cost associated with the service.

With regard to employee privacy, the data was reported in aggregate and the video segments were not used to observe individual employees. In addition, our employees are required to sign a contract that hand hygiene is an expected part of their job and is one of many indicators that will be used in their evaluations. There was no secret about what we were doing. There were signs up and everyone was aware that we were observing them. In other institutions, observing HCWs for hand hygiene compliance is often done by third-party observers. In those cases, it is usually done without letting people know ahead of time. Security cameras are used extensively in other industries to improve employee practices and for traffic enforcement. Healthcare seems to be very slow in adopting its use.

There were a number of limitations in this study. The hand hygiene observations did not include nonclinical personnel; only physicians, nurses, aides, and house staff and attending physicians were included. The facility’s hand hygiene rate with the use of remote video auditing could not be compared with hand hygiene compliance rates with the use of human observation, since the auditing rules were not exactly the same. The results cannot be generalized, since this is a single site study and a control unit with no video cameras was not conducted due to resources that would be required to observe every hand hygiene event. We also filtered out simultaneous events registered by the doorway motion sensors and events when a HCW was in the room for 60 seconds or less, therefore missing pass/fail events, and 2 of the rooms contained 4 beds where quality of the hand hygiene was not assessed.

Our experience shows that third-party remote video auditing with continuous near-real-time feedback may be more consistent than traditional auditing techniques. The technology may have other applications in quality auditing. Examples include compliance with proper technique during resuscitation, rapid responses, central line catheter placement, select nursing care (such as turning and positioning to prevent decubiti), as well as patient interactions.

Third-party remote video auditing combined with real-time feedback of HCW hand hygiene rates produced a significant and sustained improvement in hand hygiene compliance, and has the potential to improve the quality of patient care. Expanding the technology in additional patient care settings would enable us to test the efficiency of remote video auditing and its importance in reducing adverse events.

Notes

Acknowledgments.

The authors would like to acknowledge the administrative support of Michael Dowling, President and Chief Executive Officer for the North Shore-LIJ Health System, and Susan Somerville, Executive Director for North Shore University Hospital. We also want to thank all of the healthcare professionals working in the MICU for their commitment to patient safety.

The study was designed by the first and senior author. The data were collected and organized by the first author, and the statistical analysis was performed by the authors from the Krasnoff Quality Management Institute.

Financial support.

This work was supported by the New York State Department of Health (C702049) for the one-time grant of $50 000 to fund the capital equipment.

Potential conflicts of interest.

All authors: No reported conflicts.

All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.