Abstract

Background

Doffing protocols for personal protective equipment (PPE) are critical for keeping healthcare workers (HCWs) safe during care of patients with Ebola virus disease. We assessed the relationship between errors and self-contamination during doffing.

Methods

Eleven HCWs experienced with doffing Ebola-level PPE participated in simulations in which HCWs donned PPE marked with surrogate viruses (ɸ6 and MS2), completed a clinical task, and were assessed for contamination after doffing. Simulations were video recorded, and a failure modes and effects analysis and fault tree analyses were performed to identify errors during doffing, quantify their risk (risk index), and predict contamination data.

Results

Fifty-one types of errors were identified, many having the potential to spread contamination. Hand hygiene and removing the powered air purifying respirator (PAPR) hood had the highest total risk indexes (111 and 70, respectively) and number of types of errors (9 and 13, respectively). ɸ6 was detected on 10% of scrubs and the fault tree predicted a 10.4% contamination rate, likely occurring when the PAPR hood inadvertently contacted scrubs during removal. MS2 was detected on 10% of hands, 20% of scrubs, and 70% of inner gloves and the predicted rates were 7.3%, 19.4%, 73.4%, respectively. Fault trees for MS2 and ɸ6 contamination suggested similar pathways.

Conclusions

Ebola-level PPE can both protect and put HCWs at risk for self-contamination throughout the doffing process, even among experienced HCWs doffing with a trained observer. Human factors methodologies can identify error-prone steps, delineate the relationship between errors and self-contamination, and suggest remediation strategies.

(See the Major Article by Casanova et al on pages 945–9.)

The 2014–2015 Ebola virus disease (EVD) outbreak highlighted the importance of personal protective equipment (PPE) in shielding healthcare workers (HCWs) from highly infectious diseases. While the primary purpose of PPE is to protect skin, mucous membranes, and clothing from contamination during patient care, the implementation of protocols that facilitate safe and easy doffing further minimizes the risk of self-contamination and transmission of pathogens [1, 2]. For example, despite wearing more than the minimum PPE recommended by the Centers for Disease Control and Prevention (CDC) at the time [3, 4], 2 nurses in Texas contracted EVD while caring for a patient returning from West Africa with EVD [4], likely from a breach in PPE protocol. Subsequently, CDC refined its guidelines for donning and doffing Ebola-level PPE, which now include frequent hand hygiene and a trained observer (TO) [5].

Recently, several studies have examined human factors issues such as the usability of various Ebola-level PPE ensembles and their potential for error [6] as well as protocol deviations and self-contamination during doffing [7, 8]. To illuminate the complex relationships between self-contamination and errors made while doffing, a formal human factors risk analysis is still needed. Our study analyzed risk with 2 complementary tools, failure modes and effects analysis (FMEA) and fault tree analysis (FTA), which systematically identified and quantified the risk of errors in a doffing protocol and generated a probabilistic model that mapped behaviors to self-contamination data.

METHODS

All protocols were approved by Emory University’s Institutional Review Board. Eleven HCWs (10 nurses, 1 physician) marked with surrogate viruses were each observed once in a simulation [9]. Before the simulations, HCWs completed a questionnaire about their experience donning and doffing Ebola-level PPE during training and patient care. This facility’s PPE comprised disposable scrubs, Tyvek suit, shoe booties, 2 pairs of gloves, tape (to secure inner gloves to the coverall sleeves), apron, powered air purifying respirator (PAPR), and PAPR hood.

The same TO participated in every simulation and used a checklist to guide each HCW and ensure adherence to the donning and doffing protocol. After donning PPE, enveloped (Φ6) and nonenveloped (MS2) bacteriophages were applied to areas of PPE likely to be contaminated during patient care using a standard protocol [9]. The HCW then performed a standard clinical task (emptying a urinary catheter bag) on a mannequin, and then doffed following the facility’s protocol.

This protocol used the “beaking” method for removing gloves and alcohol-based hand rub (ABHR) for all hand hygiene except after removing the inner gloves (final doffing step), when soap and water were used. HCWs used manual (patient’s room) and automatic (anteroom) foam dispensers. HCWs could use a support bar for bootie removal, sanitizing the bar with ABHR and performing hand hygiene before and after bootie removal. The entire simulation was video recorded from different angles using 4 stationary cameras and 1 hand-held camera.

Risk Analyses

Failure Modes and Effects Analysis

This engineering technique is used to identify, quantify the risk of, and eliminate problems in a process, called “failure modes” (FMs) [10]. Two human factors experts reviewed video recordings of the simulations to identify FMs in each major doffing step by considering the facility’s protocol and knowledge of the PPE likely to be contaminated (the outermost items of PPE), as well as evident human factors missteps, such as errors of execution (eg, losing balance when removing booties). Before including a behavior as an FM, both reviewers needed to agree that a behavior was not redundant with another FM.

Five judges (human factors experts, infectious disease physicians, and nurses experienced in doffing and providing care for patients with EVD) independently rated the severity of the FMs using a 5-point scale (Table 1). The scale was defined over multiple dimensions including how contamination is spread to the HCW, TO, or environment, the effect on the doffing process, and damage to physical resources and PPE. If an FM had multiple consequences, the rating of the severest consequence was used (eg, not rubbing hands until dry could spread contamination as well as, less critically, prolong glove removal).

Table 1.

Scale Used to Rate the Severity of Failure Modes

Dimensions
ValueLabelContamination (PPE)Compromised PPEEffect on ProcessEquipment DamageContamination (Environment)
1NegligibleNo appreciable spread of contamination to PPENANo appreciable effect on the processNo appreciable damage to equipmentNo appreciable spread of contamination to environment
2MarginalDirty PPE contacts dirty PPE of HCWNAProcess is delayed, but not disruptedMinor equipment damageContaminated area becomes further contaminated
3SignificantDirty PPE contaminates clean PPE of HCW or TOPPE compromised (eg, wrists exposed)Minor disruption to processRepairable equipment damageMinor uncontaminated area becomes contaminated
4CriticalDirty PPE contaminates skin of HCW or TOMajor PPE compromise (eg, scrubs tear)Major, but recoverable, disruption to processPermanent equipment damageTask-critical uncontaminated area becomes contaminated
5CatastrophicDirty PPE contaminates face of HCW or TONAMajor unrecoverable disruption to processNAAnteroom becomes contaminated
Dimensions
ValueLabelContamination (PPE)Compromised PPEEffect on ProcessEquipment DamageContamination (Environment)
1NegligibleNo appreciable spread of contamination to PPENANo appreciable effect on the processNo appreciable damage to equipmentNo appreciable spread of contamination to environment
2MarginalDirty PPE contacts dirty PPE of HCWNAProcess is delayed, but not disruptedMinor equipment damageContaminated area becomes further contaminated
3SignificantDirty PPE contaminates clean PPE of HCW or TOPPE compromised (eg, wrists exposed)Minor disruption to processRepairable equipment damageMinor uncontaminated area becomes contaminated
4CriticalDirty PPE contaminates skin of HCW or TOMajor PPE compromise (eg, scrubs tear)Major, but recoverable, disruption to processPermanent equipment damageTask-critical uncontaminated area becomes contaminated
5CatastrophicDirty PPE contaminates face of HCW or TONAMajor unrecoverable disruption to processNAAnteroom becomes contaminated

Abbreviations: HCW, healthcare worker; NA, not applicable; PPE, personal protective equipment; TO, trained observer.

Table 1.

Scale Used to Rate the Severity of Failure Modes

Dimensions
ValueLabelContamination (PPE)Compromised PPEEffect on ProcessEquipment DamageContamination (Environment)
1NegligibleNo appreciable spread of contamination to PPENANo appreciable effect on the processNo appreciable damage to equipmentNo appreciable spread of contamination to environment
2MarginalDirty PPE contacts dirty PPE of HCWNAProcess is delayed, but not disruptedMinor equipment damageContaminated area becomes further contaminated
3SignificantDirty PPE contaminates clean PPE of HCW or TOPPE compromised (eg, wrists exposed)Minor disruption to processRepairable equipment damageMinor uncontaminated area becomes contaminated
4CriticalDirty PPE contaminates skin of HCW or TOMajor PPE compromise (eg, scrubs tear)Major, but recoverable, disruption to processPermanent equipment damageTask-critical uncontaminated area becomes contaminated
5CatastrophicDirty PPE contaminates face of HCW or TONAMajor unrecoverable disruption to processNAAnteroom becomes contaminated
Dimensions
ValueLabelContamination (PPE)Compromised PPEEffect on ProcessEquipment DamageContamination (Environment)
1NegligibleNo appreciable spread of contamination to PPENANo appreciable effect on the processNo appreciable damage to equipmentNo appreciable spread of contamination to environment
2MarginalDirty PPE contacts dirty PPE of HCWNAProcess is delayed, but not disruptedMinor equipment damageContaminated area becomes further contaminated
3SignificantDirty PPE contaminates clean PPE of HCW or TOPPE compromised (eg, wrists exposed)Minor disruption to processRepairable equipment damageMinor uncontaminated area becomes contaminated
4CriticalDirty PPE contaminates skin of HCW or TOMajor PPE compromise (eg, scrubs tear)Major, but recoverable, disruption to processPermanent equipment damageTask-critical uncontaminated area becomes contaminated
5CatastrophicDirty PPE contaminates face of HCW or TONAMajor unrecoverable disruption to processNAAnteroom becomes contaminated

Abbreviations: HCW, healthcare worker; NA, not applicable; PPE, personal protective equipment; TO, trained observer.

The frequency of each FM in the simulations was obtained by raters independently tallying their occurrence(s) in the video recordings. Raw frequencies were transformed into quintiles so that frequency and severity were scaled comparably [11]. For each FM, a risk index (RI) was then obtained by multiplying the average of the judge’s severity ratings for that FM by its quintile frequency score [11].

Behavioral Coding

Video recordings were coded for frequency and duration of the major doffing steps (eg, remove apron), substeps (eg, roll apron into a ball before disposing), and frequency of FMs by 2 independent investigators using The Observer XT version 12.5 (Noldus Information Technology, Leesburg, Virginia). Interrater reliability was assessed with Cohen κ [12]. Duration of doffing steps was assessed using box plots and compared using the Wilcoxon signed-rank test.

Fault Tree Analysis

This technique predicts the probability of an undesirable event (top of the “tree”) based on the sequence and combinations of events leading to the top event. At the bottom of the tree are the basic causes of the top event. FTAs were performed with Relyence Fault Tree (Release 2, Relyence Corporation, Greensburg, Pennsylvania) to suggest the most likely sequence of events that resulted in contamination of the HCWs during the simulations [9].

Events in the FTAs were largely FMs; however, new events were posited to complete a logical sequence and their frequency was subsequently obtained by a rater blind to their role in the FTA. Frequencies of events were converted to probabilities, which flowed up the tree through Boolean (“AND” or “OR”) decision gates. Last, some events in the fault tree were included only for theoretical thoroughness and were either never observed or were observed but did not contribute to the probability of the top event (eg, events involving PPE that was not contaminated in the simulation).

For both bacteriophages, a fault tree was constructed for each site of observed self-contamination. The fault trees were constructed assuming that (1) performing hand hygiene with ABHR reduced the probability of enveloped ɸ6 contamination, but not nonenveloped MS2 contamination [13–16]; (2) the probability of inadequate hand hygiene is the probability that hands were not rubbed until dry and were not rubbed thoroughly (not rubbing wrists, thumbs, or in between fingers) [17]; and (3) touching any contaminated surface resulted in transfer of contamination.

Log-likelihood ratio tests (LRTs) were used to determine how well the predicted contamination rates from the models fit the observed contamination rates. Here, the LRT statistic is approximately χ2 distributed with 1 degree of freedom [18]. Significant differences would indicate that the predicted rates poorly fit the observed data. All statistical tests were 2-sided; a P value <.05 was considered statistically significant.

RESULTS

Demographics

Seven HCWs reported receiving training within 2 months prior to participating, 3 HCWs within 3–4 months, and 1 HCW more than a year prior. The median number of times doffing during training and patient care were 11 (range, 3–101) and 21 (range, 0–144), respectively. Ten HCWs reported having cared for a patient with confirmed EVD.

Failure Modes and Effects Analysis

In the FMEA (Table 2), 51 FMs were identified and grouped by major doffing step, as defined by the protocol. Steps varied in the number (median, 6 [range, 1–13]) and the RI of their FMs (median, 7 [range, 2–18]). Hand hygiene (ΣRIs = 111) and removing the PAPR hood (ΣRIs = 70) had the greatest summative RIs owing, in part, to the number of different ways failure occurred (n = 9 and 13, respectively). The summative RIs were moderately high for removing coveralls (60), booties (40), tape (39), and apron (37). The summative RIs for removing gloves (20), engaging the TO’s attention to begin doffing (12), and removing the PAPR helmet (4) were lower.

Table 2.

Failure Modes and Effects Analysis

Doffing StepFailure ModeEffect(s)Severity ScoreFrequencyRisk Index
Engage TOTO does not inspect HCW for visible contaminationDisrupt process sequence/delays process3.00412
Hand hygieneDoes not disinfect alcohol pump after usingSpread contamination to environment3.17516
Hand hygieneDoes not disinfect between fingersSpread contamination to PPE and environment3.17516
Hand hygieneDoes not disinfect wristsSpread contamination to PPE and environment3.50518
Hand hygieneDoes not disinfect thumbsSpread contamination to PPE and environment3.67415
Hand hygieneDoes not rub hands until drySpread contamination to PPE and environment, disrupt process sequence/delays process3.33517
Hand hygieneHand hygiene truncated by TO giving instructionsDisrupt process sequence/delays process, spread contamination to PPE and environment2.83411
Hand hygieneShaking hands to drySpread contamination to environment and PPE2.83411
Hand hygieneStretching to reach alcohol dispenserOccupational injury1.5035
Hand hygieneSteps back onto coverall/mat after stepping off to reach alcohol dispenserSpread contamination to PPE2.3312
Remove apronGrabs front of apronSpread contamination to PPE2.50410
Remove apronTouches coverall sleeves to front of apronSpread contamination to PPE2.6738
Remove apronTouches apron excessivelySpread contamination to PPE2.5038
Remove apronSnaps apron roughlySpread contamination to PPE and environment3.1726
Remove apronApron touches wall when removingSpread contamination to environment3.0013
Remove apronOuter gloves touch front of coverall when rolling apron upSpread contamination to PPE1.6712
Remove bootiesCrosses leg in front of selfSpread contamination to PPE2.17511
Remove bootiesTouches bootie excessivelySpread contamination to PPE2.3337
Remove bootiesTouches back of bootie to front of coverall legSpread contamination to PPE2.1737
Remove bootiesUnstable posture (loss of balance)Occupational injury2.3337
Remove bootiesSwings legs excessively while removing bootiesSpread contamination to PPE1.8324
Remove bootiesTouches same bootie with >1 handSpread contamination to PPE2.1712
Remove bootiesTouches support stool with >1 handSpread contamination to environment2.6713
Remove tapeWrist exposed after removing tapePPE compromised3.17413
Remove tapeRoughly removes tapeSpread contamination to PPE and environment3.00412
Remove tapeCoverall sleeves tearPPE compromised2.83514
Remove glovesDifficulty pinching cuff with beaked hand (requires multiple attempts)Disrupt process sequence/delays process2.17511
Remove glovesGlove snaps when removing gloveSpread contamination to PPE and environment2.5038
Remove coverallsInner gloves coming off when removing coverall sleevesPPE compromised3.67518
Remove coverallsTouches outside of coverall sleeve with inner glovesSpread contamination to PPE2.8339
Remove coverallsLower back is exposed after removing coverallPPE compromised2.1737
Remove coverallsCoverall is off of disinfecting matSpread contamination to PPE2.3337
Remove coverallsPushes coverall down legs with inner glovesSpread contamination to PPE2.8326
Remove coverallsSteps off mat into red zone, then enters anteroomSpread contamination to environment3.1713
Remove coverallsUnstable posture (loss of balance)Occupational injury2.5025
Remove coverallsTouches front of coverall with inner glovesSpread contamination to PPE3.3313
Remove coverallsGrabs PAPR hood tiesDisrupt process sequence/delays process2.8313
Remove PAPR hoodPAPR hood contacts exposed armsSpread contamination to HCW3.33413
Remove PAPR hoodTouches ties excessivelySpread contamination to PPE2.50410
Remove PAPR hoodSqueezes front of face shield to remove from pegSpread contamination to PPE2.3349
Remove PAPR hoodPulls PAPR hood off by grabbing near front rather than the backSpread contamination to PPE, disrupt process sequence/delays process2.3337
Remove PAPR hoodTouches face shield excessivelySpread contamination to PPE2.3337
Remove PAPR hoodHCW almost hands PAPR hood to TOSpread contamination to PPE (TO), disrupt process sequence/delays process3.5014
Remove PAPR hoodTouches PAPR hood excessivelySpread contamination to PPE2.8313
Remove PAPR hoodBumps into door (eg, with PAPR hood, scrub shoulder)Spread contamination to environment2.8326
Remove PAPR hoodTO’s arm contacts PAPR battery cordSpread contamination to PPE (TO)1.8324
Remove PAPR hoodDrops PAPR helmet onto floorEquipment damage2.0012
Remove PAPR hoodGrabs PAPR hood too far backDisrupt process sequence/delays process2.3312
Remove PAPR hoodTO says “unsnap PAPR hood” before “untie PAPR hood”Disrupt process sequence/delays process1.8312
Remove PAPR hoodUnsnaps hood before untying tiesDisrupt process sequence/delays process1.6712
Remove PAPR helmetWipe face with scrub shoulderSpread contamination to HCW3.8314
Doffing StepFailure ModeEffect(s)Severity ScoreFrequencyRisk Index
Engage TOTO does not inspect HCW for visible contaminationDisrupt process sequence/delays process3.00412
Hand hygieneDoes not disinfect alcohol pump after usingSpread contamination to environment3.17516
Hand hygieneDoes not disinfect between fingersSpread contamination to PPE and environment3.17516
Hand hygieneDoes not disinfect wristsSpread contamination to PPE and environment3.50518
Hand hygieneDoes not disinfect thumbsSpread contamination to PPE and environment3.67415
Hand hygieneDoes not rub hands until drySpread contamination to PPE and environment, disrupt process sequence/delays process3.33517
Hand hygieneHand hygiene truncated by TO giving instructionsDisrupt process sequence/delays process, spread contamination to PPE and environment2.83411
Hand hygieneShaking hands to drySpread contamination to environment and PPE2.83411
Hand hygieneStretching to reach alcohol dispenserOccupational injury1.5035
Hand hygieneSteps back onto coverall/mat after stepping off to reach alcohol dispenserSpread contamination to PPE2.3312
Remove apronGrabs front of apronSpread contamination to PPE2.50410
Remove apronTouches coverall sleeves to front of apronSpread contamination to PPE2.6738
Remove apronTouches apron excessivelySpread contamination to PPE2.5038
Remove apronSnaps apron roughlySpread contamination to PPE and environment3.1726
Remove apronApron touches wall when removingSpread contamination to environment3.0013
Remove apronOuter gloves touch front of coverall when rolling apron upSpread contamination to PPE1.6712
Remove bootiesCrosses leg in front of selfSpread contamination to PPE2.17511
Remove bootiesTouches bootie excessivelySpread contamination to PPE2.3337
Remove bootiesTouches back of bootie to front of coverall legSpread contamination to PPE2.1737
Remove bootiesUnstable posture (loss of balance)Occupational injury2.3337
Remove bootiesSwings legs excessively while removing bootiesSpread contamination to PPE1.8324
Remove bootiesTouches same bootie with >1 handSpread contamination to PPE2.1712
Remove bootiesTouches support stool with >1 handSpread contamination to environment2.6713
Remove tapeWrist exposed after removing tapePPE compromised3.17413
Remove tapeRoughly removes tapeSpread contamination to PPE and environment3.00412
Remove tapeCoverall sleeves tearPPE compromised2.83514
Remove glovesDifficulty pinching cuff with beaked hand (requires multiple attempts)Disrupt process sequence/delays process2.17511
Remove glovesGlove snaps when removing gloveSpread contamination to PPE and environment2.5038
Remove coverallsInner gloves coming off when removing coverall sleevesPPE compromised3.67518
Remove coverallsTouches outside of coverall sleeve with inner glovesSpread contamination to PPE2.8339
Remove coverallsLower back is exposed after removing coverallPPE compromised2.1737
Remove coverallsCoverall is off of disinfecting matSpread contamination to PPE2.3337
Remove coverallsPushes coverall down legs with inner glovesSpread contamination to PPE2.8326
Remove coverallsSteps off mat into red zone, then enters anteroomSpread contamination to environment3.1713
Remove coverallsUnstable posture (loss of balance)Occupational injury2.5025
Remove coverallsTouches front of coverall with inner glovesSpread contamination to PPE3.3313
Remove coverallsGrabs PAPR hood tiesDisrupt process sequence/delays process2.8313
Remove PAPR hoodPAPR hood contacts exposed armsSpread contamination to HCW3.33413
Remove PAPR hoodTouches ties excessivelySpread contamination to PPE2.50410
Remove PAPR hoodSqueezes front of face shield to remove from pegSpread contamination to PPE2.3349
Remove PAPR hoodPulls PAPR hood off by grabbing near front rather than the backSpread contamination to PPE, disrupt process sequence/delays process2.3337
Remove PAPR hoodTouches face shield excessivelySpread contamination to PPE2.3337
Remove PAPR hoodHCW almost hands PAPR hood to TOSpread contamination to PPE (TO), disrupt process sequence/delays process3.5014
Remove PAPR hoodTouches PAPR hood excessivelySpread contamination to PPE2.8313
Remove PAPR hoodBumps into door (eg, with PAPR hood, scrub shoulder)Spread contamination to environment2.8326
Remove PAPR hoodTO’s arm contacts PAPR battery cordSpread contamination to PPE (TO)1.8324
Remove PAPR hoodDrops PAPR helmet onto floorEquipment damage2.0012
Remove PAPR hoodGrabs PAPR hood too far backDisrupt process sequence/delays process2.3312
Remove PAPR hoodTO says “unsnap PAPR hood” before “untie PAPR hood”Disrupt process sequence/delays process1.8312
Remove PAPR hoodUnsnaps hood before untying tiesDisrupt process sequence/delays process1.6712
Remove PAPR helmetWipe face with scrub shoulderSpread contamination to HCW3.8314

Abbreviations: HCW, healthcare worker; PAPR, powered air purifying respirator; PPE, personal protective equipment; TO, trained observer.

Table 2.

Failure Modes and Effects Analysis

Doffing StepFailure ModeEffect(s)Severity ScoreFrequencyRisk Index
Engage TOTO does not inspect HCW for visible contaminationDisrupt process sequence/delays process3.00412
Hand hygieneDoes not disinfect alcohol pump after usingSpread contamination to environment3.17516
Hand hygieneDoes not disinfect between fingersSpread contamination to PPE and environment3.17516
Hand hygieneDoes not disinfect wristsSpread contamination to PPE and environment3.50518
Hand hygieneDoes not disinfect thumbsSpread contamination to PPE and environment3.67415
Hand hygieneDoes not rub hands until drySpread contamination to PPE and environment, disrupt process sequence/delays process3.33517
Hand hygieneHand hygiene truncated by TO giving instructionsDisrupt process sequence/delays process, spread contamination to PPE and environment2.83411
Hand hygieneShaking hands to drySpread contamination to environment and PPE2.83411
Hand hygieneStretching to reach alcohol dispenserOccupational injury1.5035
Hand hygieneSteps back onto coverall/mat after stepping off to reach alcohol dispenserSpread contamination to PPE2.3312
Remove apronGrabs front of apronSpread contamination to PPE2.50410
Remove apronTouches coverall sleeves to front of apronSpread contamination to PPE2.6738
Remove apronTouches apron excessivelySpread contamination to PPE2.5038
Remove apronSnaps apron roughlySpread contamination to PPE and environment3.1726
Remove apronApron touches wall when removingSpread contamination to environment3.0013
Remove apronOuter gloves touch front of coverall when rolling apron upSpread contamination to PPE1.6712
Remove bootiesCrosses leg in front of selfSpread contamination to PPE2.17511
Remove bootiesTouches bootie excessivelySpread contamination to PPE2.3337
Remove bootiesTouches back of bootie to front of coverall legSpread contamination to PPE2.1737
Remove bootiesUnstable posture (loss of balance)Occupational injury2.3337
Remove bootiesSwings legs excessively while removing bootiesSpread contamination to PPE1.8324
Remove bootiesTouches same bootie with >1 handSpread contamination to PPE2.1712
Remove bootiesTouches support stool with >1 handSpread contamination to environment2.6713
Remove tapeWrist exposed after removing tapePPE compromised3.17413
Remove tapeRoughly removes tapeSpread contamination to PPE and environment3.00412
Remove tapeCoverall sleeves tearPPE compromised2.83514
Remove glovesDifficulty pinching cuff with beaked hand (requires multiple attempts)Disrupt process sequence/delays process2.17511
Remove glovesGlove snaps when removing gloveSpread contamination to PPE and environment2.5038
Remove coverallsInner gloves coming off when removing coverall sleevesPPE compromised3.67518
Remove coverallsTouches outside of coverall sleeve with inner glovesSpread contamination to PPE2.8339
Remove coverallsLower back is exposed after removing coverallPPE compromised2.1737
Remove coverallsCoverall is off of disinfecting matSpread contamination to PPE2.3337
Remove coverallsPushes coverall down legs with inner glovesSpread contamination to PPE2.8326
Remove coverallsSteps off mat into red zone, then enters anteroomSpread contamination to environment3.1713
Remove coverallsUnstable posture (loss of balance)Occupational injury2.5025
Remove coverallsTouches front of coverall with inner glovesSpread contamination to PPE3.3313
Remove coverallsGrabs PAPR hood tiesDisrupt process sequence/delays process2.8313
Remove PAPR hoodPAPR hood contacts exposed armsSpread contamination to HCW3.33413
Remove PAPR hoodTouches ties excessivelySpread contamination to PPE2.50410
Remove PAPR hoodSqueezes front of face shield to remove from pegSpread contamination to PPE2.3349
Remove PAPR hoodPulls PAPR hood off by grabbing near front rather than the backSpread contamination to PPE, disrupt process sequence/delays process2.3337
Remove PAPR hoodTouches face shield excessivelySpread contamination to PPE2.3337
Remove PAPR hoodHCW almost hands PAPR hood to TOSpread contamination to PPE (TO), disrupt process sequence/delays process3.5014
Remove PAPR hoodTouches PAPR hood excessivelySpread contamination to PPE2.8313
Remove PAPR hoodBumps into door (eg, with PAPR hood, scrub shoulder)Spread contamination to environment2.8326
Remove PAPR hoodTO’s arm contacts PAPR battery cordSpread contamination to PPE (TO)1.8324
Remove PAPR hoodDrops PAPR helmet onto floorEquipment damage2.0012
Remove PAPR hoodGrabs PAPR hood too far backDisrupt process sequence/delays process2.3312
Remove PAPR hoodTO says “unsnap PAPR hood” before “untie PAPR hood”Disrupt process sequence/delays process1.8312
Remove PAPR hoodUnsnaps hood before untying tiesDisrupt process sequence/delays process1.6712
Remove PAPR helmetWipe face with scrub shoulderSpread contamination to HCW3.8314
Doffing StepFailure ModeEffect(s)Severity ScoreFrequencyRisk Index
Engage TOTO does not inspect HCW for visible contaminationDisrupt process sequence/delays process3.00412
Hand hygieneDoes not disinfect alcohol pump after usingSpread contamination to environment3.17516
Hand hygieneDoes not disinfect between fingersSpread contamination to PPE and environment3.17516
Hand hygieneDoes not disinfect wristsSpread contamination to PPE and environment3.50518
Hand hygieneDoes not disinfect thumbsSpread contamination to PPE and environment3.67415
Hand hygieneDoes not rub hands until drySpread contamination to PPE and environment, disrupt process sequence/delays process3.33517
Hand hygieneHand hygiene truncated by TO giving instructionsDisrupt process sequence/delays process, spread contamination to PPE and environment2.83411
Hand hygieneShaking hands to drySpread contamination to environment and PPE2.83411
Hand hygieneStretching to reach alcohol dispenserOccupational injury1.5035
Hand hygieneSteps back onto coverall/mat after stepping off to reach alcohol dispenserSpread contamination to PPE2.3312
Remove apronGrabs front of apronSpread contamination to PPE2.50410
Remove apronTouches coverall sleeves to front of apronSpread contamination to PPE2.6738
Remove apronTouches apron excessivelySpread contamination to PPE2.5038
Remove apronSnaps apron roughlySpread contamination to PPE and environment3.1726
Remove apronApron touches wall when removingSpread contamination to environment3.0013
Remove apronOuter gloves touch front of coverall when rolling apron upSpread contamination to PPE1.6712
Remove bootiesCrosses leg in front of selfSpread contamination to PPE2.17511
Remove bootiesTouches bootie excessivelySpread contamination to PPE2.3337
Remove bootiesTouches back of bootie to front of coverall legSpread contamination to PPE2.1737
Remove bootiesUnstable posture (loss of balance)Occupational injury2.3337
Remove bootiesSwings legs excessively while removing bootiesSpread contamination to PPE1.8324
Remove bootiesTouches same bootie with >1 handSpread contamination to PPE2.1712
Remove bootiesTouches support stool with >1 handSpread contamination to environment2.6713
Remove tapeWrist exposed after removing tapePPE compromised3.17413
Remove tapeRoughly removes tapeSpread contamination to PPE and environment3.00412
Remove tapeCoverall sleeves tearPPE compromised2.83514
Remove glovesDifficulty pinching cuff with beaked hand (requires multiple attempts)Disrupt process sequence/delays process2.17511
Remove glovesGlove snaps when removing gloveSpread contamination to PPE and environment2.5038
Remove coverallsInner gloves coming off when removing coverall sleevesPPE compromised3.67518
Remove coverallsTouches outside of coverall sleeve with inner glovesSpread contamination to PPE2.8339
Remove coverallsLower back is exposed after removing coverallPPE compromised2.1737
Remove coverallsCoverall is off of disinfecting matSpread contamination to PPE2.3337
Remove coverallsPushes coverall down legs with inner glovesSpread contamination to PPE2.8326
Remove coverallsSteps off mat into red zone, then enters anteroomSpread contamination to environment3.1713
Remove coverallsUnstable posture (loss of balance)Occupational injury2.5025
Remove coverallsTouches front of coverall with inner glovesSpread contamination to PPE3.3313
Remove coverallsGrabs PAPR hood tiesDisrupt process sequence/delays process2.8313
Remove PAPR hoodPAPR hood contacts exposed armsSpread contamination to HCW3.33413
Remove PAPR hoodTouches ties excessivelySpread contamination to PPE2.50410
Remove PAPR hoodSqueezes front of face shield to remove from pegSpread contamination to PPE2.3349
Remove PAPR hoodPulls PAPR hood off by grabbing near front rather than the backSpread contamination to PPE, disrupt process sequence/delays process2.3337
Remove PAPR hoodTouches face shield excessivelySpread contamination to PPE2.3337
Remove PAPR hoodHCW almost hands PAPR hood to TOSpread contamination to PPE (TO), disrupt process sequence/delays process3.5014
Remove PAPR hoodTouches PAPR hood excessivelySpread contamination to PPE2.8313
Remove PAPR hoodBumps into door (eg, with PAPR hood, scrub shoulder)Spread contamination to environment2.8326
Remove PAPR hoodTO’s arm contacts PAPR battery cordSpread contamination to PPE (TO)1.8324
Remove PAPR hoodDrops PAPR helmet onto floorEquipment damage2.0012
Remove PAPR hoodGrabs PAPR hood too far backDisrupt process sequence/delays process2.3312
Remove PAPR hoodTO says “unsnap PAPR hood” before “untie PAPR hood”Disrupt process sequence/delays process1.8312
Remove PAPR hoodUnsnaps hood before untying tiesDisrupt process sequence/delays process1.6712
Remove PAPR helmetWipe face with scrub shoulderSpread contamination to HCW3.8314

Abbreviations: HCW, healthcare worker; PAPR, powered air purifying respirator; PPE, personal protective equipment; TO, trained observer.

FMs were often about theoretically spreading contamination, either to an HCW’s PPE (n = 31 [60%]), skin (n = 2 [4%]), or the environment, (n = 14 [27%]); fewer about delaying or disrupting the process (n = 11 [21%]), compromising PPE (n = 4 [8%]), or resulting in occupational injury (n = 3 [6%]) or damaging equipment (n = 1 [2%]). Some FMs, such as not rubbing hands until dry, had >1 effect (eg, spreading contamination and prolonging later processes).

Duration of Doffing Steps

The raters showed substantial agreement [19] for coding doffing steps, substeps, and the frequencies of FMs (mean κ = 0.77). Box plots showed variation in the duration of each doffing step (Figure 1), with complete doffing requiring a median of 5.7 minutes (range, 3.7–9.9 minutes). Removing coveralls was the most time-consuming step (median, 83.4 seconds [range, 41.7–116.5 seconds]). Moreover, coverall and outer glove removal had the largest interquartile ranges (46.7 and 39.4, respectively; range, 1.5–46.7). Contributors to variability in glove removal duration and their relation to contamination are discussed in the following sections.

Box plots of the duration of each major doffing step during simulations with Ebola-level personal protective equipment. For each step, the maximum (top whisker), 75th percentile (top line), median (dark line), 25th percentile (bottom line), and minimum (bottom whisker) values are shown. Asterisks represent the mean. Abbreviation: PAPR, powered air purifying respirator.
Figure 1.

Box plots of the duration of each major doffing step during simulations with Ebola-level personal protective equipment. For each step, the maximum (top whisker), 75th percentile (top line), median (dark line), 25th percentile (bottom line), and minimum (bottom whisker) values are shown. Asterisks represent the mean. Abbreviation: PAPR, powered air purifying respirator.

HCWs performed hand hygiene with ABHR for a median duration of 7.3 seconds (range, 0.7–39.2 seconds), with higher median durations occurring after apron removal (median, 14.1 seconds [range, 3.2–21.9 seconds]) and inner glove removal using soap and water (median, 25.6 seconds [range, 14.5–31.8 seconds]) (Figure 2).

Box plots of the duration of hand hygiene after each major step of doffing during simulations with Ebola-level personal protective equipment. For each step, the maximum (top whisker), 75th percentile (top line), median (dark line), 25th percentile (bottom line), and minimum (bottom whisker) values are shown. Asterisks represent the mean. Abbreviations: PAPR, powered air purifying respirator; TO, trained observer.
Figure 2.

Box plots of the duration of hand hygiene after each major step of doffing during simulations with Ebola-level personal protective equipment. For each step, the maximum (top whisker), 75th percentile (top line), median (dark line), 25th percentile (bottom line), and minimum (bottom whisker) values are shown. Asterisks represent the mean. Abbreviations: PAPR, powered air purifying respirator; TO, trained observer.

Highest Risks in the Failure Modes and Effects Analysis

Among the FMs with the highest RIs were several related to hand hygiene (RIs > 73%–94% of all FMs). These were not disinfecting ABHR dispenser after use (64% of hand hygiene instances); not rubbing hands until dry (36%); not cleaning wrists (30%), between fingers (15%), or thumbs (5%); hand hygiene truncated by TO moving to next step (11%); and shaking hands to dry (4%). Of the few FMs specific to the TO, truncating hand hygiene had the clearest negative impact on HCWs, accounting for 15% of hand hygiene instances when hands were not rubbed until dry.

An insufficient duration of hand hygiene (until gloves were dry) appeared to be related to FMs during glove removal. Specifically, if gloves were slick from ABHR, firmly gripping 1 glove with the other gloved hand became challenging, particularly with the reduced dexterity from double-gloving. Across the 11 HCWs, there was a total of 111 attempts at pinching the cuff of 1 glove with the “beaked” glove (5 times more than what should be necessary), 71% of which occurred during outer glove removal. Removal of the outer gloves (median, 21.5 seconds [range, 7.4–70.1 seconds]) took statistically longer than inner gloves (median, 11.3 seconds [range, 6.8–64.7]) (P = .02). Moreover, poor grip can lead to glove-snapping and, although rare, snapping the inner gloves during removal emerged as part of a critical pathway for hand contamination in the FTAs.

Another family of FMs concerned compromised PPE (RIs, >62%–96% of all FMs), particularly of the hands and wrists during tape and coverall removal. One of these FMs appeared to be related to another; coverall sleeves are tucked into the inner gloves, which can pull the gloves off when HCWs remove their hands from their sleeves. Some HCWs anticipated this and loosened their sleeves with their inner gloves before removing their coveralls.

A final family of FMs emerged that can be characterized as “mishandling PPE,” comprising grabbing the front of the apron (36% of HCWs) and several FMs specific to PAPR hood removal (RIs, >63%–83% of all FMs): squeezing front of face shield (45% of HCWs), fumbling with PAPR hood ties (36%), and PAPR hood shroud contacting exposed arms (36%).

Fault Tree Analyses

Ten HCWs contributed behavioral and contamination data for the FTAs [9]; contamination data were unavailable for 1 HCW.

The surrogate virus ɸ6 was detected on 10% of HCWs’ scrubs. The fault tree predicted a 10.4% contamination rate for scrubs (P = .96), which most likely occurred when the PAPR hood inadvertently contacted scrubs during removal or, less likely, during coverall removal, when contaminated inner gloves contacted the inside of the PAPR hood shroud, which later rests against the scrubs (Figure 3). ɸ6 was not detected on bare hands (predicted rate, 0.15%, P = .87). The model predicted some inner glove contamination (predicted rate, 10.3%, P = .14), but none was observed (Figure 4). Although the FMs used in our definition of inadequate hand hygiene [17] were among the riskiest in the FMEA, the FTAs show that the conjunction of these behaviors was unlikely (0.14; Figures 3 and 4). This suggests that, overall, hand hygiene was not as poor as the FMEA indicates, corroborating the low ɸ6 contamination rates.

Fault tree analysis of 10 healthcare workers during doffing of Ebola-level personal protective equipment for Φ6 self-contamination of scrubs (highlighted in red). Events highlighted in yellow did not involve PPE contaminated in the simulation and did not contribute to the probability of the top event. Abbreviations: FTA, fault tree analysis; IG, inner gloves; PAPR, powered air purifying respirator; PPE, personal protective equipment.
Figure 3.

Fault tree analysis of 10 healthcare workers during doffing of Ebola-level personal protective equipment for Φ6 self-contamination of scrubs (highlighted in red). Events highlighted in yellow did not involve PPE contaminated in the simulation and did not contribute to the probability of the top event. Abbreviations: FTA, fault tree analysis; IG, inner gloves; PAPR, powered air purifying respirator; PPE, personal protective equipment.

Fault tree analysis of 10 healthcare workers during doffing of Ebola-level personal protective equipment for Φ6 self-contamination of hands and inner gloves (highlighted in red). Gates highlighted in green are decomposed in Supplementary Figure 1. Abbreviations: FTA, fault tree analysis; IG, inner glove; PAPR, powered air purifying respirator; PPE, personal protective equipment.
Figure 4.

Fault tree analysis of 10 healthcare workers during doffing of Ebola-level personal protective equipment for Φ6 self-contamination of hands and inner gloves (highlighted in red). Gates highlighted in green are decomposed in Supplementary Figure 1. Abbreviations: FTA, fault tree analysis; IG, inner glove; PAPR, powered air purifying respirator; PPE, personal protective equipment.

MS2 was detected on 20% of scrubs, 70% of inner gloves, and 10% of hands. The predicted contamination rates for scrubs, inner gloves, and hands were 19.38% (P = .96), 73.40% (P = .81), and 7.34% (P = .76), respectively. FTAs for MS2 suggest that the pathways for scrub contamination were similar to those for ɸ6 (during PAPR hood or coverall removal; Figure 5), but for MS2, both pathways were equally culpable. The most likely source of inner glove contamination was touching the PAPR hood face shield during PAPR hood removal, although other routes, such as the ABHR dispenser, were also possible. Moreover, hand contamination of the one HCW was likely due to glove-snapping during the removal of inner gloves, for which the probability of being contaminated was high (0.73; Figure 6).

Fault tree analysis of 10 healthcare workers during doffing of Ebola-level personal protective equipment for MS2 self-contamination of scrubs (highlighted in red). Events highlighted in yellow did not involve PPE contaminated in the simulation and did not contribute to the probability of the top event. Abbreviations: FTA, fault tree analysis; IG, inner gloves; PAPR, powered air purifying respirator; PPE, personal protective equipment.
Figure 5.

Fault tree analysis of 10 healthcare workers during doffing of Ebola-level personal protective equipment for MS2 self-contamination of scrubs (highlighted in red). Events highlighted in yellow did not involve PPE contaminated in the simulation and did not contribute to the probability of the top event. Abbreviations: FTA, fault tree analysis; IG, inner gloves; PAPR, powered air purifying respirator; PPE, personal protective equipment.

Fault tree analysis of 10 healthcare workers during doffing of Ebola-level personal protective equipment for MS2 self-contamination of hands and inner gloves (highlighted in red). Gates highlighted in green are decomposed in Supplementary Figure 2. Abbreviations: FTA, fault tree analysis; IG, inner gloves; PPE, personal protective equipment.
Figure 6

Fault tree analysis of 10 healthcare workers during doffing of Ebola-level personal protective equipment for MS2 self-contamination of hands and inner gloves (highlighted in red). Gates highlighted in green are decomposed in Supplementary Figure 2. Abbreviations: FTA, fault tree analysis; IG, inner gloves; PPE, personal protective equipment.

The FTAs suggest that touching the PAPR hood’s face shield, if contaminated, can be a critical route for contaminating the inner gloves. While FMs related to PAPR hood removal were not particularly risky in and of themselves, the FTA implicates them as a major cause of self-contamination that would have been underestimated by the FMEA alone. The FTA also suggests that while inner gloves can protect HCWs from hand contamination, they may inadvertently contaminate clean items of PPE, such as scrubs during coverall removal.

Last, the FTAs reveal a potentially serious “near-miss” in the doffing protocol; the probability that some part of HCWs’ hands are exposed in the steps before inner glove removal is 0.82. Fortunately, these events did not propagate further up the tree because a contaminated object never happened to touch exposed hands.

DISCUSSION

We observed a range of errors with varying degrees of risk among highly experienced HCWs doffing Ebola-level PPE with a TO. Among the highest risk errors identified by the FMEA were those related to hand hygiene, compromised PPE (exposed hands and wrists), and mishandling PPE. The extent to which these errors may have contributed to self-contamination was characterized by the FTAs. Despite observing deficiencies in hand hygiene technique, the probability of committing those errors together was fortunately rather small, which agrees with the low self-contamination rates with the enveloped virus. Still, deficiencies in hand hygiene technique are concerning as nonenveloped viruses may be more resistant to ABHR and consequently result in higher contamination. Compromised PPE led to a near miss whereas mishandling PPE, particularly the PAPR hood face shield, emerged as a major source of contamination of the inner gloves.

PPE can both protect and endanger HCWs throughout the doffing process. For example, inner gloves were largely effective at protecting HCWs’ hands, but may have spread contamination to clean items of PPE (eg, when unzipping coveralls). Thus, opportunities for incidental contact during doffing should be minimized. Our findings also suggest that some protections may be conditional. For example, the “beaking” method minimizes contact with the contaminated outer surface of a glove. If gloves are slick from ABHR, however, HCWs may be at risk for self-contamination via glove-snapping. Thus, “beaking” may be most effective only when ABHR has dried completely.

Our findings highlight the importance of considering self-contamination during doffing as a probabilistic event. Although only 2 of all the HCWs who treated EVD patients in the United States contracted the disease, this achievement may have involved a certain amount of luck as our simulation approach revealed that a confluence of random events is often necessary for self-contamination to occur. Existing protocols include redundancies, such as frequent hand hygiene [5], to reduce the chances that an error will propagate contamination forward. However, our results suggest that these steps may be abbreviated in practice and, when combined with other errors, may result in pathogen transmission.

Despite our sample’s extensive training and experience with PPE, many of our findings were related to hand hygiene and glove removal, which are performed routinely by HCWs in any clinical setting [20]. Many HCWs, however, receive PPE training on the job rather than from a standardized, rigorous process [21]. Moreover, PPE elements vary across facilities, which may result in suboptimal use of PPE when combined with the mobility of HCWs and lack of standardized training. The optimal type and frequency of training for PPE remains unclear, although education and practice appear to decrease self-contamination when doffing routine PPE, such as gowns and gloves [8].

Our findings, however, are not without their limitations. The contamination pathways identified are plausible but hypothetical, requiring further empirical testing to confirm. Moreover, our analyses accounted only for simulations with controlled application of contamination, and other pathways may emerge during actual patient care. Nonetheless, human factors methodologies can provide valuable insights and solutions for optimizing PPE doffing, and resources exist to help medical professionals utilize them [11, 22]. When designing a PPE protocol, stakeholders should not only test the usability of various ensembles [6], but also combine these assessments with a formal risk analysis to identify specific objectives for testing [11]. Beginning with the riskiest behaviors, stakeholders should develop remediation strategies and then test the effectiveness of those solutions. For example, the problems of coverall sleeves pulling inner gloves off and snapping gloves during removal may be remediated by using extended-cuff inner gloves and a different glove removal technique, respectively. Afterward, the change in severity, frequency, or probabilities of these FMs and their consequences should be reassessed iteratively until effective control measures are established. To ensure the safety of HCWs, these tools should be integrated and individualized to different settings, ideally before providing direct patient care.

Supplementary Data

Supplementary materials are available at Clinical Infectious Diseases online. Consisting of data provided by the authors to benefit the reader, the posted materials are not copyedited and are the sole responsibility of the authors, so questions or comments should be addressed to the corresponding author.

Notes

Financial support. This work was supported by the Centers for Disease Control and Prevention Epicenters Program (award number U54CK000164). Emory University Hospital’s participation was in part supported by the National Center for Advancing Translational Sciences of the National Institutes of Health (award number UL1TR000454; Atlanta Clinical and Translational Science Institute).

Acknowledgments. We thank Golda Nguyen, Robert Cummings, and Faith Aisien for their help with coding and analyzing data. We also thank the participating hospitals and healthcare workers, as well as their administrative teams, and the members of the Prevention Epicenter of Emory and Atlanta Consortium Hospitals.

Potential conflicts of interest. All authors: No reported conflicts. All authors have submitted the ICMJE Form for Disclosure of Potential Conflicts of Interest. Conflicts that the editors consider relevant to the content of the manuscript have been disclosed.

References

1.

Edmond
MB
,
Diekema
DJ
,
Perencevich
EN
.
Ebola virus disease and the need for new personal protective equipment
.
JAMA
2014
;
312
:
2495
6
.

2.

Fischer
WA
2nd ,
Hynes
NA
,
Perl
TM
.
Protecting health care workers from Ebola: personal protective equipment is critical but is not enough
.
Ann Intern Med
2014
;
161
:
753
4
.

3.

Centers for Disease Control and Prevention
.
CDC tightened guidance for U.S. healthcare workers on personal protective equipment for Ebola
. Available at: https://www.cdc.gov/media/releases/2014/fs1020-ebola-personal-protective-equipment.html. Accessed
18 May 2017
.

4.

Liddell
AM
,
Davey
RT
Jr
,
Mehta
AK
et al.
Characteristics and clinical management of a cluster of 3 patients with Ebola virus disease, including the first domestically acquired cases in the United States
.
Ann Intern Med
2015
;
163
:
81
90
.

5.

Centers for Disease Control and Prevention
.
Guidance on personal protective equipment (PPE) to be used by healthcare workers during management of patients with confirmed Ebola or persons under investigation (PUIs) for Ebola who are clinically unstable or have bleeding, vomiting, or diarrhea in U.S. hospitals, including procedures for donning and doffing PPE
. Available at: https://www.cdc.gov/vhf/ebola/healthcare-us/ppe/guidance.html. Accessed
18 May 2017
.

6.

Herlihey
TA
,
Gelmi
S
,
Flewwelling
CJ
et al.
Personal protective equipment for infectious disease preparedness: a human factors evaluation
.
Infect Control Hosp Epidemiol
2016
;
37
:
1022
8
.

7.

Kwon
JH
,
Burnham
CAD
,
Reske
K
et al.
Assessment of healthcare worker protocol deviations and self-contamination during personal protective equipment donning and doffing
.
Infect Control Hosp Epidemiol
2017
;
1
7
.

8.

Tomas
ME
,
Kundrapu
S
,
Thota
P
et al.
Contamination of health care personnel during removal of personal protective equipment
.
JAMA Intern Med
2015
;
175
:
1904
10
.

9.

Casanova
LM
,
Erukunuakpor
K
,
Kraft
CS
et al.
Assessing viral transfer during doffing of Ebola-level personal protective equipment in a biocontainment unit
.
Clin Infect Dis
2018
;
66
:
945
9
.

10.

Stamatis
DH.
Failure mode and effect analysis: FMEA from theory to execution
.
Milwaukee, WI
:
ASQC Quality Press
,
1995
.

11.

Israelski
EW
,
Muto
WH
.
Human factors risk management for medical products
. In:
Carayon
P
, ed.
Handbook of human factors and ergonomics in health care and patient safety
, 2nd ed.
Boca Raton, FL
:
CRC Press
,
2011
:
475
506
.

12.

Cohen
J
.
A coefficient of agreement for nominal scales
.
Educ Psychol Meas
1960
;
20
:
37
46
.

13.

Grayson
ML
,
Melvani
S
,
Druce
J
et al.
Efficacy of soap and water and alcohol-based hand-rub preparations against live H1N1 influenza virus on the hands of human volunteers
.
Clin Infect Dis
2009
;
48
:
285
91
.

14.

Kampf
G
,
Grotheer
D
,
Steinmann
J
.
Efficacy of three ethanol-based hand rubs against feline calicivirus, a surrogate virus for norovirus
.
J Hosp Infect
2005
;
60
:
144
9
.

15.

Larson
EL
,
Cohen
B
,
Baxter
KA
.
Analysis of alcohol-based hand sanitizer delivery systems: efficacy of foam, gel, and wipes against influenza A (H1N1) virus on hands
.
Am J Infect Control
2012
;
40
:
806
9
.

16.

Sickbert-Bennett
EE
,
Weber
DJ
,
Gergen-Teague
MF
,
Sobsey
MD
,
Samsa
GP
,
Rutala
WA
.
Comparative efficacy of hand hygiene agents in the reduction of bacteria and viruses
.
Am J Infect Control
2005
;
33
:
67
77
.

17.

Boyce
JM
,
Didier
P
.
Guideline for hand hygiene in health-care settings: recommendations of the Healthcare Infection Control Practices Advisory Committee and the HICPAC/SHEA/APIC/IDSA Hand Hygiene Task Force
.
Infect Control Hosp Epidemiol
2002
;
23
:
S3
40
.

18.

Agresti
A.
Categorical data analysis
.
Hoboken, NJ
:
John Wiley & Sons
,
2002
.

19.

Landis
JR
,
Koch
GG
.
The measurement of observer agreement for categorical data
.
Biometrics
1977
;
33
:
159
74
.

20.

Mitchell
R
,
Roth
V
,
Gravel
D
et al.
Are health care workers protected? An observational study of selection and removal of personal protective equipment in Canadian acute care hospitals
.
Am J Infect Control
2013
;
41
:
240
4
.

21.

John
A
,
Tomas
ME
,
Cadnum
JL
et al.
Are health care personnel trained in correct use of personal protective equipment
?
Am J Infect Control
2016
;
44
:
840
2
.

22.

DeRosier
J
,
Stalhandske
E
,
Bagian
JP
,
Nudell
T
.
Using health care failure mode and effect analysis: the VA National Center for Patient Safety’s prospective risk analysis system
.
Jt Comm J Qual Improv
2002
;
28
:
248
67
.

Supplementary data