Pooled analysis of WHO Surgical Safety Checklist use and mortality after emergency laparotomy

Background The World Health Organization (WHO) Surgical Safety Checklist has fostered safe practice for 10 years, yet its place in emergency surgery has not been assessed on a global scale. The aim of this study was to evaluate reported checklist use in emergency settings and examine the relationship with perioperative mortality in patients who had emergency laparotomy. Methods In two multinational cohort studies, adults undergoing emergency laparotomy were compared with those having elective gastrointestinal surgery. Relationships between reported checklist use and mortality were determined using multivariable logistic regression and bootstrapped simulation. Results Of 12 296 patients included from 76 countries, 4843 underwent emergency laparotomy. After adjusting for patient and disease factors, checklist use before emergency laparotomy was more common in countries with a high Human Development Index (HDI) (2455 of 2741, 89·6 per cent) compared with that in countries with a middle (753 of 1242, 60·6 per cent; odds ratio (OR) 0·17, 95 per cent c.i. 0·14 to 0·21, P < 0·001) or low (363 of 860, 42·2 per cent; OR 0·08, 0·07 to 0·10, P < 0·001) HDI. Checklist use was less common in elective surgery than for emergency laparotomy in high‐HDI countries (risk difference −9·4 (95 per cent c.i. −11·9 to −6·9) per cent; P < 0·001), but the relationship was reversed in low‐HDI countries (+12·1 (+7·0 to +17·3) per cent; P < 0·001). In multivariable models, checklist use was associated with a lower 30‐day perioperative mortality (OR 0·60, 0·50 to 0·73; P < 0·001). The greatest absolute benefit was seen for emergency surgery in low‐ and middle‐HDI countries. Conclusion Checklist use in emergency laparotomy was associated with a significantly lower perioperative mortality rate. Checklist use in low‐HDI countries was half that in high‐HDI countries.


Introduction
The volume of surgical procedures performed worldwide is large 1 and, although many advances have been made in the past several decades, surgical care exposes patients to substantial risk of morbidity and mortality. The safety of surgical care has gained traction within the global health landscape, yet it remains a pressing concern in both resource-rich and -poor settings 2 . In 2009, the WHO released a 19-item surgical safety checklist for implementation in countries around the world 3 . The checklist was designed to promote understanding and cohesive communication, and to ensure good practice among all surgical team members at three specific intervals: before induction of anaesthesia, before skin incision, and before the patient leaves the operating theatre 3 . Although the improvement in outcomes was dramatic 4 , uptake of this safety tool remains to be quantified on a large scale. Attention surrounding use of the checklist in resource-limited settings is of particular relevance to surgical care and outcomes 4 .
Process-related discrepancies, such as lack of a particular safe practice protocol, are chief contributors to adverse surgical events 5 . Despite the potential benefits of checklist use, there are numerous barriers to implementation 6,7 . Dynamic educational and social factors, such as ambiguity and confusion around the purpose of the checklist and negative attitudes to checklist adoption among some team members, contribute to poor checklist uptake 8,9 . In addition, in settings where resources are limited, completion of a checklist that focuses on unavailable items can seem pointless, for instance in the absence of pulse oximetry or antibiotics 10,11 . Despite these barriers to implementation and completion, it has been suggested 5 that absence of the checklist itself may serve as a major contributor to adverse surgical events.
Although data supporting the effectiveness of the checklist in fostering improved surgical outcomes are encouraging, studies in globally representative populations are uncommon. Furthermore, checklist outcomes have been studied largely within elective general surgery and subspecialty settings, with only a few studies examining checklist use in emergency care 12,13 . Attitudes to checklist use by providers working in emergency settings can be negative 12 . A survey of obstetric care providers found that one-third believed a checklist would be an inconvenience in emergencies 14 . Despite this, the benefit of checklist use does extend to emergency surgical care, as shown in an analysis of the original WHO checklist study in urgent operations across eight countries 13 . Importantly, it is also possible that the benefit of the checklist may be greatest in emergency situations, given the increased risks 15 .
Using a large, validated, global data set, this study aimed to compare reported use of the WHO Surgical Safety Checklist in patients undergoing emergency laparotomy and elective gastrointestinal surgery. Associations were sought between checklist use and perioperative mortality, accounting for country developmental level as well as patient and disease factors.

Study design and participants
Data were collected prospectively within two international, multicentre, observational cohort studies: GlobalSurg 1 16 and GlobalSurg 2 17 . Both studies were performed by the GlobalSurg Collaborative group using prespecified published protocols (NCT02179112 18 , NCT02662231 19 ). This collaborative methodology has been described elsewhere 20 . A UK National Health Service Research Ethics review considered both GlobalSurg 1 and Glob-alSurg 2 exempt from formal research registration (South East Scotland Research Ethics Service, references NR/1404AB12 and NR/1510AB5). Individual centres obtained their own audit, ethical or institutional approval. Details of data validation have been described and published previously 16,17 . Results of this analysis are reported according to Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) guidelines 21 .
In both contributing studies, investigators from healthcare facilities worldwide that fulfilled the inclusion criteria were invited to participate and submit data using an online platform 22 . Small teams of investigators were recruited via email, social media and through personal contacts. Investigators collected data during at least one 2-week interval during study windows. In GlobalSurg 1, investigators identified consecutive patients between 1 July 2014 and 21 December 2014 16 . This study included any patient undergoing emergency intraperitoneal surgery, defined in the study methods 16 . In GlobalSurg 2, investigators included consecutive patients between 4 January 2016 and 31 July 2016 17 . This study included all patients undergoing elective or emergency gastrointestinal surgery 17 . In both studies, patients were followed to day 30 after surgery, or for the duration of their inpatient stay in locations where follow-up was not feasible. Local investigators uploaded records to a secure online database, using the Research Electronic Data Capture (REDCap) system 23 .
Data from both GlobalSurg 1 and GlobalSurg 2 were combined to create a multicentre data set. Variables were cross-referenced and streamlined for coding consistency. Patients were then selected as having undergone emergency laparotomy or elective gastrointestinal surgery. Emergency laparotomy was captured using the definition from the UK National Emergency Laparotomy Audit 24 , adapted for global settings. Trauma laparotomy was included, whereas laparoscopic procedures and individuals aged less than 18 years were not included in the analysis.

Variables
The primary outcome measure was 30-day perioperative mortality, expressed as a proportion. Perioperative mortality was defined as 'any death, regardless of cause, occurring within 30 days of surgery in or out of the hospital' 25 . The metric was calculated by dividing the number of perioperative deaths by the total number of included operations performed 26 . The primary explanatory variable was reported use of the WHO Surgical Safety Checklist. Checklist use was recorded as 'no, not available', 'no, but available', 'yes', or 'unknown' for each patient in the study. Reported use of the checklist was calculated as a proportion recorded as 'yes' of the total number of patients included. Countries were stratified into three tertiles according to the Human Development Index (HDI) rank 27 . This is a composite statistic of life expectancy, education and income indices published by the United Nations. HDI was chosen over purely economic measures of country development Surgical Safety Checklist use and mortality after emergency laparotomy e105 on the principle that 'people and their capabilities should be the ultimate criteria for assessing the development of a country, not economic growth alone' 27 .
Patient-level variables included age, sex, diabetes, smoking status and ASA fitness grade. For simplification, ASA was grouped as a score of less than III and one of III or more. Disease-level variables included six major diagnostic groups: abdominal wall, benign foregut, benign midgut/hindgut, malignancy, trauma/injury and other. Operative characteristics, including the requirement for a bowel resection and the level of contamination, were also included.

Power considerations
GlobalSurg 1 and GlobalSurg 2 both included a priori sample size calculations that accounted for the uncertainty in the data coming from collaborating countries 18,19 . In making between-group comparisons of checklist use by HDI country or urgency, a difference from 10 to 5 per cent can be shown with α = 5 per cent and β = 10 per cent (90 per cent power) with group sizes of 582.

Statistical analysis
Differences between HDI tertiles were tested with Pearson's χ 2 test and Kruskal-Wallis test for categorical and continuous variables respectively. Multivariable logistic regression models were used to adjust for confounding in analyses of checklist use and 30-day perioperative mortality. Coefficients are expressed as odds ratios (ORs) with 95 per cent confidence intervals and P values derived from percentiles of 10 000 bootstrap replications. Models were constructed using the following principles: variables associated with outcome measures in previous studies were accounted for; demographic variables were included in model exploration; all first-order interactions were checked and included in final models if found to be influential; final model selection was informed using a criterion-based approach minimizing the Akaike information criterion and discrimination determined using the c-statistic (area under the receiver operating characteristic (ROC) curve). Hierarchical models accounting for clustering within countries were not used in this analysis owing to colinearity between explanatory variables of interest and country. Model residuals were checked (residuals versus fitted values; normality plot of standardized deviance residuals; and residuals versus leverage), and goodness of model fit determined using the Hosmer-Lemeshow test.
To help translate model outputs to real-life quantities of interest, bootstrapped simulations of model predictions were performed at specified co-variable levels. This enables the results of regression analyses to be expressed as probabilities, with the intention of these being more intuitive to interpret than ORs.
All analyses were undertaken using the R Foundation statistical program R 3.4.14, with finalfit 28 and dplyr packages (R Project for Statistical Computing, Vienna, Austria).

Data sharing
The data set can be explored using an online visualization application at http://data.globalsurg.org.

Results
A total of 26 228 patient records were sourced from the GlobalSurg 1 (10 745, 41⋅0 per cent) and GlobalSurg 2 (15 483, 59⋅0 per cent) data sets (Fig. 1). For 552 patients (2⋅1 per cent) 30-day mortality and/or checklist use was not known; these were removed. Patterns of missing data were examined and not considered to influence final results (Tables S1 and S2, supporting information). For 13 380 patients (51⋅0 per cent) the inclusion criteria were not fulfilled, and these were removed; reasons included undergoing an emergency procedure other than emergency laparotomy and age less than 18 years. The included procedures by country HDI are shown in Table S3 (supporting information). The final data set represents 12 296 patients from 76 countries, with 326 centres from GlobalSurg 1 16 and 356 centres from GlobalSurg 2 17 .

Demographics
Extensive data relating to patient and operative characteristics by HDI group and surgery type are provided in Tables S4-S6 (supporting information) to allow a full understanding of the population. Patients were distributed across HDI groups as shown in Fig. 1. Patients undergoing emergency laparotomy were older, more likely to be men, had a higher ASA score, were less likely to have cancer, and had a higher rate of wound contamination (Table S4, supporting information). Those undergoing emergency laparotomy in low-HDI compared with high-HDI countries were younger, more likely to be men, had a lower ASA score, were less likely to have had a bowel resection or cancer, and more likely to have wound contamination (Table S5, supporting information). Similar differences were seen in the elective surgery group (Table S6, supporting information). 12

Use of WHO Surgical Safety Checklist
Reported WHO Surgical Safety Checklist use across the entire cohort was 8881 of 12 296 (72⋅2 per cent). There was little difference in checklist use overall when comparing emergency laparotomy with elective surgery (73⋅7 versus 71⋅2 per cent respectively). Checklist use differed in high-(84⋅5 per cent) compared with middle-(59⋅4 per cent) and low-(47⋅3 per cent) HDI country groups ( Table 1). A multivariable regression model was used to adjust for confounding and identify predictors of checklist use ( Table S7, supporting information). A significant interaction was found between type of surgery and country HDI for checklist use: the patterns of checklist use by surgery type were different across HDI groups. After adjusting for patient and operative characteristics, checklist use continued to be lower in low-and middle-HDI countries (Fig. 2a). To convey differences in checklist use more intuitively, bootstrapped predicted probabilities of checklist use were determined ( Table 2). Absolute risk differences for emergency laparotomy versus elective surgery differed by HDI group. Checklist use was less common for elective surgery than emergency laparotomy in the high-HDI group (absolute risk difference −9⋅4 (95 per cent c.i. −11⋅9 to −6⋅9) per cent; P < 0⋅001), no different in the middle-HDI group (1⋅9 (−2⋅3 to 6⋅5) per cent; P = 0⋅357) and more common in the low-HDI group (12⋅1 (7⋅0 to 17⋅3) per cent; P < 0⋅001).

Use of checklist and 30-day mortality
Overall, reported use of the WHO Surgical Safety Checklist was associated with a lower 30-day mortality (471 of 8881, 5⋅3 per cent) compared with reported non-use (244 of 3415, 7⋅1 per cent) ( Table S8, supporting information). In models adjusting for confounders, reported use of the checklist was still associated with a significantly lower 30-day mortality (OR 0⋅60, 95 per cent c.i. 0⋅50 to 0⋅73; P < 0⋅001). Again, to create a more intuitive interpretation of the mortality model, adjusted predicted probabilities of 30-day mortality were created to allow comparisons across HDI group, surgery type and reported checklist use ( Fig. 3; Table S9, supporting information). No interaction between checklist use and 30-day mortality was seen for HDI or type of surgery. Thus, as expected, a significant checklist effect was seen for each combination of surgery type and HDI group, with magnitudes of effect shown in Fig. 3. The greatest absolute risk difference for checklist use was seen in emergency surgery in low-HDI (absolute risk reduction 4⋅6 (95 per cent c.i. 2⋅7 to 7⋅0) per cent; P < 0⋅001) and middle-HDI (5⋅1 (2⋅9 to 7⋅8) per cent; P < 0⋅001) countries, by virtue of the higher baseline 30-day mortality (  Values in parentheses are *percentages and †95 per cent confidence intervals. A multivariable logistic regression model was specified for checklist use by Human Development Index (HDI) group, type of surgery and confounders (see Fig. 2 and Table S4, supporting information). Bootstrapped adjusted predictions of the probability of checklist use were performed for different HDI groups and surgery type, with other co-variable levels specified: age 52 years; ASA grade less than III; no diabetes; malignancy disease classification; clean/contaminated wound status. Absolute risk differences for the probability of checklist use were determined and a two-sided P value was calculated. §χ 2 test (within HDI group between surgery type and checklist).   Table S8, supporting information) was used to generate adjusted predicted probabilities of death using bootstrap replication, with other co-variable levels specified: age 52 years, ASA grade less than III, malignancy disease classification, and contamination. Absolute risk differences for 30-day mortality are presented with 95 per cent confidence intervals, and two-sided P values for the absolute risk difference (Table S9, supporting information). HDI, Human Development Index

Discussion
In this large multinational prospective cohort study, reported use of the WHO Surgical Safety Checklist was associated with a significant reduction in 30-day perioperative mortality. This relationship was consistent and independent of key patient and disease-related variables. Checklist use in low-HDI countries was half that of high-HDI countries, and this effect persisted after accounting for differences in patient and disease characteristics. Checklist use was lower for elective surgery than for emergency laparotomy in high-HDI countries; this finding was unexpected. The association between checklist use and lower mortality was consistent across HDI groups and type of surgery, even after adjustment for case mix. The greatest absolute benefits were found in emergency surgery in low-and middle-HDI countries, owing to the higher baseline mortality rate.
Evidence supporting use of the surgical safety checklist in hospital practice is widely positive and supports the promotion of the checklist in patient safety programmes worldwide 2 . A strength of this study is the breadth of countries and hospitals that contributed prospectively collected data. Use of the WHO Surgical Safety Checklist in low-and middle-income countries was 2928 of 5248 (55⋅8 per cent), the same as that reported in the recent African Surgical Outcomes Study (6183 of 10 836, 57⋅1 per cent) 29 . In the present study, checklist use was found to be significantly lower in some low-and middle-HDI countries compared with high-HDI countries, yet the association with lower mortality was still seen. This clearly highlights an area for practice change. Fostering awareness to motivate local checklist champions is important, but may be difficult in more remote environments. In regions with less established local organizational infrastructure, the checklist may simply have not come to the attention of providers. Supportive governmental and academic institutions are crucial in facilitating the process, through continued professional development and ministry of health-accredited programmes 30 . Successful implementation of the checklist requires careful thought and local adaptation. Avoiding the perception of the checklist as just a 'tickbox' exercise is crucial for success 9 .
This study explored the hypothesis that checklist use is lower in emergency settings due to the particular challenges therein. Focus on emergency laparotomy provided a more homogeneous study group, rather than including all patients undergoing emergency surgery. Checklist use has been studied extensively in elective surgery, but has received less attention in emergency surgery, particularly in global settings 13 . Narratives that highlight time pressures, low staffing, inflexible hierarchies and lack of resources during times of emergency promote an idea that the checklist should not be prioritized in urgent care. The present study shows that the checklist can be used in emergency settings, and commonly is. Moreover, associations between checklist use and better outcomes are just as evident in emergency laparotomy as in elective surgery. The capacity for checklist implementation in settings providing emergency surgical care should not be undervalued. On the contrary, there may be much to gain for emergency surgery in low-and middle-income settings, given the higher baseline mortality rate.
A number of weaknesses to the approach taken here may be discussed. In general, the methodology is subject to selection bias at hospital level. Collaborators self-select to take part, which may reflect better resourced institutions contributing than is seen in the general population. During the study period, consecutive patients must be recruited so that selection bias at patient level is minimized. Data were validated particularly carefully in the Global-Surg 2 study 17 . As described previously, patient recruitment together with a subset of variables were re-collected by an independent team in contributing hospitals. In centres with the fewest resources, this can be a challenge, particularly where there are no formal written records. Some collaborators described the GlobalSurg data as of better quality than what was otherwise available within their hospital.
With regard to checklist use itself, collaborators simply reported whether a WHO Surgical Safety Checklist had been used before surgery. No review was undertaken of what form the checklist took, what local adaptations may have been introduced, whether there was broad team acceptance of the process, or whether the implementation was deemed successful. Self-reported checklist use is a poor reflection of meaningful compliance with checklist items 31 , and partial compliance reduces the positive benefits on outcomes derived from this 32 . Furthermore, the use and appropriate implementation of a surgical safety checklist may represent a wide range of health service system characteristics, including organizational and management attributes that support good clinical practices. Facilities reporting high use may also be those with significant resources, staffed by individuals familiar with key patient safety concepts who work together to ensure reliable systems are in place to deliver consistent patient care.
This study shows that the WHO Surgical Safety Checklist can be used in emergency surgery in resource-poor settings. The association with lower mortality is likely to reflect broader health system differences that prioritize safe and effective surgical care, yet the checklist plays an important part. Much of the benefit is likely to come from behaviours that can be difficult to measure, such as improved communication, better team work, identification of potential problems before they occur, and empowerment of members of staff at all levels. The checklist likely helps improve surgical safety by providing a framework for focusing teams on specific critical safety standards that are frequently assumed to have occurred but may not be adhered to. It can also help identify specific lapses and process weaknesses that can be the focus of improvement efforts. Where standards are either not known or not clear, the checklist can raise awareness of them and help guide hospital policies and protocols. It can even create a 'team-generated Hawthorne effect', whereby all perioperative personnel are involved in the responsibility of ensuring compliance with standards, and observe completion together using the checklist.
The data reported here have important implications for policy-makers. Ten years after the introduction of the checklist, there is much work to be done in promoting its adoption worldwide. Local adaptation and ownership are clearly important in ensuring long-term sustainable change 33 . Further studies around the details of implementation in resource-constrained settings will help tailor checklist procedures to local needs, thereby ensuring greatest effect. Strong compliance and effective implementation are challenging, but have the potential to save many lives and should be a priority for surgical safety.