Journal Logo

Collective review

Impact of the WHO Surgical Safety Checklist Relative to Its Design and Intended Use: A Systematic Review and Meta-Meta-Analysis

Sotto, Kenji T. BAa,*; Burian, Barbara K. PhDb; Brindle, Mary E. MDc,d

Author Information
Journal of the American College of Surgeons: December 2021 - Volume 233 - Issue 6 - p 794-809e8
doi: 10.1016/j.jamcollsurg.2021.08.692
  • Free
  • Open

Abstract

Abbreviations and Acronyms: CD: cannot determine; DVT: deep vein thrombosis; NA: not applicable; NR: not reported; OR: odds ratio; PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analysis; SSC: Surgical Safety Checklist; SSI: surgical site infection; UTI: urinary tract infection.

The operating room is a fast-paced and detail-oriented environment requiring highly skilled multidisciplinary staff to deliver the care necessary to treat a wide variety of patients and conditions. The technical skills required to safely perform complex procedures, including the setup and use of numerous items of equipment, and coordination of the surgical team to ensure the safety and care of each patient, provide multiple opportunities for errors. Approximately 313 million surgical procedures are performed annually worldwide,1 with an estimated adverse event rate of 10% and mortality rate of 7.3%. Half of these adverse events are likely preventable.2

As part of a global effort to reduce the number of adverse events and postoperative deaths, the World Health Organization (WHO) developed and implemented the Surgical Safety Checklist (SSC).3 The WHO SSC is composed of 19 items divided into 3 sections, 1 for use at 3 critical perioperative moments—before induction of anesthesia, before skin incision, and before the patient leaves the room. The items within the checklist are a mix of process checks, intended to reinforce accepted safety practices, and items to foster better communication between clinical disciplines. The initial trial of the WHO SSC at 8 hospitals worldwide resulted in significant reductions of complication rates from 11% at baseline to 7% (p < 0.001) after WHO SSC implementation, as well as a reduction in mortality rates from 1.5% to 0.8% (p = 0.003).4

Similar studies5,6 have demonstrated comparable results across a wide variety of settings. However, these substantial claims and the sustainability of the SSC overall effectiveness has been challenged.7,8 Indeed, several studies9-11 found no statistically significant reductions in mortality. Reasons for this variability in outcomes can be ascribed, in part, to inconsistency in implementation across facilities.12

Since its inception, the WHO SSC has been implemented in healthcare facilities in 70% of countries13 and has become standard practice in most. The outcomes of a majority of studies give insight to the SSC’s effectiveness at an institutional level. This study was conducted as part of a larger project focused on accentuating facilitators and minimizing barriers of WHO SSC implementation, modification, training, and evaluation in high-resource, English-speaking countries. The aim of this study was to perform a systematic review, thematic analysis, and meta-analyses of other systematic reviews and meta-analyses (ie, a meta-meta-analysis) on WHO SSC impacts and outcomes across multiple sites to identify what parts are working, what can be done to make it more effective, and to determine if it achieved its intended effect relative to its design and intended use.

Methods

We conducted an analysis of systematic reviews and meta-analyses reporting on impacts and clinical outcomes associated with WHO SSC use, through a qualitative thematic analysis14 and quantitative meta-analyses (ie a meta-meta-analysis), following the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) guidelines (Fig. 1).15

F1-20
Figure 1:
Preferred reporting items for systematic review and meta-analyses flow diagram.

Data sources

Author KS conducted a systematic search of all articles from January 2008 until February 2021 contained in PubMed and the World Health Organization Global Index Medicus using the following search terms: “WHO Surgical Safety Checklist Literature Review,” “WHO Surgical Safety Checklist Systematic Review,” “WHO Surgical Safety Checklist Thematic Analysis,” “World Health Organization Surgical Safety Checklist Review,” “WHO Surgical Checklist AND Barriers,” “Surgical Safety Checklist,” “Perioperative AND Checklist AND Systematic Review,” “Safety AND Checklist AND Systematic Review,” and “Surgical AND Checklist AND Systematic Review.”

Study selection

Titles and abstracts of citations between January 2008 and February 2021 were identified by an author (KS) for full text screening. Full-length articles were retrieved for abstracts that did not provide sufficient information to determine eligibility, and were reviewed for inclusion by 2 reviewers (KS and BB). Disagreements were settled by a third reviewer (MB) if consensus was not met between the reviewers. Systematic reviews that contained extractable data on performance of the WHO SSC were included for full-text review. Any outcome measures were considered (eg clinical outcomes, process measures, attitudes, etc). For the search, we also included systematic reviews that explored implementation considerations, which we retained for future study. Those excluded were duplicates, non-English articles, those that were not systematic reviews, and studies whose focus was on other surgical safety checklists (eg SURPASS Checklist, Universal Protocol). See Figure 1 for a PRISMA diagram depicting the study selection process.

Analyses

Quality assessment and risk of bias in individual studies

Risk of bias within each systematic review was independently assessed by 2 reviewers (KS and MB) using the Quality Assessment of Systematic Reviews and Meta-Analyses tool developed by the National Institutes of Health.16 The tool consists of 8 items focusing on key concepts for critical appraisal of the internal validity of each systematic review. Each systematic review was assessed for the following criteria: a focused question, eligibility criteria, search strategy, dual review for inclusion/exclusion of studies, quality appraisal for internal validity, key characteristics of included studies, assessment of publication bias, and assessment of heterogeneity if a meta-analysis was conducted. Reviewers selected either “yes,” “no,” or “cannot determine (CD)/not applicable (NA)/not reported (NR)” for each item. Items for which “no/CD/NR” was selected were noted as potential risk of bias and/or potential flaws in study design. An overall quality rating of “good” (least risk of bias), “fair” (susceptible to some bias), or “poor” (significant risk of bias and considered for exclusion) was given to each systematic review based on the overall assessment of the 8 items. Disagreements were settled by a third reviewer (BB) if consensus was not met between the reviewers.

Quantitative analysis

All quantifiable outcome measures were identified and categorized. Unique studies within meta-analyses were identified and collated within each outcome measure. Outcome measures for which data were available from 2 or more individual studies across systematic review were identified for meta-analysis.

Each outcome was pooled using a random-effects model. Pooled analyses were reported as odds ratios with a 95% confidence interval. Tests for overall effect were reported as Z-scores. A value of p < 0.05 indicated statistical significance. Heterogeneity was calculated using Higgin’s I2. I2 values of 25%, 50%, and 75% were interpreted as low, moderate, and high heterogeneity, respectively.17 All analyses were performed using Comprehensive Meta-Analysis software (Version 3.0, Biostat, Inc.).

Risk of bias across studies (publication bias)

Risk of bias across studies (publication bias) was evaluated through visual inspection of funnel plot symmetry, if there were a sufficient number of trials (n ≥ 10).18 Egger’s regression tests were conducted to further assess for funnel plot asymmetry. A 2-tailed p value < 0.05 indicated statistical significance and potential for publication bias. Funnel plots and Egger’s regression tests were performed using Comprehensive Meta-Analysis software (Version 3.0, Biostat, Inc.).

Thematic analysis

We included all identified systematic reviews that specifically explored outcomes measures in our thematic analysis of the impact of the WHO SSC. Themes were developed both deductively and inductively through line-by-line coding of each systematic review using QSR International NVivo software (Version 12, QRS International), performed by author KS and refined with author BB. Similar themes were further grouped into 4 major areas or categories, described below. A 1-way, chi-square goodness-of-fit test was conducted for each major area or category and each theme using IBM SPSS Statistics for Macintosh (Version 25.0, IBM Corp). This study will report the results and findings on the categories and themes identified that focus on the impact and outcomes of the WHO SSC.

Results

We identified 1,048 articles for tile and abstract screening. After applying inclusion and exclusion criteria, we excluded 1,012 articles. The remaining 36 full-text articles were reviewed, and 20 systematic reviews were included for our review (Fig. 1).

Risk of bias in individual studies

Table 1 displays a summary of quality assessment ratings for each systematic review and meta-analysis included in our study. Risk of bias was low for each systematic review included in our study. Of the 20 systematic reviews included in our study, 15 systematic reviews19-33 were given an overall quality of “good” (least risk of bias) and 5 systematic reviews34-38 were rated as “fair” (susceptible to some bias).

T1-20
Table 1:
Quality Assessment of Systematic Reviews and Meta-Analyses Included in our Study

Quantitative analysis

The 20 systematic reviews included, contained 24 unique observational cohort studies that reported pre-post data on a total of 18 clinical outcomes (Table 2). Each outcome was assessed in anywhere from 2 to 19 studies. The most common outcomes evaluated were surgical site infection (n = 19 studies), mortality (n = 18), and morbidity or any complication (n =17). The overall effect for each outcome was further categorized into 3 categories: favors WHO SSC, favors control, and no impact. Table 3 displays a summary of the results for each outcome analyzed.

T2A-20
Table 2:
Characteristics of Studies Included for Quantitative Analysis
T2B-20
Table 2:
Continued.
T3-20
Table 3:
Forest Plot Statistics Summary by Outcome

Postoperative outcomes that favor the use of the WHO SSC

Nine postoperative outcomes—surgical site infection (SSI; Fig. 2), mortality (Fig. 3), morbidity (eFig. 1), pneumonia (eFig. 2), unplanned return to the operating room (eFig. 3), urinary tract infection (UTI; eFig. 4), blood loss requiring transfusion (eFig. 5), unplanned intubation (eFig. 6), and sepsis (eFig. 7)—favored the use of the WHO SSC. For example, 19 studies reported data on postoperative SSI, which included a total of 289,845 patients. Pooled analysis revealed that 1,750 of 142,743 (1.2%) patients in the intervention group compared with 2,512 of 147,102 (1.7%) patients in the control group experienced an SSI. A meta-analysis (Fig. 2) revealed that WHO SSC use was associated with a significant reduction of SSIs [odds ratio (OR) 0.62 (0.47–0.81), Z = -3.51 (p < 0.001)], and there was a high level of heterogeneity across trials (I2 = 92%).

FE1-20
eFigure 1:
Clinical outcome: any complication (morbidity). OR, odds ratio; SSC, surgical safety checklist.
FE2-20
eFigure 2:
Clinical outcome: pneumonia. OR, odds ratio; SSC, surgical safety checklist.
FE3-20
eFigure 3:
Clinical outcome: unplanned return to operating room. OR, odds ratio; SSC, surgical safety checklist.
FE4-20
eFigure 4:
Clinical outcome: urinary tract infection (UTI). OR, odds ratio; SSC, surgical safety checklist.
FE5-20
eFigure 5:
Clinical outcome: blood loss requiring transfusion. OR, odds ratio; SSC, surgical safety checklist.
FE6-20
eFigure 6:
Clinical outcome: unplanned intubation. OR, odds ratio; SSC, surgical safety checklist.
FE7-20
eFigure 7:
Clinical outcome: sepsis. OR, odds ratio; SSC, surgical safety checklist.
F2-20
Figure 2:
Forest plot: surgical site infection (SSI). OR, odds ratio; SSC, surgical safety checklist.
F3-20
Figure 3:
Forest plot: mortality. OR, odds ratio; SSC, surgical safety checklist.

Eighteen studies reported data on mortality, including a total of 373,385 patients. Pooled analysis revealed that 1,587 of 187,196 (0.8%) patients in the intervention group compared to 1,853 of 186,189 (1.0%) patients in the control group died. A meta-analysis (Fig. 3) revealed that WHO SSC use was associated with a significant reduction of deaths [odds ratio (OR) 0.82 (0.72–0.94), Z = -2.90 (p = 0.004)] with a moderate level of heterogeneity across trials (I2 = 44%). The most significant impact of the checklist was seen in a composite measure of morbidity [OR 0.60 (0.49–0.74), Z = -4.88 (p < 0.001)].

Postoperative outcomes that do not favor use of the WHO SSC

Deep vein thrombosis (DVT) was the only postoperative outcome assessed that did not favor use of the WHO SSC. Five studies reported DVT occurrence data. Pooled analysis revealed that 81 of 121,625 (0.07%) patients in the intervention group, compared with 53 of 126,762 (0.04%) patients in the control group, experienced DVT. A meta-analysis (Fig. 4) revealed that WHO SSC use was associated with a significant increase of DVT cases [OR 2.00 (1.37–2.92), Z = 3.61 (p < 0.001)]; there was a low heterogeneity across trials (I2 = 0%).

F4-20
Figure 4:
Forest plot: deep vein thrombosis (DVT). OR, odds ratio; SSC, surgical safety checklist.

Postoperative outcomes on which the WHO SSC had no impact

We found that the WHO SCC had no impact on the remaining 8 postoperative outcomes evaluated in the studies we included—renal complications (eFig. 8), ventilation >48 hours (eFig. 9), cardiac arrest (eFig. 10), myocardial infarction (eFig. 11), stroke/cerebrovascular accident (eFig. 12), pulmonary embolism (eFig. 13), DVT combined with pulmonary embolism and embolism (eFig. 14), and septic shock (eFig. 15). For example, 5 studies reported rates of ventilation greater than 48 hours, and included 224,090 patients. Pooled analysis revealed that 154 of 109,574 (0.14%) patients in the intervention group compared with 198 of 114,516 (0.17%) patients in the control group were ventilated for more than 48 hours. A meta-analysis revealed that the WHO SCC had no significant impact on prolonged ventilation rates [OR 0.72 (0.36–1.44), Z = -0.94 (p = 0.349)]; there was a moderate level of heterogeneity between trials (I2 = 73%). With the exception of renal complications, postoperative outcomes for which the WHO SSC had no impact included data from 5 or fewer studies.

FE8-20
eFigure 8:
Clinical outcome: renal complication. OR, odds ratio; SSC, surgical safety checklist.
FE9-20
eFigure 9:
Clinical outcome: ventilation >48 hours. OR, odds ratio; SSC, surgical safety checklist.
FE10-20
eFigure 10:
Clinical outcome: cardiac arrest. OR, odds ratio; SSC, surgical safety checklist.
FE11-20
eFigure 11:
Clinical outcome: myocardial infarction. OR, odds ratio; SSC, surgical safety checklist.
FE12-20
eFigure 12:
Clinical outcome: stroke/cerebrovascular accident (CVA). OR, odds ratio; SSC, surgical safety checklist.
FE13-20
eFigure 13:
Clinical outcome: pulmonary embolism (PE). OR, odds ratio; SSC, surgical safety checklist.
FE14-20
eFigure 14:
Clinical outcome: deep vein thrombosis (DVT), pulmonary embolism (PE), and embolism (combined). OR, odds ratio; SSC, surgical safety checklist.
FE15-20
eFigure 15:
Clinical outcome: septic shock. OR, odds ratio; SSC, surgical safety checklist.

Risk of bias across studies (publication bias)

Visual evaluation of funnel plots and Egger’s tests for mortality (p = 0.453; Fig. 5), SSI (p = 0.077), pneumonia (p = 0.118), and unplanned return to the operating room (p = 0.057) showed no signs of asymmetry, indicating no presence of publication bias. A funnel plot and significant Egger’s test for morbidity (p = 0.004) showed signs of asymmetry, indicating a potential presence of publication bias. Funnel plots for all other outcomes—UTI, blood loss, unplanned intubation, sepsis, DVT, renal complications, ventilation >48 hours, cardiac arrest, myocardial infarction, stroke/cerebrovascular accident, pulmonary embolism, septic shock, and DVT combined with pulmonary embolism, and embolism were not conducted due to a small number of studies (n < 10) included for meta-analyses.

F5-20
Figure 5:
Funnel plot of standard error by log odds ratio (observed values) for mortality.

Thematic analysis

Narrative information from the original studies included in the 20 systematic reviews19-38 used in this study were deductively coded in 4 primary areas of interest to the investigators: impact of the WHO SSC; barriers and facilitators to checklist adoption, implementation, or use; operational issues and compliance with checklist use; and local modifications made to the checklist. In this article, we focus only on the first area—studies that explore checklist impact on outcomes related to the patient, healthcare team, and health system.

Subthemes under the umbrella theme of impact were identified inductively, from the data, and deductively, based on previous work.39 These included: Clinical Outcomes, Process Measures (eg time tracking), Effects on Team Dynamics and Communication, Influence of or on Safety culture, Efficiency and Workload Involved in Using the Checklist, and Checklist Impact on Institutional Practices. The number of positive, neutral, and negative items coded in these themes and related subthemes are shown in Figure 6; Table 4 shows the key findings for each impact theme, as well as the designs of the studies from which the narrative data were obtained and the relationship of the impact findings to the WHO SSC.

F6-20
Figure 6:
Themes, subthemes, and frequencies of positive, neutral, and negative impacts of the WHO Surgical Safety Checklist.
T4A-20
Table 4:
Impact of WHO Surgical Safety Checklist: Key Findings and Study Design
T4B-20
Table 4:
Continued.
T4C-20
Table 4:
Continued.

Overall impact

Within the publications, more than twice as many items pertaining to the impact of the WHO SSC were positive (n = 58), as compared to neutral (n = 24) or negative (n = 23). A 1-way chi-square goodness of fit test revealed that the direction of the qualitative themes for overall impact of the SSC was statistically more positive, X2 (2, N = 105) = 22.686, p < 0.001.

Themes associated with specific aims of SSC

Four of the 6 themes pertained directly to the aims or purposes behind the development of the SSC3: Clinical Outcomes, Process Measures, Team Dynamics and Communication, and Safety Culture. In all 4 themes, narrative information described the SSC as having overwhelming positive effects rather than neutral or negative effects.

Under the Clinical Outcomes theme, the SSC was credited with greatly reducing mortality, morbidity, and never events40 (eg wrong patient, wrong site surgery, and retained foreign objects), X2 (2, N = 61) = 28.557, p < 0.001 (Table 4). However, some studies included in the systematic reviews we analyzed found no statistically significant reductions in mortality and morbidity, though these were few in number.

Process measures in the operating room are closely associated with surgical safety. In this study, we found mostly positive (n = 15), but a few neutral (n = 2) and negative (n = 2) reports of the WHO SSC on clinical processes, such as decreased nontechnical surgical errors and increased situation awareness and knowledge about patients, X2 (2, N = 19) = 17.789, p < 0.001.

Specifically, team members reported that the SSC provided brief pertinent information about patient’s history and risk, as well as the required procedure, increasing overall situational awareness.32(p.44)

Our thematic analysis also found overwhelmingly positive reports (Table 4) of the checklist on Team Dynamics and Communication within the operating room, X2 (2, N = 24) = 42.250, p < 0.001. The checklist was reported to improve both teamwork and communication, as its developers intended.

Finally, more than twice as many items reported a positive effect of the SCC on Safety Culture in the operating room (n = 13), as compared with negative effects (n = 5); X2 (2, N = 19) = 11.789, p = 0.003. Interestingly, some who reported negative cultural effects (n = 5) felt the use of the SSC might actually widen professional hierarchies and accentuate power differentials among operating room staff—the opposite effect that SSC developers intended.

Furthermore, if OR staff place differing importance to SSC adherence, SSC completion might actually antagonize team relationships/interactions and widen pre-existing power differentials.”32(p.45)

Importantly in some studies, SSC implementation did not mitigate the professional hierarchy, but can actually accentuate the power differential due to its perceived ‘staged’ nature.32(p.45)

Themes not directly associated with SSC aims

The other 2 themes we identified—Efficiency and Workload involved in using the checklist and Checklist Impact on Institutional Practices—are associated with SSC use, but were not focal areas considered during its development.

In contrast with the findings of the 4 themes addressed in the SSC’s development, more negative (n = 11) than positive (n = 5) items related to Efficiency in the operating room and Perceived Workload were found; this difference was statistically significant, X2 (2, N = 17) = 8.941, p = 0.011. For example, individuals felt that introduction of the checklist was disruptive to established work routines and did not fit into workload flow, especially during emergency operations. Additionally, some found the checklist to duplicate existing safety procedures.

The introduction of a checklist in the operating theatre involves doctors and nurses changing existing routines. Workflows on the individual, professional, or team level have to be altered and aligned in order to create a brief moment of reflection to review the safety checks collectively. The workflow introduced by the checklist often collides with existing routines. This creates conflicts as existing workflows are established in function of different priorities (e.g., efficiency or productivity).21(p.6)

The last theme—Checklist Impact on Institutional Practices—was developed deductively based on reviews of the larger SSC literature, but no items pertaining to this theme were identified in the systematic reviews included in this study.

Discussion

The aim of the WHO SCC is to reduce the frequency of surgical morbidities and mortality globally by “reinforcing accepted safety practices and fostering better communication between clinical disciplines.”3 Our systematic review and meta-analysis analyzed impacts and clinical outcomes associated with WHO SSC use through a thematic analysis and meta-meta-analyses. Our principal findings can be summarized as: the WHO SSC positively impacts the things it was explicitly designed to address and does not positively impact the things it was not explicitly designed for.

The thematic analysis revealed 4 themes that are emphasized by the authors—clinical outcomes, process measures, team dynamics and communication, and safety culture—in which the direction of the narrative of the overall impact of the WHO SSC was positive; these are directly related to the original aims of the WHO SSC. Our meta-meta-analysis directly supports these findings and the primary goal of the WHO SSC to decrease surgical complications and mortality. Not surprisingly, postoperative outcomes that favored the use of the WHO SSC were addressed by specific items within the checklist (eg SSI, unplanned intubation). All infectious outcomes (ie SSI, pneumonia, UTI, and sepsis), with the exception of septic shock, favored the use of the WHO SSC. This is unsurprising because the WHO SSC contains a check for antibiotic prophylaxis administration. Conversely, postoperative outcomes that did not favor the use of WHO SSC (DVT) or where the WHO SSC had no impact (renal complications) were conditions for which no specific items appeared in the SSC. Surprisingly, a majority of these items are both directly and indirectly related to cardiovascular functioning (eg cardiac arrest, myocardial infarction, stroke/cerebrovascular accident, and pulmonary embolism). These findings suggest that content—the specific items included in the checklist—does matter; it is not just the change in culture induced through checklist use that influences positive surgical outcomes.

Our thematic analysis also revealed another theme—efficiency and workload—in which the direction of the narrative was negative. For example, checklist users often felt that the checklist slowed down processes within the operating room. This effect is common when a new quality improvement program is initiated and workflow must be adjusted to accommodate. This concern, however, also indicates that users may not be aware that the WHO SSC is actually intended to slow down processes in the operating room—the very notion of a “time out” is to set aside other tasks to focus attention on something else, in this case, addressing items and issues of concern related to patient safety and surgical outcomes.

Based on the studies included for analysis, it is evident that there is widespread use of the WHO SSC across countries with both low and high human development indices. The findings from our thematic analysis were mostly positive. Neutral and negative findings (eg efficiency and workload) indicate that there is room for improvement in training and implementation of the WHO SSC. The content of the SSC may also need to be reconsidered in some types of circumstances, such as emergency operations, in which some of the patient data (eg allergies, comorbidities) may not be known. Furthermore, the urgent or emergent nature of emergency conditions may take greater precedence over pausing for a “time out” to conduct the checklist.

The majority of clinical outcomes analyzed in our meta-analyses also favor the use of the WHO SSC. Though these findings were significant, in most outcomes, the pooled effect comes considerably close to the line of no effect (eg Fig. 2); we had expected to see a more robust effect in favor of the WHO SSC.

To our knowledge, this study is the first meta-meta-analysis consisting of a thematic analysis and meta-analysis of the WHO SSC. Consistent findings discovered through both qualitative and quantitative analyses indicate that faith can be placed in the robustness of this study’s results.

There are, however, some limitations to this study. There is moderate to high heterogeneity in the pooled results, which could limit the generalizability of our findings. However, some degree of heterogeneity is always anticipated in meta-analyses and can reflect differences in study populations, methods of outcome measurement, analytic methods, and numerous other factors.

Many of the original studies included in the systematic reviews analyzed are relatively old at this point and were conducted not long after the WHO SSC was first introduced. Furthermore, the effectiveness of the checklist in developing or low-resource countries, as compared to in developed or high-resource countries, is an important consideration, but beyond the scope of this study.

Conclusions

Although largely positive, there was some variability in impact and outcome findings in both the thematic analysis and meta-analyses. The variability across outcomes implies that there is no single determinant for SSC success; rather, there are multiple determinants that affect checklist impact and outcomes, such as the specific content included, the way it is implemented, its training, and how it is used in practice. Our findings suggest that users may need to revisit their WHO SSC to determine if there is a need for modification and changes to implementation strategies and methods for ensuring compliance and its use as intended. The overall purposes and goals of the WHO SSC must remain at the forefront during these processes; however, improving operating room team communication and reducing errors leading to patient mortality or morbidity are critical.

Author Contributions

Study conception and design: Sotto, Burian, Brindle

Acquisition of data: Sotto

Analysis and interpretation of data: Sotto, Burian, Brindle

Drafting of manuscript: Sotto, Burian, Brindle

Critical revision: Sotto, Burian, Brindle

Acknowledgement

This study is part of a research program aiming to optimize the performance of the Surgical Safety Checklist, funded by the Canadian Institute for Health Research.

References

1. Nepogodiev D Nepogodiev, Martin J Martin, Biccard B Biccard, et al. Global burden of postoperative death. Lancet. 2019;393:401.
2. Schwendimann R Schwendimann, Blatter C Blatter, Dhaini S Dhaini, et al. The occurrence, types, consequences and preventability of in-hospital adverse events – a scoping review. BMC Health Serv Res. 2018;18:521.
3. World Health Organization. Implementation Manual WHO Surgical Safety Checklist 2009: Safe Surgery Saves Lives. 2009, World Health Organization.
4. Haynes AB Haynes, Weiser TG Weiser, Berry WR Berry, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491-499.
5. Weiser TG Weiser, Haynes AB Haynes, Dziekan G Dziekan, et al. Effect of a 19-item surgical safety checklist during urgent operations in a global patient population. Ann Surg. 2010;251:976-980.
6. Prakash P Prakash, Baduni N Baduni, Sanwal MK Sanwal, et al. Effect of World Health Organization surgical safety checklist on patient outcomes in a tertiary care hospital of Delhi. Int Med J. 2014;21:376-378.
7. White MC White, Randall K Randall, Ravelojaona VA Ravelojaona, et al. Sustainability of using the WHO surgical safety checklist: a mixed-methods longitudinal evaluation following a nationwide blended educational implementation strategy in Madagascar. BMJ Glob Health. 3, 2018, e001104
8. Urbach DR Urbach, Dimick JB Dimick, Haynes AB Haynes, Gawande AA Gawande. Is WHO’s surgical safety checklist being hyped? BMJ. 2019;366:l4700.
9. Haugen AS Haugen, Sφfteland E Sφfteland, Almeland SK Almeland, et al. Effect of the World Health Organization checklist on patient outcomes: a stepped wedge cluster randomized controlled trial. Ann Surg. 2015;261:821-828.
10. Urbach DR Urbach, Govindarajan A Govindarajan, Saskin R Saskin, et al. Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370:1029-1038.
11. van Klei WA van Klei, Hoff RG Hoff, van Aarnhem EE van Aarnhem, et al. Effects of the introduction of the WHO "Surgical Safety Checklist" on in-hospital mortality: a cohort study. Ann Surg. 2012;255:44-49.
12. Delisle M Delisle, Pradarelli JC Pradarelli, Panda N Panda, et al. Variation in global uptake of the Surgical Safety Checklist. Br J Surg. 2020;107:e151-e160.
13. Buissonniere M Buissonniere. Checking in on the checklist: uptake, impact, and opportunities for the next decade. Available at: https://www.lifebox.org/wp-content/uploads/2020/01/Checking-In-On-the-Checklist-web.pdf Accessed 1-12-2020
14. Braun V Braun, Clarke V Clarke. Using thematic analysis in psychology. Qual Res Psychol. 2006;3:77-101.
15. Moher D Moher, Liberati A Liberati, Tetzlaff J Tetzlaff, Altman DG Altman. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009;339:b2535.
16. National Heart, Lung, and Blood Institute, National Institutes of Health, US Department of Health and Human Services. Quality Assessment of Systematic Reviews and Meta-Analyses. Available at: https://www.nhlbi.nih.gov/health-topics/study-quality-assessment-tools Accessed 10-8-2021
17. Higgins JP Higgins, Thompson SG Thompson, Deeks JJ Deeks, Altman DG Altman. Measuring inconsistency in meta-analyses. BMJ. 2003;327:557-560.
18. Higgins J Higgins, Thomas J Thomas, Chandler J Chandler, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.2 (updated February 2021). Cochrane. Available at: https://training.cochrane.org/handbook/current/chapter-10 2021. Accessed 10-8-2021
19. Abbott TEF Abbott, Ahmad T Ahmad, Phull MK Phull, et al. The surgical safety checklist and patient outcomes after surgery: a prospective observational cohort study, systematic review and meta-analysis. Br J Anaesth. 2018;120:146-155.
20. Bergs J Bergs, Hellings J Hellings, Cleemput I Cleemput, et al. Systematic review and meta-analysis of the effect of the World Health Organization surgical safety checklist on postoperative complications. Br J Surg. 2014;101:150-158.
21. Bergs J Bergs, Lambrechts F Lambrechts, Simons P Simons, et al. Barriers and facilitators related to the implementation of surgical safety checklists: a systematic review of the qualitative evidence. BMJ Qual Saf. 2015;24:776-786.
22. Biccard BM Biccard, Rodseth R Rodseth, Cronje L Cronje, et al. A meta-analysis of the efficacy of preoperative surgical safety checklists to improve perioperative outcomes. S Afr Med J 2016; 106.
23. Borchard A Borchard, Schwappach DL Schwappach, Barbir A Barbir, Bezzola P Bezzola. A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Ann Surg. 2012;256:925-933.
24. Boyd J Boyd, Wu G Wu, Stelfox H Stelfox. The impact of checklists on inpatient safety outcomes: a systematic review of randomized controlled trials. J Hosp Med. 2017;12:675-682.
25. de Jager E de Jager, McKenna C McKenna, Bartlett L Bartlett, et al. Postoperative adverse events inconsistently improved by the World Health Organization Surgical Safety Checklist: A systematic literature review of 25 studies. World J Surg. 2016;40:1842-1858.
26. Gillespie BM Gillespie, Chaboyer W Chaboyer, Thalib L Thalib, et al. Effect of using a safety checklist on patient complications after surgery: a systematic review and meta-analysis. Anesthesiology. 2014;120:1380-1389.
27. Lagoo J Lagoo, Lopushinsky SR Lopushinsky, Haynes AB Haynes, et al. Effectiveness and meaningful use of paediatric surgical safety checklists and their implementation strategies: a systematic review with narrative synthesis. BMJ Open. 7, 2017, e016298
28. Patel J Patel, Ahmed K Ahmed, Guru KA Guru, et al. An overview of the use and implementation of checklists in surgical specialities - a systematic review. Int J Surg. 2014;12:1317-1323.
29. Russ S Russ, Rout S Rout, Sevdalis N Sevdalis, et al. Do safety checklists improve teamwork and communication in the operating room? A systematic review. Ann Surg. 2013;258:856-871.
30. Tang R Tang, Ranmuthugala G Ranmuthugala, Cunningham F Cunningham. Surgical safety checklists: a review. ANZ J Surg. 2014;84:148-154.
31. Thomassen O Thomassen, Storesund A Storesund, Softeland E Softeland, Brattebo G Brattebo. The effects of safety checklists in medicine: a systematic review. Acta Anaesthesiol Scand. 2014;58:5-18.
32. Wangoo L Wangoo, Ray RA Ray, Ho Y-H Ho. Compliance and surgical team perceptions of WHO Surgical Safety Checklist; systematic review. Int Surg. 2016;101:35-49.
33. White MC White, Peven K Peven, Clancy O Clancy, et al. Implementation strategies and the uptake of the World Health Organisation Surgical Safety Checklist in low and middle income countries: a systematic review and meta-analysis. Ann Surg. 2020;273:e196-e205.
34. Lau CSM Lau, Chamberlain RS Chamberlain. The World Health Organization Surgical Safety Checklist improves post-operative outcomes: A meta-analysis and systematic review. Surg Sci. 2016;7:12-65946.
35. Mafra C Mafra, Soares C Soares. Surgical safety checklist: An integrative review of the benefits and importance. Revista de Pesquisa: Cuidado é Fundamental Online. 2018;10:268.
36. McDowell DS McDowell, McComb SA McComb. Safety checklist briefings: a systematic review of the literature. AORN J. 2014;99:125-137.
37. Treadwell JR Treadwell, Lucas S Lucas, Tsou AY Tsou. Surgical checklists: a systematic review of impacts and implementation. BMJ Qual Saf. 2014;23:299-318.
38. Westman M Westman, Takala R Takala, Rahi M Rahi, Ikonen TS Ikonen. The need for surgical safety checklists in neurosurgery now and in the future-a systematic review. World Neurosurg. 2020;134:614-628.e3.
39. Patel K Patel, Haynes AB Haynes, Hannenberg A Hannenberg, et al. 10 years of the Surgical Safety Checklist: Current perspectives from an international survey. J Am Coll Surg. 2020;231:e118.
40. NHS Improvement. Never Events List 2018. 2018, NHS Improvement.
41. Kwok AC Kwok, Funk LM Funk, Baltaga R Baltaga, et al. Implementation of the World Health Organization surgical safety checklist, including introduction of pulse oximetry, in a resource-limited setting. Ann Surg. 2013;257:633-639.
    42. Yuan CT Yuan, Walsh D Walsh, Tomarken JL Tomarken, et al. Incorporating the World Health Organization Surgical Safety Checklist into practice at two hospitals in Liberia. Jt Comm J Qual Patient Saf. 2012;38(6):254-260. http://dx.doi.org/10.1016/s1553-7250(12)38032-x.
      43. Lübbeke A Lübbeke, Hovaguimian F Hovaguimian, Wickboldt N Wickboldt, et al. Effectiveness of the surgical safety checklist in a high standard care environment. Med Care. 2013;51:425-429.
        44. Tillman M Tillman, Wehbe-Janek H Wehbe-Janek, Hodges B Hodges, et al. Surgical care improvement project and surgical site infections: can integration in the surgical safety checklist improve quality performance and clinical outcomes? J Surg Res. 2013;184:150-156.
          45. Boaz M Boaz, Bermant A Bermant, Ezri T Ezri, et al. Effect of surgical safety checklist implementation on the occurrence of postoperative complications in orthopedic patients. Isr Med Assoc J. 2014;16:20-25.
            46. Chaudhary N Chaudhary, Varma V Varma, Kapoor S Kapoor, et al. Implementation of a surgical safety checklist and postoperative outcomes: a prospective randomized controlled study. J Gastrointest Surg. 2015;19:935-942.
              47. Kim RY Kim, Kwakye G Kwakye, Kwok AC Kwok, et al. Sustainability and long-term effectiveness of the WHO surgical safety checklist combined with pulse oximetry in a resource-limited setting: two-year update from Moldova. JAMA Surg. 2015;150:473-479.
                48. Rodrigo-Rincon I Rodrigo-Rincon, Martin-Vizcaino MP Martin-Vizcaino, Tirapu-Leon B Tirapu-Leon, et al. The effects of surgical checklists on morbidity and mortality: a pre- and post-intervention study. Acta Anaesthesiol Scand. 2015;59:205-214.
                  49. Biskup N Biskup, Workman AD Workman, Kutzner E Kutzner, et al. Perioperative safety in plastic surgery: Is the World Health Organization checklist useful in a broad practice? Ann Plast Surg. 2016;76:550-555.
                    50. Lacassie HJ Lacassie, Ferdinand C Ferdinand, Guzmán S Guzmán, et al. World Health Organization (WHO) surgical safety checklist implementation and its impact on perioperative morbidity and mortality in an academic medical center in Chile. Medicine (Baltimore). 95, 2016, e3844
                      51. Mayer EK Mayer, Sevdalis N Sevdalis, Rout S Rout, et al. Surgical checklist implementation project: the impact of variable WHO checklist compliance on risk-adjusted clinical outcomes after national implementation: a longitudinal study. Ann Surg. 2016;263:58-63.
                        52. O’Leary JD O’Leary, Wijeysundera DN Wijeysundera, Crawford MW Crawford. Effect of surgical safety checklists on pediatric surgical complications in Ontario. CMAJ. 2016;188:e191-e198.
                          53. Askarian M Askarian, Kouchak F Kouchak, Palenik CJ Palenik. Effect of surgical safety checklists on postoperative morbidity and mortality rates, Shiraz, Faghihy Hospital, a 1-year study. Qual Manag Health Care. 2011;20:293-297.
                            54. Bliss LA Bliss, Ross-Richardson CB Ross-Richardson, Sanzari LJ Sanzari, et al. Thirty-day outcomes support implementation of a surgical safety checklist. J Am Coll Surg. 2012;215:766-776.
                              55. Lepänluoma M Lepänluoma, Takala R Takala, Kotkansalo A Kotkansalo, et al. Surgical safety checklist is associated with improved operating room safety culture, reduced wound complications, and unplanned readmissions in a pilot study in neurosurgery. Scand J Surg. 2013;103:66-72.
                                56. Toor AA Toor, Farooka MW Farooka, Ayyaz M Ayyaz, et al. Pre-operative antibiotic use reduces surgical site infection. J Pak Med Assoc. 2015;65:733-736.
                                  57. Westman M Westman, Marttila H Marttila, Rahi M Rahi, et al. Analysis of hospital infection register indicates that the implementation of WHO surgical safety checklist has an impact on early postoperative neurosurgical infections. J Clin Neurosci. 2018;53:188-192.
                                    58. Lepänluoma M Lepänluoma, Rahi M Rahi, Takala R Takala, et al. Analysis of neurosurgical reoperations: use of a surgical checklist and reduction of infection-related and preventable complication-related reoperations. J Neurosurg. 2015;123:145-152.
                                      © 2021 by Lippincott Williams & Wilkins, Inc.