Over the past 2 decades, following publication of the Institute of Medicine Report “To Err is Human,”1 health professionals have become increasingly interested in patient safety. Much progress has been made in improving perioperative quality and safety. Process improvement interventions such as checklists, national clinical audits with data feedback, adherence to evidenced care pathways, and multidisciplinary team training all improve perioperative outcomes.2,3 However, knowledge of how to implement evidenced interventions effectively, sustainably and at scale remains lacking. Therefore, the focus of patient safety science is shifting from addressing an evidence gap to an implementation gap—understanding how best to promote the uptake of clinically proven safety interventions into routine healthcare practice.
In low- and middle-income countries (LMICs), the imperative to close the implementation gap is even greater, since resources are limited, surgical quality and safety is poor, and outcomes remain significantly worse than in high-income countries (HICs).4–6 Understanding the importance of the implementation gap in patient safety led the 2019 Global Ministerial Patient Safety Summit (GMPSS) to declare that healthcare systems, especially in LMICs, must urgently focus on implementation strategies designed to reduce the gap if the momentum of the global patient safety movement is to be realized.7
Implementation of the World Health Organization (WHO) Surgical Safety Checklist (SSC) is a good example of an implementation gap waiting to be addressed. The SSC is a safety improvement intervention that has demonstrated substantial improvements in surgical outcomes.8–10 but poor implementation has caused limited effectiveness.11 Globally, widespread SSC implementation has met with mixed results. The WHO unsuccessfully attempted nationwide implementation of the SSC in 15 African countries in 2011.12 After 1 year, only 1 hospital in the 15 countries had managed to implement the SSC and none had started nationwide implementation. In the UK, after a nationwide implementation in 2009–2010, variable implementation success was reported which was attributed to different contextual barriers.13,14 A recent study by Delisle et al on worldwide uptake of the SSC based on data from 85,957 patients in 1464 facilities in 94 countries showed that on average, facilities used the SSC in 75% of operations.15 However, this global usage rate should be interpreted with caution as most of the data comes from HIC where SSC is much more common. The study is made up of 77% HIC, 22% middle income, and 5% low income country data, and subgroup analysis showed that SSC use in HICs was 88 to 89%, whereas use in low-income countries was less than 30%. Therefore, there is much work to be done in informing the knowledge gap in SSC implementation in LMICs and responding to the GMPSS call to focus on implementation strategies to reap the global benefits of well-evidenced patient safety improvements.
The goal of this systematic review is to identify the implementation strategies used in LMICs and examine any association of implementation strategies with implementation effectiveness and secondarily to evaluate the clinical impact of the SSC.
Search Strategy and Selection Criteria
This systematic review and meta-analysis was conducted according to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA). The review protocol was published on PROSPERO (CRD42018100034).
We used a predefined search strategy. The Cochrane library, MEDLINE, EMBASE, and PsycINFO databases were searched systematically for all publications between June 2008 (when the SSC was officially launched) and February 2019. The search strategy used a combination of the key word “surg∗” combined using the Boolean operator “AND” with any of the following key words, “surg∗ checklist∗,” “safe∗ checklist∗,” “WHO checklist∗.”
Hand-search was further carried out, of reference lists of previously published systematic reviews and of included studies for additional relevant references. There were no language restrictions. Translators translated studies in languages other than English and French.
Studies were included if they were original human studies reporting primary data on the introduction and/or impact of the WHO SSC on surgical care in LMICs. If studies included both HIC and LMIC data, they were included but only LMIC outcome data were extracted. Studies were excluded if they were reviews, other secondary data reports, HIC studies or reported on surgical checklists other than the SSC. This review was limited to the WHO SSC or adapted versions thereof rather than other surgical checklists. Other surgical checklists were excluded because we wanted to maintain a level of coherence in what was evaluated for both clinical and implementation effectiveness. The 2018 World Bank lending definitions were used to define LMICs and HICs (Supplementary_Material_Page_1, http://links.lww.com/SLA/C149).
After deduplication checks, 4 reviewers (MCW, SR, NS, AJML) screened titles and abstracts in duplicate for eligibility, and full texts of potentially relevant articles were retrieved.
In pairs, authors (OC, IO) independently extracted data and conducted a risk of bias assessment for each included study. Authors compared results and resolved disagreements by consensus with 2 other authors (KP, MCW). Data were extracted using a standardized, prepiloted form that included year, study country, study population, implementation strategies used, implementation outcomes, clinical outcomes, number of authors with LMIC affiliation, and risk of bias.
Risk of bias was assessed using the QualSyst tool,16 which is designed to assess both quantitative and qualitative studies, including observational studies. Mixed methods studies underwent both quantitative and qualitative assessments. Since a scoping search did not identify many randomized trials, we decided a priori that no study would be excluded based on study design or risk of bias. The focus of the review was on SSC implementation strategies used and outcomes reported. We used the study quality assessment to inform our analysis of the strength of evidence. Low quality may reflect poor study design or poor reporting, and since these 2 cannot be distinguished we made a judgment based on the information presented.
The coprimary objectives were to identify the implementation strategies used and to assess implementation effectiveness by examining the association between the strategies used and the implementation outcomes reported. Implementation strategies were assessed based on the Expert Recommendations for Implementing Change (ERIC) programme,17 which defines 73 individual strategies grouped into 9 domains.18 The 9 domains (and number of individual strategies identified per domain) are:
- 1. use of evaluative and iterative strategies (10)
- 2. provision of interactive assistance (4)
- 3. adapt and tailor to context (4)
- 4. development of stakeholder inter-relationships (17)
- 5. training and education of stakeholders (11)
- 6. support of clinicians (5)
- 7. engagement with consumers (5)
- 8. use of financial strategies (9)
- 9. change of infrastructure (8)
Further definitions of the 73 distinct strategies are shown in Supplementary_Material_Pages_2-5, http://links.lww.com/SLA/C149. The implementation outcomes assessed were 8 internationally recognized outcomes defined in the implementation science evidence-base19 and adopted by the WHO,20 which represent the current gold-standard for evaluating implementation effectiveness (Table 1). We further defined sustainability as at least 12 months having elapsed between the beginning of implementation and the end of the outcome evaluation. As implementation outcomes were only measured in intervention groups, no effect size measure could be calculated. The 2 main implementation outcomes were decided a priori as fidelity and penetration because from our previous work on checklist implementation we hypothesized that these would be the most commonly reported. Both fidelity and penetration were reported as percentages. Due to limited power to conduct regression analysis, we assessed the strength of the association between number of implementation strategies and percentage fidelity and penetration, with the use of descriptive graphs (scatterplots) and correlation coefficient values.
TABLE 1 -
Eight Gold-standard Implementation Outcomes as Defined by the Implementation Science Evidence Base19
and Adopted by the WHO20
||Perception among implementation stakeholders that a given evidence-based practice is agreeable or satisfactory
||Intention, initial decision, or action to try to employ an evidence-based practice
||Perceived fit, relevance, or compatibility of the evidence-based practice for a given practice setting, provider, or consumer; perceived fit to address problem
||Extent to which a new evidence-based practice can be successfully used or carried out within a given agency or setting
||Degree to which an evidence-based practice was implemented as it was prescribed in the original protocol or intended by the practice developers
||Cost impact of an implementation effort.
||Integration of a practice within a service setting and its subsystems.
||Extent to which a newly implemented evidence-based practice is maintained or institutionalized within a service setting's on going, stable operations (outside the context of a research study)
The secondary objective was to analyze the clinical effectiveness of the SSC by extracting data relating to clinical outcomes (this was an addition to the PROSPERO protocol). Morbidity and mortality were included if they were reported before and after implementation of the SSC or in an intervention control group such that a risk ratio could be calculated. Relative risks (RR) with 95% CIs from individual studies were weighed and combined to produce an overall pooled RR across the studies. A random effects model with the use of DerSimonian and Laird method was employed to account for heterogeneity between studies.21 The percentage of heterogeneity between studies was quantified by the I-squared statistic (I2) and evidence of heterogeneity was tested with a chi-squared test. Each study was weighed and the RRs and 95% CIs were displayed in forest plots, along with the overall pooled overall effect estimate. Publication bias was assessed through funnel plots.
Where authors had multiple affiliations, if at least one was an LMIC affiliation, we counted the LMIC affiliation. If a paper did not specify a HIC or LMIC affiliation, we hand-searched for other studies by the same author to identify the affiliation.
All statistical analyses were conducted with R, version 3.5.0.
Role of the Funding Source
The funders of the study had no role in study design, data collection, data analysis, data interpretation, or writing of the report. The corresponding author had full access to all the data in the study and final responsibility for the decision to submit for publication.
We screened 1562 citations and included 47 papers reporting 42 unique studies in the final analysis.8,22–67 Details of the selection process are shown in Figure 1. Of the 47 papers, 39 were quantitative studies, 4 qualitative and 4 mixed methods. Median quality score for quantitative studies was 22 out of a maximum score of 28 (IQR: 18–22; range: 14–28), and for qualitative studies 19.5 out of a maximum score of 20 (IQR: 14–20; range: 5–20).
Characteristics of included studies are summarized in (Supplementary_Material_Page_6-13, http://links.lww.com/SLA/C149). Most (n = 34) were observational studies, of which 18 used an uncontrolled pre/post intervention design. Studies covered SSC implementation in 26 LMICs (Supplementary_Material_Page_14, http://links.lww.com/SLA/C149). Fourteen countries reported data from a single study in a single paper, whereas other countries, such as India and Brazil, reported multiple reports for some or all of their studies. Thirty-five studies (74%) had a first author with an LMIC affiliation, and 38 studies (81%) had either a first or last author with LMIC affiliation.
Included studies were conducted in a variety of settings, most commonly in 1 hospital or surgical service, although 5 papers report 3 national-level implementations in Madagascar, Benin, and Thailand.29,40,60,62,63 Hospitals ranged from large referral hospitals with multiple operating rooms performing over 21,000 procedures per year,39 to studies limited to a single service such as otorhinolaryngology in a single facility in India.30
The spread of ERIC strategies used across included studies is shown in Figure 2. The most commonly used strategy domains were “train and educate stakeholders,” “adapt and tailor to context,” “provide interactive assistance,” “develop stakeholder relationships,” and “support clinicians.” No strategies in the “engage consumers” domain were used and only 1 study used strategies from the “financial strategies” domain.
Further analysis of the reported implementation strategies demonstrated that the median number of strategies used per study was 4 (IQR: 1–14, range 0–28). Twenty-eight of the 73 ERIC-defined strategies were not reported in any study. The most frequently reported individual strategy was “conduct educational meetings,” described in 27 studies, usually as one of a series of meetings or training sessions to introduce the SSC. “Promote adaptability” and “make training dynamic” were each described in 20 studies. The manner in which the “promote adaptability” strategy was used varied widely. For example, Melekie et al reported that a multidisciplinary team adapted the SSC to local circumstances, whereas Bashford et al described a 5-month multidisciplinary consultation process of adapting, reviewing, revising until consensus was achieved.26,48 Twelve studies did not describe any implementation strategies.
The most commonly reported implementation outcomes19 were fidelity (31/47 studies) and penetration (27/47 studies). No study reported the cost of implementation (Fig. 3). Penetration ranged from 44 to 100% with a mean of 81% (SD 20%). Fidelity was most often measured as the proportion of 6 basic safety procedures (verification of patient identity, assessment of risk of difficult intubation, assessment of the risk of major blood loss, use of a pulse oximeter, appropriate administration of antibiotics, and completion of the surgical count) that were completed (28/47). Mean fidelity was 85% (SD 13%) but varied across the safety processes measured. Pulse oximeter use had the highest fidelity, whereas assessment of the risk of major blood loss had the lowest fidelity. Measurements of fidelity for nontechnical skills, such as teamwork and communication, were rare.
The median length of time between implementation and outcome evaluation was 7 months (IQR: 4–22, range 0–64 mo (Supplementary_Material_Page_15, http://links.lww.com/SLA/C149). Twenty studies reported on sustainability.
For studies reporting at least 1 implementation strategy (35/47), we did not detect an association between strategy domain used and implementation effectiveness (quantitatively assessed as either SSC penetration or fidelity of SSC implementation). Scatter plots and correlation coefficients are shown in Supplementary_Material_Pages_16-17, http://links.lww.com/SLA/C149.
The associations of the SSC with mortality and morbidity are presented in Figures 4–6. When an overall pooled effect was estimated from a random effects model, there was a negative association between the SSC and mortality (n = 8; RR 0.77; 95% CI 0.76–0.89; I2 0%) (Fig. 4). Funnel plots showed little evidence of asymmetry and publication bias (Supplementary_Material_Pages_18-20, http://links.lww.com/SLA/C149). When combining all the effects of the studies that investigated morbidity from infectious complications as an outcome or when a result for “all complications” was provided, the overall direction remained the same. Overall, SSC implementation was associated with a 44% decrease in risk of all complications (n = 9; RR 0.56; 95% CI 0.45–0.71; I2 77.8%) and a 53% decrease in risk of infectious complications (n = 12; RR 0.47; 95% CI 0.40–0.55; I2 42.2%) (Figs. 5 and 6).
This is the first systematic review evaluating SSC implementation strategies, implementation effectiveness, and clinical impact in LMICs. We did not detect evidence of an association between the implementation strategies used to introduce the SSC in LMICs and the effectiveness of implementation as assessed by the fidelity of SSC use or the level of penetration. Our meta-analysis shows that SSC use is associated with reductions in: mortality by 23%, infectious complications by 56% and any complications by 44%, although the number of included studies was only small (n = 9). The pattern of clinical effectiveness results in our study replicates other reviews of the impact of the SSC in HICs, which report associated reductions in mortality and morbidity of 23 to 51% and 27 to 41%, respectively.9,10,68
We identified no implementation strategies in a quarter (12/47) of included studies, and the median number reported was only 4. This finding may explain why we did not identify a statistical relationship between strategies used and implementation or clinical effectiveness. It may also reflect the fact that the implementation of the SSC (in both HICs and LMICs) is often left to frontline clinical staff that traditionally have little, if any, training in implementation methods, and that many journals and lay media have traditionally placed more emphasis on clinical outcomes compared with implementation. This explanation is supported by a recent Health Policy paper69 and review,70 which suggest that the lack of implementation science methodology represents a neglected opportunity to bridge the gap between research, policy, and practice. This knowledge gap in implementation exists in both high- (HIC) and low- and middle-income countries (LMIC). A recent high-profile example in a HIC context is the Enhanced Peri-Operative Care for High-risk patients (EPOCH) trial.71 The EPOCH trial attempted to implement a multimodal care pathway for patients undergoing emergency laparotomy across 93 United Kingdom (UK) hospitals, but was unable to demonstrate any improvement in survival or length of hospital stay. The null outcome was attributed to challenges in implementation, such as under-estimating the local context and under-estimating the social aspects of change.72 In 2013, a similar null outcome was reported for the “Matching Michigan” national improvement which aimed to reduce central venous catheter infections across 223 UK intensive care units.73,74 Both examples used an intervention with a strong evidence base, but failed to effectively implement the intervention at scale. Therefore if the declarations of the 2019 GMPSS are to be realized, one of which was an urgent call to focus on implementation strategies in LMICs,7 further application of implementation science methodology is required among the global surgical community.
Although this review could not find an association between strategy and outcome in SSC implementation, this does not mean that one might not exist. More likely lack of evidence of an association was due to limited power in conducting this type of analysis. The most frequently used strategies were chosen from the following 5 domains: “train and educate stakeholders,” “adapt and tailor to context,” “provide interactive assistance,” “develop stakeholder relationships,” and “support clinicians.” Our interpretation of this finding is that the use of relatively few implementation strategies from these 5 domains is important for successful SSC implementation. This interpretation is supported by 2 further systematic reviews on implementing healthcare interventions. Firstly, Rowe et al75 reported a review of implementation strategies to improve healthcare workers’ practices in LMICs. While large numbers of multifaceted strategies had large effects, they were not always more effective than simpler ones. Simply combining training and supervision, with tools such as group problem solving, significantly increased the chance of successful implementation. Second, a Cochrane review76 on implementation strategies showed that the use of educational meetings; tailored interventions; practice facilitation; local opinion leaders; and audit and feedback were all effective in changing healthcare workers’ behavior. Therefore, the overall body of evidence suggests that specific tailored strategies chosen from the 5 domains highlighted by this review can assist SSC implementation in LMICs.
This review has some limitations. We did not exclude any study on the basis of quality and therefore several potential biasing and confounding elements must be considered. Although our review showed high fidelity (85%), the fidelity measures used focussed mainly on adherence to process. Only 2/47 studies used validated measures for nontechnical skills such as teamwork and communication that are thought to partially account for the improved surgical outcomes associated with SSC use. Therefore while overall fidelity was high, this does not necessarily equate to appropriate use of the SSC and should be interpreted with caution. There was considerable heterogeneity of included studies especially in those reporting “all complications” (I2 = 77.8%). The range of methodologies used, the reporting of the implementation strategies, and the reporting of implementation and clinical outcomes, varied widely. There was likely under-reporting of the implementation strategies used, which limited the power of the study to evaluate the correlation between strategy and outcome. This also prevented meta-regression being performed to account for some of the heterogeneity. It is possible that the search may have inadvertently missed WHO-inspired checklists that were not reported in the literature as such.
This review also has a number of strengths. This is the first systematic review to examine in detail the methodology of SSC implementation in LMICs. It is also the first meta-analysis of the clinical impact of SSC use in exclusively LMICs and shows that the SSC is associated with a reduction in surgical mortality and morbidity. This means that despite the lack of resources in LMICs, the SSC is still effective, and therefore improving our understanding of how best to scale up implementation across all LMICs will be critical to improving surgical safety for those who need it most.
The evidence from our review suggests that a few specific tailored strategies chosen from 5 commonly used domains can assist SSC implementation: “train and educate stakeholders,” “adapt and tailor to context,” “provide interactive assistance,” “develop stakeholder relationships,” and “support clinicians.” Therefore, we hypothesize that these 5 strategies might be best to employ first in future SSC implementation efforts. Other areas for future study include: the involvement of patients in implementing and sustaining SSC use, since no studies in our review used any strategy from the “engage consumers” domain; and implementation cost that was not reported by any included study. As LMICs develop National Surgical Obstetric and Anaesthesia Plans (NSOAPs),77 the cost-effectiveness of one intervention over another will be important for identifying NSOAP funding and implementation priorities.
In summary, our review shows that SSC use in LMICs is associated with a reduction in the risk of mortality, infectious, and all complications by 23%, 56%, and 44% respectively. While we cannot make specific recommendations about the currently best method of SSC implementation in LMICs, we do recommend that clinicians and policy-makers undertaking SSC implementation commit to report their implementation strategies and outcomes adequately so that the global surgical community can close the knowledge gap on SSC implementation in LMICs.
1. Institute of Medicine. To Err Is Human: Building a Safer Health System, Kohn LT, Corrigan JM, Donaldson MS editors Washington, DC, The National Academies Press, 2000.
2. Howell AM, Panesar SS, Burns EM, et al. Reducing the burden of surgical harm: a systematic review of the interventions used to reduce adverse events in surgery. Ann Surg
3. Montroy J, Breau RH, Cnossen S, et al. Change in adverse events after enrollment in the national surgical quality improvement program: a systematic review and meta-analysis. PLoS One
4. Biccard BM, Madiba TE, Kluyts HL, et al. Perioperative patient outcomes in the African Surgical Outcomes Study: a 7-day prospective observational cohort study. Lancet
5. GlobalSurg CollaborativeSurgical site infection after gastrointestinal surgery in high-income, middle-income, and low-income countries: a prospective, international, multicentre cohort study. Lancet Infect Dis
6. GlobalSurg CollaborativeMortality of emergency abdominal surgery in high-, middle- and low-income countries. Br J Surg
7. Global Ministerial Summit on Patient Safety
. Jeddah Declaration on Patient Safety
, Jeddah, Saudi Arabia, Global Ministerial Summit on Patient Safety
8. Haynes AB, Weiser TG, Berry WR, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med
9. Bergs J, Hellings J, Cleemput I, et al. Systematic review and meta-analysis of the effect of the World Health Organization surgical safety checklist on postoperative complications. Br J Surg
10. Borchard A, Schwappach DL, Barbir A, et al. A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Ann Surg
11. Urbach DR, Govindarajan A, Saskin R, et al. Introduction of surgical safety checklists in Ontario. Canada N Engl J Med
12. Kariyoi PC, Highteoweri J, Ndihokubwayo JB, et al. Challenges facing the introduction of the WHO surgical safety checklist: a short experience in African countries. African Health Monitor
13. Russ S, Rout S, Caris J, et al. Measuring variation in use of the WHO surgical safety checklist in the operating room: a multicenter prospective cross-sectional study. J Am Coll Surg
2015; 220: 1–11.e4.
14. Russ SJ, Sevdalis N, Moorthy K, et al. A qualitative evaluation of the barriers and facilitators toward implementation of the WHO surgical safety checklist across hospitals in England: lessons from the “Surgical Checklist Implementation Project. Ann Surg
15. Delisle M, Pradarelli JC, Panda N, et al. Variation in global uptake of the Surgical Safety Checklist. Br J Surg
16. Kmet LM, Lee RC, Cook LS. Standard Quality Assessment Criteria for Evaluating Primary Research Papers from a Variety of Fields (QualSyst), Alberta Heritage Foundation for Medical Research, 2004.
17. Powell BJ, Waltz TJ, Chinman MJ, et al. A refined compilation of implementation strategies: results from the Expert Recommendations for Implementing Change (ERIC) project Implementation science: IS 2015; 10; 21.
18. Waltz TJ, Powell BJ, Matthieu MM, et al. Use of concept mapping to characterize relationships among implementation strategies and assess their feasibility and importance: results from the Expert Recommendations for Implementing Change (ERIC) study. Implement Sci
19. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health
20. Peters DH, Tran NT, Adam T Implementation Research in Healthcare: A practical Guide, World Health Organization, 2013.
21. DerSimonian R, Laird N. Meta-analysis in clinical trials. Controll Clin Trials
22. Ambulkar R, Ranganathan P, Salunke K, et al. The World Health Organization Surgical Safety Checklist: an audit of quality of implementation at a tertiary care high volume cancer institution. J Anaesthesiol Clin Pharmacol
23. Anwer M, Manzoor S, Muneer N, et al. Compliance and Effectiveness of WHO Surgical Safety Check list: a JPMC Audit. Pak J Med Sci
24. Askarian M, Kouchak F, Palenik CJ. Effect of surgical safety checklists on postoperative morbidity and mortality rates, Shiraz Faghihy Hospital, a 1-year study. Quality Manage Health Care
25. Baradaran Binazir M, Alizadeh M, Jabbari Bayrami H, et al. The effect of a modified world health organization surgical safety checklist on postoperative complications in a tertiary hospital in iran Iranian. J Public Health
26. Bashford T, Reshamwalla S, McAuley J, et al. Implementation of the WHO Surgical Safety Checklist in an Ethiopian Referral Hospital. Patient Saf Surg
27. Becret A, Clapson P, Andro C, et al. [Feasibility and relevance of an operating room safety checklist for developing countries: study in a French hospital in Djibouti]. Med Sante Trop
28. Chaudhary N, Varma V, Kapoor S, et al. Implementation of a surgical safety checklist and postoperative outcomes: a prospective randomized controlled study. J Gastrointest Surg
29. Close KL, Baxter LS, Ravelojaona VA, et al. Overcoming challenges in implementing the WHO Surgical Safety Checklist: lessons learnt from using a checklist training course to facilitate rapid scale up in Madagascar. BMJ Global Health
30. Dabholkar Y, Velankar H, Suryanarayan S, et al. Evaluation and customization of WHO safety checklist for patient safety
in otorhinolaryngology. Indian J Otolaryngol Head Neck Surg
31. Delgado Hurtado JJ, Jimenez X, Penalonzo MA, et al. Acceptance of the WHO Surgical Safety Checklist among surgical personnel in hospitals in Guatemala city. BMC Health Serv Res
32. Diedhiou M, Manyacka P, Dieng M, et al. [Feasibility and relevance of safety checklist in the central operating block at the Regional Medical Center, Saint Louis, Senegal]. Pan Afr Med J
33. Ellis R, Izzuddin Mohamad Nor A, Pimentil I, et al. Improving surgical and anaesthesia practice: review of the use of the WHO Safe Surgery Checklist in Felege Hiwot Referral Hospital, Ethiopia. BMJ Qual Improv Rep
34. Fatima I, Humayun A, Anwar MI, et al. Evaluating quality standards’ adherence in surgical care: a case study from Pakistan. Int J Qual Health Care
35. Freitas MR, Antunes AG, Lopes BN, et al. [Assessment of adherence to the WHO surgical safety checklist in urological and gynecological surgeries at two teaching hospitals in Natal, Rio Grande do Norte State, Brazil]. Cad Saude Publica
36. Garland NY, Kheng S, De Leon M, et al. Using the WHO Surgical Safety Checklist to Direct Perioperative Quality Improvement at a Surgical Hospital in Cambodia: the importance of objective confirmation of process completion. World J Surg
37. Haynes AB, Weiser TG, Berry WR, et al. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf
38. Igaga EN, Sendagire C, Kizito S, et al. World Health Organization Surgical Safety Checklist: compliance and associated surgical outcomes in Uganda's Referral Hospitals. Anesth Analg
39. Kasatpibal N, Senaratana W, Chitreecheur J, et al. Implementation of the World Health Organization surgical safety checklist at a university hospital in Thailand. Surg Infect (Larchmt)
40. Kasatpibal N, Sirakamon S, Punjasawadwong Y, et al. An exploration of surgical team perceptions toward implementation of surgical safety checklists in a non-native English-speaking country. Am J Infect Control
41. Khorshidifar A, Kadkhodaee H, Zamen Z. Degree of observance of the WHO Surgical Safety Checklist. Trauma Mon
42. Kim RY, Kwakye G, Kwok AC, et al. Sustainability and long-term effectiveness of the WHO surgical safety checklist combined with pulse oximetry in a resource-limited setting: two-year update from Moldova. JAMA Surg
43. Kwok AC, Funk LM, Baltaga R, et al. Implementation of the World Health Organization surgical safety checklist, including introduction of pulse oximetry, in a resource-limited setting. Ann Surg
44. Lacassie HJ, Ferdinand C, Guzman S, et al. World Health Organization (WHO) surgical safety checklist implementation and its impact on perioperative morbidity and mortality in an academic medical center in Chile. Medicine
45. Lilaonitkul M, Kwikiriza A, Ttendo S, et al. Implementation of the WHO Surgical Safety Checklist and surgical swab and instrument counts at a regional referral hospital in Uganda—a quality improvement project. Anaesthesia
46. Maziero EC, de Camargo Silva AE, de Fatima Mantovani M, et al. [Adherence to the use of the surgical checklist for patient safety
]. Rev Gaucha Enferm
47. McGinlay D, Moore D, Mironescu A. A prospective observational assessment of Surgical Safety Checklist use in Brasov Children's Hospital, barriers to implementation and methods to improve compliance. Rom J Anaesth Intensive Care
48. Melekie TB, Getahun GM. Compliance with Surgical Safety Checklist completion in the operating room of University of Gondar Hospital Northwest Ethiopia. BMC Res Notes
49. Oak SN, Dave NM, Garasia MB, et al. Surgical checklist application and its impact on patient safety
in pediatric surgery. J Postgrad Med
50. Pancieri AP, Santos BP, de Avila MA, et al. [Safe surgery checklist: analysis of the safety and communication of teams from a teaching hospital]. Rev Gaucha Enfermagem
51. Prakash P, Baduni N, Sanwal MK, et al. Effect of World Health Organisation Surgical Safety Checklist on patient outcomes in a tertiary care hospital of Delhi. Int Med J
52. Ribeiro H, Quites HFO, Bredes AC, et al. [Adherence to completion of the safe surgery checklist]. Cad Saude Publica
53. Santana HT, de Freitas MR, Ferraz EM, et al. WHO Safety Surgical Checklist implementation evaluation in public hospitals in the Brazilian Federal District. J Infect Public Health
54. Santana HT, Rodrigues MC, do Socorro Nantua Evangelista M. Surgical teams’ attitudes and opinions towards the safety of surgical procedures in public hospitals in the Brazilian Federal District. BMC Res Notes
55. Sayed HA, Zayed M, El Qareh NM, et al. Patient safety
in the operating room at a governmental hospital. J Egypt Public Health Assoc
56. Shankar R. Implementation of the WHO Surgical Safety Checklist at a teaching hospital in India and evaluation of the effects on perioperative complications. Int J Health Plann Manage
57. Toor AA, Farooka MW, Ayyaz M, et al. Pre-operative antibiotic use reduces surgical site infection. J Pak Med Assoc
58. Verwey S, Gopalan PD. An investigation of barriers to the use of the World Health Organization Surgical Safety Checklist in theatres. S Afr Med J
59. Weiser TG, Haynes AB, Dziekan G, et al. Effect of a 19-item surgical safety checklist during urgent operations in a global patient population. Ann Surg
60. White MC, Baxter LS, Close KL, et al. Evaluation of a countrywide implementation of the world health organisation surgical safety checklist in Madagascar. PLoS One
61. White MC, Peterschmidt J, Callahan J, et al. Interval follow up of a 4-day pilot program to implement the WHO surgical safety checklist at a Congolese hospital. Global Health
62. White MC, Randall K, Capo-Chichi NFE, et al. Implementation and evaluation of nationwide scale-up of the Surgical Safety Checklist. Br J Surg
63. White MC, Randall K, Ravelojaona VA, et al. Sustainability of using the WHO surgical safety checklist: a mixed-methods longitudinal evaluation following a nationwide blended educational implementation strategy in Madagascar. BMJ Glob Health
64. Yu X, Huang Y, Guo Q, et al. Clinical motivation and the surgical safety checklist. Br J Surg
65. Yuan CT, Walsh D, Tomarken JL, et al. Incorporating the World Health Organization Surgical Safety Checklist into practice at two hospitals in Liberia. Jt Comm J Qual Patient Saf
66. Rivero GMD, Nolasco GA, Puntunet BML, et al. Nivel de cumplimiento y factores que influyen en la aplicación de la lista de verificación de cirugía segura. Rev Mexicana Enfermería Cardiol
67. El Mhamdi S, Letaief M, Cherif Y, et al. [Implementation of the safe surgery checklist of the World Health Organization at the University Hospital of Monastir (Tunisia)]. Tunis Med
68. Abbott TEF, Ahmad T, Phull MK, et al. The surgical safety checklist and patient outcomes after surgery: a prospective observational cohort study, systematic review and meta-analysis. Br J Anaesth
69. Theobald S, Brandes N, Gyapong M, et al. Implementation research: new imperatives and opportunities in global health. Lancet
70. Hull L, Athanasiou T, Russ S. Implementation science: a neglected opportunity to accelerate improvements in the safety and quality of surgical care. Ann Surg
71. Peden CJ, Stephens T, Martin G, et al. Effectiveness of a national quality improvement programme to improve survival after emergency abdominal surgery (EPOCH): a stepped-wedge cluster-randomised trial. Lancet
72. Stephens TJ, Peden CJ, Pearse RM, et al. Improving care at scale: process evaluation of a multi-component quality improvement intervention to reduce mortality after emergency abdominal surgery (EPOCH trial). Implement Sci
73. Bion J, Richardson A, Hibbert P, et al. ‘Matching Michigan’: a 2-year stepped interventional programme to minimise central venous catheter-blood stream infections in intensive care units in England. BMJ Qual Saf
74. Dixon-Woods M, Leslie M, Tarrant C, et al. Explaining Matching Michigan: an ethnographic study of a patient safety
program. Implement Sci
75. Rowe AK, Rowe SY, Peters DH, et al. Effectiveness of strategies to improve health-care provider practices in low-income and middle-income countries: a systematic review. Lancet Glob Health
76. Pantoja T, Opiyo N, Lewin S, et al. Implementation strategies for health systems in low-income countries: an overview of systematic reviews. Cochrane Database Syst Rev
77. Albutt K, Sonderman K, Citron I, et al. Healthcare leaders develop strategies for expanding national surgical, obstetric, and anaesthesia plans in WHO AFRO and EMRO Regions. World J Surg