The overall in-hospital mortality rate during the whole study period was 1.3% and decreased from 1.6% in the control steps to 1.0% in the steps after SSC implementation (P = 0.151). The result did not change after controlling for possible confounders including sex, age, comorbidity (American Society of Anesthesiologists score), surgical specialty, urgency of surgery, type of anesthesia, and times (study time points from August 2009 to June 2010) (Supplemental Digital Content 2, available at http://links.lww.com/BRS/A868). Analysis of mortality by hospital revealed a significant decrease from 1.9% to 0.2% (P = 0.020) post-SSC implementation in the smaller community hospital of the study.
Patients' LOS was compared at control and SSC intervention stages of the study. The total in-hospital LOS for both study hospitals was significantly reduced from 7.8 days to 7.0 days after introduction of the Checklist, with a mean difference of −0.8 days, t = 2.30 (95% CI, 0.11–1.43). Furthermore, there were no significant changes in length of surgery or in total time spent in the operating room.
To our knowledge, this is the first stepped wedge cluster RCT on the clinical effectiveness of the WHO SSC. The study showed substantial improvements in surgical outcomes. Across 2 hospitals of a well-developed and funded health care system (Norway) including 5 surgical specialties, complication rates fell by 42% on average when the SSC was introduced. The effect was largest when all 3 parts of the SSC were conducted. The effect was significant even when surgical procedures included “intention to treat” with the SSC (in all 3083 surgical procedures postintervention, Table 2). The findings support our hypothesis and are consistent with previous pre-/poststudies having found similar effects of the WHO Checklist use.14–16 Our results of reduction in morbidity also correspond to findings on use of the comprehensive “surgical patient safety checklist system” (SURPASS) in The Netherlands.13
The in-hospital stay decreased significantly in this study by almost a day. This is the first time the WHO SCC is shown to reduce LOS. The finding is consistent with a reduction in LOS by 0.6 days previously obtained after introducing the SURPASS checklist, which, however, did not reach statistical significance.13 Furthermore, our study reflects similar findings in intensive care units, where LOS has been significantly reduced after use of a daily checklist (goal sheet).38 LOS reduction provides a potential of significant cost savings in surgical care by improved patient outcome, as costs of complications and unplanned returns to operating room are reduced.39 Although the WHO SSC was designed for quality improvement, a secondary effect—cost savings—should further encourage health care leaders to adopt and support its use.
After implementation of the WHO SSC, the overall study mortality deceased from 1.6% to 1.0% but did not improve significantly (P = 0.151). However, we observed a highly significant reduction of mortality from 1.9% to 0.2% (P = 0.02) in the smaller community hospital (albeit on fewer cases due to small hospital size), with a relative risk reduction of 91%. The Checklist effect on mortality was thus present but weaker in our RCT than in previous reports from pre-/postintervention studies.13–16
In our view, this study's major contribution to our better understanding of Checklist effects lies in its stepped wedge cluster RCT design. Such designs have been considered unfeasible because in countries such as the United Kingdom, the WHO SSC is now national policy (and hence a control arm is not available) and also due to contamination and biases resulting from “control” operating room teams treating control patients as patients assigned to the checklist arm.13 However, such contaminations and biases were minimized by randomization of the study clusters in “stepped wedges.”33 Each cluster acted as its own control and hence provided data in both the control and SSC stages, comparable with a crossover design, with all data being compared between the control and SSC stages. To reduce uncertainty of variation in surgical procedure complication rates and complexity within each cluster from pre- to postintervention, we adjusted for possible risk factors as age, sex, comorbidity, surgical specialty, emergency or planned surgery, type of anesthesia, and time (study time points). The stepped wedges provided the possibility to control complication and morbidity for time effects during the study period. Complications rates varied at different study time points but when controlled for, time was not a confounding factor for the Checklist effect on complications (Table 3). The stepped wedge cluster RCT design is considered particularly appropriate for studying patient safety interventions.30,32 To control for leakage and possible contamination of surgeons between the 2 hospitals and the 5 surgical specialties, we did not include the same surgical specialty in both hospitals. The SSC was first introduced to the intervention groups. Hence, any possible contamination would have leaked from the intervention group to improve care in the controls, eventually. The results do not suggest that this was apparent.
The degree of blinding is important for the validity of RCTs, and in our study, operating room staff were not informed of the study outcomes, as they routinely registered the patient data on the electronic data system of their operating rooms. To further prevent information bias, the outcome assessors were masked to which cohort (control and SSC stages) patients were enrolled. Furthermore, to reduce the risk of performance and information bias, all recovery and ward staff carried out care as usual and were blinded to the study cohorts and outcomes, following the extended CONSORT statement for nonpharmacological randomized trials.40
Our study has several limitations. First, the clusters that had not yet received the intervention could have been contaminated by possible enthusiasm for the SSC from colleagues in other specialties that were in the SSC study stage. Such bias would have likely minimized any positive effects of the Checklist. The substantial and robust decrease of complications that we found suggests that such bias did not affect the study significantly. A second limitation is the way in which the data were registered. A selection of ICD-10 codes was used to identify complications. It is possible that surgeons and ward doctors reported the ICD-10 codes variably. As far as we could account for, there were no changes in the ICD-10 code implemented during the study period. Furthermore, variable recording would equally affect the control and the SSC stages of the study. A final limitation is that recording of complications was confined to the in-hospital admission period. Data on complications after discharge were not recorded or obtained. The total number of postoperative complications could, therefore, be higher. A more extensive follow-up of the patients after discharge would be beneficial in future studies, though costly.
Further research should investigate how use of the SSC and other checklists achieves its positive impact on patient outcomes. Improved outcomes post–checklist implementation have been explained by improvements in communication and teamwork in the operating room27 and a wider improvement in safety attitudes.20,22,24–26 In a concurrent with this study evaluation of the impact of the introduction of the WHO SSC on patient safety climate in operating rooms, we did not find the hypothesized improvement in culture—although we did find that operating room teams reported being better able to handle a complex situation when the Checklist is used.34 We also anecdotally observed that the introduction of the WHO SSC drove behavior change, as the team members paused, introduced themselves, and carried out team briefings prior to the operative list. Such behavioral changes may precede deeper changes in organizational safety culture—which may in turn underline the sustainability of long-term appropriate implementation of a checklist and improved patient outcomes. These questions require longitudinal controlled research designs to be addressed.
This stepped wedge cluster RCT adds to this growing body of evidence on the positive effects on patient outcomes driven by the WHO SSC. We conclude that the use of the WHO Checklist prevents complications and reduces in-hospital length of stay and potentially also mortality across a wide range of patients undergoing simple or complex surgical procedures in hospitals within a well-developed and funded health care system.
The authors thank the Norwegian Knowledge Center for Patient Safety for collaboration with translating the WHO Surgical Safety Checklist. The study was endorsed by the Norwegian National Knowledge Center for Patient Safety and the Patient Safety Office of the World Health Organization.
1. Weiser TG, Regenbogen SE, Thompson KD, et al. An estimation of the global volume of surgery: a modelling strategy based on available data. Lancet. 2008;372:139–144.
2. Finks JF, Osborne NH, Birkmeyer JD. Trends in hospital volume and operative mortality for high-risk surgery. N Engl J Med. 2011;364:2128–2137.
3. Gawande AA, Thomas EJ, Zinner MJ, et al. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery. 1999;126:66–75.
4. Jencks SF, Williams MV, Coleman EA. Rehospitalizations among patients in the Medicare fee-for-service program. N Engl J Med. 2009;360:1418–1428.
5. Pearse RM, Moreno RP, Bauer P, et al. Mortality after surgery in Europe: a 7-day cohort study. Lancet. 2013;380:1059–1065.
6. Ghaferi AA, Birkmeyer JD, Dimick JB. Variation in hospital mortality associated with inpatient surgery. N Engl J Med. 2009;361:1368–1375.
7. Davies P, Lay-Yee R, Briant R, et al. Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J. 2002;15:U271.
8. de Vries EN, Ramrattan MA, Smorenburg SM, et al. The incidence and nature of in-hospital adverse events: a systematic review. Qual Saf Health Care. 2008;17:216–223.
9. Kable AK, Gibberd RW, Spigelman AD. Adverse events in surgical patients in Australia. Int J Qual Health Care. 2002;14:269–276.
10. Arriaga AF, Bader AM, Wong JM, et al. Simulation-based trial of surgical-crisis checklists. N Engl J Med. 2013;368:246–253.
11. Bliss LA, Ross-Richardson CB, Sanzari LJ, et al. Thirty-day outcomes support implementation of a surgical safety checklist. J Am Coll Surg. 2012;215:766–776.
12. de Vries EN, Prins HA, Bennink MC, et al. Nature and timing of incidents intercepted by the SURPASS checklist in surgical patients. BMJ Qual Saf. 2012;21:503–508.
13. de Vries EN, Prins HA, Crolla RMPH, et al. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010;363:1928–1937.
14. Haynes A, Weiser T, Berry W, et al. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–499.
15. van Klei WA, Hoff RG, van Aarnhem EEHL, et al. Effects of the introduction of the WHO “Surgical Safety Checklist” on in-hospital mortality: a cohort study. Ann Surg. 2012;255:44–49.
16. Weiser TG, Haynes AB, Dziekan G, et al. Effect of a 19-item surgical safety checklist during urgent operations in a global patient population. Ann Surg. 2010;251:976–980.
17. Weiser TG, Haynes AB, Lashoher A, et al. Perspectives in quality: designing the WHO Surgical Safety Checklist. Int J Qual Health Care. 2010;22:365–370.
18. Borchard A, Schwappach DLB, Barbir A, et al. A systematic review of the effectiveness, compliance, and critical factors for implementation of safety checklists in surgery. Ann Surg. 2012;256:925–933.
19. Fudickar A HK, Wiltfang J, Bein B. The effect of the WHO Surgical Safety Checklist on complication rate and communication. Dtsch Arztebl Int. 2012;109:6.
20. Kearns RJ, Uppal V, Bonner J, et al. The introduction of a surgical safety checklist in a tertiary referral obstetric centre. BMJ Qual Saf. 2011;20:818–822.
21. Nilsson L, Lindberget O, Gupta A, et al. Implementing a pre-operative checklist to increase patient safety: a 1-year follow-up of personnel attitudes. Acta Anaesthesiol Scand. 2010;54:176–182.
22. Takala RSK, Pauniaho SL, Kotkansalo A, et al. A pilot study of the implementation of WHO Surgical Checklist in Finland: improvements in activities and communication. Acta Anaesthesiol Scand. 2011;55:1206–1214.
23. Böhmer AB, Kindermann P, Schwanke U, et al. Long-term effects of a perioperative safety checklist from the viewpoint of personnel. Acta Anaesthesiol Scand. 2013;57:150–157.
24. Böhmer AB, Wappler F, Tinschmann T, et al. The implementation of a perioperative checklist increases patients' perioperative safety and staff satisfaction. Acta Anaesthesiol Scand. 2012;56:332–338.
25. Helmiö P, Blomgren K, Takala A, et al. Towards better patient safety: WHO Surgical Safety Checklist in otorhinolaryngology. Clin Otolaryngol. 2011;36:242–247.
26. Haynes AB, Weiser TG, Berry WR, et al. Changes in safety attitude and relationship to decreased postoperative morbidity and mortality following implementation of a checklist-based surgical safety intervention. BMJ Qual Saf. 2011;20:102–107.
27. Russ S, Rout S, Sevdalis N, et al. Do safety checklists improve teamwork and communication in the operating room? A systematic review. Ann of Surg. 2013;258:856–871.
28. Birkmeyer JD. Strategies for improving surgical quality—checklists and beyond. N Engl J Med. 2010;363:1963–1965.
30. Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6:54.
32. Mdege ND, Man MS, Taylor CA, et al. Systematic review of stepped wedge cluster randomized trials shows that design is particularly used to evaluate interventions during routine implementation. J Clin Epidemiol. 2011;64:936–948.
33. Brown C, Hofer T, Johal A, et al. An epistemology of patient safety research: a framework for study design and interpretation. Part 2. Study design. Qual Saf Health Care. 2008;17:163–169.
34. Haugen AS, Søfteland E, Eide GE, et al. Impact of the World Health Organization's Surgical Safety Checklist on safety culture in the operating theatre: a controlled intervention study. Br J Anaesth. 2013;110:807–815.
36. Cohen J. Statistical Power Analysis for the Behavioural Sciences. 2nd ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
37. Laupacis A, Sackett DL, Roberts RS. An assessment of clinically useful measures of the consequences of treatment. N Engl J Med. 1988;318:1728–1733.
38. Pronovost P, Berenholz S, Dorman T, et al. Improving communication in the ICU using daily goals. J Crit Care. 2003;18:71–75.
39. Semel ME, Resch S, Haynes AB, et al. Adopting a surgical safety checklist could save money and improve the quality of care in U.S. hospitals. Health Aff (Millwood). 2010;29:1593–1599.
40. Boutron I, Moher D, Altman DG, et al. Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration. Ann Intern Med. 2008;148:295–309.