There was no statistically significant association between compliance with the WHO SSC and perioperative surgical outcomes: 30-day mortality and length of hospital stay. There was a significant association between compliance with the WHO SSC and the incidence of pain and loss of consciousness. We also found that the odds of death were significantly higher compared to the rest in hospital 4 (Tables 3–5).
The WHO SSC was performed in 43.5% of the enrolled surgeries and a copy was present in only 5% of the patient’s files of the recruited patients.
In about 80% of the enrolled surgeries, pulse oximeter placement was adhered to during the “sign in” component of the WHO SSC. Ascertaining whether the patient had a known allergy was the worst performed (4.4%). Antibiotic prophylaxis was not given in the recommended time as per WHO standards in about half (49.6%) of the enrolled surgeries. Instrument, needle, and sponge counts were done at “sign out” in less than half (41%) of the enrolled surgeries.
We found low levels of compliance across the hospitals with no significant association between this compliance and perioperative surgical outcomes. Mean compliance with the WHO SSC was about 40% with a wide variation across the different participating hospitals and across the African hospitals that found a 48.5% mean use of the WHO SSC with variation between 10% and 90%. Uganda’s compliance was 10%.9 Despite the fact that compliance with the WHO SSC was low, we found no significant association with adverse events, length of hospital stay, and 30-day mortality. This is similar to van Klei et al’s5 study that demonstrated unchanged mortality rates in the patients for whom the checklist was incomplete and significantly lower mortality rates in those patients for whom the SSC was completed in the perioperative period.
The reasons for low compliance with the WHO SSC include lack of leadership, teamwork, and enforcement of the use of the checklist; lack of training and awareness on WHO SSC use, socioeconomic norms, and cultural barriers where hierarchy prevents some team members from initiating the WHO SSC, and resource limitations in terms of stationery, inadequate staffing, and time constraints.9,10 The variation in compliance is similar to a retrospective study conducted across 28 European countries that found that the average use of the checklist was about 1.5 times higher (65.7%) than that in our study, with a variation of 0%–99.6% among participating hospitals.11
Van Schoten’s12 study showed hospital type was one of the factors influencing compliance with the WHO SSC. In our study, we found that the private not for profit hospital had a mean compliance of 60% which we attributed to better funding, leadership, and availability of resources to train and ensure continued use of the SSC. Surprisingly, the regional referral hospital had a compliance of nearly 90% despite the limitations public hospitals have in terms of availability of resources and funding. This remarkably high compliance could be attributed to either the Hawthorne effect or observer bias or both.
Compliance was better with the “sign in” and “time out” phases but worse with the “sign out” phase. From experience, priorities at the end of surgery differ for the different team members. This result is comparable to the findings from van Klei et al’s5 before and after study with a similar case mix. Vogts et al13 speculated that poor compliance with the “sign out” domain could be explained by the fact that it is not linked to a particular point in the patient’s care unlike the other domains of the checklist. Compliance was worse in particular surgical specialties; orthopedic surgery, neurosurgery, and ear, nose, and throat surgery, with compliance to the “sign out” phase of the WHO SSC being the worst done even for these surgical specialties. This is in contrast with van Schoten et al’s12 study where compliance with the WHO SSC was high in ENT surgery. The reason for our results remains unclear.
Previous studies have proposed direct and indirect mechanisms by which the checklist improves surgical outcomes. An indirect mechanism would be improved teamwork and communication among the entire surgical team which will likely improve safety in the perioperative period.14 Timely administration of the correct antibiotic would be a direct mechanism. Although we did not study quality of checklist performance, we found that nurses were present more frequently during the performance of the SSC compared to other team members, indicating a lack of teamwork during the use of the SSC.
Unlike in previous studies,15 we found no significant association with either 30-day mortality or length of stay. Because this was a secondary objective, our study was not powered to detect the association between compliance and the aforementioned outcomes due to resource constraints. These results should therefore be interpreted with caution. Possible explanations could be low American Society of Anaesthesiologists physical status, lower risk surgical procedure as the majority were cesarean sections and use of regional anesthesia, all of which reduce the risk of perioperative adverse events.16,17 There was, however, a statistically significant association between compliance and some perioperative adverse events as defined by the American College of Surgeons National Surgical Quality Improvement Program18 particularly pain and loss of consciousness. The reason for the association of better compliance with the presence of pain and loss of consciousness postoperatively remains unclear. The reason for the increased odds of death at hospital 4 remains uncertain and warrants investigation.
Strengths of this study include the fact that a prospective multicentre cohort study reduces the risk of recall bias and improves the generalizability of the results. As one of the larger studies of its kind in sub-Saharan Africa, it adds valuable information to the pool of data on the WHO SSC. Our main limitation was the small sample size compared to similar studies that may have prevented recognition of some adverse events that occur less frequently. A sample size of about 3500 patients would have been sufficient to achieve this objective. The Hawthorne effect cannot be ruled out in a directly observed study such as ours. We did try, however, to reduce its effect by ensuring that the investigators were familiar to the surgical team and trained them to document their observations in a manner that would not raise suspicion although in doing this, we could have increased the risk of observer bias. We were unable to collect data on age and sex of the study participants, an oversight on our part and this limited our analysis.
We set out to find the extent of compliance with the WHO SSC and the association of this compliance with perioperative surgical outcomes. We found that compliance with its use in Uganda’s referral hospitals showed an association with the increased incidence of pain and loss of consciousness postoperatively. Following these results, we recommend that a larger study, well powered to detect an association of compliance with perioperative adverse events, is done to demonstrate generalizability of the results. We also recommend that further research associating pain scores with use of the WHO SSC be considered. E
The authors thank the Government of Uganda through the Ministry of Health who sponsored the first author’s Master of Medicine in Anaesthesia and Critical care. Anaesthesia and Intensive Care consultants partially funded the study. Many thanks go to the following doctors; Dr Stephen Ssenyonjo Ttendo, Dr Joseph Kyobe Kiwanuka, Dr George Kateregga, Dr Davidson Ocen, Dr Amanda Akatukunda, Dr Fred Bulamba, and Ms Bernadette Kamaria who were of great help during the data collection process at the different study sites and Dr John Baptist Kiggundu who was instrumental in data management and result analysis. We are grateful to Dr Janat Tumukunde, Dr Peter Kaahwa Agaba, Dr Mary Theresa Nabukenya, Dr Emmanuel Timarwa Ayebale, Dr John Mark Kasumba, Professor Joan Kalyango, Dr Pauline Bakibinga, and Dr Abdhalah Ziraba for their invaluable contribution to this final report.
Name: Elizabeth N. Igaga, MBChB.
Contribution: This author helped with designing the research, sought ethical approval, trained the data collectors and data entrants, wrote the report, and read and approved the final manuscript.
Name: Cornelius Sendagire, MBChB.
Contribution: This author helped with designing the study protocol, analyzing the data, writing the final report, and read and approved the final manuscript.
Name: Samuel Kizito, MBChB.
Contribution: This author helped with conceiving the study, obtaining ethical approval, cleaning and analyzing the data, as well as writing the final report, and read and approved the final manuscript.
Name: Daniel Obua, MBChB.
Contribution: This author helped with designing the study, obtaining ethical approval, writing the final report, and read and approved the final manuscript.
Name: Arthur Kwizera, MBChB.
Contribution: This author helped with designing the study, obtaining ethical approval, data collection, writing the final report, and read and approved the final manuscript.
This manuscript was handled by: Angela Enright, MB, FRCPC.
1. Patel J, Ahmed K, Guru KA, et al. An overview of the use and implementation of checklists in surgical specialities - a systematic review. Int J Surg. 2014;12:1317–1323.
2. de Vries EN, Prins HA, Crolla RM, et al; SURPASS Collaborative Group. Effect of a comprehensive surgical safety system on patient outcomes. N Engl J Med. 2010;363:1928–1937.
3. Weiser TG, Haynes AB, Dziekan G, Berry WR, Lipsitz SR, Gawande AA; Safe Surgery Saves Lives Investigators and Study Group. Effect of a 19-item surgical safety checklist during urgent operations in a global patient population. Ann Surg. 2010;251:976–980.
4. Haynes AB, Weiser TG, Berry WR, et al; Safe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–499.
5. van Klei WA, Hoff RG, van Aarnhem EE, et al. Effects of the introduction of the WHO “Surgical Safety Checklist” on in-hospital mortality: a cohort study. Ann Surg. 2012;255:44–49.
6. Walker IA, Reshamwalla S, Wilson IH. Surgical safety checklists: do they improve outcomes? Br J Anaesth. 2012;109:47–54.
7. Russ S, Rout S, Caris J, et al. Measuring variation in use of the WHO surgical safety checklist in the operating room: a multicenter prospective cross-sectional study. J Am Coll Surg. 2015;220:1–11.e4.
8. Lilaonitkul M, Kwikiriza A, Ttendo S, et al. Implementation of the WHO Surgical Safety Checklist and surgical swab and instrument counts at a regional referral hospital in Uganda - a quality improvement project. Anaesthesia. 2015;70:1345–1355.
9. Kariyo P, Hightower J, Ndihokubwayo J, Tumusiime P, Mwikisa C. Challenges facing the introduction of the WHO surgical safety checklist: a short experience in African countries. Afr Health Monitor. 2013;36:9.
10. Ratcliffe A, Kintu A, Byamugisha J. Barriers to the use of the World Health Organisation Surgical Safety Checklist at Mulago National Referral Hospital. 2016.Mbale Uganda Association of Anesthesiologists of Uganda, Makerere University.
11. Jammer I, Ahmad T, Aldecoa C, et al; European Surgical Outcomes Study (EuSOS) group. Point prevalence of surgical checklist use in Europe: relationship with hospital mortality. Br J Anaesth. 2015;114:801–807.
12. van Schoten SM, Kop V, de Blok C, Spreeuwenberg P, Groenewegen PP, Wagner C. Compliance with a time-out procedure intended to prevent wrong surgery in hospitals: results of a national patient safety programme in the Netherlands. BMJ Open. 2014;4:e005075.
13. Vogts N, Hannam JA, Merry AF, Mitchell SJ. Compliance and quality in administration of a Surgical Safety Checklist in a tertiary New Zealand hospital. N Z Med J. 2011;124:48–58.
14. Jager E, McKenna C, Bartlett L, Gunnarsson R, Ho Y-H. Postoperative adverse events inconsistently improved by the World Health Organization Surgical Safety Checklist: a systematic literature review of 25 studies. World J Surg. 2016;40:1842–1858.
15. Urbach DR, Govindarajan A, Saskin R, Wilton AS, Baxter NN. Introduction of surgical safety checklists in Ontario, Canada. N Engl J Med. 2014;370:1029–1038.
16. Rodgers A, Walker N, Schug S, et al. Reduction of postoperative mortality and morbidity with epidural or spinal anaesthesia: results from overview of randomised trials. BMJ. 2000;321:1493.
17. Kettner SC, Willschke H, Marhofer P. Does regional anaesthesia really improve outcome? Br J Anaesth. 2011;107suppl 1i90–i95.
© 2018 International Anesthesia Research Society
18. Khuri SF, Daley J, Henderson W, et al. The National Veterans Administration Surgical Risk Study: risk adjustment for the comparative assessment of the quality of surgical care. J Am Coll Surg. 1995;180:519–531.