- Question: How can the unacceptably high maternal mortality rates in low- and middle-income countries be improved?
- Findings: We developed and evaluated a context-relevant checklist for the management of obstetric crisis in low- and middle-income countries, demonstrated the need for this tool in a real-world setting, and confirmed its potential efficacy through a pilot simulation study.
- Meaning: Ongoing efforts are underway to fully complete checklist implementation, and future research should reproduce this work for other priorities for safe surgery and anesthesia outlined in the Disease Control Priorities, third edition, and supported by the Lancet Commission on Safe Surgery.
The World Health Organization (WHO) continues to report unacceptably high maternal mortality rates in low- and middle-income countries (LMICs).1 The maternal mortality rate in developing countries is 239 per 100,000 live births compared to 12 per 100,000 live births in developed countries.1 Globally, 99% of maternal deaths occur in developing countries, with half of these in Sub-Saharan Africa (SSA). According to the WHO, SSA is reported to be the most dangerous place in the world for a woman to bear a child.2 The third edition of Disease Control Priorities (DCP) reported that scaling up surgical services to include the performance of safe cesarean delivery (CD), in addition to laparotomy and open treatment of a fracture, could prevent 3.2% of annual deaths and 3.5% of disability-adjusted life years lost due to poor perioperative care.3 Based on these findings, the DCP, third edition, considers CD an “essential surgery,” and the Lancet Commission on Global Surgery categorizes it as a “must do” procedure at all first-level care hospitals in LMICs.3,4
The dominant causes of maternal death in the developing world are peripartum hemorrhage (PPH), hypertensive diseases of pregnancy (eg, preeclampsia [Pre-E] and eclampsia), sepsis, and uterine rupture.5 Since 1997, detailed information about maternal deaths in South Africa has been published in the form of the “Saving Mothers” reports.5,6 In these reports, each death was examined by a medical expert for the presence of avoidable factors, which were defined as mistakes or omissions by health care workers, contributing to the death. In all of the yearly reports, substandard management by the health care professionals during emergency events led to death in a large proportion of cases, which was often attributed to failure to follow standard protocols of management.5,6 As such, there are likely significant gains to be made globally, and specifically in SSA, with regard to the knowledge of and adherence to management protocols for obstetric emergencies.
In parallel with these findings, there is increasing interest in and evidence for cognitive aids as a means to improve adherence to published guidelines and prevent errors during acute perioperative events, including obstetric crises.7–16 In 2009, the WHO released the Surgical Safety Checklist, 1 type of cognitive aid. After widespread implementation in surgical systems across a range of economic backgrounds, the application of these care principles resulted in a >30% reduction in perioperative mortality and complications.12 Numerous additional trials in developed countries have shown the benefit of cognitive aids for improving adherence to guidelines in perioperative crisis situations.7,10,11,14,17 However, those previously published findings are not inclusive of, or directly applicable to, maternal surgery and anesthesia for CD in low-resource settings.
With existing published data showing the effectiveness of surgical checklists and with data showing that cognitive aids improve adherence to evidence-based guidelines in perioperative crisis management (both in the simulation laboratory and in in situ settings), it is reasonable to expect that an obstetric crisis checklist with context-relevant management algorithms could improve adherence to best practice in LMICs. Acknowledging the effectiveness of surgical and crisis checklists, we developed a context-sensitive obstetric anesthesia checklist for CD in East Africa. We hypothesized that the use of a context-sensitive obstetric anesthesia checklist for CD would improve adherence to best practice guidelines in the management of anesthesia for elective and urgent/emergent CD in LMICs.
Each section presents the methodology used in that section. This manuscript adheres to the applicable Standards for Quality Improvement Reporting Excellence guidelines.
Create: Elements and Attributes, Checklist Development
We first created an evidence-based obstetric crisis checklist for CD that was context relevant in LMICs. The “Safe Cesarean Delivery Checklist” was developed using a modified Delphi technique during an annual anesthesia conference in June 2014 with >60 East African anesthesia and obstetric professionals (Figure 1). Based on the elements and attributes recommended for optimal usability, the checklist was designed to be: (1) complete but succinct; (2) relevant for the clinical environment; and (3) organized for easy readability, even during a crisis situation.10 Throughout an 8-hour conference, attendees received education surrounding core principles of checklist creation and participated in several rounds of facilitated small-group discussions to produce and refine the checklist content.
The final version of the Safe Cesarean Delivery Checklist is a full-color document designed to be readable as a pocket-size version that individual clinicians can carry in their scrub top or as a large poster that can be placed in a visible location on the operating room wall or hanging on the anesthesia machine (Figure 1). The checklist contains 4 main steps. The first 3 include actions for preparing the operating theater, patient, and neonatal team for any CD, routine or urgent/emergent. Step 4 outlines additional care for urgent or emergent CD, with separate columns for the 3 most commonly encountered life-threatening obstetrical emergencies in East Africa: PPH, Pre-E, and obstructed labor/fetal distress. The steps included for the urgent and emergent CD were only those determined to be “critical steps.” These were defined as a patient management action with a high likelihood of affecting maternal and fetal outcomes and one that could be accomplished at facilities represented by the caregivers present at the conference, which ranged from county-referral hospitals (level 5 hospital in the Kenyan system) to subcounty and district hospitals (levels 4 and 3 in the Kenyan system, respectively).
Create: Testing, Baseline Clinical Observations
After the checklist was developed, an observational study was completed at a private tertiary referral center in Kenya (Africa Inland Church [AIC] Kijabe Hospital, Kijabe, Kenya). The observational study was performed to evaluate baseline clinical performance without the aid of the checklist and confirm that there was a clinical need for improvement in adherence to guidelines. After receipt of AIC Kijabe Hospital Institutional Review Board approval, 25 consecutive CDs were observed without interference or input by the study personnel. Of note, study personnel were trained on how to observe and record whether each component of the checklist was completed, including completion of the item before it would be of benefit to the patient undergoing CD. For example, to receive credit for “prepare airway equipment,” all components had to be ready on the anesthesia workstation within easy reach of the anesthetist “before commencing the anesthetic,” whether spinal or general anesthesia. Presence of airway devices in a drawer on the anesthesia cart or machine was not considered sufficient.
Create: Simulation Testing
Having established a need for the checklist through clinical observation, we evaluated the usability of the checklist by testing the hypothesis that the use of obstetric crisis checklists in the management of high-fidelity–simulated obstetrical emergency scenarios will improve adherence to published guidelines as compared to management from memory alone. This was specifically tested using 2 high-fidelity–simulated obstetric crisis scenarios of urgent CD for severe Pre-E and PPH. This study was performed at Kijabe Hospital in the simulation laboratory that opened in October 2015 as part of an educational capacity-building grant.
After receipt of AIC Kijabe Hospital Institutional Review Board approval and informed consent, 11 Kenyan registered nurse anesthetist (KRNA) students were oriented to the simulation environment and then engaged in context-relevant simulation. The simulation environment was created within the KRNA student classrooms at the hospital. It was representative of all items the KRNAs would have in a typical AIC Kijabe operating theater setting, including a high-fidelity simulation mannequin, a digital monitor displaying real-time electrocardiogram, blood pressure, EtCO2, and plethysmography. The participants were senior KRNA students who were familiar with high-fidelity simulations as part of their training curriculum. Of note, at Kijabe Hospital, the KRNA students are the primary in-room providers for most CDs and have personally performed anesthesia for >20 CDs before the simulation study.
The simulation scenarios (Pre-E and PPH) were created by a team that included faculty with extensive experience in simulation and simulation curriculum development and also East African anesthetists who analyzed whether the scenarios were context appropriate. Of note, multiple clinical stems were created for each scenario to prevent recall bias with pre- and posttesting and also to prevent bias introduced from participants discussing their simulations with one another. Standardized team members were used to simulate other roles in the care team. Performance during the simulations was assessed using the items from the Safe CD Checklist as a scoring tool.
After baseline simulation sessions in which each participant managed both scenarios without any cognitive aid, the participants received individual debriefing. Then, all participants received small-group didactic sessions that provided training on the content and use of the Safe CD Checklist. Two weeks later, at a time unknown to the participants, posttraining testing was performed in which the participants could use the Safe CD Checklist.
For the pilot simulation study, the primary outcome measured was the percentage (%) of checklist actions completed for scenario management pre- and posttraining. Data were analyzed using a paired t test and reported as a mean ± standard deviation.
For the observational component of the study, on average, roughly 75% of the general steps applicable to a routine CD (boxes indicated under steps 1–3 in Figure 1) were performed by anesthesia providers. For urgent and emergent cases, however, there was a sharp decline in completion of indicated steps (boxes listed in individual columns under step 4 in Figure 1). These results are shown in Figure 2. Very few participants (1) checked to see if blood or additional intravenous fluid was available before starting the CD; (2) performed general anesthesia with rapid sequence induction (instead of spinal) when indicated by maternal or fetal condition; (3) ensured the presence of antihypertensive medications in the setting of severe Pre-E; or (4) checked for latest coagulation studies in preeclamptic patients. These observations, although only from a pilot study, highlight the actual clinical need for improvement in adherence to guidelines and the real-world need for the cognitive aid that we developed specifically in the setting of urgent and emergent CD cases in a referral hospital in East Africa.
For both simulation scenarios, our results demonstrated a significant improvement in mean percentage of steps completed. The mean percentage of completed actions for the pretraining scenarios was 22.6% ± 5.6% for Pre-E and 22.0% ± 12.8% for PPH, while posttraining means were 75% ± 9.4% for Pre-E and 69% ± 8.6% for PPH (P < .0001 pre- versus posttraining for both scenarios; Figure 3).
The current high maternal mortality rate in SSA mandates improvement in systems, processes, resource development, and education. Providing education and training in the management of common obstetrical crises may not only improve care delivery but also decrease maternal and neonatal mortality secondary to avoidable causes of death. Although recognized academic and professional entities have published management guidelines concerning these clinical conditions, clinicians in low-resource settings are faced with constraints that make the published guidelines contextually irrelevant. The Lancet Commission highlighted this need by calling for tertiary care hospitals, like AIC Kijabe Hospital, to serve as role models for education, support, and research and to establish context-relevant clinical practice guidelines for LMICs.4 In light of this clinical need, a context-relevant “Safe Anesthesia Cesarean Delivery Checklist” was developed and tested with the goal of improving adherence to best practice guidelines in the management of elective and urgent/emergent CD in LMICs, specifically East Africa. Our present study has several notable findings, each of which will be discussed with respect to the current literature.
First, we developed and evaluated a context-relevant checklist for management of obstetrical crises in LMICs. It was created in a rigorous fashion, using a well-described model that incorporated input from experts in checklist creation as well as a large number of health care professionals who currently provide obstetrical care in East Africa. The American College of Obstetricians and Gynecologists,18 the WHO,19 and the Royal College of Obstetricians and Gynaecologists20 have published surgical safety checklists for planned CDs, but none covers critical management steps in commonly encountered urgent or emergent CDs. Furthermore, each of these checklists is context relevant for operative environments in tertiary referral hospitals in developed countries. As such, many checklist items do not apply when translated to LMICs. Finally, data on the actual clinical implementation and uses of these checklists for obstetrical urgent and emergency cases also remain sparse.10 We believe that the checklist we present, developed through a rigorous process with input from a large group of East African health care professionals fulfills a current need in SSA that other existing cognitive aids do not.
Second, after creating the checklist, we confirmed the clinical need for such a tool, even at a referral hospital that performs >500 CDs per year. Specifically, clinical observations of CDs revealed that roughly 75% of items were completed that are common to all CDs, most of which involve preparing for the case. However, many fewer items were completed concerning real-time, emergency-specific management in cases of PPH, Pre-E, and obstructed labor/fetal distress. This is in line with what is known in the literature about skilled clinicians forgetting important steps in patient management during high-stake events.21,22 Furthermore, our study was conducted at a high-volume tertiary referral hospital with well-developed anesthesia, surgery, and obstetrical training programs.23 It is reasonable to assume that performance at this center is at least representative of, if not better than, many centers in Kenya. In either case, the testing of the checklist at this location highlights its likely need and usefulness at other institutions throughout Kenya specifically and SSA in general. After introduction and implementation of the checklist at Kijabe Hospital, we found a barrier to implementation over the subsequent 6 months was that most of the checklist training was delivered to anesthesia students because they have more time and freedom to be relieved from clinical duties, whereas practicing clinicians are busier throughout the day. Future improvements for complete implementation will require that training is viewed as a mandatory component for all clinicians practicing in the operating theaters.
Third, we found that after training with the Safe CD Checklist, KRNA students showed a statistically significant improvement in performance in a high-fidelity simulation environment. Of note, in comparison to the practice observed in the clinical setting without the checklist, improvement in the preparation for and management of CD was noted, as well as an improvement in the management steps specific to 2 commonly encountered obstetric emergencies in LMICs, specifically Pre-E and PPH. We believe this demonstrates that simulation training followed by frequent clinical use of a checklist could likely result in increased adherence to guidelines for routine and emergency CDs; however, any hypothesis of translation to the clinical arena needs to be confirmed with future clinical research.24,25
Our article has several limitations. First, the results of our clinical observations and simulation training were small in number of cases and participants and only come from 1 institution. While they likely represent the overall performance that could be assessed elsewhere, future study will need to confirm this. Second, the effect of repeated experiential learning exercises, including simulation and small-group didactic sessions, cannot be isolated from effects of any clinical checklist use in the operating room during the 2 weeks after baseline testing and training. Thus, the specific pedagogical intervention responsible for any improvement cannot be known based on our study. However, it is likely that the overall curriculum involving repetitive teaching and deliberate practice led to the improvement.26–28 Although these are pilot data, the education intervention bundle that consisted of a context-relevant checklist, simulation training and debriefing, and small-group didactics led to a marked improvement in outcomes.25
Future research should include a roadmap for how to expand the initial work that we have described. Multiple educational interventions (eg, simulation versus small-group didactics)29 should be tested to isolate the most effective pedagogical approach in SSA and to evaluate whether obstetric crisis checklists and simulation education can have lasting results on maternal mortality in SSA. This evidence should be applied across a broad spectrum of participants and teams in SSA, which should then be paired with baseline clinical assessments of performance, as well as posttraining clinical assessments. If the research is successful in improving process metrics, then large-scale studies should be undertaken in cooperation with the Ministry of Health in multiple countries in SSA to see if maternal and neonatal outcomes can be improved through systematized training and implementation of a checklist for safe CD. Finally, this process will need to be reiterated for other care priorities outlined in the DCP, third edition, for safe surgery and anesthesia, such as laparotomy (particularly related to traumatic events) and fixation of open fractures.
In conclusion, we report on the successful creation and initial testing of a checklist for safe CD in SSA. Ongoing efforts are underway to complete checklist implementation. Future research should reproduce this work for other priorities for safe surgery and anesthesia outlined in the DCP, third edition, and supported by the Lancet Commission on Safe Surgery.
Name: Louise A. Alexander, MD.
Contribution: This author helped plan and execute the simulation studies at Kijabe Africa Inland Church hospital, helped synthesize the data obtained, and was the primary author of the manuscript.
Name: Mark W. Newton, MD.
Contribution: This author helped organize the East African anesthesia conference where the safe cesarean delivery checklist was developed, implement the observational and simulation studies, synthesize and organize results, and write the manuscript.
Name: Kendall G. McEvoy.
Contribution: This author helped gather data for the observational portion of the study.
Name: Micah J. Newton, BS.
Contribution: This author helped gather data for the observational portion of the study.
Name: Mary Mungai, BSN.
Contribution: This author helped coordinate checklist distribution to the operating theaters and coordinate nurse anesthetist participation in simulation sessions.
Name: Mary DiMiceli-Zsigmond, MD.
Contribution: This author helped write the obstetric simulation scenarios and train the nurse anesthetist staff in simulation production.
Name: Bantayehu Sileshi, MD.
Contribution: This author helped develop the safe cesarean checklist and distribute it at Kijabe Hospital.
Name: Scott C. Watkins, MD.
Contribution: This author helped design the observational and simulation study and design the safe cesarean checklist.
Name: Matthew D. McEvoy, MD.
This author helped with all stages of study design and implementation for the safe cesarean checklist creation, observational and simulation studies, data gathering, and statistical analysis.
This manuscript was handled by: Angela Enright, MB, FRCPC.
3. Mock CN, Donkor P, Gawande A, et alEssential surgery: key messages from Disease Control Priorities, 3rd edition. Lancet. 2015;385:2209–2219.
4. Meara JG, Leather AJ, Hagander LGlobal surgery 2030: evidence and solutions for achieving health, welfare, and economic development. Lancet. 2015;386:569–624.
5. Moran NF, Naidoo M, Moodley JReducing maternal mortality on a countrywide scale: the role of emergency obstetric training. Best Pract Res Clin Obstet Gynaecol. 2015;29:1102–1118.
6. Moodley J, Pattinson RC, Fawcus S, Schoon MG, Moran N, Shweni PMNational Committee on Confidential Enquiries Into Maternal Deaths in South Africa. The confidential enquiry into maternal deaths in South Africa: a case study. BJOG. 2014;121suppl 453–60.
7. Arriaga AF, Bader AM, Wong JMSimulation-based trial of surgical-crisis checklists. N Engl J Med. 2013;368:246–253.
8. Behrens V, Dudaryk R, Nedeff N, Tobin JM, Varon AJThe Ryder cognitive aid checklist for trauma anesthesia. Anesth Analg. 2016;122:1484–1487.
9. Burden AR, Carr ZJ, Staman GW, Littman JJ, Torjman MCDoes every code need a “reader?” Improvement of rare event management with a cognitive aid “reader” during a simulated emergency: a pilot study. Simul Healthc. 2012;7:1–9.
10. Goldhaber-Fiebert SN, Howard SKImplementing emergency manuals: can cognitive aids help translate best practices for patient care during acute events? Anesth Analg. 2013;117:1149–1161.
11. Goldhaber-Fiebert SN, Pollock J, Howard SK, Bereknyei Merrell SEmergency manual uses during actual critical events and changes in safety culture from the perspective of anesthesia residents: a pilot study. Anesth Analg. 2016;123:641–649.
12. Haynes AB, Weiser TG, Berry WR, et alSafe Surgery Saves Lives Study Group. A surgical safety checklist to reduce morbidity and mortality in a global population. N Engl J Med. 2009;360:491–499.
13. Marshall SThe use of cognitive aids during emergencies in anesthesia: a review of the literature. Anesth Analg. 2013;117:1162–1171.
14. McEvoy MD, Hand WR, Stoll WD, Furse CM, Nietert PJAdherence to guidelines for the management of local anesthetic systemic toxicity is improved by an electronic decision support tool and designated “Reader.” Reg Anesth Pain Med. 2014;39:299–305.
15. Mohammed A, Wu J, Biggs TDoes use of a World Health Organization obstetric safe surgery checklist improve communication between obstetricians and anaesthetists? A retrospective study of 389 caesarean sections. BJOG. 2013;120:644–648.
16. Ziewacz JE, Arriaga AF, Bader AMCrisis checklists for the operating room: development and pilot testing. J Am Coll Surg. 2011;213:212.e10–217.e10.
17. Field LC, McEvoy MD, Smalley JCUse of an electronic decision support tool improves management of simulated in-hospital cardiac arrest. Resuscitation. 2014;85:138–142.
21. Augoustides JG, Atkins J, Kofke WAMuch ado about checklists: who says I need them and who moved my cheese? Anesth Analg. 2013;117:1037–1038.
22. Gaba DMPerioperative cognitive aids in anesthesia: what, who, how, and why bother? Anesth Analg. 2013;117:1033–1036.
23. Newton M, Bird PImpact of parallel anesthesia and surgical provider training in sub-Saharan Africa: a model for a resource-poor setting. World J Surg. 2010;34:445–452.
24. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DBTranslational educational research: a necessity for effective health-care improvement. Chest. 2012;142:1097–1103.
25. McGaghie WC, Draycott TJ, Dunn WF, Lopez CM, Stefanidis DEvaluating the impact of simulation on translational patient outcomes. Simul Healthc. 2011;6supplS42–S47.
26. Weller JM, Nestel D, Marshall SD, Brooks PM, Conn JJSimulation in clinical teaching and learning. Med J Aust. 2012;196:594.
27. Wayne DB, Didwania A, Feinglass J, Fudala MJ, Barsuk JH, McGaghie WCSimulation-based education improves quality of care during cardiac arrest team responses at an academic teaching hospital: a case-control study. Chest. 2008;133:56–61.
28. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DBDoes simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011;86:706–711.
© 2019 International Anesthesia Research Society
29. Sheakley ML, Gilbert GE, Leighton K, Hall M, Callender D, Pederson DA brief simulation intervention increasing basic science and clinical knowledge. Med Educ Online. 2016;21:30744.