Daniels, Kay MD; Arafeh, Julie RN, MSN; Clark, Ana RN; Waller, Sarah MD; Druzin, Maurice MD; Chueh, Jane MD
During 1991–1999, a total of 4200 deaths were determined to be pregnancy related. The overall pregnancy-related mortality ratio was 11.8 deaths per 100,000 live births and ranged from 10.3 in 1991 to 13.2 in 1999.1 In fact, the maternal mortality in the United States has not improved in two decades and is higher than in most developed countries.2 A crucial factor for maternal mortality in obstetrical medicine is that labor and delivery is a critical care area. Obstetrical emergencies are often unexpected, marked by significant time pressure, high stakes, and technical and ethical challenges associated with caring simultaneously for two patients (mother and fetus). This represents a unique situation in medicine, which requires excellent teamwork and superior communication skills between multiple medical teams. To date, the best method to provide crisis training has not been established. Learning in a classroom followed by observation has been the primary technique for medical training. But is there a better way?
The Joint Commission on Accreditation of Healthcare Organizations released a Sentinel Event Alert 30 dated July 21, 2004, titled: “Preventing infant death and injury during delivery.” In the cases studied, communication failures topped the list of identified root causes (72%).3 In the report, Joint Commission on Accreditation of Healthcare Organizations states:
Since the majority of perinatal death and injury cases reported root causes related to problems with organizational culture and with communication among caregivers, it is recommended that organizations
1. Conduct team training in perinatal areas to teach staff to work together and communicate more effectively.
2. For high-risk events, such as shoulder dystocia, emergency cesarean delivery, maternal hemorrhage and neonatal resuscitation, conduct clinical drills to help staff prepare for when such events actually occur, and conduct debriefings to evaluate team performance and identify areas for improvement.
In its landmark, 1999 report on medical error and patient safety, the Institute of Medicine wrote, “The Committee believes that health care organizations should establish team training programs for personnel in critical care areas using proven methods such as crew resource management techniques employed in aviation including simulation.”4
Simulation refers to the recreation of an actual event that has previously occurred or could potentially occur.5 Advances in information technology have led to the successful introduction of simulator-based training in many safety- critical industries such as aviation and nuclear power.
The use of simulation training as an educational modality has been studied in other fields of medicine. Its use in obstetrical training has been developing rapidly over the last few years. Specifically, studies have explored the use of simulation as a training tool to improve the technical skills of obstetricians.6–9 Draycott et al9 have shown improvement in actual clinical outcomes with the use of simulation technique to train labor and delivery units on the management of shoulder dystocia. However, there are limited studies directly comparing simulation to traditional teaching in an academic residency training program.
Shoulder dystocia and eclampsia were chosen for this comparison study because of their impact on maternal morbidity and malpractice claims. The reported incidence of shoulder dystocia ranges from 0.6% to 1.4% among vaginal deliveries of fetuses in the vertex presentation; however, despite this, relatively rare occurrence lawsuits involving brachial plexus injuries after shoulder dystocia are now the second most common type of lawsuit in obstetrics.10,11 Eclampsia was chosen because it represents a medical as well as an obstetrical emergency, requiring rapid multidisciplinary care of the patient to protect the maternal airway while simultaneously reacting to fetal compromise. Of all the reported maternal deaths in the United States for the years 1979–1992, 19.6% were considered due to preeclampsia-eclampsia with 49% of those due to eclampsia.12
The objective of our prospective, randomized study was to determine whether simulation training is more effective than traditional didactic instruction in teaching obstetrical teams the management of two obstetric crises, shoulder dystocia, and eclampsia in an academic setting.
Stanford University Institutional Review Board committee approved the study. Institutional Review Board approval emphasized that any performance data generated in the study were confidential and not applicable to job promotion/demotions. This study was conducted from March of 2006 to March of 2007.
This was a randomized, prospective study. The study population consisted of labor and delivery nurses from one institution, Lucile Packard Children's Hospital (LPCH) with >1 year and <5 years of labor and delivery experience and obstetric residents from two institutions: Stanford University Medical Center (SUMC) and Santa Clara Valley Medical Center with no >5 years of labor and delivery experience.
Subjects were recruited by flyers and announcements in staff and resident meetings. All eligible staff was offered the opportunity to enter the study. Participation was voluntary. Participants who met criteria were selected at random. The residents and nurses were placed in teams. Each team consisted of two residents and two labor and delivery nurses. The teams were evenly divided between the didactic/control (Did) or simulation/experimental (Sim) group based on years of experience and previous exposure to simulation training to assure equal experience levels between the groups. Previous simulation experience was defined as having taken part in one 2-hour high-fidelity simulation course on team training conducted in 2005 (Table 1).
After the teams were formed, they were randomly assigned to either the Did or Sim group using a computer randomization program. Eight teams were created, each consisting of two obstetric residents and two labor and delivery nurses (Sim = 16, Did = 16).
The teams retained the same members throughout the study. All team members performed within their discipline. For those teams that had a member not arrive for the performance testing, (two simulation teams and three didactic teams) a substitute was provided. The substitute was a member of the faculty and would respond to commands but would not initiate activity.
Exclusion criteria were <6 months or >5 years experience on labor and delivery. Of note, all but two of the obstetric residents at Stanford Hospital participated in the study.
All subjects completed a multiple-choice questionnaire (MCQ) with 20 queries. The MCQ was produced by two experts; one in maternal fetal medicine and the other in advanced practice nursing using both American College of Obstetricians and Gynecologist (ACOG) and expert developed guidelines.10,13–15 Before the use of the MCQ in the study, an experienced obstetrical physician was independently administered the MCQ, and achieved a 95% score. It did not undergo validation testing. However, because it was derived from published ACOG and expert guidelines, we believe it addressed areas critical to the proper management of shoulder dystocia and eclampsia.
The information provided in both the lecture and simulation training sessions were based on the same ACOG and expert guidelines as the MCQ used for cognitive testing. Both control and experimental groups were administered the same questionnaire. The knowledge scores were individual scores but presented as combined data for each intervention group by discipline. Participants were not told their scores on the MCQ.
Educational Intervention Sessions
The course was conducted at The Center for Advanced Pediatric and Perinatal Education at LPCH. Center for Advanced Pediatric and Perinatal Education is a high-fidelity, simulated labor and delivery suite, with working programmable fetal monitors (FetalSim Advanced Medical Simulations, Inc, Binghamton, NY), a pelvic model (Simulaids Inc, Saugerties, NY), a full-body simulated patient mannequin (SimMan; Laerdal Medical, Wappingers Falls, NY), and a video-recording system.
The Sim group underwent 3 hours of team training on eclampsia and shoulder dystocia in a high-fidelity simulation studio. The Sim group did not receive structured lectures. All information was conveyed through the simulation scenario followed by the debriefing sessions. Teaching of crisis resource management (CRM) principles was included in the prebriefing and debriefing segments of the training.
The principles of crew resource management originated in the aviation field.16 They were then adapted to the medical field by Gaba et al17 for use in anesthesia and termed CRM. CRM addresses the nontechnical skills that a medical personnel need to maintain safety and work effectively as a team. CRM principles were explained to all the participants at the introduction to the simulation course and were reinforced at the debriefing sessions.
Each course began with a brief overview of simulation-based training and a discussion of the use of videotapes for debriefing followed by a detailed orientation to the simulated labor and delivery suite, mannequin, and equipment. All participants then took part in both scenarios. The scenarios were followed by a 40-minute debrief. Debriefing consisted of self-review by using the videotape of the simulation and a facilitated discussion of the individual's performance by trained faculty.18 Faculty instructors were physicians and advanced practice nurses from SUMC. The faculty had been trained by local experts in the field of debriefing and had been actively participating in simulation training courses for the previous 2 years.
The didactic group underwent the following: 1.5 hour of classroom lecture on management of eclampsia, observed a 26-minute videotape on shoulder dystocia produced by ACOG followed by 0.5 hour of “hands on” demonstration and practice of shoulder dystocia maneuvers on a pelvic model (Simulaids Inc, Saugerties, NY). This format represents our traditional teaching methods of management of shoulder dystocia and eclampsia in the department of obstetrics and gynecology at SUMC.
Simulation and Didactic Groups
After the completion of the educational interventions, participants in both groups received two peer-reviewed articles13,14 and two ACOG clinical guidelines on shoulder dystocia and eclampsia.10,15
One month after the training, all participants were administered the same MCQ used for the initial assessment. This was followed by a performance test.
Performance testing was performed as a labor and delivery drill. The testing drills were designed so as not to repeat the simulation training experience by making the following changes. Instead of a full-body mannequin, the drills used a standardized patient. Using a standardized patient added verisimilitude to the drill. An alternative pelvic model was used (Prompt Trainer by Limbs and Things, Inc. Savannah, GA) that neither group had practiced on previously. Crucially, the drills were performed on the labor and delivery unit at LPCH, a setting equally familiar to all the participants. Performance testing was performed for both eclampsia and shoulder dystocia. All scenes were videotaped for future scoring.
Checklist and Scoring
A checklist for scoring the team performances was designed by two experts: a maternal fetal medicine specialist and an experienced labor and delivery nurse. Team performances were scored by a single-maternal fetal medicine specialist who reviewed the videotapes and comprised a team score by the use of the predesigned checklist. The checklist included scoring for application of the maneuvers: for correct execution, efficiency as well as teamwork used during the drill. A score of 0 was given if the correct maneuver was either not performed or the performance lacked teamwork and efficiency. A score of 1 was given if the procedure was performed correctly, and the overall performance was efficient with adequate teamwork. A score of −1 was given if an incorrect maneuver was performed (Table 2). The reviewer was familiar with some of the participants but was completely blinded to the type of training provided to each team.
The difference between the performances of two groups was analyzed using independent samples Student t test and analysis of variance (one way). Mean scores and standard deviations were calculated (post minus pre) for the MCQ. P value of ≤0.05 was considered to be statistically significant. All data were analyzed using a statistical software package (SPSS version 12.0; Chicago, IL).
Thirty-two medical personnel were selected to participate in the study. Three nurses from the lecture group and one nurse and one resident from the Sim group did not complete the study leaving a total of 27 participants (Fig. 1). Their initial MCQ scores were excluded from analysis.
Comparison of pretraining and pretesting MCQ was compared. Results showed that there was improvement in overall scores, but it did not reach statistical significance at either baseline or after training. These scores were independent of the performance scores (Table 3).
Performance Testing Results
Performance evaluation of the shoulder dystocia drill found that the simulation trained teams scored significantly higher than the didactic trained teams (Sim = 11.75, Did= 6.88, P = 0.002). This was also found to be true for the eclampsia performance scores (Sim =13.25, Did = 11.38, P = 0.032; Table 4).
The individual team scores are provided (Table 5). No single team score overly impacted the overall performance results for either the Did or the Sim group.
Although both teams appeared to know the correct sequence of maneuvers, the simulation-trained teams consistently scored higher due to their efficacy, team work, and correct execution of maneuvers.
The leading causes of pregnancy-related deaths are embolism (20%), hemorrhage (17%), and pregnancy-induced hypertension (16%).1 Therefore, most morbidities and mortalities that occur in the obstetric setting are acute crises. Many of these crises on the labor and delivery ward are survivable—provided that we teach our teams to perform effectively during the crisis.
Our study demonstrated that cognitive knowledge as measured by the pre- and posttraining MCQ testing remained equal between the didactic and simulation teams.
The information tested in the MCQ was based on the current standard guidelines available on the specific subject matter. It was assumed that providers, at this experience level, should have obtained their baseline knowledge previously. By conducting the MCQ pretraining, we were able to confirm that the cognitive knowledge level among all team members was equal and is reflected by equal baseline scores. The equal improvement in MCQ scores, pre- and posttraining, suggests that information previously presented in the traditional lecture format can as effectively be taught with simulation training.
However, the ability to perform accurately and efficiently in a simulation of shoulder dystocia or eclampsia in a labor and delivery drill was significantly better after simulation training. Most importantly, we found that the simulation-trained teams consistently executed the maneuvers more smoothly, accurately, and with greater efficiency as a team than the didactic-trained groups.
Besides the obvious “practice makes perfect” explanation, previous studies have also documented the crucial role that good teamwork plays in clinical execution of procedures that require more than one person. Ellis et al19 found that the introduction of simulation training in the obstetrical emergencies of eclampsia “resulted in enhanced performance with higher rates of completion for basic tasks, shorter times to administration of magnesium sulfate, and improved teamwork.”
We believe that the findings in this study further support the principle that these elusive teamwork skills cannot be taught in a didactic setting. Simulation training is the only forum in which those roles can be assigned and perfected.
Our study has potential limitations. The overall number of trainees was very limited. There was an uneven experience level drop out of participants, which may have biased the results. All of the participants were relatively inexperienced, so it is unknown whether the same effect would exist if simulation training was given to seasoned providers.
Another possible limitation was we chose not to perform a preintervention measure of skills performance. Instead, we specifically selected the least experienced providers (<5 years) in an attempt to have equally skilled participants. Our goal was to assess the use of simulation as a teaching modality. We felt that a preintervention skills assessment would prematurely expose the participants to the subject matter and thereby mitigate the full impact of simulation as a teaching tool.
One unexpected finding was the lack of significant change between pre- and posttraining MCQ test scores in both Did and Sim groups. This may have been due to unexplored factors such as length of time elapsed between training and testing, but the fact that the Sim group performed as well as the Did group in both pre- and posttraining tests indicated that the Sim group experienced no handicaps in cognitive training.
The teams during the performance testing were identical to the teams during the Sim intervention. Therefore, there exists the potential effect of increased intrateam familiarity in the Sim group. Whether this team familiarity alone is the basis for the improved performance is not clear.
Another limitation to our study is the use of only one professional evaluator. Because of the complicated nature of each scenario, it was felt that the grader needed to have an in depth knowledge in the management of obstetrical emergencies. The rater who was chosen was a specialist in high-risk obstetrics, board certified in Maternal Fetal Medicine.
Finally, not specific to our study, but of concern for simulation in general is whether testing performance in a simulated setting, however “life-like,” reflects skills in an actual clinical event.
Future areas of investigation include: is simulation of value only for the inexperienced provider or can experienced providers benefit? What is the rate of retention or decay of the knowledge and CRM skills in simulation training versus the traditional didactic teaching modalities? What skills are best taught in a simulator?
Our study showed that simulation-trained individuals performed equally well in written tests of cognitive information when compared with didactic-trained individuals, but simulation training allowed health care teams to “experience” obstetrical crises and incorporate that learning when tested in a life-like setting. Simulation training seems to be a valuable alternative to traditional curriculum and should be used to enhance resident education in obstetrical emergencies.
1.Chang J, Elam-Evans LD, Berg CJ, et al. Pregnancy-related mortality surveillance—United States, 1991–1999. MMWR Surveill Summ 2003;52:1–8.
2.Danel I, Berg C, Johnson CH, Atrash H. Magnitude of maternal morbidity during labor and delivery: United States, 1993–1997. Am J Public Health 2003;93:631–634.
3.Sentinel event # 30 in Joint Commission for Accreditation of Healthcare Organizations Sentinel Event Alert Issue 30, 2004.
4.Kohn L, Corrigan J, Donaldson M, eds. To Error is Human: Building a Safer Health System. Washington DC: National Academy Press; 1999.
5.Hunt EA, Shilkofski NA, Stavroudis TA, Nelson KL. Simulation: translation to improved team performance. Anesthesiol Clin 2007;25:301–319.
6.Deering S, Poggi S, Macedonia C, Gherman R, Satin AJ. Improving resident competency in the management of shoulder dystocia with simulation training. Obstet Gynecol 2004;103:1224–1228.
7.Maslovitz S, Barkai G, Lessing JB, Ziv A, Many A. Recurrent obstetric management mistakes identified by simulation. Obstet Gynecol 2007;109:1295–1300.
8.Crofts JF, Bartlett C, Ellis D, Hunt LP, Fox R, Draycott TJ. Management of shoulder dystocia: skill retention 6 and 12 months after training. Obstet Gynecol 2007;110:1069–1074.
9.Draycott TJ, Crofts JF, Ash JP, et al. Improving neonatal outcomes through practical shoulder dystocia training. Obstet Gynecol 2008;112:14–20.
10.ACOG Committee on Practice Bulletins-Gynecology, The American College of Obstetrician and Gynecologists. ACOG practice bulletin clinical management guidelines for obstetricians-gynecologists. Number 40, November 2002. Obstet Gynecol 2002;100:1045–1050.
11.Professional Insurance Association of America risk management data, 2005.
12.MacKay AP, Berg CJ, Atrash HK. Pregnancy-related mortality from preeclampsia and eclampsia. Obstet Gynecol 2001;97:533–538.
13.Sibai BM. Diagnosis, prevention, and management of eclampsia. Obstet Gynecol 2005;105:402–410.
14.Gherman RB, Chauhan S, Ouzounian JG, Lerner H, Gonik B, Goodwin TM. Shoulder dystocia: the unpreventable obstetric emergency with empiric management guidelines. Am J Obstet Gynecol 2006;195:657–672.
15.ACOG Committee on Practice Bulletins—Obstetrics. ACOG practice bulletin. Diagnosis and management of preeclampsia and eclampsia. Number 33, January 2002. Obstet Gynecol 2002;99:159–167.
16.Helmreich RL, Merritt AC, Wilhelm JA. The evolution of Crew Resource Management training in commercial aviation. Int J Aviat Psychol 1999;9:19–32.
17.Gaba DM, Fish KJ, Howard SK. Crisis Management in Anesthesiology. New York: Churchill Livingstone; 1994:XV;294.
18.Fanning R, Gaba D. The Role of Debriefing in Simulation-Based Learning. Simul Healthc 2007;2:115–125.
19.Ellis D, Crofts J, Hunt F, Read LP, Fox R, James M. Hospital, simulation center and teamwork training for eclampsia management: a randomized controlled trial. Obstet Gynecol 2008;111:723–731.
© 2010 Lippincott Williams & Wilkins, Inc.