Simulation training, originally developed for nonmedical high-risk performance environments, such as aviation and military training, provides comprehensive learning opportunities in a risk-free setting.1 As an educational adjunct, medical simulation training offers the opportunity to make mistakes in a controlled setting, participate in a variety of simulated exercises, and use repetition to aid in learning.1–3 As opposed to the “see one, do one” approach, with simulation students can learn, practice, and repeat clinical skills or procedures until proficiency is achieved before ever touching a real patient.
Focus on medical errors and need for improved patient safety, limited patient availability for educational purposes, and outcomes-based educational requirements make simulation training an attractive approach in addition to traditional teaching methods for medical students and residents. Simulated experiences, such as virtual anatomy and CPR training on simulated mannequins, are currently well integrated into undergraduate medical education. Simulation training lends itself well to the field of obstetrics4,5 and, in particular, to the teaching of vaginal delivery maneuvers. The procedural skills needed to perform a vaginal delivery are not readily gained through reading texts or listening to lectures. Further, the profound experience of obstetric delivery generally is not easily transformed into a learner-centered experience.
Although simulation training seems ideal for use in various obstetric clinical scenarios, surprisingly little is known about its application in obstetric undergraduate education. To date, two studies have been published addressing medical student simulation training in obstetrics.6,7 Jude et al6 reported increased student confidence in performance of vaginal delivery after simulated experiences. However, no aspects of student performance were studied. Clinical skills assessment tools are being incorporated in general and specialty surgical fields,8,9 including gynecology10 to assess skills performance. To our knowledge, these tools have not been applied or modified to obstetric undergraduate or postgraduate training for assessment of delivery facility or to evaluate whether gaining delivery skills improves participation in live deliveries. The purpose of this project was to determine whether any relationship exists between simulation training for delivery maneuvers and participation in live deliveries during the clinical rotation, to assess student self-confidence in performance of these maneuvers, and to investigate whether training on a mannequin-based simulator improves performance of the maneuvers required for vaginal delivery.
This study was performed in accordance with guidelines and regulations of the Montefiore Medical Center Institutional Review Board. The third year medical students were approached for voluntary participation in the study at the start of their clinical rotation in Obstetrics & Gynecology and Women’s Health (Ob/Gyn) at Montefiore Medical Center and the Albert Einstein College of Medicine, Bronx, NY. A competency-based obstetric skills assessment tool for simulated vaginal delivery was designed and tested before initiation of the study of student performance. This tool was then used to assess medical students’ competency in performing vaginal delivery maneuvers on a macro NOELLE Vaginal Delivery Simulator (Gaumard Scientific, Coral Gables, FL).
The competency-based skills assessment tool was designed in conjunction with the eight members of the Ob/Gyn clerkship teaching committee. The members identified medical student learning objectives for vaginal delivery by reviewing the Medical Student Educational Objectives published by the Association of Professors of Gynecology and Obstetrics.11 The Educational Objectives: Core Curriculum in Obstetrics and Gynecology, published by the Council on Resident Education for Obstetrics and Gynecology was used as an additional reference.12 The assessment tool was formulated by consensus from this experienced obstetric clinical faculty to evaluate essential skills and standards of performance for obstetric maneuvers and delivery, in the third year medical students. Eleven critical skills were identified (Fig. 1). These skills were incorporated into an assessment tool by the author (A.D.) and assessed as present or absent. The assessment tool was tested with nine second year Ob/Gyn residents from within the institution and five third year medical students from an outside medical school, performing live deliveries. No changes to the assessment tool were made after this test.
Figure 2 describes the sequence of randomization, instruction, and testing for the project. On the orientation day of the clerkship, students (eighty-eight), from all clerkship sites were invited to participate in this study to assess basic obstetric delivery skills on the Noelle vaginal delivery simulator. Students who signed and returned the consent form (33) were randomized into either a simulation group (SIM) or a control group (CON). Allocation was determined with sequential opaque envelopes containing allocation cards, designating group assignment. The allocation cards had been randomly inserted into the envelopes by a physician who was not involved in the study. Assignment was concealed until after enrollment was complete.
On the day of randomization, both test groups shared a lecture on delivery maneuvers and demonstration with a pelvic skeleton and doll that was part of the core clerkship lectures for all students. Then the group assigned to simulation training participated in a 60-minute session that included a lecture and demonstration of a normal vaginal delivery on the Noelle vaginal delivery simulator. Next, the SIM had a preceptor-led teaching and practice session using Noelle, with individual and group instruction, and received corrective feedback on the delivery maneuvers. The SIM practiced on the simulator for approximately 1 to 2 hours, and the number of simulated deliveries was dependent on student comfort and skill acquisition. The CON received no training in vaginal delivery with the Noelle at any time during the study. Within 3 days after randomization, both the simulation and the CONs, after orientation to the mannequin and test procedure, were assessed for skill with vaginal delivery maneuvers on the simulator. No feedback or details of the evaluation tool were provided to participants. The researcher (A.D.) was familiar with the assessment tool, performed the observation and assessment, and was blinded to randomization. All study participants were reassessed in a similar fashion, on the simulator, at the conclusion of the clerkship with an identical vaginal delivery scenario, and the same competency-based assessment tool was used.
All students received “traditional teaching” throughout the course of the clerkship. Traditional teaching consisted of didactic lectures describing pelvic anatomy, labor, cardinal movements of labor, and vaginal delivery maneuvers. Additionally, all students received preceptor instruction during live deliveries. The required 6-week clerkship is offered throughout the year, with approximately 30 students rotating per 6-week block. The clerkship provides experience and instruction for students in six different clinical sites, located throughout New York City. Core didactic teaching occurs at the medical college, 1 day/week, with all rotating students attending. The learning objectives and expectations for clinical exposure are standardized across all sites. On each labor and delivery unit, students are encouraged and expected to follow selected patients in labor, participate in clinical activities, and assist and perform deliveries under direct supervision. Students are assessed on their working knowledge, history, and physical examination skills and their participation in patient care. All students complete required patient/procedure logs during the clerkship and are required to participate in at least five vaginal deliveries. Currently, students are not specifically assessed on their ability to perform either obstetric maneuvers or full vaginal delivery.
Study participants in both groups were e-mailed previously published nonanonymous surveys, measuring the medical student’s individual level of confidence in conducting a vaginal delivery. These identical surveys were administered immediately after randomization and at the completion of the clerkship.4 Questions included comfort with the ability to deliver the head, shoulders, body, and placenta, to inspect the perineum, and to participate in a delivery. Comfort with “hands on” (ie, manually guided) and “hands off” (ie, verbally guided) levels of supervision were also asked. The possible self-confidence responses (“I am comfortable with….”) were in a 5-point Likert scale ranging from strongly disagree to strongly agree. Students were asked to e-mail the completed confidence surveys to an author (N.F.), and reminders were sent to achieve a high rate of return. After rotation completion, an author (N.F.) evaluated student participants’ obstetric patient logs to determine the number of deliveries the student participated in during the clerkship.
No background data were available to perform a power calculation a priori. Data were collected by direct observation of the participant performing obstetric maneuvers of vaginal delivery on the simulator. Each maneuver skill was evaluated as being present or absent during the simulated delivery test. For purposes of analysis, skills were assigned a value of 1 if present and 0 if not present (Fig. 1). It was decided before initiating the study that a delivery should be judged on more than individual delivery maneuvers and that some assessment of overall performance was necessary; therefore, a final assessment score based on the simulated delivery was also assigned. The final score was not a composite of individual skill components but rather was reflective of a combination of overall delivery fluidity and skill. The grading rubric for the skills components of the final delivery score assigned eligibility for a score of 5 if none or one maneuver skill element was missing, eligibility for a score of 4 was assigned if two skills were not present, and eligibility for a score of 3 was assigned if three skills were missing. Eligibility for a score of 2 was assigned if four maneuvers were missing, and a score of 1 was possible if five maneuvers of vaginal delivery were not observed. A 0 score was assigned if greater than five maneuvers of delivery were not observed. Scores for each student were entered into an Excel database, and statistical analysis was performed using SPSS 2007 (Chicago, IL). A P value of <0.05 was considered significant. For comparisons within each group, a Wilcoxon Signed Rank test was used, and for comparisons between groups, a Mann-Whitney U test was used.
Thirty-three students participated during the course of four clerkship rotations, beginning in July 2007. Seven of the 33 students were in the first clerkship of the third year, 14 of the 33 were in the second clerkship, seven in the third rotation, and five in the fourth. Two of the 33 medical students were the fourth year students, returning to complete the core clinical clerkships after leaves of absence during the third year. This was their first and only obstetrics exposure. Fifty-five students declined to enter the study (37.5% rate of participation). Eighteen students were in the SIM, seven men and 11 women, and 15 students were in the CON, four men and 11 women.
Self-reported obstetric patient clerkship logs demonstrate that the SIM participated in significantly more supervised live deliveries than CON (9.8 ± 3.7 versus 6.2 ± 2.8, P < 0.005). Sixty-six self-assessed confidence surveys (100% return rate) were received. The first collected survey demonstrated no significant differences between the simulated and the nonsimulated groups, in self-reported confidence for performing individual skills or complete vaginal delivery. Measures of self-confidence were low for both. The second survey of student confidence, completed at the conclusion of the clerkship, demonstrated significant differences between the simulated and the nonsimulated groups. The SIM expressed more confidence in response to all questions of comfort, when compared with CON, but statistical significance was found with delivery of the shoulders (4.25 ± 0.856 versus 3.36 ± 1.1, respectively, P < 0.05) and delivery of the body (4.56 ± 0.63 versus 3.91 ± 0.54, P < 0.05). Similarly, a significant difference was seen in the second survey between the simulated and the nonsimulated groups in the perceived confidence to deliver with “hands off” supervision (3.81 ± 0.83 versus 3.00 ± 1.0, P = 0.034). See Table 1.
The overall delivery score in the first maneuvers’ skills assessment was significantly greater in the SIM than in the CON (3.94 ± 0.94 versus 2.07 ± 1.22, P < 0.001). At the conclusion of the clerkship, the overall delivery score in the second skills assessment was also significantly greater in the SIM than the CON (4.88 ± 0.33 versus 4.31 ± 0.63, P = 0.007). The SIM demonstrated increased scores from the first to the second skills assessment, (3.94 ± 0.94 versus 4.88 ± 0.33, P = 0.002). In the CON, there was also a significant difference in mean score, (2.07 ± 1.22 versus 4.31 ± 0.63, P = 0.002) from the first to the second assessment. Significant differences were seen between the groups at the early assessment for the following skills: control of the fetal head, check nuchal cord, deliver posterior shoulder, placental inspection, and perineum inspection, P < 0.05. In the second assessment, only placental inspection demonstrated a significant difference, P < 0.05.
This study demonstrates that medical students trained on a mannequin-based simulator showed increased self-assessed confidence in basic delivery maneuver skills and, more importantly, seemed to translate this into increased participation in live clinical deliveries. We believed this increased confidence, and participation was based on demonstrated improved performance in basic maneuvers for a simulated vaginal delivery.
We postulate that increased confidence, based on increased competence with “basic” skills, led to increased initiative and comfort in the live clinical situation. Although the minimum number of supervised deliveries required for the rotation (five) is similar for all students, the number of deliveries that the students who underwent simulation training participated in was higher than those who did not undergo training. Students demonstrating more confidence and a higher level of initial skills may be incorporated into the obstetric team earlier, and patient response toward medical student participation might be more favorable. Certainly, students apply to medical school to gain clinical, not simulated, skills. However, providing early simulation training in our clerkships may facilitate a positive clinical experience, improve skill acquisition and learning in the clinical environment, and increase clinical confidence. Additionally, achieving these goals during the obstetrics and gynecology clerkship could be quite beneficial toward recruitment of obstetrics and gynecology residents.
Overall performance of delivery maneuvers was significantly higher in the SIM compared with the CON at the beginning and end of the clerkship. The CON did have particular difficulty with the delivery of the shoulders, both at weeks 1 and 5. Future studies with simulation could target this aspect of a delivery to increase comfort. The CON did show an improvement in overall scores from the first assessment to the second assessment for vaginal delivery maneuvers. The SIM showed an improvement in scores and maintained their skills throughout the clerkship. We also found that performance level was still lower at the end of the rotation in the CON compared with those who had extensive simulation training, thus suggesting that the simulation training had primed the learners in this group to be able to achieve a greater level of skill by the end of the rotation. Significant score improvement could not be shown independently for each competency skill, possibly because of the small sample size in each group.
A significant limitation to this and many simulation-based projects is the possibility that increased familiarity with the simulation environment contributed to the differences seen in delivery maneuver scores between the SIM and the CON. However, familiarity with the simulator does not account for the differences we observed in confidence or clinical participation.
Another limitation to this project was the rate of participation by the students. It was lower than we originally anticipated. The timing of the study may have contributed to the lower participation, because the study was conducted in the first clerkships of the third year of medical school. Students may have been too overwhelmed to commit to participation. We do not believe that students who participated were significantly different than those who did not, because the number of deliveries in the logs was similar in range. The timing of the study may have also contributed to the increase in confidence seen in the SIM. Perhaps, if the study had been repeated later in the academic year, differences in confidence would be less apparent.
The benefits of medical simulation training extend beyond repetitive skills training and may be used to record a variety of performance metrics that can be used to assess skill acquisition and competency, which has become increasingly important part of medical education. We designed a competency-based obstetric skills assessment tool for simulated vaginal delivery to assess medical students’ competency in performing vaginal delivery maneuvers. We did not assess competency in the maneuvers during an actual patient delivery because of the inability to standardize each clinical situation and observer. To the novice learner, the variation among “simple” deliveries can be tremendous. Nevertheless, our assessment tool could be applied to live deliveries.
The use of medical simulation training continues to grow, but barriers to its implementation still exist. One of the limitations of using a simulation for student education is cost. A relatively low-tech simulator, such as the one used here, costs approximately $5000, and faculty time necessary to prepare and conduct a training session amounts to approximately 3 to 4 hours per clerkship group. More advanced models that mimic complex pathophysiology are more costly. In an era of cost containment, spending on simulation may be difficult to justify when the evidence of effectiveness (as it applies to patient safety) is limited. However, given the nonmonetary benefits of practice without fear of doing harm, learning from mistakes without fear of punitive responses and standardizing the educational experience, the educational value may justify the expense.
Medical simulation training presents many benefits in Obstetrics, a high-risk specialty uniquely attuned to patient safety. The use of simulation training gives medical students the ability to learn and practice necessary skills for vaginal delivery in a controlled setting. We believe that we have shown that simulation training for vaginal delivery leads to a higher level of participation during the clinical clerkship, better performance of delivery maneuvers, and improves self-assessed confidence. Future studies are needed to address whether simulation training may make the clinical obstetrics rotation more enjoyable for students and promote recruitment to the specialty.
1. Macedonia CR, Gherman RB, Satin AJ. Simulation laboratories for training in obstetrics and gynecology. Obstet Gynecol
2. Johannsson H, Ayida G, Sadler C. Faking it? Simulation in the training of obstetricians and gynaecologists. Curr Opin Obstet Gynecol
3. Advanced Initiatives in Medical Simulation. Building a National Agenda for Simulation Based Medical Education
. In: Advanced Initiatives in Medical Simulation; Washington, DC: 2004.
4. Crofts JF, Ellis D, Draycott TJ, et al. Change in knowledge of midwives and obstetricians following obstetrical emergency training: a randomised control trial of local hospital simulation centre and teamwork training. BJOG
5. Gardner R, Raemer DB. Simulation in obstetrics and gynecology. Obstet Gynecol Clin N Am
6. Jude DC, Gilbert GG, Magrane D. Simulation training in the obstetrics and gynecology clerkship. Am J Obstet Gynecol
7. Deering SH, Hodor J, Wylen M, et al. Additional training with an obstetric simulator improves medical student comfort with basic procedures. Simul Healthc
8. Reznick RK, MacRae H. Teaching surgical skills-changes in the wind. N Engl J Med
9. Kneebone R. Simulation in surgical training: educational issues and practical implications. Med Educ
10. Banks EH, Chudnoff S, Karmin I, et al. Does a surgical simulator improve resident operative performance of laparoscopic tubal ligation? Am J Obstet Gynecol
11. APGO. Medical Student Educational Objectives
. 8th ed. MD: Association of Professors of Gynecology and Obstetrics; 2004.
12. Council on Resident Education in Obstetrics and Gynecology. Educational Objectives: Core Curriculum in Obstetrics and Gynecology
. 8th ed. Washington: Council on Resident Education in Obstetrics and Gynecology; 2005.