In medical emergencies, the patient is often conscious, and there remains a direct interaction between the patient and carers. Patients' perceptions of this interaction are important to study. In a report, the patient's perception was the stronger predictor not only of health outcomes but also of efficiency of health care.1 Emergencies demand excellent communication, but published evidence suggests that doctors vary widely in their ability to communicate to the satisfaction of patients.2
Any interaction with the patient is particularly important in maternity care because the woman often has an active part to play. However, there has been an acknowledgment that nontechnical skills are not properly addressed in obstetric training programs.3 The result can be poor communication with women and with their partners and relatives who might also attend. Experience of difficult interaction with maternity staff during labor has been shown to be associated with significantly increased risk of women having negative views of their delivery.4–6 This can contribute to postnatal depression and posttraumatic stress disorder.7 Communication training might be an answer for this problem and hybrid simulation (HYB), which includes patient-actors, an effective way to deliver it.
Properly trained patient-actors are valid and valuable proxies for developing and assessing communication skills. One study demonstrated that patient-actors working to an agreed protocol are reliable reporters of physicians' practice. Their reports of physicians' quality of care compare well (kappa = 0.81) with independent external assessments of audiovisual recordings of patient-doctor interactions,8 which means that only one actor may be adequate for assessments at exploratory-level studies. A study concluded that their use as a “gold standard” in comparing the quality of care across interventions or sites is justified.8 In contrast, computerized mannequins may be inappropriate for developing nontechnical skills. Advanced technology allows mannequins to “speak,” but in a large study of qualified doctors and midwives, the Simulation and Fire-drill Evaluation (SaFE) study, training with high-tech mannequins did not seem to improve communication skills as much as training with patient-actors.9 When designing obstetric teaching interventions that try to imitate the demands of real-life labor ward crises, behavioral fidelity may be as important as the technical quality of the equipment. Rehearsals that are based solely on a mannequin run the risk of focusing too much on the maneuvers alone, at the risk of ignoring the woman. Bad habits could then be inculcated early on in development. To the contrary, integration of patient-actors and mannequins (HYB) is an inexpensive, simple, and effective training method that allows both manual and verbal skills to be learnt and practiced simultaneously. We have demonstrated previously that simulation training for an obstetric emergency [shoulder dystocia (SD)] that included patient communication resulted in a significant increase in patient-actors' perception of care, as measured using a validated scale, the patient perception score (PPS).9
The PPS is a simple tool that was used in the SaFE study. It had construct and consequential validity for simulated obstetric emergencies9; after training, there was a significant increase in every aspect of the patient-actor's perception of their care, given by qualified staff, during SD [median (mean, range)]: respect from 4 (3.7, 2–5) to 4 (4.3, 2–5) P < 0.001, safety from 3 (3.4, 1–5) to 4 (4.2, 2–5) P < 0.001, and communication from 4 (3.7, 2–5) to 4 (4.2, 2–5) P < 0.001. In the same study, perception of safety and communication was significantly improved after training with patient-actors compared with training with mannequins (safety P = 0.048 and communication P = 0.035).9
The primary aim of this study was to investigate whether students' communication skills during simulated emergencies, as assessed with the PPS, are improved by rehearsals that include patient-actors. Secondary aims were to investigate whether high-fidelity simulation is useful for improving students' manual skills, and to assess their views about this form of training.
This was a randomized controlled trial with anonymized data collection. The assessor was blind to the assigned group. The chief investigator (DS) conducted the random allocation using an electronic random-number generator. The CONSORT statement was followed to report the manuscript10 and the CONSORT extension for the preparation of the abstract.11 The study was conducted as an exploratory phase II trial12 with the purpose to inform the design of future definitive RCTs.
Research Setting and Recruitment
The study was conducted in a large maternity unit within a University Hospital in the Southwest of England. During 2007–2008, four cohorts of fourth-year medical students, with 12 in each group, were attached to the Department of Obstetrics & Gynaecology (O&G) for 8 weeks. All four groups went through the SD training program. The first two groups were used as pilots to allow the training method to be learnt and refined. The last two cohorts were recruited to the study proper and were subjected to assessment before and after the training schedule.
Simulated SD was used for the standardized emergency scenario. SD is an emergency that arises as the head delivers. One of the fetal shoulders becomes stuck behind the maternal pubic bone preventing delivery of the body. SD is an uncommon and largely unpredictable event that carries risk of serious permanent injury to both mother and baby. The fetus is at risk for both physical injury (nerve root damage, bone fractures, cerebral palsy, etc) and of death due to hypoxia-acidosis.
Specific obstetric maneuvers have been shown to be associated with a good outcome in the large majority of cases, and these have been adopted into a national guideline.13
The first phase of the study consisted of a pretraining assessment at the beginning of the clinical attachment. The participants had no knowledge that it was a SD scenario. They were asked to deliver the body of a baby with simulated SD, using a high-fidelity mannequin, using as much force and for as long as they felt appropriate to complete the delivery if left alone in a birth room. We compared the peak force they used to that observed for labor ward staff in the Department-of-Health funded SaFE study.14
All participants then attended a simulation session (SIM) on management of SD that included practical demonstration using a high-fidelity mannequin. All information given to participants verbally was derived from national guidelines.13 The mannequin (Limbs and Things Ltd., Bristol, UK), which was specifically designed for SD training, consists of accurately modeled maternal and fetal components.
Toward the end of the attachment (5–6 weeks later), a refresher session was conducted for which the participants were randomly allocated to one of two training interventions. Half attended a small-group tutorial (SGT) and the other half attended a SIM with a patient-actor (HYB). For both sessions, we used standardized training15 materials adapted for undergraduate use. For HYB, the actor was integrated with a part-task trainer; she was kneeling behind the pelvic model (Fig. 1). The training materials include specific instructions to the patient-actor on how to interact with the learners and when to provide verbal cues. In addition to the standardized script, the manual also contains instructions on how to provide feedback to learners about aspects of the communication that are perceived as good or as in need of improvement.
The control group (SGT) was not given such communication teaching. They were shown again how to manage SD, having had already attended the initial SIM. An MS Powerpoint presentation was used that focused on theory and practical management and was developed from standardized evidence-based training material.15 The Powerpoint slides contained images depicting the correct management of SD and which errors to avoid, based on findings from 450 simulations.16
Within a week from the refresher seminar, participants from both groups (SGT + HYB) were asked to manage a SD simulation scenario while communicating with a patient-actor, again kneeling behind the pelvic model. A healthcare professional played the role of the partner.
We assessed communication skills of the two groups (simple tutorial and workshop with patient-actor) using a validated scale derived from the SaFE study (PPS).2,9 The communication skills were assessed by an experienced assessor (CB) blinded to the intervention group, who had attended a formal “patient-actor” training day for the SaFE study. The purpose of that study day had been to standardize both delivery of the scenario and assessment. For each drill, the patient-actor assessed the quality of communication using three five-point Likert scales (5, strongly agree; 4, agree; 3, neutral; 2, disagree; and 1, strongly disagree): Communication – “I felt well informed due to good communication,” Respect – “I felt I was treated with respect at all times,” and Safety – “I felt safe at all times.” All participants also completed a six-item, five-point Likert-scale survey to assess their reaction to the training intervention. The statements were (1) simulators are valuable tools for teaching medical students; (2) I enjoyed the simulation teaching session; (3) the drill evoked a realistic response from me; (4) my knowledge has improved after the session; (5) I feel more confident to deal with similar situations now; and (6) this SIM should be part of the attachment's curriculum. For both the SaFE-PPS and the reaction survey, the five-point Likert scale was 1, strongly disagree; 2, disagree; 3, neutral; 4, agree; and 5, strongly agree.
The Kirkpatrick model17 was used to evaluate the success of our teaching using our chosen outcome measures: level 1 reaction—five-point Likert-scale from six-item questionnaire; level 2 skills—success in achieving delivery, reduction in average and peak applied force, use of safe maneuvers, time from diagnosis to delivery within safe limits; and level 3 clinical behavior—SaFE-PPS. For level 4, outcome, measuring patient or organizational outcomes and associating them with medical students teaching is not realistic. Although the outcomes of medical simulation could be used as a proxy for real-life outcome in such circumstances, we decided to report these as a measure of skill, evaluation level 2.
To evaluate these skills, force and time measurements were collected with a 20-Hz transducer. The data were transferred to a bespoke computer program by the umbilical cord and then stored as a Comma-Separated-Variable file in an Excel spreadsheet for later analysis. Force was measured in Newtons (N). Appropriate and safe use of obstetric maneuvers was assessed and agreed by three independent assessors, two doctors and a midwife, using a checklist derived from published national guidance.13
Data Collection and Statistical Analysis
All data were anonymized. Participants chose a password at the beginning of the attachment for all studies and used the same password in every questionnaire or study record. The consent to research forms contained a tear-off slip on which participants noted their password to keep for future reference in any study throughout the attachment.
We treated communication scores as ordinal and used Mann-Whitney U to compare their distribution and also Fisher exact test to compare the number of participants with good scores [4 = agree, 5 = strongly agree (that I was respected/ informed/felt safe)] between the two groups. We compared the force they used posttraining (HYB + SGT) with that they had used pretraining using paired t test. To inform futures studies, we also calculated effect size as risk ratio with 95% confidence intervals (CI). Statistical significance was set at P < 0.05.
Ethical approval was granted by the University of Bristol, Faculty of Medicine & Dentistry, Committee for Ethics (reference number 070809, January 2008). Participants were asked by the chief researcher during an introductory tutorial at the start of their attachment whether they would consent in writing to participate in the studies and agree to the random allocation process. They were given ethically approved information leaflets before recruitment. They were informed that they were entirely free to decline to enter the study and that if they chose not to participate it would have no bearing on their teaching or assessment.
All eligible subjects consented to study and signed participation agreement forms. Only 12 subjects (first cohort) participated in the first phase (pretraining unprepared simulation). Feedback suggested that they perceived the session as stressful. Thereafter, we decided to stop the baseline assessment and to undertake only the training (SIM) and refresher (SGT or HYB) sessions with the second cohort and beyond. Force data were collected from the 11 (of 12) subjects of the first cohort who had participated in the first phase. One of the 12 subjects had been rostered to a night attachment.
All 24 subjects recruited for the second phase attended the main training workshop (SIM) at the end of the first week of their attachment. Twenty subjects received the refresher intervention in their 7th or 8th week of the attachment: 11 attended the simulation workshop with a patient-actor (HYB) and nine attended the tutorial (SGT). A further four subjects who had consented to the study were unable to attend due to other commitments (rostered for night attachment). All 20 subjects were assessed for communication and clinical skills. The official CONSORT diagram is depicted in Figure 2.
The subjects whose training had included the simulation workshop with a patient-actor (HYB) had significantly higher median communication score than subjects in group SGT (Fig. 3, median = 4 HYB versus 3 SGT; P = 0.0128, Mann-Whitney U). Their total PPS was also significantly higher (11 versus 9; P = 0.0239, Mann-Whitney U). They had higher median safety score (4 versus 3; P = 0.31, Mann-Whitney U), but this was not statistically significant. The respect scores were not significantly different (3 versus 3; P = 0.19, Mann-Whitney U).
The percentage of participants with good communication scores (≥4/5) was also significantly higher for the patient-actor workshop group (74% HYB versus 22% SGT). The likelihood ratio for participants of group HYB to have good communication scores (≥4/5) compared with participants of group SGT was RR = 3.2727, 95% CI = 0.9148–11.7089, P = 0.03 (Fisher exact test). For the other scales, the effect size was smaller but of consistent direction (toward improvement): Respect: 64% HYB versus 45% SGT, RR = 1.4318, 95% CI = 0.6082–3.3709, P > 0.05 and Safety: 44% HYB versus 33% SGT, RR = 1.3636, 95% CI = 0.4413–4.2136, P > 0.05.
Of the 11 participants in the preliminary study, none achieved delivery of the baby. On average, they spent 54 seconds (range 20–259 seconds) attempting delivery before abandoning and/or calling for help. The geomean peak force used by medical students (61 N, range 17–255 N) was similar to that reported for trainee doctors before training (55 N, minimum 5 N, and maximum 236 N) in the SaFE study.18 Some were tentative; others used very high forces up to 250 N.
Of the 20 participants in the second phase of the study who underwent assessment (B), all achieved delivery with correct performance of all basic actions, within a geometrical mean of 149 seconds (range 96–212 seconds).
For the 11 participants in the first cohort who were assessed with force monitoring before and after training, the average force they used throughout the drill reduced significantly (geometrical mean 16 N pretraining to 1 N posttraining; P = 0.005, one-way-paired t test). They also tended to use less peak (geometrical mean 61 N pretraining to 30 N posttraining; P = 0.298, one-way-paired t test) and total force (40,090–11,600 N; P = 0.169, one-way-paired t test). Only two of 11 (18%) participants exceeded applied force of more than 100 N after training, the threshold beyond which the fetus is thought more likely to suffer permanent injury,14 when compared with 42% pretraining (P = 0.221, χ2).
We received 16 completed questionnaires, a completion rate of 67%. Reaction to training was very positive. Ninety-four percent enjoyed the sessions, 100% thought that simulation was realistic and 88% that it was valuable, 88% felt more knowledgeable after training, and 81% more confident.
Rehearsals of an obstetric emergency were used to train practical maneuvers and communication skills in 2 cohorts of fourth year medical students during a clinical attachment. It has previously been shown that this training package can improve the management of simulated SD by doctors and midwives.9,18,19 Furthermore, introduction of the training program has been associated with improvement in real-life outcomes for babies.20–22 As such, it provided a potentially good example to evaluate the results of an educational intervention in medical students.
To measure the success of the intervention in improving communication skills, we used the PPS. The PPS is a simple instrument and loses some content validity because it cannot explore issues in depth. However, it gains in external validity, because, in real life, women with any educational background are likely to complete it. Indeed, a larger real-life study2 demonstrated that response rates are excellent, and PPS scores had very good internal consistency (Cronbach alpha = 0.83). The PPS could identify outliers, and scores differed between accoucheurs of different seniority. PPS scores also correlated strongly2 with scores in another established maternal satisfaction questionnaire (Spearman r = 0.64, P < 0.001).7 Therefore, PPS scores in HYB may be valuable proxy measures for real-life communication skills.
Using the PPS, we found that HYB training results in better communication skills with patient actors than standard methods without special communication training. Additionally, the results from this exploratory study suggest that the medical student response to medical simulation training may be very similar to that of doctors and midwives in that they may be able to learn practical skills quickly. The high attendance rate and very positive feedback indicate that such training is also popular.
Overall, our pilot intervention seems to be successful in the context of medical simulation. However, this study focused on quantitative assessments. Future studies should include qualitative evaluation methods and explore other types of communication training. Some may object to the design that used no communication training for the control group. We believe this was necessary, because the latter was standard practice for undergraduates until our study. Later studies could compare the efficacy and efficiency of various methods of communication training instead. One further criticism could be that the study groups are relatively small, but this was an exploratory study, designed to inform development of a whole-scale intervention. Statistical significance was set at P < 0.05. This was considered adequate as both the number of statistical comparisons (tests) performed and the number of participants was small; moreover, there were no subgroup analyses. The statistical significance level is only indicative. We encourage readers to consider the size of the effect in preference: students who had HYB training were three times (RR = 3.2727) more likely to have good communication scores. However, our study relied on the perception of a single rater, which might have been biased by her personal disposition. We acknowledge that several raters and repeated measurements might be preferable in future studies to reduce measurement bias, as well as assessments of intrarater and interrater reliability. We did not have access to full demographic data for both groups; furthermore, we did not have data on their previous exposure to communication training. To our knowledge, there had not been any formal training with patient-actors at any other time in the curriculum. Therefore, we expected that their skills at baseline would be similar after the random allocation, but residual confounding was possible. Finally, completion rate was 67% for the reaction questionnaire. There is possibility of response bias, if less satisfied students had been less likely to complete or return the questionnaire, but response rates of ∼70% are usually considered satisfactory for surveys.
We acknowledge that the benefit of this training for medical students is more tenuous than for qualified maternity staff because the time between training and clinical practice will be more than 2 years, and our study demonstrated only short-term retention of skills. Medical students no longer gain hands-on experience in obstetrics, decreasing satisfaction with clinical attachments.23,24 This may be partly responsible for poor recruitment into obstetrics, which has recently been a significant problem for the speciality.25–27 Simulation might be a means of making clinical attachments more interesting, with active learning and the facility for direct feedback, and may increase interest in obstetrics as a career. In a more general sense, this study suggests that rehearsals might help prepare all medical students for true-life emergencies and perhaps allow some to see that they have a particular aptitude for the management of serious medical crises.
In conclusion, this exploratory study showed that simulation of medical emergencies seems to carry important educational benefit for medical students in a way very similar to that for qualified obstetricians and midwives. In addition to having been an exercise that stimulated interest, it enabled the participants in this study to improve both manual skills and the ability to communicate with patient-actors. What should be aimed for in future definitive studies is to confirm this improvement and assess whether better skills in simulation can be retained and translated into improved management of real-life emergencies after qualification.
1.Stewart M, Brown JB, Donner A, et al. The impact of patient-centered care on outcomes. J Family Pract
2.Siassakos D, Clark J, Sibanda T, et al. A simple tool to measure maternal perceptions of operative birth. BJOG
3.Kitzinger S. Sheila Kitzinger's letter from Europe: the making of an obstetrician. Birth
4.Rijnders M, Baston H, Schonbeck Y, et al. Perinatal factors related to negative or positive recall of birth experience in women 3 years postpartum in the Netherlands. Birth
5.Kirke PN. Mothers' views of care in labour. Br J Obstet Gynaecol
6.Kirke PN. Mothers' views of obstetric care. Br J Obstet Gynaecol
7.Goodman P, Mackey MC, Tavakoli AS. Factors related to childbirth satisfaction. J Adv Nurs
8.Luck J, Peabody JW. Using standardised patients to measure physicians' practice: validation study using audio recordings. BMJ
9.Crofts JF, Bartlett C, Ellis D, et al. Patient-actor perception of care: a comparison of obstetric emergency training using manikins and patient-actors. Qual Saf Health Care
10.Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials. Lancet
11.Hopewell S, Clarke M, Moher D, et al. CONSORT for reporting randomised trials in journal and conference abstracts. Lancet
12.MRC Health Services and Public Health Research Board. A Framework for Development and Evaluation of RCTs for Complex Interventions to Improve Health
. Medical Research Council; 2000. Available at: http://www.mrc.ac.uk/Utilities/Documentrecord/index.htm?d=MRC003372
. Accessed July 9, 2009.
13.Draycott T, Fox R, Montague I. Royal College of Obstetricians and Gynaecologists. Shoulder Dystocia
. Clinical Guideline No. 42. London: RCOG Press; 2005.
14.Crofts JF, Ellis D, James M, Hunt LP, Fox R, Draycott TJ. Pattern and degree of forces applied during simulation of shoulder dystocia
. Am J Obstet Gynecol
15.Draycott T, Winter C, Crofts J, Barnfield S. PRactical Obstetric Multiprofessional Training (PROMPT) Trainer's Manual
. London: RCOG Press; 2008.
16.Crofts JF, Fox R, Ellis D, Winter C, Hinshaw K, Draycott TJ. Observations from 450 shoulder dystocia
simulations: lessons for skills training. Obstet Gynecol
17.Freeth D, Hammick M, Koppel I, Reeves S, Barr H. Interprofessional Education Joint Evaluation Team. A Critical Review of Evaluations of Interprofessional Education: Commissioned by the Learning and Teaching Support Network Centre for Health Sciences and Practice; 2002. Available at: http://www.health.heacademy.ac.uk/publications/occasionalpaper
. Accessed July 9, 2009.
18.Crofts JF, Bartlett C, Ellis D, Hunt LP, Fox R, Draycott TJ. Training for shoulder dystocia
: a trial of simulation using low-fidelity and high-fidelity mannequins. Obstet Gynecol
19.Crofts JF, Bartlett C, Ellis D, Hunt LP, Fox R, Draycott TJ. Management of shoulder dystocia
: skill retention 6 and 12 months after training. Obstet Gynecol
20.Draycott T, Crofts JF, Ash JP, et al. Improving neonatal outcome through practical shoulder dystocia
training. Obstet Gynecol
21.Draycott T, Sibanda T, Owen L, et al. Does training in obstetric emergencies improve neonatal outcome? BJOG
22.Siassakos D, Hasafa Z, Sibanda T, et al. Retrospective cohort study of diagnosis-delivery interval with umbilical cord prolapse; the effect of team training. BJOG
23.Royal College of Obstetricians and Gynaecologists & Royal College of Midwives. The Clinical Learning Environment and Recruitment Report of a Joint Working Party
. London: RCOG Press; 2008.
24.O'Flynn N, Rymer J, Wass V. Women's attitudes to the sex of medical students in a gynaecology clinic: cross sectional survey * Commentary: Patients as partners in medical education. BMJ
25.Higham J. How can we make our medical students enthusiastic about a future in obstetrics and gynaecology? BJOG
26.Whitten SM, Higham J. Recruitment and retention into obstetrics and gynaecology: the influencing factors. Br J Hosp Med (Lond)
27.Turner G, Lambert TW, Goldacre MJ, Barlow D. Career choices for obstetrics and gynaecology: national surveys of graduates of 1974–2002 from UK medical schools. BJOG