OBJECTIVE: To compare the effectiveness of training for eclampsia in local hospitals and a regional simulation center, with and without teamwork theory.
METHODS: This study is a randomized controlled trial of training in local hospitals and in a simulation center in the United Kingdom. Midwives and obstetricians working at participating hospitals were randomly assigned to 24 teams. Teams were randomly allocated to training in local hospitals or at a simulation center, and to teamwork theory or not. Performance was evaluated before and after training with a standardized eclampsia scenario captured on video. Outcome measures were completion of tasks, time to completion of tasks, administration of magnesium sulfate, and quality of teamwork.
RESULTS: Training was associated with an increase in completion of basic tasks; 87% before training and 100% afterward. Basic tasks were completed more quickly; 55 seconds compared with 27 seconds, P=.012. The magnesium sulfate loading dose was administered by 61% of teams before training and by 92% afterward (P=.040). There was a shorter median time to administration (116 seconds less; P=.011). Training at the simulation center was not associated with additional improvement. Teamwork generally improved (median global score rose from 2.5 to 4.0; P<.001) but there was no additional benefit from teamwork training.
CONCLUSION: Training resulted in enhanced performance with higher rates of completion for basic tasks, shorter times to administration of magnesium sulfate, and improved teamwork. There was no additional benefit from training in a simulation center, and none from teamwork theory.
CLINICAL TRIAL REGISTRATION: ISRCTN, http://isrctn.org, ISRCTN67906788, reference number 0270030
LEVEL OF EVIDENCE: I
Training for eclampsia management results in enhanced performance with higher rates of completion for basic tasks, shorter times to administration of magnesium, and improved teamwork.
From the 1Department of Obstetrics and Gynaecology, North Bristol NHS Trust, Southmead Hospital, Bristol; 2University Department of Clinical Sciences at South Bristol, Institute of Child Life and Health, UBHT Education Centre, Bristol; 3Department of Obstetrics and Gynaecology, Gloucestershire Hospitals NHS Trust, Gloucester; and 4Department of Obstetrics and Gynaecology, Taunton and Somerset NHS Trust, Musgrove Park Hospital, Taunton, United Kingdom.
Funded by the Patient Safety Research Programme, Department of Health, United Kingdom.
The authors thank the following people for their contributions to training and evaluation of the program: Christine Bartlett, Karen Cloud, Maureen Harris, Sarah Fitzpatrick, Derek Tuffnell, Bryony Strachan, Stephanie Withers, Valentine Akande, Cathy Winter, Fiona Donald, Tim Draycott, Imogen Montague, Penny Watson, Anne McCrum, Sarah Read, Heather Smart, Melanie Robson, Katie Harrison, Neil Liversedge, Joanne Crocker, and Simon Grant.
Corresponding author: Dr. Mark James, Consultant Obstetrician and Gynaecologist, Department of Obstetrics and Gynaecology, Gloucestershire Hospitals NHS Trust, Gloucester, GL1 3NN, United Kingdom; e-mail: email@example.com.
Financial Disclosure The authors have no potential conflicts of interest to disclose.
Eclampsia is an uncommon obstetric emergency in developed countries, occurring in just 1 in 2,000 maternities, but it is associated with high rates of maternal and perinatal morbidity and mortality.1 Effective management demands immediate life support together with magnesium sulfate for fit control and prevention.2
United Kingdom Confidential Enquiries into Maternal Deaths and Stillbirths and Deaths in Infancy have both highlighted that substandard care of obstetric emergencies is implicated in a substantial proportion of maternal and fetal deaths.3,4 The UK Clinical Negligence Scheme for Trusts first mandated annual multi-professional skills training for these obstetric emergencies in 2001. Nationally approved courses have been developed (Managing Obstetric Emergencies and Trauma Course, Advanced Life Support for Obstetrics). Eclampsia drills have been described,5 but the effectiveness and pitfalls of this type of training have not been precisely defined. In a systematic review of training in 2003, Black and Brocklehurst6 concluded that few training methods have been properly evaluated, and they recommended that research into training of emergencies be undertaken.6
Multi-professional skills training for eclampsia management is potentially important because of the infrequent occurrence, unpredictability, and rapid onset of eclampsia. The aim of the study was to estimate the effectiveness of skills training for eclampsia, including a comparison of simulation training conducted on the delivery suites of local hospitals or at a regional simulation center using a high-technology mannequin, with and without instruction in teamwork theory.
MATERIALS AND METHODS
The Simulation and Fire-drill Evaluation study was a multi-faceted randomized controlled trial commissioned by the Department of Health for England and Wales to investigate the effectiveness of training for obstetric emergencies. Using a 2×2 factorial design, the study compared high-technology training at a simulation center with low-tech training in local units. The effect of instruction in teamwork theory was evaluated.
There were no data on training effectiveness upon which to base power calculations, so the study was powered to detect differences in overall multiple choice question scores (reported separately); sample size of 36 per intervention was estimated to give 89% power to detect a difference of 20 in the mean multiple choice questionnaire score (estimated within-group standard deviation [SD] was ±35).
Participants were recruited to the study from six large District Hospitals in the South West of England. Recruitment took place from September to November 2004 in three hospitals and from December 2004 to February 2005 in the remaining three hospitals. Target recruitment was 144 participants; 48 junior and 48 senior midwives (junior is 5 years or less experience; senior is more than 5 years experience); 24 junior and 24 senior obstetricians (junior is 3 years or less experience; senior is more than 3 years experience). All midwives (hospital and community based), and all doctors working in the maternity departments of the six hospitals were considered for entry into the study. Members of staff were excluded if any of the following applied; participation at a nationally accredited obstetric emergencies course within 12 months, trainer for the study, participation in the pilot study, and long-term leave. Each of the participating hospitals provided four lists of eligible staff; categorized by profession and experience. Computer-generated randomized lists of the four staff groups for each hospital were drawn up by the study's regional co-coordinator. A study midwife at each hospital recruited 24 members of staff using the randomization sequence.
The individual participants were randomly allocated to one of four simulation teams using a computerized random-number generator. Each team comprised one senior doctor, one junior doctor, two senior midwives, and two junior midwives. After recruitment, the teams were randomly assigned to one of four training arms: 1) One-day local hospital training course without teamwork training; 2) two-day local hospital training course including teamwork training; 3) one-day simulation center training course without teamwork training; or 4) two-day simulation center training course including teamwork training, using a computer-generated number sequence.
After completion of the preevaluation assessment (see below), all teams attended the obstetric emergencies training course to which they had been assigned. All training courses included a 20-minute lecture and 40-minute drill (with feedback) on the management of eclampsia. All participants received a manual on the management of obstetric emergencies that contained detailed information on the general management and treatment of eclampsia. Training in each of the local hospitals used basic mannequins and patient–actors, with local senior obstetricians and midwives as the trainers. Training at the regional simulation center was given by expert senior obstetric and midwifery trainers using an advanced human patient simulator (SimMan; Laerdal Medical Corporation,Orpington, Kent, UK). Half the teams were chosen at random to undergo additional instruction in team-work theory that focused on roles and responsibilities, communication, and situational awareness. To standardize training, all the trainers attended a “Training-the-Trainers” course and all received a trainer's manual together with a CD-ROM (containing a digital slide presentation; PowerPoint, Microsoft Corporation, Edmond, WA) and lecture notes. Trainers also attended sessions on teamwork training and were given guidance on how to deliver the teamwork lectures.
Baseline evaluations were undertaken in participating hospitals 1 to 3 weeks before training, which included simulated drills and a multiple-choice questionnaire. Participants were made aware they would take part in a simulation of an obstetric emergency as a member of a team but were not aware of the specific topic. They were given generic drill instructions before the simulation and were asked to carry out all actions as if for real. To aid evaluation, participants wore colored sashes to identify grades of staff because this had been found to be an issue in a pilot study. Simulated eclampsia evaluation drills, using a patient– actor, were conducted in a delivery room and videotaped using four ceiling-mounted cameras connected to a digital recorder. A scripted handover of a standardized scenario was given to a junior midwife while the remainder of the group waited in an adjacent room. A partogram was available in the room as an additional source of information and for documentation. A member of the evaluation team, who read from a standardized script, provided further clinical information when appropriate (such as blood pressure) by an intercom system. The patient–actor was instructed to have a seizure for 1 minute, starting 60 seconds from the end of the handover. The drill was terminated after a period of 10 minutes from the end of the handover, or earlier if the team began to transfer the patient to the operating room, because there was no facility to continue to record the simulation outside the delivery room. The teams were then reassessed using the same evaluation, 1 to 3 weeks after completion of the training intervention. The team members received training in several obstetric emergencies, and they were deliberately not made aware that the posttraining evaluation was a repeat eclampsia scenario. In the posttraining evaluation, the handover was given to the second junior midwife in the group.
Participants completed a 185-question, negatively- marked, true/false/don't-know, multiple-choice questionnaire on obstetric emergencies. This included 30 questions about preeclampsia/eclampsia. Maximal and minimal scores for the eclampsia section were 30 and –30. The participants were not aware of their scores between evaluations.
Two trained assessors (senior midwife and obstetrician) independently reviewed all video recordings of the eclampsia scenario. They were blind to the training intervention and timing of the simulation (pretraining or posttraining), and they viewed the recordings in different sequences randomly generated by computer. A checklist of key actions, based on local practice professional guidelines (derived from evidence-based sources, including the Cochrane Library of Systematic Reviews and guidance notes of the Royal College of Obstetricians and Gynaecologists) was used to evaluate management of the eclampsia scenarios. Basic tasks were identified that each simulation team should achieve; emergency call for help, statement of the problem, lowering of headrest, adoption of left-lateral position, and administration of oxygen. The timings of these key events and protocol violations (omissions and inappropriate actions) were documented.
A validated 5-point Likert scale was used to assess teamwork.7 Trained teamwork assessors, who were blind to the training intervention and timing of the simulation (pretraining or posttraining), viewed the video recordings in different sequences (randomly generated by computer). If the scores of the two assessors were discordant, the tape was sent to an external expert assessor for a third assessment. The average of the three scores was then calculated and rounded to the nearest integer.
Ethical approval was granted by the Regional Research Ethics Committee, and five Local Research Ethics Committees granted site-specific approval. Research and Development approval was granted by each Healthcare Trust. Consent was taken from participants. A 5% level of statistical significance was used.
There were 975 members of staff working in the six participating hospitals, of which 912 (93.5%) met the eligibility criteria. 240 staff were approached, 158 consented to participate, and 140 entered the study (18 staff withdrew before the commencement of the study after giving initial consent). Although the target had been to recruit 144 staff, this was not achieved; three doctors were unable to attend due to clinical commitments, and a midwife withdrew on the morning of the assessment due to illness (Fig. 1). Of the 140 staff who entered the study, 132 completed the posttraining assessment, a drop out rate of 5.7%, all due to illness (Fig. 2).
Participants answered 8,165 of the 8,190 (99.7%) questions related to knowledge of eclampsia. Overall, there was a statistically significant 32% increase in the mean multiple-choice question score for this section: 12.9 (SD ±5.5) to 16.7 (SD ±5.1) following training (two-tailed paired Student t test, P<.001); mean change for 133 completed pairs +3.8 (95% confidence interval 2.9–4.7). There was no difference according to the type of training the participants underwent (local hospital or simulation center, P=.46, team training P=.37; two-way analysis of variance).
One pretraining scenario was not recorded because of a fault in recording equipment; 23 of 24 (96%) of pretraining videos and 24 of 24 (100%) of posttraining videos were available for analysis. The data have been analyzed for all 24 teams before and after training, and separately the posttraining data for the four training intervention groups (Table 1).
Overall, 20 of 23 (87%) teams completed all the basic tasks in the pretraining simulation compared with 24 of 24 (100%) teams posttraining (not statistically significant). There was a statistically significant improvement in the time taken to complete all five basic tasks for the 20 teams that achieved all tasks on both occasions (median values 55 compared with 27 seconds, P=.012; Wilcoxon matched-pairs signed ranks test). The data for the individual tasks are presented in Tables 1 and 2. Subgroup analysis for the four training-intervention groups (local hospital compared with simulation center, and team theory compared with no team theory) are given in Table 3. The subgroup sizes are small but there is no obvious pattern of clinically important differences.
Data for requisition, preparation, and administration times of the magnesium loading dose are presented in Table 2, together with time to achieve venous access and blood sampling. The loading dose was drawn up by 17 of 23 (74%) cases before training and 23 of 24 (96%) after training, but this difference was not statistically significant (McNemar test for change, P=.062). For the 17 teams who prepared the magnesium in both preassessments and postassessments, there was a statistically significant reduction in the median time taken to do this (95 seconds quicker, P=.001; Wilcoxon matched-pairs signed ranks test). The loading dose of magnesium was administered by 14 of 23 (61%) teams before training and by 22 of 24 (92%) after training (P=.040, McNemar test for change). For the 13 teams that had administration times recorded before and after training, there was a statistically significant shorter median time to administration (116 seconds quicker; P=.011, Wilcoxon matched-pairs signed ranks test), with the median time reduced by 30%.
Analysis of the posttraining data for the four training intervention groups (local hospital or simulation center and team and no team theory) revealed no consistent pattern of improvement for time to administration of the loading dose of magnesium for place of training or instruction in team work theory (Table 3).
Overall, the percentage of teams that prepared the maintenance dose more than doubled with training (4 of 23 [17%] before compared with 9 of 24 [38%] afterward), but this was not statistically significant (P=.062, McNemar test for change). Numbers are too few for valid analysis of administration time of the maintenance dose.
The median time to achieve all key tasks significantly improved in all teams, but there was no significant difference between the training methods employed. There were a lower number of protocol violations (omissions and inappropriate actions) after training (39 compared with 11 after training). In the preevaluation sessions 12 of 23 (52%) teams failed to restrict fluids compared with 3 of 24 (12.5%) teams after training (0.039, McNemar test for change).
Twenty of the 47 video recordings were sent to the external assessor. For 18 of the 20 teams with discordant scores, the original scores differed by one, and for 2 teams differed by 2. The proportion of teams that scored well (score of 4 or 5) increased for all measures of team behavior: 35% to 92% for clinical skills score (P=.001 two-tailed McNemar test for change); 30% to 96% for team behavior score (P<.001); and 17% to 88% for global rating (P<.001). The median scores for clinical skill, team behavior, and global rating for the 24 teams, before and after training, showed a statistically significant improvement for all modalities (Table 4); means are given because they provide better representation of the observed changes.
The median posttraining scores for the four subgroups revealed remarkably similar results with no clinically significant advantage from training in the simulation center or from teamwork theory (Table 5). All subgroups showed an improvement in the proportion with good scores (4 or 5) but the numbers are too small for statistical comparison, so the data are not presented.
Eclamptic convulsions are associated with a high rate of serious maternal and perinatal complications; 2% of women died after eclampsia in the United Kingdom BEST survey.1 In part this relates to the severity of the associated vascular disorder but it is likely that eclampsia per se increases the risk to mother and fetus, hence the rationale for primary prevention of eclampsia.8 The mechanism for this could include avoidance of the consequences of hypoxia–acidosis and aspiration of gastric contents. Moreover, if it occurs at a critical time, eclampsia can hinder the provision of effective care such as rapid delivery of an acidotic fetus. In addition to the administration of anticonvulsant therapy, simple measures such as airway protection and administration of oxygen might play a crucial role in preventing clinical problems. It follows that eclampsia demands rapid mobilization and effective coordination of the emergency maternity team. The infrequency of eclampsia means that a doctor on a five-year training program might see just one case before certification, hence the need to consider rehearsal. The baseline evaluation data from this study confirmed that need.
This was a large multi-center study of practical training for eclampsia across a health region involving a variety of professions with a wide range of experience. All groups underwent rehearsals as part of the evaluation, so drill repetition has not been subjected to a controlled trial. Nevertheless, one benefit of our study is that it contributes a large body of observational data on the value of rehearsal for obstetric emergencies.9 The principle aims of the study were to address questions about preferred location of training and the value of instruction in teamwork theory. These issues have been appropriately tested by a randomized controlled trial.
There were some practical issues that arose during the study; the number of participants and the skill mix of some teams changed slightly due to a small number of dropouts. These occurred despite reminder letters and despite local midwives being used to negotiate off-duty requests, maintain enthusiasm, and champion the study. Sickness, the commonest factor, was inevitable, but this happens in true life. Moreover, the dropout rate is so low as to have little effect on the interpretation of the findings. One other consideration is the argument that simulation cannot replicate real-life scenarios. There is some truth in that, but it was clear that participants were both anxious and intent on managing things correctly, as occurs with real emergencies. Overall, we believe that the findings provide useful data on which to base power calculations for future studies, direct future research, and guide the development of training programs.
The initial evaluations provide a guide to the current standard of the management of eclampsia across a health region. Compliance rates for individual tasks were generally very high (Table 1), but the composite data show that 1 in 6 (17%) of the teams failed to complete all basic tasks. Although only one team failed to get the magnesium from the eclampsia box, only 60% had commenced the loading dose by the end of the simulation, and just 17% had prepared the maintenance dose. There was also a high rate of errors, with one half of the teams failing to restrict fluids.10 As well as these quantitative measures, a number of qualitative issues were identified. It became clear that a significant proportion of team members were unsure of how to draw up the magnesium, and others were unfamiliar with the syringe driver used to administer it. The latter highlights the need for equipment training as part of orientation on appointment to a new hospital.
Training in general seems to have improved performance. Not only did more teams complete basic tasks, but those that did were able to complete them more quickly. The time range for some actions remained very wide, however, suggesting that median values could be improved much further with further practice. It is difficult to know the value of these improvements if they were translated into clinical practice, but it is logical to conclude that reduction of the times for repositioning by an average of 50% (21 seconds) and for oxygen administration by 43% (30 seconds) are likely to benefit a small proportion of women with eclampsia (and their fetuses). Similarly, quicker administration of the magnesium loading dose (average reduction of almost 2 minutes) is also likely to improve outcome for a small proportion of cases. Importantly, this reduction consisted entirely of improved preparation time. The drug delivery time was not reduced, thus maintaining patient safety; overly rapid infusion might cause cardiac arrest.11 It is crucial that any reduction in intervention times is achieved without compromising safety. One further area for concern was the failure to call the anesthetist. This actually worsened from three to four teams after training, and the time to do this almost doubled to 80 seconds. A similar issue was found in the Simulation and Fire-drill Evaluation shoulder dystocia training trial in which more failed to call a pediatrician after training.12 Training programs need to address this specific issue.
Comparison of the teams found that there was no consistent difference according to the location of the training. This suggests that training in base hospitals is entirely appropriate, provided, of course, that there is sufficient expertise in training methods. The advantages of local training are that of lower cost with no travel time or expense, and perhaps that of the local team developing together, including healthcare assistants, receptionists, and porters. Part of this team development could also be the capacity to analyze its own strengths and weaknesses, particularly if simulations are videotaped. The role of regional simulation center–based courses should perhaps be prioritized to the development of specialist trainers who then set up local programs. One additional advantage of local trainers meeting centrally is that practical initiatives can be shared. As a direct result of the “training-the-trainers” day for this study, eclampsia boxes13 were introduced into several units.
The data show that teamwork behaviors were much enhanced, with the large majority of teams showing improvement. There was more than a 2.5-fold increase in the proportion of good scores, and after training, about 90% of the teams scored well for all three teamwork measures. Interestingly, the teams that received specific instruction on teamwork theory fared no better than those that did not, suggesting that improved teamwork was a general effect of rehearsal. Shapiro and colleagues14 studying the effect of rehearsals for Emergency Department scenarios also found evidence of improved teamwork behaviors despite no specific teamwork theory being given.
What is not clear from our study is whether the improvement in teamwork only applied to these specifically constructed teams or whether the effect would continue if the individual clinicians were randomly reallocated to new teams. This is important because there is no single emergency team in maternity units but rather a number of team combinations that continually change according to duty rostering. It is possible that teamwork theory might be of benefit for clinical teams that are perpetually changing but that hypothesis was not tested in this study. Any future projects could test these points by random allocation and reallocation to teams before and after serial evaluations.
The reason for the apparent failure of team theory training is not clear. It could be that the instruction of teamwork theory was inadequate but equally it might be that this task is of relatively limited complexity such that simple rehearsal training is sufficient. We recommend that further research be undertaken into more complex clinical scenarios, such as patients with major trauma, for which several teams often work in concert.
This study shows that multi-professional skills training was associated with improved management of eclampsia, both in terms of completion of tasks and the times taken to perform them. If translated into clinical practice, these improvements are likely to improve outcomes for a proportion of women (and fetuses) affected. Eclampsia training in a regional simulation center seemed to confer no advantage over local training, which we believe is likely to have several advantages, including lower cost. Similarly, instruction in teamwork theory provided no additional benefit beyond that of simple rehearsal. We propose that these data strongly support the introduction of local rehearsals for the management of eclampsia. The ultimate challenge will be to prove that improved performance in simulation translates into better outcome for mothers and fetuses.15
1. Douglas KA, Redman CW. Eclampsia in the United Kingdom. BMJ 1994;309:1395–400.
2. Duley L. Magnesium sulphate in eclampsia. Eclampsia Trial Collaborative Group. Lancet 1998;352:67–8.
3. Why mothers die 2000–2003. The sixth report of the Confidential Enquiries into Maternal Deaths in the United Kingdom. London (UK): RCOG Press; 2004.
4. Maternal and Child Health Consortium: Confidential Enquiry into Stillbirths and Deaths in Infancy: 8th Annual Report. London (UK): RCOG Press; 2001.
5. Thompson S, Neal S, Clark V. Clinical risk management in obstetrics: eclampsia drills. Qual Saf Health Care 2004;13:127–9.
6. Black RS, Brocklehurst P. A systematic review of training in acute obstetric emergencies. BJOG 2003;110:837–41.
7. Weller JM, Bloch M, Young S, Maze M, Oyesola S, Wyner J, et al. Evaluation of high fidelity patient simulator in assessment of performance of anaesthetists. Br J Anaesth 2003;90:43–7.
8. Altman D, Carroli G, Duley L, Farrell B, Moodley J, Neilson J, et al. Do women with pre-eclampsia, and their babies, benefit from magnesium sulphate? The Magpie Trial: a randomized placebo-controlled trial. Lancet 2002;359:1877–90.
9. Thompson S, Neal S, Clark V. Clinical risk management in obstetrics: eclampsia drills [published erratum appears in BMJ 2004;328:762]. BMJ 2004;328;269–71.
10. Royal College of Obstetricians and Gynaecologists. Management of Severe Pre-eclampsia/Eclampsia. Green Top Guideline No. 10(A). London (UK): RCOG; 2006. Available at: http://www.rcog.org.uk/index.asp?PageID=1542
. Retrieved December 7, 2007.
11. Crowther C. Magnesium sulphate versus diazepam in the management of eclampsia: a randomized controlled trial. Br J Obstet Gynaecol 1990;97:110–7.
12. Crofts JF, Bartlett C, Ellis D, Hunt LP, Fox R, Draycott TJ. Training for shoulder dystocia: a trial of simulation using low-fidelity and high-fidelity mannequins. Obstet Gynecol 2006;108:1477–85.
13. Draycott T, Broad G, Chidley K. The development of an eclampsia box and fire drill. Br J Midwifery 2000;8:26–30.
14. Shapiro MJ, Morey JC, Small SD, Langford V, Kaylor CJ, Jagminas L, et al. Simulation based teamwork training for emergency department staff: does it improve clinical team performance when added to an existing didactic teamwork curriculum? Qual Saf Health Care 2004;13:417–21.
15. Draycott T, Sibanda T, Owen L, Akande V, Winter C, Reading S, et al. Does training in obstetric emergencies improve neonatal outcome? BJOG 2006;113:177–82.
Figure. No caption available.
© 2008 The American College of Obstetricians and Gynecologists
This article has been cited