Skip Navigation LinksHome > October 2012 - Volume 120 - Issue 4 > Objective Structured Assessment of Technical Skills Evaluati...
Obstetrics & Gynecology:
doi: 10.1097/AOG.0b013e31826af9a9
Original Research

Objective Structured Assessment of Technical Skills Evaluation of Theoretical Compared With Hands-On Training of Shoulder Dystocia Management: A Randomized Controlled Trial

Buerkle, Bernd MD; Pueth, Julia; Hefler, Lukas A. MD, MBA; Tempfer-Bentz, Eva-Katrin MD; Tempfer, Clemens B. MD, MBA

Free Access
Article Outline
Collapse Box

Author Information

From the Department of Obstetrics and Gynecology, Ruhr University Bochum, Bochum, Germany; and the Karl Landsteiner Institute of Gynecologic Surgery and Oncology and the Department of Obstetrics and Gynecology, Medical University of Vienna, Vienna, Austria.

Corresponding author: Clemens B. Tempfer, MD, Department of Obstetrics and Gynecology, Ruhr University Bochum-Marienhospital Herne, Hoelkeskampring 40, D-44625 Herne, Germany; e-mail: clemens.tempfer@marienhospital-herne.de.

Financial Disclosure The authors did not report any potential conflicts of interest.

Collapse Box

Abstract

OBJECTIVE: To compare the skills of performing a shoulder dystocia management algorithm after hands-on training compared with demonstration.

METHODS: We randomized medical students to a 30-minute hands-on (group 1) and a 30-minute demonstration (group 2) training session teaching a standardized shoulder dystocia management scheme on a pelvic training model. Participants were tested with a 22-item Objective Structured Assessment of Technical Skills scoring system after training and 72 hours thereafter. Objective Structured Assessment of Technical Skills scores were the primary outcome. Performance time, self-assessment, confidence, and global rating scale were the secondary outcomes. Statistics were performed using Mann-Whitney U test, χ2 test, and multiple linear regression analysis.

RESULTS: Two hundred three participants were randomized. Objective Structured Assessment of Technical Skills scores were significantly higher in group 1 (n=103) compared with group 2 (n=100) (17.95±3.14 compared with 15.67±3.18, respectively; P<.001). The secondary outcomes global rating scale (GRS; 10.94±2.71 compared with 8.57±2.61, respectively; P<.001), self-assessment (3.15±0.94 compared with 2.72±1.01; P=.002), and confidence (3.72±0.98 compared with 3.34±0.90, respectively; P=.005), but not performance time (3:19±0:48 minutes compared with 3:31±1:05 minutes; P=.1), were also significantly different, favoring group 1. After 72 hours, Objective Structured Assessment of Technical Skills scores were still significantly higher in group 1 (n=67) compared with group 2 (n=60) (18.17±2.76 compared with 14.98±3.03, respectively; P<.001) as were GRS (10.80±2.62 compared with 8.15±2.59; P<.001) and self assessment (SA; 3.44±0.87 compared with 2.95±0.94; P=.003). In a multiple linear regression analysis, group assignment (group 1 compared with 2; P<.001) and sex (P=.002) independently influenced Objective Structured Assessment of Technical Skills scores.

CONCLUSION: Hands-on training helps to achieve a significant improvement of shoulder dystocia management on a pelvic training model.

CLINICAL TRIAL REGISTRATION: www.ClinicalTrials.gov, NCT01618565.

LEVEL OF EVIDENCE: I

Shoulder dystocia is an obstetric emergency with potentially serious sequelae for the neonate including fracture of the clavicula and humerus, Erb's palsy, and asphyxia.1,2 There is no uniform definition of shoulder dystocia and therefore the prevalence varies in the literature. In studies of large patient cohorts, the prevalence of shoulder dystocia is cited as between 0.7% and 2.5% of vaginal deliveries.3,4 Management of shoulder dystocia is difficult because it may involve a number of technically challenging maneuvers such as McRobert's maneuver, Wood's corkscrew maneuver, delivery of the posterior shoulder, and Zavanelli's maneuver.5 Initial maneuvers such as McRobert's maneuver have a high success rate, making it difficult to gain practical experience with second-line and third-line maneuvers. In addition to the technical challenge, these maneuvers have to be performed under the stress of an obstetric emergency and, in many cases, unanticipated.

Therefore, training of shoulder dystocia management is important and regular trainings are recommended by obstetric guidelines.6 It is, however, unclear, what the best training methods are and assessment of the validity of different training methods in controlled trials is lacking (PubMed search; June 8, 2012; search terms: shoulder dystocia, training, model, randomized, pelvic trainer, shoulder dystocia management).

Objective Structured Assessment of Technical Skills is a well-established, reliable, and reproducible instrument for measuring technical skills.7 Increasing evidence suggests that training models and simulators are effective means to improve technical performance of medical staff.711 In a previous study, for example, we demonstrated that hands-on training of cervical conization using a porcine tissue training model improves surgical performance.12 In the present study, we attempted to compare hands-on training compared with expert demonstration of shoulder dystocia management on a pelvic training model using Objective Structured Assessment of Technical Skills scores.

We hypothesized that hands-on training of shoulder dystocia management on a pelvic training model is more effective than expert demonstration with effectiveness defined as high Objective Structured Assessment of Technical Skills scores. We designed a randomized, controlled study measuring Objective Structured Assessment of Technical Skills scores of participants working through a shoulder dystocia management algorithm after a 30-minute hands-on training compared with a 30-minute expert demonstration. To test for immediate as well as short-term training effects, we tested participants immediately after training and retested them after 72 hours.

Back to Top | Article Outline

MATERIALS AND METHODS

A randomized, double-blind, single-institution study was carried out between January 2011 and April 2012 at the Department of Obstetrics and Gynecology, Ruhr University Bochum, in a population of consecutive medical students, who took part in an obstetrics rotationship. Approval for this study was obtained by the ethics committee of the Ruhr University Bochum. Informed consent was obtained from participants. Figure 1 shows a Consolidated Standards of Reporting Trials diagram of the patient flow through this study. Inclusion criteria were informed consent and medical student. Exclusion criteria were unwillingness to participate and presence of a language barrier. Group assignment was performed by one of the two instructors (B.B. and C.B.T.) using a computer-generated randomization list with a 1:1 allocation ratio without blocking. This was done in a separate room. Then, participants were allocated to separate rooms, where the training took place. The evaluator was not present. Thus, participants and the evaluator who tested the participants after training (J.P.) were unaware of the group assignment. After randomization, participants in group 1 underwent a 30-minute training session with one instructor demonstrating shoulder dystocia management on a pelvic training model and all participants subsequently performed five maneuvers hands-on once. The five maneuvers were as follows: vaginal digital examination, McRobert's maneuver, Wood's corkscrew maneuver, release of the posterior shoulder, and Zavanelli's maneuver. Participants in group 2 underwent a 30-minute training session with one instructor demonstrating all five maneuvers for shoulder dystocia management on the pelvic training model. The pelvic model and the model of the newborn used in this study allowed the determination of the fetal head position.

Fig. 1
Fig. 1
Image Tools

Participants were not allowed to film procedures or to take photographs or written notes. Immediately after the training session as well as 72 hours later, participants were tested by the evaluator (J.P.), who was unaware of the group assignment and was not involved in or present at the training procedures. Testing was done using an Objective Structured Assessment of Technical Skills. To grade each proband's performance, the evaluator calculated an Objective Structured Assessment of Technical Skills score by adding points given for each of the 22 items on a task-specific check list (Table 1) with 1 point for correctly performing each item and 0 point for not performing or not correctly performing the item. Thus, a higher score indicates greater proficiency. Specifically, McRobert's maneuver, Wood's corkscrew maneuver, delivery of the posterior shoulder, and Zavanelli's maneuver as well as logistic items such as diagnosis of shoulder dystocia, information of a senior obstetrician and an anesthesiologist, or stopping of oxytocin were part of the Objective Structured Assessment of Technical Skills.

Table 1
Table 1
Image Tools

We chose to use medical students, because they were naïve to any form of previous shoulder dystocia training or practical experience of shoulder dystocia management. Thus, in our view, in contrast to residents, medical students were ideal participants in the sense that they would not introduce bias in terms of practical experience, ie, various years of residency experience, previous exposure to shoulder dystocia, and previous shoulder dystocia trainings.

Categorical variables were analyzed by χ2 test and continuous variables were compared using the Mann-Whitney U test with a significance level of .05. We performed a multiple linear regression model to test whether the training effect, as measured by Objective Structured Assessment of Technical Skills scores, was independent of the participants' sex (male compared with female) and type of curriculum (model curriculum compared with regular curriculum). For the calculation of odds ratios, we used a logistic regression model with Objective Structured Assessment of Technical Skills scores as a categorical variable (greater than 90% of Objective Structured Assessment of Technical Skills scores achieved compared with greater than 90% of Objective Structured Assessment of Technical Skills scores achieved). The “model curriculum” is an experimental form of the medical school curriculum offered at the Ruhr University Bochum. In contrast to the regular medial curriculum, it puts more emphasis on practical aspects and problem-oriented learning methods. Values are given as means. A power calculation demonstrated that, with a sample size of 200, the study has a power of greater than 95% to detect an absolute 20% difference in Objective Structured Assessment of Technical Skills scores at a significance level of 0.05 assuming the Mann-Whitney U test. An Objective Structured Assessment of Technical Skills score difference of 20%, ie, a median score difference of 4, range 0–8, was estimated to be achievable by hands-on training and to be clinically relevant based on previous literature.13 We used the statistical software SPSS 11.0 for Windows for statistical analysis.

Back to Top | Article Outline

RESULTS

Two hundred three participants were randomized. Group-specific characteristics of study participants are listed in Table 2. Primary and secondary outcome parameters are summarized in Table 3. Objective Structured Assessment of Technical Skills scores (primary outcome) were significantly higher in group 1 (n=103) compared with group 2 (n=100) (17.95±3.14 compared with 15.67±3.18, respectively; P<.001). Figure 2 shows a box plot of Objective Structured Assessment of Technical Skills scores in both groups. The secondary outcomes GRS (10.94±2.71 compared with 8.57±2.61, respectively; P<.001), SA (3.15±0.94 compared with 2.72±1.01; P=.002), and CON (3.72±0.98 compared with 3.34±0.90, respectively; P=.005) were also significantly different between groups favoring group 1. This was not the case for PT, which was shorter in group 1 compared with group 2 (3:19±0:48 minutes compared with 3:31±1:05 minutes; P=.1), but this difference failed to reach statistical significance. The number of participants achieving 90%, 80%, and 70% of the maximum Objective Structured Assessment of Technical Skills scores in group 1 and group 2 were 41 of 103, 69 of 103, and 81 of 103 and 14 of 100, 30 of 100, and 51 of 100, respectively.

Table 2
Table 2
Image Tools
Table 3
Table 3
Image Tools
Fig. 2
Fig. 2
Image Tools

To test for a short-term training effect, we retested participants after 72 hours. Sixty-seven participants in group 1 and 60 participants in group 2 took part in retesting. There was no significant difference in the lost to follow-up rate between group 1 and group 2 (36/103 compared with 40/100; P=nonsignificant). After 72 hours, Objective Structured Assessment of Technical Skills scores were still significantly higher in group 1 compared with group 2 (18.17±2.76 compared with 14.98±3.03, respectively; P<.001) as were the secondary outcomes GRS (10.80±2.62 compared with 8.15±2.59; P<.001) and SA (3.44±0.87 compared with 2.95±0.94; P=.003). No significant differences were seen after 72 hours regarding the secondary outcomes PT (3:01±0:40 minutes compared with 2:58±0:46 minutes; P=.7) and CON (3.66±0.89 compared with 3.36±0.91, respectively; P=.06).

Participants differed regarding their sex (female, n=138 compared with male, n=65) and regarding the type of medical school curriculum (regular curriculum; n=166 compared with an experimental “model” curriculum; n=37). Although the distribution of sex and type of curriculum was not significantly different in groups 1 and 2 (P=.2 and P=.7, respectively), males and students of the experimental “model” curriculum achieved higher Objective Structured Assessment of Technical Skills scores compared with females and students of the regular curriculum (data not shown). Therefore, we tested whether the positive effect of hands-on training on the proficiency to manage a shoulder dystocia on a pelvic model, as measured by Objective Structured Assessment of Technical Skills scores, was independent of sex and type of curriculum. In a multiple linear regression analysis, group assignment (group 1 compared with 2; odds ratio [OR] 1.85, 95% confidence interval [CI] 1.12–1.97; P<.001) as well as sex (OR 1.21, 95% CI 1.18–2.36; P=.002) and type of curriculum (OR 1.12, 95% CI 1.01–2.09; P=.006) were significantly associated with Objective Structured Assessment of Technical Skills scores. Therefore, group assignment was an independent predictor of Objective Structured Assessment of Technical Skills scores as were sex and type of curriculum.

Back to Top | Article Outline

DISCUSSION

In a randomized trial, we found that Objective Structured Assessment of Technical Skills scores for shoulder dystocia management were significantly higher after hands-on training compared with demonstration using a pelvic training model with this effect lasting up to 72 hours after training. In addition, secondary outcomes such as GRS, SA, and CON were also significantly improved by hands-on training. This suggests that hands-on training helps to achieve a significant improvement of shoulder dystocia management skills. We suggest to implement hands-on training and Objective Structured Assessment of Technical Skills scoring of shoulder dystocia management into obstetrics–gynecology medical staff training programs.

Training models and simulators can be used to improve technical performance.811 In a previous study we demonstrated that hands-on training of cervical conization using a porcine tissue training model can improve surgical performance.12 The results of the present study are in accordance with these data demonstrating that hands-on training improves technical skills, in this case solving an obstetric emergency on a pelvic training model. Thus, our data and the data of others support the use of training models and simulators in clinical practice.

Compared with theoretical training and expert demonstration, hands-on simulator trainings are more time-consuming and resource-consuming. Therefore, institutions may be reluctant to invest into this form of education, precluding widespread use of these training techniques. In this respect, it is imperative to produce high-level evidence to establish the superiority of hands-on simulator trainings compared with traditional expert demonstration and “learning by doing” approaches. Our study adds to the growing body of evidence supporting the implementation of hands-on simulator training schemes into medical staff training programs.

Regular training of rare obstetric complications such as breech presentation and shoulder dystocia is recommended.6 It is, however, unclear by what means training success should be ideally measured. Our data support the notion that technical proficiency in the management of shoulder dystocia can be objectively measured by Objective Structured Assessment of Technical Skills. Our data support the use of Objective Structured Assessment of Technical Skills to assess group differences in technical skills studies and to identify trainees who fall behind and may benefit from retraining in clinical practice.

Our study has limitations. First, we cannot comment on the long-term durability of the study results, because participants were only tested immediately after the training session and 72 hours thereafter. The advantage of hands-on training over expert demonstration may decrease over time and with greater experience. Also, we do not know how often retrainings should be implemented for all or selected trainees such as those who have reached low Objective Structured Assessment of Technical Skills marks. On the other hand, it is reasonable to assume that regular retrainings will steadily improve technical skills and will thus emphasize the advantage of hands-on training over demonstration seen in this study. In addition, the external validity of our study is limited, because we have used medical students as participants and the results of their trainings may not translate to residents and staff.

Regarding the generalizability and clinical implications of our study, the data have to be interpreted with caution. We have tested the participants' skills on a training model. Performance may be different in a real-life emergency and training success may not or only in part be transferrable to actual emergencies. On the other hand, the only way to prepare for emergencies is regular training and simulator training has been demonstrated to result in better real-life performance in some situations, eg, laparoscopic surgery.14 Because shoulder dystocia is a rare event, it will not be possible to test real-life performance differences based on training method. Another problem is that not all Objective Structured Assessment of Technical Skills items used in this study such as the presence of a midwife or the performance of an episiotomy are consistent with standard procedures in other institutions. Thus, training settings in other institutions may differ from the one used in this study.

In summary, the data presented in this prospective, randomized study provide level I evidence that hands-on training helps to achieve a significant and sustained improvement of shoulder dystocia management on a training model. We suggest to implement hands-on training and Objective Structured Assessment of Technical Skills scoring of shoulder dystocia management into medical staff training programs.

Back to Top | Article Outline

REFERENCES

1. Gilby JR, Williams MC, Spellacy WN. Fetal abdominal circumference measurements of 35 and 38 cm as predictors of macrosomia. A risk factor for shoulder dystocia. J Reprod Med 2000;45:936–8.

2. Leung TY, Stuart O, Suen SS, Sahota DS, Lau TK, Lao TT. Comparison of perinatal outcomes of shoulder dystocia alleviated by different type and sequence of manoeuvres: a retrospective review. BJOG 2011;118:985–90.

3. Revicky V, Mukhopadhyay S, Morris EP, Nieto JJ. Can we predict shoulder dystocia? Arch Gynecol Obstet 2012;285:291–5.

4. Overland EA, Vatten LJ, Eskild A. Risk of shoulder dystocia: associations with parity and offspring birthweight. A population study of 1 914 544 deliveries. Acta Obstet Gynecol Scand 2012;91:483–8.

5. Hoffman MK, Bailit JL, Branch DW, Burkman RT, Van Veldhusien P, et al.. Consortium on Safe Labor. A comparison of obstetric maneuvers for the acute management of shoulder dystocia. Obstet Gynecol 2011;117:1272–8.

6. Sokol RJ, Blackwell SC; American College of Obstetricians and Gynecologists. Committee on Practice Bulletins–Gynecology. Shoulder dystocia. ACOG Practice Bulletin No. 40, November 2002 (Replaces practice pattern number 7, October 1997). Int J Gynaecol Obstet 2003;80:87–92.

7. Goff BA, VanBlaricom A, Mandel L, Chinn M, Nielsen P. Comparison of objective, structured assessment of technical skills with a virtual reality hysteroscopy trainer and standard latex hysteroscopy model. J Reprod Med 2007;52:407–12.

8. Gurusamy KS, Aggarwal R, Palanivelu L, Davidson BR. Virtual reality training for surgical trainees in laparoscopic surgery. The Cochrane Database of Systematic Reviews 2009, Issue 1. Art. No.: CD006575. DOI: 10.1002/14651858.CD006575.pub2.

9. Polterauer S, Grimm C, Hanzal E, et al.. Surgical skills training—results of a prospective trial. Geburtsh Frauenheilk 2010;70:990–9.

10. Lentz GM, Mandel LS, Lee D, Gardella C, Melville J, Goff BA. Testing surgical skills of obstetric and gynecologic residents in a bench laboratory setting: validity and reliability. Am J Obstet Gynecol 2001;184:1462–8.

11. Reznick RK, MacRae H. Teaching surgical skills—changes in the wind. N Engl J Med 2006;355:2664–9.

12. Hefler L, Grimm C, Kueronya V, Tempfer C, Reinthaller A, Polterauer S. A novel training model for the loop electrosurgical excision procedure. Am J Obstet Gynecol 2012;206:535.e1–4.

13. Price J, Naik V, Boodhwani M, Brandys T, Hendry P, Lam BK. A randomized evaluation of simulation training on performance of vascular anastomosis on a high-fidelity in vivo model: the role of deliberate practice. J Thorac Cardiovasc Surg 2011;142:496–503.

14. Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, et al.. Effect of virtual reality training on laparoscopic surgery: randomized controlled trial. BMJ 2009;338:b1802.

Cited By:

This article has been cited 1 time(s).

Obstetrics and Gynecology Clinics of North America
Shoulder Dystocia
Grobman, W
Obstetrics and Gynecology Clinics of North America, 40(1): 59-+.
10.1016/j.ogc.2012.11.006
CrossRef
Back to Top | Article Outline

© 2012 The American College of Obstetricians and Gynecologists

Login

Article Tools

Images

Share