Secondary Logo

Journal Logo

Decay in Blood Loss Estimation Skills After Web-Based Didactic Training

Toledo, Paloma MD, MPH; Eosakul, Stanley T. BS, MS; Goetz, Kristopher BA; Wong, Cynthia A. MD; Grobman, William A. MD, MBA

Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: February 2012 - Volume 7 - Issue 1 - p 18–21
doi: 10.1097/SIH.0b013e318230604f
Empirical Investigations
Free

Introduction Accuracy in blood loss estimation has been shown to improve immediately after didactic training. The objective of this study was to evaluate retention of blood loss estimation skills 9 months after a didactic web-based training.

Methods Forty-four participants were recruited from a cohort that had undergone web-based training and testing in blood loss estimation. The web-based posttraining test, consisting of pictures of simulated blood loss, was repeated 9 months after the initial training and testing. The primary outcome was the difference in accuracy of estimated blood loss (percent error) at 9 months compared with immediately posttraining.

Results At the 9-month follow-up, the median error in estimation worsened to −34.6%. Although better than the pretraining error of −47.8% (P = 0.003), the 9-month error was significantly less accurate than the immediate posttraining error of −13.5% (P = 0.01).

Conclusion Decay in blood loss estimation skills occurs by 9 months after didactic training.

From the Department of Anesthesiology (P.T., S.T.E., C.A.W.), Institute for Healthcare Studies (P.T., W.A.G.), Department of Obstetrics and Gynecology (W.A.G.), Northwestern University Feinberg School of Medicine; and Department of Quality Management (K.G.), Northwestern Memorial Hospital, Chicago, IL.

The authors declare no conflict of interest.

This study was completed while the first author was a National Research Service Award postdoctoral fellow at the Institute for Healthcare Studies under an institutional award from the Agency for Healthcare Research and Quality, T-32 HS 000078 (PI: Jane L. Holl, MD, MPH) and was supported by a grant from Ken and Anne Griffin (Wong and Grobman). Stan Eosakul was supported by a Foundation for Anesthesia Education and Research (FAER) Medical Student Anesthesia Research Fellowship.

Reprints: Paloma Toledo, MD, MPH, Department of Anesthesiology, Northwestern University Feinberg School of Medicine, 251 E. Huron Street, F5-704, Chicago, IL 60611 (e-mail: p-toledo@md.northwestern.edu).

Postpartum hemorrhage is a leading cause of maternal death in the United States and worldwide.1,2 Recent studies have shown that the incidence of postpartum hemorrhage is increasing.3,4 Postpartum hemorrhage is traditionally defined as blood loss greater than 500 mL after vaginal delivery or 1000 mL after cesarean delivery and is typically estimated through a visual assessment of blood loss.

Visual assessment has remained standard practice despite the availability of other techniques to estimate blood loss because these alternatives are costly, time consuming, and often require the use of specialized equipment.5–7 Several studies have shown that providers tend to underestimate large amounts of blood loss, with the magnitude of underestimation worsening with increasing volumes of blood lost.8–12 In the obstetric setting, appropriate recognition of small amounts of blood loss may also have significance as inappropriate assessment of blood loss may lead to unnecessary interventions, such as a cesarean delivery in the setting of a placenta previa. While studies have shown that there is an improvement in blood loss estimation after didactic training and simulation, the long-term retention of the blood loss estimation skills has not been well studied.13–15 The objective of this study was to test the hypothesis that blood loss estimation skills would decay by 9 months after didactic web-based training.

Back to Top | Article Outline

METHODS

This study was approved by the Northwestern University Institutional Review Board. Participants were recruited from a cohort of 141 labor and delivery providers who underwent web-based training as part of a study evaluating the effect of live versus web-based didactic training in blood loss estimation.15 We have previously reported our findings of improvement in blood loss estimation accuracy immediately after training.

Briefly, before training, participants estimated blood loss at five simulated blood loss stations (pretest). The stations were photographs of three small blood loss scenarios (laparotomy sponge, perineal pad, and under-the-buttocks pad) and two large blood loss scenarios (vaginal delivery drape and blood spill on mannequin and bed). After the pretest, actual blood volumes were revealed; participants then completed a 20-minute, web-based didactic training program on postpartum hemorrhage and estimation of blood loss. Participants then took a five-station posttest that had different blood volumes than the pretest. Participants were not informed of the correct values for the posttest stations after they had completed the test.

As most studies of skill decay in the surgical and obstetrical literature have assessed for skill decay between 6 months and 1 year after training, we chose to evaluate decay 9 months after training.16–18 All participants from the web-based portion of the study were invited to participate in the follow-up study evaluating retention of skills. Three attempts were made to contact participants for study recruitment. Written informed consent was obtained before participation. Consenting participants completed the same five-station posttest as during the initial training. Blood loss volumes for each test are shown in Table 1.

Table 1

Table 1

Back to Top | Article Outline

Statistical Analysis

The primary outcome variable was accuracy of the blood loss estimation. Volume estimates were converted to percent errors [(estimated blood loss − actual blood loss) × 100/actual blood loss]. Provider characteristics and volume estimates between participants who did and did not complete the follow-up study were compared using a Student t test and χ2 analysis. The percentage error between testing periods (pretest − posttest, posttest − 9 month follow-up, and pretest − 9 month follow-up) and overall aggregate error was compared using the Wilcoxon matched pairs signed rank test as the data were not normally distributed. The effects of participant characteristics [provider type (obstetrician, anesthesiologist, or nurse), years of clinical experience, and formal training in blood loss estimation before the original training session] on the accuracy of blood loss estimation were compared using either the Kruskal-Wallis test or Wilcoxon signed rank test. All tests were two-tailed, and a P < 0.05 was considered significant. Data were analyzed using Stata SE (Version 10, College Station, TX).

Back to Top | Article Outline

RESULTS

Of the 141 labor and delivery providers who participated in the original web-based training program, 52 participated in the 9-month follow-up study. Eight of these participants did not give permission to use their test results; their data were excluded from the analysis. Participant characteristics are shown in Table 2. Comparison of the participants who completed the follow-up study with those who did not complete the follow-up study (n = 89) found no differences in provider characteristics or pre- or posttest blood loss estimation accuracy (data not shown).

Table 2

Table 2

The median (interquartile range) aggregate pretest accuracy of blood loss estimation was −47.8% (−60.9% to −28.7%) with an improvement to −13.5% (−18% to −9.8%) in the immediate posttest (P < 0.001). At the 9-month follow-up, accuracy had decreased to −34.6% (−53.4% to −14.3%), which was worse than the immediate posttest (P = 0.01) but better than the pretest value (P = 0.003) (Fig. 1).

Figure 1

Figure 1

At the 9-month follow-up, posttest accuracy was unchanged in three of the stations (laparotomy sponge, perineal pad, blood spill on mannequin and bed). Underestimation was greater compared with the immediate posttest in the vaginal delivery drape station; overestimation was greater in the under-the-buttocks pad station (Table 3). There were no significant differences in accuracy of estimation at 9-month follow-up based on provider type, duration of clinical experience, or previous formal training on estimates of blood loss accuracy (P = 0.12, 0.16, and 0.35, respectively).

Table 3

Table 3

Back to Top | Article Outline

DISCUSSION

While training in blood loss estimation initially improves accuracy, the important finding from this study is that decay in these skills occurs by 9 months after didactic training including simulated blood loss training. To our knowledge, no previous study has evaluated whether decay occurs after blood loss estimation training.

Accurate estimation of blood loss is important, especially as the volume of blood loss increases. In a previous study, we showed that providers underestimated by 41% a 2-L blood loss after simulated vaginal delivery.11 Although postpartum hemorrhage may not be preventable, improved outcomes from early recognition and initiation of treatment may be possible. Berg et al,19 in an evaluation of maternal deaths in North Carolina over a 5-year period, found that 93% of the deaths related to postpartum hemorrhage were preventable. Similarly, in a retrospective analysis of data from the Hospital Corporation of America, Clark et al20 found that 72% of maternal deaths from hemorrhage were preventable and suggested that prompt attention to clinical signs of hemorrhage and hypovolemia could have prevented death in these patients.

Studies on retention of knowledge and skills after training have been conflicting. Rapid skill decay has been shown after both advanced cardiac life support and advanced trauma life support (ATLS) training.17,21–24 Ali et al17 evaluated ATLS-trained physicians at 6 months and 2, 4, and 6 years after training. Participants were given a validated multiple-choice questionnaire to assess knowledge. By 6 months after training, only 50% of the participants were able to meet the minimum passing requirements of ATLS. Furthermore, at 2, 4, and 6 years after training, none of the physicians met passing requirements. In the obstetric literature, one study evaluated skill retention after shoulder dystocia training.18 All study participants had attended a 40-minute workshop on shoulder dystocia management. At 3 weeks and 6 and 12 months after training, participants were tested using a shoulder dystocia simulation. Participants were stratified by proficiency, and those who successfully delivered the neonate before training had no deterioration in skills at any of the follow-up points; conversely, the participants who enhanced their delivery skills during the training did have some decrement in the frequency of achieving a successful delivery during simulation at 6 and 12 months.

A meta-analysis of factors that influence skill decay found that there was substantial skill loss with nonpractice.25 The authors estimated that after more than 1 year without practicing, the average participant would perform at less than 92% of their performance level before the nonpractice interval began. While blood loss estimation is a daily occurrence for most labor and delivery practitioners, feedback on the actual blood loss is not; therefore, skill decay might occur even in the setting of exposure to blood loss. Other important factors that were identified in the meta-analysis include the type of task that was being evaluated (physical vs. cognitive, natural vs. artificial), the method of testing used, and degree of overlearning (ie, deliberate practice after completion of training).25

In the current study, we found that three of the individual stations did not have skill decay compared with the immediate posttest. One possible explanation is that during the didactic training, maximum saturated capacities were given for the laparotomy sponge, and a general rule was given for how to estimate blood loss hemorrhage in a bed, thus making retention easier for these scenarios. An alternative explanation is that providers tend to estimate certain volumes, such as 100 or 1000 mL regularly. This may explain why the perineal pad, whose actual volume was 75 mL, had an overestimation of 33% in both the immediate posttest and 9-month follow-up. In contrast, the under-the-buttocks pad station and vaginal delivery drape stations had significant skill decay over 9 months. The marked over- and underestimation in each of these stations, respectively, likely is a reflection of providers’ tendency to overestimate small volumes of blood loss and underestimate large volumes of blood loss.10,11

There are several limitations to our study. The scenarios were all isolated photographs of simulated blood loss situations. It is possible that providers would be more accurate at blood loss estimation if they had clinical information such as the rate of blood loss, hemodynamic data, or the ability to view the simulated blood loss from multiple angles. The use of the same photographs in the 9-month follow-up and immediate posttest may have introduced some bias; however, we feel that this is unlikely as providers were never provided with the true blood volumes after completion of the immediate posttest. We chose to evaluate only the subset of the participants in the original study who underwent web-based, but not live, training. Most hospitals do not have the time or resources to do live blood loss estimation training and testing with multiple scenarios, therefore web-based didactic training and testing are more feasible. It is possible that participants who underwent the live training and testing would have better retention due to blood loss being displayed in its natural settings. An additional limitation is that there was a large range of experience in providers who participated in the follow-up study. However, we feel that the effect of this variability is minimal as we found no difference in accuracy of blood loss estimates either by experience or by provider type in this dataset or in previous studies assessing accuracy of blood loss estimation.11,15 Finally, not all of the participants who participated in the original web-based training module participated in the follow-up study. It is possible that there was responder bias, and the participants who felt more (or less) confident in blood loss estimation skills or who had greater interest in hemorrhage participated in the 9-month follow-up study and are therefore not representative of the overall sample.

Further studies are needed to determine not only when skill decay occurs but also when the decay is clinically significant, as well as the optimal interval for retraining. In addition, alternative web-based modules should be developed and tested to determine the optimal method for teaching and retaining blood loss estimation skills. Finally, research is needed to determine whether improved accuracy of blood loss estimation in simulated deliveries translates into improved clinical outcomes such as a reduction in the number of blood transfusions, hysterectomies, and maternal deaths due to hemorrhage.

Back to Top | Article Outline

REFERENCES

1. Chang J, Elam-Evans LD, Berg CJ, et al.. Pregnancy-related mortality surveillance—United States, 1991–1999. MMWR Surveill Summ 2003; 52: 1–8.
2. World Health Organization, UNICEF, UNFPA. Maternal Mortality in 2000: Estimates Developed by WHO, UNICEF, and UNFPA. Geneva: WHO, UNICEF, and UNFPA; 2004.
3. Ford JB, Roberts CL, Simpson JM, Vaughan J, Cameron CA. Increased postpartum hemorrhage rates in Australia. Int J Gynaecol Obstet 2007; 98: 237–243.
4. Knight M, Callaghan WM, Berg C, et al.. Trends in postpartum hemorrhage in high resource countries: a review and recommendations from the International Postpartum Hemorrhage Collaborative Group. BMC Pregnancy Childbirth 2009; 9: 55.
5. Chua S, Ho LM, Vanaja K, Nordstrom L, Roy AC, Arulkumaran S. Validation of a laboratory method of measuring postpartum blood loss. Gynecol Obstet Invest 1998; 46: 31–33.
6. Quinlivan WL, Brock JA. Blood volume changes and blood loss associated with labor. I. Correlation of changes in blood volume measured by I-131 albumin and Evans blue dye, with measured blood loss. Am J Obstet Gynecol 1970; 106: 843–849.
7. Nelson GH, Ashford C, Williamson R, Amburn SD. Method for calculating blood loss at vaginal delivery. South Med J 1981; 74: 550–552.
8. Patel A, Goudar SS, Geller SE, et al.. Drape estimation vs. visual assessment for estimating postpartum hemorrhage. Int J Gynaecol Obstet 2006; 93: 220–224.
9. Prasertcharoensuk W, Swadpanich U, Lumbiganon P. Accuracy of the blood loss estimation in the third stage of labor. Int J Gynaecol Obstet 2000; 71: 69–70.
10. Bose P, Regan F, Paterson-Brown S. Improving the accuracy of estimated blood loss at obstetric haemorrhage using clinical reconstructions. BJOG 2006; 113: 919–924.
11. Toledo P, McCarthy RJ, Hewlett BJ, Fitzgerald PC, Wong CA. The accuracy of blood loss estimation after simulated vaginal delivery. Anesth Analg 2007; 105: 1736–1740.
12. Stafford I, Dildy GA, Clark SL, Belfort MA. Visually estimated and calculated blood loss in vaginal and cesarean delivery. Am J Obstet Gynecol 2008; 199: 519.e1–e7.
13. Dildy GA III, Paine AR, George NC, Velasco C. Estimating blood loss: can teaching significantly improve visual estimation? Obstet Gynecol 2004; 104: 601–606.
14. Luegenbiehl DL. Improving visual estimation of blood volume on peripads. MCN Am J Matern Child Nurs 1997; 22: 294–298.
15. Toledo P, McCarthy RJ, Burke CA, Goetz K, Wong CA, Grobman WA. The effect of live and web-based education on the accuracy of blood-loss estimation in simulated obstetric scenarios. Am J Obstet Gynecol 2010; 202: 400.e1–e5.
16. Stefanidis D, Acker C, Heniford BT. Proficiency-based laparoscopic simulator training leads to improved operating room skill that is resistant to decay. Surg Innov 2008; 15: 69–73.
17. Ali J, Cohen R, Adam R, et al.. Attrition of cognitive and trauma management skills after the Advanced Trauma Life Support (ATLS) course. J Trauma 1996; 40: 860–866.
18. Crofts JF, Bartlett C, Ellis D, Hunt LP, Fox R, Draycott TJ. Management of shoulder dystocia: skill retention 6 and 12 months after training. Obstet Gynecol 2007; 110: 1069–1074.
19. Berg CJ, Harper MA, Atkinson SM, et al.. Preventability of pregnancy-related deaths: results of a state-wide review. Obstet Gynecol 2005; 106: 1228–1234.
20. Clark SL, Belfort MA, Dildy GA, Herbst MA, Meyers JA, Hankins GD. Maternal death in the 21st century: causes, prevention, and relationship to cesarean delivery. Am J Obstet Gynecol 2008; 199: 36.e1–e5.
21. Blumenfeld A, Ben Abraham R, Stein M, et al.. Cognitive knowledge decline after Advanced Trauma Life Support courses. J Trauma 1998; 44: 513–516.
22. O’Steen DS, Kee CC, Minick MP. The retention of advanced cardiac life support knowledge among registered nurses. J Nurs Staff Dev 1996; 12: 66–72.
23. Moser DK, Dracup K, Guzy PM, Taylor SE, Breu C. Cardiopulmonary resuscitation skills retention in family members of cardiac patients. Am J Emerg Med 1990; 8: 498–503.
24. Kaye W, Mancini ME. Retention of cardiopulmonary resuscitation skills by physicians, registered nurses, and the general public. Crit Care Med 1986; 14: 620–622.
25. Arthur WBW, Stanush PL, McNelly TL. Factors that influence skill decay and retention: a quantitative review and analysis. Hum Perform 1998; 11: 57–101.
Keywords:

Education; Estimated blood loss; Postpartum hemorrhage; Skill decay

© 2012 Society for Simulation in Healthcare