Journal Logo

Empirical Investigations

Effects of Simulation Versus Traditional Tutorial-Based Training on Physiologic Stress Levels Among Clinicians: A Pilot Study

Bong, Choon Looi MBChB, FRCA; Lightdale, Jenifer R. MD, MPH; Fredette, Meghan E. BS; Weinstock, Peter MD, PhD

Author Information
Simulation in Healthcare: The Journal of the Society for Simulation in Healthcare: October 2010 - Volume 5 - Issue 5 - p 272-278
doi: 10.1097/SIH.0b013e3181e98b29
  • Free

Abstract

High-fidelity medical simulation is fast becoming a standard of health care training throughout undergraduate and postgraduate education. The teaching tool emphasizes experiential learning and capitalizes on well-established principles of “andragogy” or adult learning.1 High-fidelity simulation- based training (SBT) has also been commonly described as a high-stress experience.2–4 It has been hypothesized that the stress and anxiety SBT evokes are central to its effectiveness as a teaching tool.

Numerous studies in rodents and humans have demonstrated that adrenal stress hormones can influence learning and memory performance in a variety of ways depending on timing and profile of hormone release.5–7 For example, human studies have found that learning and memory can be enhanced when adrenal stress hormones are released postlearning.8–10 In addition, a subject's perception of stress results in differing patterns of neuroendocrine activation. An easily handled and successful challenge elicits norepinephrine and testosterone release, whereas with increased anxiety and difficulty, epinephrine, prolactin, renin, and fatty acids are the responsive mediators. As distress grows, rises in cortisol become prominent.11

A small pilot study by Muller et al4 among adult intensivists reported elevated markers of physiologic stress after high-fidelity SBT. In particular, salivary amylase and cortisol levels increased significantly within subjects during the course of all simulated sessions independent of specific course or session objectives. Interestingly, salivary alpha-amylase was also shown to be reduced and performance to be improved after repeat exposure to simulation training, although the exact relationship remains unclear.

To begin to better understand the dose-response relationship of simulation and stress and its effects on learning, we measured and compared physical and biochemical markers of stress in participants undergoing standardized SBT with those undergoing traditional “sit-down” didactic/tutorial-based sessions. Our primary study hypothesis was that participants engaged in SBT would experience increased stress levels, manifested by increased heart rate (HR) and salivary cortisol (SC), compared with participants engaged in traditional “sit-down” interactive-education training (IET). We also explored secondary study hypotheses including whether participants of SBT would experience differing levels of stress, depending on (1) their role and perceived level of involvement in the SBT sessions and (2) the number of sessions they participated in.

Our primary study outcomes were physiologic and biochemical measures of stress response. Tachycardia, because of sympathetic stimulation, is a well-recognized sign of the stress response. Therefore, we chose subject's HR as an objective and noninvasive physical measure. For more than 30 years, cortisol levels has been used as a measure and marker of physiologic stress.12 Recently, the salivary form of cortisol has been shown to be synchronous with the serum concentration across the 24-hour time frame13 and easily measured with a simple enzyme immunoassay,14 making it an optimal assay for psychoneuroendocrinology research, clinical environments, and behavioral sciences. SC was, therefore, chosen as the biochemical marker of stress response for our study.

METHODS

The study was independently approved by the Children's Hospital Boston Institutional Review Board and dovetailed with an ongoing study protocol, where physicians in the Division of Gastroenterology were randomized to undergo crisis resource management (CRM) training either through a more traditional “sit-down” IET session or in a fully integrated SBT session with other relevant clinical staff (nurses and technicians) in the Gastroenterology Procedure Unit. Physician subjects were stratified into three groups by experience level (Fellows, Junior endoscopists—less than 5-year experience, and Senior endoscopists—greater than 5-year experience) and were randomized to either training arm (SBT or IET) by opening consecutive sealed opaque envelopes containing computer-generated random numbers. Demographic information, including age and endoscopy experience, and written consent were obtained from all participants. In the SBT arm, we also noted individual participation by scenario and tabulated numbers of scenarios per participant.

In total, four SBT courses and two IET courses, both 3 to 4 hours in duration, were delivered on 4 different days. The learning objectives of both courses including review of the same four case scenarios (Table 1). To minimize the effects of diurnal variations in cortisol production, all the sessions were conducted at the same time of the day (1–5 PM). Before being divided into randomly assigned IET and SBT groups, all participants underwent a combined 30-minute introductory lecture focusing on the principles of CRM and participated in a team exercise in which they were given the task of passing tennis balls among one another in no particular order. The exercise was designed to demonstrate elements of a crisis (eg, multitasking) and provide an opportunity for the team to discuss principles of CRM and emphasize the importance of teamwork, resources allocation, and effective communication. After the lecture, participants were assigned to either SBT or IET training sessions. Figure 1 illustrates the study design and session content.

T1-5
Table 1:
Outline of SBT Case Scenarios and Goals
F1-5
Figure 1.:
Study design and session content. After randomization, physicians participated in either simulation-based or interactive-education training (SBT and IET, respectively). SBT and IET course content were designed to be identical with the exception of teaching modality—simulation versus tutorial. T0, T1, T2, and T3 denote the time points at which heart rate and salivary cortisol were collected.

IET Sessions

Physicians only were randomized to IET sessions, which involved traditional lecture style teaching with multimedia interactive components consisting of video, workshop, group discussions, and hands-on activities to learn and foster the development of new information and skills in endoscopy, sedation, patient monitoring, and CRM. Attendees spent approximately half the session (2 hours) sitting at a conference table, listening to presentations, and engaging in the case discussions; the remaining time was interspersed and was spent standing up or moving around, participating in multimedia activities. A capnography station was included where participants took turns having themselves monitored by capnography and learning how to interpret the capnographs. Physical exertion was minimal and involved a mixture of seated and hands-on activities. HR and SC levels were obtained 30 minutes before the IET sessions (baseline, T0) and immediately after the IET sessions (T2). There was no debriefing involved in the IET sessions.

SBT Sessions

Four SBT courses were conducted. During each, the participants were given the same brief 30-minute introduction on the principles of CRM and high-fidelity simulation. This was followed by the four high-fidelity simulation crisis scenarios (Table 1) lasting approximately 20 minutes each. In each scenario, a different physician took part as event manager and led the team (composed of a physician, nurse, and technician) in managing the crisis. The scenarios required the participants to be standing, but physical exertion was minimal. Each scenario was immediately followed by a 20-minute video-assisted seated debriefing by a facilitator who was well trained in the principles of CRM (P.W.). In this way, we sought to maintain a similar ratio of sitting and standing time in SBT to that in IET. HR and SC levels were obtained at four time points for each participant: baseline at 30 minutes before (T0), immediately before the scenario (T1), immediately after the scenario (T2), and immediately after debriefing (T3).

Data Collection

Data collection was accomplished by two investigators who were not involved in teaching the SBT or IET sessions. HR was recorded using the Mio Ultimate HR watch (Mio) and verified using pulse oximetry (Nonin Onyx II 9550). SC samples were collected using Salimetrics Oral Swabs (Salimetrics) placed under the tongue for 2 minutes, and the swabs were then placed in sample storage tubes and kept frozen at −80°C until all samples were centrifuged and analyzed after the completion of all SBT and IET sessions, at the Salimetrics laboratory (Salimetrics LLC, State College, PA) using the ELISA technique.

Statistical Analysis

As there are no comparable investigations addressing the stress response during SBT versus IET, sample size calculation was determined for this primary study hypothesis mainly by convenience and by modeling the study of Muller et al.4 All data were analyzed with SPSS software version 16.0 (SPSS Inc., Chicago, IL). Changes in HR and SC from baseline to the end of the sessions were calculated as the difference in HR or SC between T0 and T2. Nonparametric statistical analysis was performed. Median changes in (and interquartile ranges [IQRs]) HR and SC in physicians participating in the SBT and IET sessions were compared using the Mann-Whitney U test. Median changes in HR and SC from T0 to T2 were compared between the different provider groups participating in the SBT sessions using the Kruskal-Wallis test, P values <0.05 were considered significant.

RESULTS

Fourteen physicians were randomized to the IET arm, whereas 13 physicians, 11 nurses, and four technicians participated in the SBT sessions (Table 1). Physicians were balanced across study arms in terms of their experience (per the randomization scheme) and their age and gender (Table 2). Each physician subject randomized to SBT courses was assigned to be the primary event manager during only one scenario. Because of work schedules and presence in the Gastroenterology Procedure Unit on the days of the SBT courses, two of the nurses and all four of the technicians took part in more than one scenario.

T2-5
Table 2:
Demographics of Participants

Physiologic Measurements of Physician Stress After SBT Versus IET

Physicians in the SBT group had a median increase in HR from T0 to T2 of 17 (IQR, 2–24) beats/min (bpm) (Fig. 2), compared with physicians in the IET group, who had a median decrease in HR of 4 (IQR, −8 to 2) bpm, P = 0.001 (Fig. 2A). SC response (change from T0 to T2) was measured and compared among physicians engaged in SBT versus IET (Fig. 2B). Physicians in the SBT group had a median increase in SC of 0.140 (IQR, 0.040–0.209, approximately 1.9-fold compared with baseline), whereas physicians in the IET group had a median decrease in SC of 0.015 (IQR, −0.051 to 0.005, approximately 1.2-fold) compared with baseline, P = 0.001. There were no differences in physiologic or biochemical measures of physician stress engaged in either SBT or IET when analyzed by experience (data not shown). Data from one of the physicians in the SBT group was excluded because this individual had a baseline SC that was more than 10-fold higher compared with the rest of the study population. (For safety reasons, the identity of this physician was unblinded before final data analysis, and the physician was informed and referral was made to the relevant primary care physician.)

F2-5
Figure 2.:
Physiologic measurements of physician stress after SBT versus IET. Change in HR (A) and salivary cortisol (B) from baseline to end of session. HR expressed as bpm and salivary cortisol as μg/dL. Black bars show the median, upper, and lower end of the box representing the first and third quartile. Narrow lines above and below the box represent the spread of HR and values. Circles are for values with 1.5–3-fold interquartile distance and asterisks for values with more than threefold interquartile distance.

Time Course of Stress Response During SBT

The median baseline HR increased significantly in a bell pattern from 78 (IQR, 68–84) to 88 (IQR, 78–102) just before the start of the SBT session (T1) (Fig. 3), reaching a maximum peak at 100 (IQR, 82–106) bpm at the end of the scenario (T2) and eventually decreased but remained elevated above the baseline at 80 (IQR, 78–88) bpm just after debriefing (T3) (Fig. 3A). There was a significant change in HR in all participants across the different time points (from T0 to T3) during the SBT. Overall, there was a median increase in HR from T0 to a peak at T2 of 18 bpm, (IQR, 10–28), P = 0.001.

F3-5
Figure 3.:
Time course of stress response in all participants during SBT. Change in HR (A) and salivary cortisol (B) over time for all participants. HR expressed as bpm and salivary cortisol as μg/dL. Black bars show the median, upper, and lower end of the box representing the first and third quartile. Narrow lines above and below the box represent the spread of HR and values. Circles are for values with 1.5–3-fold interquartile distance and asterisks for values with more than threefold interquartile distance.

The change in SC over time during the SBT sessions was similar to the HR changes (Fig. 3B). The median baseline SC increased significantly from 0.136 (IQR, 0.082–0.224) μg/dL to 0.186 (0.114–0.300) μg/dL just before the start of the SBT session (T1), further increasing to 0.229 (0.120–0.422) μg/dL at the end of the scenario (T2), and remained elevated at 0.232 (0.093–0.407) μg/dL just after debriefing (T3). There was a significant change in SC in all participants across the different time points (from T0 to T3) during the SBT. Overall, there was a median increase in SC from T0 to T2 of 0.089 (IQR, 0.003–0.201), P = 0.004.

Comparison of Physiologic Stress Among Different Provider Groups During SBT

There were similar increases in HR across the different groups of providers after SBT. Physicians, RNs, and Technicians had median increases in HR of 17 (1.2-fold), 27 (1.3-fold), and 16 (1.2-fold) bpm, respectively (Fig. 4). There was no statistically significant difference in the median increases in HR between provider groups. Physicians, RNs, and Technicians experienced median increases in SC of 0.140 μg/dL (twofold), 0.131 μg/dL (twofold), and 0.013 (1.1-fold) μg/dL, respectively. The technicians experienced a relatively smaller increase in SC compared with the physicians and RNs (P = 0.046).

F4-5
Figure 4.:
Comparison of physiologic stress among different provider groups during SBT. Change in HR (A) and salivary cortisol (B) at end of SBT sessions between provider groups. HR expressed as bpm and salivary cortisol as μg/dL. Black bars show the median, upper, and lower end of the box representing the first and third quartile. Narrow lines above and below the box represent the spread of HR and values. Circles are for values with 1.5–3-fold interquartile distance and asterisks for values with more than threefold interquartile distance.

DISCUSSION

The results of our study show that physicians who underwent SBT involving four scripted case scenarios experienced a significantly higher stress response—evidenced by increases in both physiologic and biochemical markers, when compared with those who engaged in tutorial-based discussion and management of the same cases. This response was similar in all team members who participated in simulation-based training and demonstrated a bell pattern in its overall time course. When analyzed by role, endoscopy technicians participated in more simulation and seemed to experience a relatively smaller increase in SC (1.1-fold versus 2-fold), when compared with physicians and RNs. Additional findings included that measures of physiologic stress among caregivers was elevated from baseline even before engaging in (in anticipation of) simulation and also remained above baseline during debriefing.

Despite rapid growth in the use of simulation-based education methodologies worldwide, little is known regarding the learner's physiologic response during simulation training, thus limiting our ability to fine tune the instrument efficiently and accurately. One notable study, as early as 1977, measured pilot stress during simulated versus actual emergency flight conditions15 and showed that high-fidelity simulation resulted in a measurable stress response as evidenced by increased catecholamine secretion in both students and instructors, independent of their previous flight experience. Our study of medical crises similarly found that regardless of experience, all physicians manifested increased physiologic and biochemical measures of stress in the SBT arm.

Our study seeks to expand on the study of Muller et al4 by comparing the simulation stress response to traditional tutorial-based medical training. We confirm their study in two ways: (1) we found increased physiologic and biochemical measures of stress in participants undergoing SBT and (2) we also found that technicians who experienced multiple SBT sessions had less overall increase in SC. Such “normalization” calibrates simulation-induced stress to baseline (IET) and sets the stage for titration of the simulation experience to optimize learning.

SBT resulted in marked increases in both physical (HR) and biochemical (cortisol) levels of stress after all SBT sessions. This finding raises the issue that simulation scenarios may be stressful in excess and not sufficiently titrated. It is possible that many apparently “authentic” simulated clinical experiences are too emotionally robust and “over shoot” the stress and anxiety that might be produced by a real event, therefore, placing all participants on the “plateau” portion of the stress curve. This potential “physiologic artifact” may be supported by our findings of increased HR among participants relative to their baseline even prior engaging in simulation-based versus tutorial-based learning. It also suggests that the intrinsic qualities of simulated experiences themselves (or even in anticipation of these experiences such as being observed, evaluated, and/or video taped) may be sufficiently anxiety provoking.

Such simulation “overshoot” was described by Quilici et al,16 who measured HR and blood pressure as stress markers in surgeons during the practical examination in an Advanced Trauma Life Support course and compared the values with those taken during real-life clinical care in the emergency room. They found that the stress values during the simulated scenario were higher than those measured in real cases. Similarly, Hassan et al17 demonstrated that insufficient stress coping strategies among novices in surgery correlated with poor performance in laparoscopic procedures. These findings suggest that achievement of authentic physiologic response in the learner is a potentially important further component of simulator reality. Our data measurements, comparing SBT with IET, and establishment of a standard response curve begins to allow us to study, manipulate, and achieve this element with the goal of optimizing the simulation experience on all fronts with the goal of improving learning.

There were several limitations to our study. First, complex logistics involving simultaneous collection of both saliva samples and measurement of HR limited our sample size, which may have limited our ability to detect differences in several areas. Two of the scenarios involved actors. Use of the Kruskal-Wallis test did not reveal any differences in stress between the scenarios with or without actors, but because of the small numbers, further analysis would not have been appropriate. The observation that physicians experienced similar patterns of stress independent of scenario content and level of clinical expertise may be the result of our small sample size and deserves further investigation with larger numbers.

Second, the fact that only some of our participants (two nurses and four technicians) took part in more than one SBT scenario makes it difficult for us to precisely test a secondary study question of the effect of repeated exposure to the SBT experiences on physiologic differences over time. In fact, we did find that technicians (who were all exposed to more than one SBT scenario) seemed to experience a lower increase in SC than the physicians and nurses. Our study also questioned whether differences in measurable stress would be found according to professional roles. Further study with larger numbers of participants is warranted to determine whether lower elevation in SC seen in our technicians was due to “blunting” as a result of repeat exposure or due to differences in perceived roles and responsibilities in the simulated crisis.

For reasons of confidentiality, we were not able to obtain information on whether our subjects (colleagues) were on beta blockers, calcium-channel blockers, or other medications. Although unlikely, this may potentially confound our results. Finally, because this initial study focused on normalization and calibration of the stress response to high-fidelity SBT versus a standard curriculum (IET), we did not measure specific outcome related to the two different types of experiences. Studies ongoing in our laboratory are aimed at modulating/titrating the simulation experience to a variety of stress levels and measuring the dose response with respect to physiology, performance/learning outcomes, and patient outcomes.

In summary, our study demonstrates that clinicians, independent of clinical role, experience significant increases in physiologic stress before, during, and after high-fidelity simulation, when compared with traditional tutorial-based education and that this response can be measured easily and noninvasively over time. By evaluating the simulation experiences at the level of the learner's physiologic milieu and response, physiologic studies such as these magnify the field to microanalysis of the simulation experience. Close attention to simulation-induced stress will allow educators to take full advantage of the teaching tool by optimizing learning and practice with the overarching goal of improving patient safety and outcomes.

ACKNOWLEDGMENTS

The authors thank the following individuals at Children's Hospital Boston for their invaluable support and contributions: Lisa A. Heard, BSN, RN, CGRN; Jeffrey Burns, MD, MPH; Alan M. Leichtner, MD; Liana Kappus, MEd; and the staff of the Gastroenterology Procedure Unit and Children's Hospital Boston Simulator Program.

REFERENCES

1. Knowles MS. The Modern Practice of Adult Education: From Pedagogy to Andragogy (Revised Edition). Englewood Cliffs: Prentice Hall/Cambridge; 1980.
2. van Dulmen S, Tromp F. Grosfled F, ten Cate O, Bensing J. The impact of assessing simulated bad news consultations on medical students' stress response and communication performance. Psychoneuroendocrinology 2007;32:943–950.
3. John B, Suri I, Hillermann C, Mendonca C. Comparison of cricothyroidotomy on manikin vs. simulator: a randomized cross-over study. Anaesthesia 2007;62:1029–1032.
4. Muller M, Hansel M, Fichtner A, et al. Excellence in performance and stress reduction during two different full scale simulator training courses: a pilot study Resuscitation 2009;80:919–924.
5. de Kloet ER, Oitzl MS, Joels M. Stress and cognition: are corticosteroids good or bad guys? Trends Neurosci 1999;22:422–426.
6. Roozendaal B. Glucocorticoids and the regulation of memory consolidation. Psychoneuroendocrinology 2000;25:213–223.
7. McGaugh JL, Roozendaal B. Role of adrenal stress hormones in forming lasting memories in the brain. Curr Opin Neurobiol 2002;12:205–210.
8. Buchanan TW, Lovallo WR. Enhanced memory for emotional material following stress-level cortisol treatment in humans. Psychoneuroendocrinology 2001;26:307–317.
9. Cahill L, Gorski L, Le K. Enhanced human memory consolidation with post-learning stress: interaction with the degree of arousal at encoding. Learn Mem 2003;10:270–274.
10. Andreano JM, Cahill L. Glucocorticoid release and memory consolidation in men and women. Psychol Sci 2006;17:466–470.
11. James P Henry. Biological basis of the stress response. News Physiol Sci 1993;8:69–73.
12. Lewis J. Steroid analysis in saliva: an overview. Clin Biochem Rev 2006;27:139–146.
13. Dorn LD, Lucke JF, Loucks TL, Berga SL. Salivary cortisol reflects serum cortisol: analysis of circadian profiles. Ann Clin Biochem 2007;44:281–284.
14. Gozansky WS, Lynn JS, Laudenslager ML, Kohrt WM. Salivary cortisol determined by enzyme immunoassay is preferable to serum total cortisol for assessment of dynamic hypothalamic–pituitary–adrenal axis activity. Clin Endocrinol 2005;63:336–341.
15. Krahenbhul Gary S, et al. Stress and Simulation in Pilot Training. Final Report. ERIC (Education Resources Information) no. ED 167842. May 1977 to December 1977.
16. Quilici AP, Pogetti RS, Fontes B, Zantut LF, Chaves ET, Birolini D. Is the advanced trauma life support simulation exam more stressful for the surgeon than emergency department trauma care? Clinics 2005;60:287–292.
17. Hassan I, Weyers P, Maschuw K, et al. Negative stress-coping strategies among novices in surgery correlate with poor virtual laparoscopic performance. Br J Surg 2006;93:1554–1559.
Keywords:

Simulation; Physiologic stress; Salivary cortisol; Gastrointestinal endoscopy

© 2010 Society for Simulation in Healthcare