Simulation is rapidly becoming a vital component of the residency education process.1 Just as simulation in aviation has revolutionized pilot training, medical simulation is expected to revolutionize healthcare.2 When residents train with simulators, they frequently work in teams, with each participant having a different role in the scenario. However, it is not known whether the participant’s role in the simulation scenario affects their physiologic or psychologic stress response or their assimilation of new clinical knowledge.
Heart rate and self-perceived stress are likely to be markers of the physiologic and psychologic stress response experienced by participants in simulation scenarios. We therefore sought to measure the heart rates and self-perceived stress and self-perceived learning of participants in two simulation scenarios, as well as evaluate their written responses to objective questions at the completion of the experience. We hypothesized that measurements of participants’ stress and learning would be similar among all team members in our simulated scenarios.
We performed a prospective, randomized, cohort trial.
The study was conducted at a single US Midwestern Level 1 Trauma and Tertiary Care Medical Center. This Medical Center supports more than 200 residents and fellows and, on a rotating basis, trains medical students from multiple medical schools.
Selection of Participants
Participants included residents in emergency (EM), family, internal, and pediatric medicine as well as medical students on the EM elective. All the participants in the study were taking the Difficult Airway Course offered by the Department of Emergency Medicine. This annual, 1-day course includes interactive workshops using simulated cases. For the course in 2006, all resident and student participants in the Difficult Airway Course were scheduled to take part in the simulation workshops. All of the EM residents and most of the other residents and medical students had already received instruction in the use of high-fidelity simulation mannequins and were familiar with simulation scenarios. Because the Difficult Airway Course is for educational purposes only, the scenarios were not used to assess, grade, or pass/fail a participant. Our institution’s investigational review board approved this study as consent exempt.
Two difficult airway scenarios were developed by a board-certified EM faculty physician, tested in a pilot program, and revised with consensus input from two other board-certified EM faculty physicians. Scenario A simulated an adult patient, and Scenario B simulated a pediatric patient. The cases were presented in a standardized fashion to each group of participants. The simulators (Sim Man, SimBaby, Laerdal, Wappingers Falls, NY) displayed standardized initial vital signs, vital sign trends, and responses to therapeutic interventions, all of which were preprogrammed in the computer software. The EM faculty physicians who developed the scenarios also designed the physiologic parameters using the scenario editing software.
The software ran the simulations in exactly the same manner for every group of participants. Characters in the scenario such as EM department personnel, consultants, and patient family members were portrayed by faculty members. The faculty involved were experienced in using case simulations for oral exams and in presenting simulated patients, staff members, and patients’ family members in a neutral manner. All faculty involved in the case presentations received a written version of the case before the scenario was presented. The written version contained scripted cues that could be given to resident or student participants during a scenario. To ensure a standard portrayal of the scenario to all participants, the faculty practiced their presentations of the case with each other before the study began. Only scripted cues (patient complaint, vital signs, physical examination findings, nurse statements, and response to therapy) were allowed to be given to the participants. Faculty were specifically requested not to deviate from the scripted cues.
Moreover, faculty were instructed to avoid providing any information to participants not specified in the case script.
Groups of three to five residents or students participated in each scenario. Participants in each group were randomized according to a random number table that assigned them to one of three roles: Team Leader, Procedure Chief, or Important Team Member. Once assigned to a role, each participant received a typed index card delineating their role.
You will lead the team by directing their actions and coordinating their efforts to manage the patient’s case. You will not perform procedures. You should take advice and input from team members in the process of making decisions. You have final say in what actions are to be taken in managing the case.
You will be the primary person to perform any needed procedure. It is your responsibility to accomplish a procedure once the Team Leader decides that it needs to be done. You may allow team members to assist you, give advice, or to attempt the procedure. If there is more than one procedure to be performed, please allow a team member to perform the second or the third procedure.
Important Team Member
The Team Leader and Procedure Chief will direct your actions in caring for the patient. Although it will not be your primary responsibility to make treatment decisions or perform procedures, your input into treatment decisions and help with procedures and patient management will be critical to the success of your team.
All participants, regardless of role, were viewed as active members of the simulation team, and all were active in the treatment of the patient, including performance of procedures, direction of procedures, acquiring equipment, assembling equipment, and contributing to decisions.
Groups of students and residents rotated through both scenarios in a continuous fashion. This limited any interaction between the groups during the study. All participants gave written agreement not to disclose the scenario to any other participant. Because participation in the scenarios was for educational purposes only, there was little incentive not to abide by this agreement. Also, we believe there is a culture of academic honesty in our department that in some measure inhibits disclosure of privileged information.
The scenarios played out are as follows.
A 30-year-old man has been involved in a motor vehicle accident. History, physical examination findings, and radiographic findings clearly indicate a severe cervical spine fracture with anterior cord syndrome. The mannequin is on a backboard with a hard cervical collar on the neck. The patient suddenly becomes cyanotic and needs a crash airway. Bagging is difficult and only somewhat effective. Neck movement to intubate the patient is relatively contraindicated because of the need to avoid a complete cord injury. As a cognitive forcing strategy, there is a delay in finding a cricothyrotomy tray.
Laryngeal mask airway (LMA) or bagging with oral and nasal airways are acceptable bridge devices. Once the LMA is placed successfully, one of the team members may perform cricothyrotomy before transferring the patient through helicopter.
A 1-year-old female infant started choking after placing a penny in her mouth. Initially, the child is conscious, coughing, and breathing with difficulty. The child then suffers acute, complete airway obstruction but is still conscious. The participants are expected to perform Heimlich abdominal thrusts. These are unsuccessful.
The child then becomes unconscious. Participants are then expected to proceed to a direct laryngoscopy to remove the coin. After coin removal, the resident can choose to either intubate or bag-valve-mask ventilate with the expectation that the child will begin to breathe on her own in a short time.
Methods of Measurement
Each group of participants worked through each scenario by interacting with the simulator and the faculty actors. After randomized assignment to a role, each participant had a pulse oximeter probe attached to one finger and was then monitored by portable continuous pulse oximetry. At two points in the study, a research nurse or attending EM physician recorded the participants’ heart rates as noted on the pulse oximeter. The first point (preprocedure) was within a 5-minute period before the scenario began (the participants were standing and ambulatory for this measurement), and the second point was at the time that each team was addressing the critical airway intervention for each case (Scenario A, LMA placement; Scenario B, removal of coin from airway). After a period of debriefing, each participant completed a standardized data collection form that included a visual analog scale (VAS) for perceived stress and for perceived learning and three objective questions related to the teaching points of each case. Anchors for the perceived stress VAS were “None” and “Most Severe.”
Anchors for learning were “None” and “Most Excellent.”
Data Collection and Processing
A standardized data collection instrument was used. This instrument gathered heart rate measurements, 10 cm VAS measurements of perceived stress and perceived learning, and responses to objective test questions covering the teaching points of each case. Data were then taken from our collection instrument and entered into an Excel spreadsheet by our research coordinator. The data were analyzed with the use of SPSS version 15.
Primary Outcome Measure
The change in participants’ heart rates from preprocedure to the time of the critical airway intervention in each scenario.
Secondary Outcome Measures
Participants’ self-perceived stress and self-perceived learning, and written responses to objective test questions covering salient learning points in each scenario.
Mean heart rates of participants were compared between preprocedure and critical airway intervention with the Wilcoxon Signed Ranks test and between roles using one-way analysis of variance. Use of the Wilcoxon Signed Ranks test enables eliminating within-group variance in favor of focusing on individual changes over time and, hence, provides a more sensitive test of changes. Correlations between self-reported stress and learning were evaluated with Spearman correlation. We calculated that a sample size of 47 heart rate measurements was required for detection of a correlation (r) of >0.40 with a two-sided α of 0.05 and power of 80%, whereas a sample size of 12 subjects in each role was required for detection of an effect size on heart rate difference between roles of 1.0 with a two-sided α of 0.05 and power of 80%.
A total of 38 unique participants took part in the scenarios, affording a total of 53 measurements. Because of the format for the 1-day course and the requirement for maintenance of anonymity on data acquisition forms, we allowed participants to perform Scenario A, Scenario B, or both scenarios as time permitted. A total of 28 subjects were measured using Scenario A, whereas 25 subjects were measured using scenario B. The number of simulations for each level of trainee was as follows: five for the medical students, 22 for the first year residents, 12 for the second year residents, and 14 for the third year residents. The role of procedure chief was performed by 14 participants, team leader by 13 participants, and team member by 26 participants. Procedures were performed by 29 participants.
Mean self-reported stress level on a 100-point VAS across all roles was 54 with a standard deviation (SD) of 20. Mean heart rate across all roles at the start of each scenario was 84, with SD of 12, minimum 46 and maximum 108. Mean heart rate across all roles at the critical airway intervention of each scenario was 88, with SD of 12, minimum 66 and maximum of 112. Figure 1 shows a graphical representation of each participant’s heart rate preprocedure and then during the critical airway intervention of each scenario. Using a Wilcoxon Signed Ranks test to account for the repeated measure on each participant, we found a small absolute but statistically significant increase in heart rate during the critical airway intervention, with a median increase of 4 beats per minute (bpm), interquartile range of −4 to 8 bpm, minimum −21 bpm, and maximum 56 bpm (P = 0.04).
On examining heart rate changes during the procedure, we found no significant differences among different roles (one-way analysis of variance, P = 0.06) (Fig. 2). Furthermore, the role a participant played in a scenario seemed to have no effect on stress level (P = 0.27) (Fig. 3). Mean self-reported learning value on a 100-point VAS across all roles was 81, with SD 12, minimum 62 and maximum 100. Self-reported learning value increased with self-reported stress level (rs = 0.37, P = 0.01); however, no significant correlation was found between heart rate increase and stress (P = 0.09) or heart rate increase and learning value (P = 0.56). The role a participant played in a scenario had no effect on perceived learning value (P = 0.57) (Fig. 4).
On examination of the effects of level of training on our measurements, we found no significant differences between medical students and residents at all three levels of training with respect to heart rate changes during the procedure or self-reported learning value; however, we found that self-reported stress levels were statistically significantly lower in residents in their last year of training when compared with all other participants (Fig. 5).
Postscenario objective test scores were uniformly high, with the majority of participants getting all three questions correct, eight participants missing one question, and three participants missing two questions. No effect of role was seen on final test score (P = 0.74).
Work-related stress is a pattern of reactions that occur when workers perceive an imbalance between the demands facing them and the knowledge and skills they have to meet those demands. Worker reactions to this perceived imbalance may include: physiologic (heart rate, blood pressure, and stress hormones), emotional (anxiety and irritation), cognitive (narrowing of attention), and behavioral (impulsiveness and mistakes made) responses.3,4
A previous study of EM residents demonstrated that participant’s heart rate and perceived stress increased during a simulated scenario of anaphylaxis.5 We found that participants’ heart rates, self-reported stress, and learning were equally affected among all roles in our limited scenarios. Our results support the concept that, despite taking part in different roles, all participants may indeed benefit from simulation scenarios. Our study was structured to evaluate heart rate measurements, self-assessments, and objective questions. It is likely that more specific measures such as leadership ability or procedural skill may be role dependent.
Physiologic parameters such as heart rate, heart rate variability, blood pressure, salivary cortisol, and white blood cell count have all been considered objective markers of a participant’s stress in studies of clinical practice or learning.6–9 In our study, residents in multiple specialties and medical students had measured increases in heart rates during our simulated cases. The mean increase in heart rate was about 4 bpm with a SD of 12. This is comparable with previously reported heart rates for nurses performing endotracheal suctioning in the intensive care unit,10 physicians taking part in the Advanced Cardiac Life Support practical test,11 plastic surgeons performing rhinoplasty,7 and surgical residents on call6 but is less than the increase reported for surgical residents taking the Advanced Trauma Life Support course9 and cardiologists performing cardiac catheterizations.8
We also found that participants had success answering standardized questions correctly postscenario regardless of their role in the scenario. In our study, participants’ self-perceived learning increased with their self-reported stress levels. It has been shown that persons in an activated autonomic state have improved memory of events.12
Previous studies have shown that more experienced operators show fewer changes in their physiologic and psychologic measures of stress.6,8 Although all our participants had trained on simulators before this study, none of them had extensive experience of training in this modality. Our data show that participants with more clinical training reported less stress during our simulated scenarios than participants with less clinical training. Blunting of the stress response as a result of experience would be another positive aspect of simulation training.
We measured preprocedure ambulatory heart rates just before the participants beginning the simulation scenarios. Consequently, heart rates may have been elevated above usual resting rates because of the anticipated challenges that the participants were about to face. This would have the potential to reduce the difference in heart rates from the start of the case to the critical intervention. Future research could improve on our model by measuring heart rates on a separate day during a traditional didactic conference to get a better measure of a participant’s preprocedure heart rate. Using holter monitors to get continuous data and assessing heart rate variability would be helpful. It would also be interesting to measure heart rates at the time of critical interventions during real resuscitations in the EM department and compare them to heart rates during simulation scenarios. The stress invoked by simulation scenarios has not been compared with real emergency department resuscitations. Moreover, the use of heart rate is a limited measure of stress because it may be affected by physical and cognitive activity.
Technical hurdles with pulse-oximetry measurements hindered our ability to measure heart rates at more than 2 points in time for each participant. During the course of the simulated cases, the finger probe would frequently become loose or dislodged and would have to be replaced. To not disrupt the simulation to any great degree, we limited our measurements to before the case began and at the point of the critical airway actions.
We used a VAS to measure the subjective aspects of participants’ stress and learning, which may have limited sensitivity in measurement. Nevertheless, the psychologic effects on participants in simulated environments have been successfully measured by other authors using VAS. Wright et al studied participants using high-fidelity simulators in a medical helicopter environment. They measured participants’ responses on VAS scales to questions regarding comfort with their level of training, the value of medical simulation, and their awareness of obstacles in the air-medical environment.13 Lapostolle et al14 used VAS to measure the certainty of prehospital providers that they felt a pulse on a high-fidelity mannequin. Seaberg et al15 used VAS to measure whether a simulated waiting room experience helped resident trainees empathize with their patients and whether the exercise was useful. Until more objective measures of self-efficacy are developed for simulation participants, VAS seems to be a reasonable alternative.
Participants’ heart rates increased from preprocedure to the time of critical intervention. Self-reported learning values increased with self-reported stress levels. Participants were successful in answering factual questions about the teaching points covered in the simulation. In our limited scenarios, measurements of stress and learning did not differ by role. Our results support the concept that all participants may indeed benefit from simulation scenarios.
1. Godwin SA, Caro D, Harwood-Nuss A. Simulation Training in an Emergency Medicine Residency
. Chicago, IL: Accreditation Council for Graduate Medical Education; 2005.
2. Dunn W, Murphy JG. Simulation: about safety, not fantasy. Chest
3. Houtman I, Jettinghoff K, Cedillo L. Raising awareness of stress
at work in developing countries: a modern hazard in a traditional working environment. Advice to employers and worker representatives. Geneva: World Health Organization; 2007.
4. Sende J, Jbeili C, Schvahn S, et al. Stress
among emergency physicians in France: stress
and consequences. Ann Emerg Med
5. Syrett JI, Rose LJ, Spillane LL. Simulating reality—the role of simulators in emergency medicine resident training. Acad Emerg Med
6. Tendulkar AP, Victorino GP, Chong TJ, et al. Quantification of surgical resident stress
“on call.” J Am Coll Surg
7. Demirtas Y, Tulmac M, Yavuzer R, et al. Plastic surgeon’s life: marvelous for mind, exhausting for body. Plast Reconstr Surg
2004;114:923–931; discussion 932–933.
8. Detling N, Smith A, Nishimura R, et al. Psychophysiologic responses of invasive cardiologists in an academic catheterization laboratory. Am Heart J
9. Quilici AP, Pogetti RS, Fontes B, Zantut LF, Chaves ET, Birolini D. Is the advanced trauma life support simulation exam more stressful for the surgeon than emergency department trauma care? Clinics
10. Smith AM, Ortiguera SA, Laskowski ER, et al. A preliminary analysis of psychophysiological variables and nursing performance in situations of increasing criticality. Mayo Clin Proc
11. Lima E Jr, Knopfholz J, Menini CM. Stress
during ACLS courses: is it important for learning skills? Arq Bras Cardiol
12. Maheu FS, Joober R, Lupien SJ. Declarative memory after stress
in humans: differential involvement of the beta-adrenergic and corticosteroid systems. J Clin Endocrinol Metab
13. Wright SW, Lindsell CJ, Hinckley WR, et al. High fidelity medical simulation in the difficult environment of a helicopter: feasibility, self-efficacy and cost. BMC Med Educ
14. Lapostolle F, Le Toumelin P, Agostinucci JM, Catineau J, Adnet F. Basic cardiac life support providers checking the carotid pulse: performance, degree of conviction, and influencing factors. Acad Emerg Med
15. Seaberg DC, Godwin SA, Perry SJ. Teaching patient empathy: the ED visit program. Acad Emerg Med