Conclusion: The results indicate that there was little difference overall in the students performance on assessment (whether by simulator or by instructor). The only variable to have a significant difference between the two groups was Inst_Pulse, which showed that students who practiced on the simulator performed significantly better when evaluated on a human subject by an instructor. The overall study findings support the transferability of the learning on the manikin to performance on individuals. They also suggest that simulation-based practice leads to no difference in assessment outcomes than practice on individuals. This is a positive finding for simulation community. Students practicing on pre-programmed simulators require much less faculty time to achieve the same results, indicating that simulation-based vital signs training may represent an equivalent, but cost-saving learning modality for teaching vital signs to pre-licensure nursing students.
1. Bern, L., Brandt, M., Mbelu, N., Asonye, U., Fisher, T., Shaver, Y. & Serrill, C. Differences in blood pressure values obtained with automated and Manual methods in medical inpatients. MEDSUR Nursing. 207; 16(6): 356-361.
2. Lockwood, C., Conroy-Hiller, T., Page, T. Systematic review: Vital signs, Joanna Briggs Institute Reports. 2004; 2: 207-230.
3. Bogan, B., Kritzer S., Dean, D. Nursing student compliance to standards for blood pressure measurement. Journal of Nursing Education. 1993; 32: 90-92.
Disclosures: Frances Lee, DBA, has the potential to receive royalties from scenarios published in the Laerdal SimStore(r). No royalties have been received to date. John Schaefer, MD is non-majority owner of Sim Tunes (outlet for Medical University of South Carolina licensed, copyrightable, simulation training products). A Medical University of South Carolina COI management policy in place. Dr. Schaefer also received patent royalties from Laerdal Medical Corp (SimMan/Baby/3G). A Medical University of South Carolina (MUSC) COI management policy is in place.
656 Evaluation of a Computer-based Educational Intervention to Improve Medical Teamwork and Performance During Simulated Patient Resuscitations
Rosemarie Fernandez, MD4, Marina Pearce, MA2, James Grand, PhD3, Tara Rench, MA2, Heather Brooks-Buza, BA5, Georgia Chao, PhD1, and Steve W. J. Kozlowski, PhD2
1MANAGEMENT, MICHIGAN STATE UNIVERSITY, EAST LANSING, MI, USA and 2PSYCHOLOGY, MICHIGAN STATE UNIVERSITY, EAST LANSING, MI, USA and 3PSYCHOLOGY, UNIVERSITY OF AKRON, AKRON, OH, USA and 4EMERGENCY MEDICINE, UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE, SEATTLE, WA, USA and 5ACADEMIC AND STUDENT PROGRAMS, WAYNE STATE UNIVERSITY, DETROIT, MI, USA
Introduction/Background: Medical resuscitation teams are ad hoc interdisciplinary action teams that can threaten patient safety due to their: (1) interdisciplinary nature, (2) complex, dynamic and time-pressured working conditions, and (3) high membership variability.1-4 These characteristics inhibit appropriate teamwork, resulting in poor communication and coordination, failure to recognize threats to patient safety, and decreased team effectiveness.5,6 Recent educational efforts to improve healthcare team effectiveness have focused on team training. While some of these efforts have demonstrated promising results, feasibility issues limit their implementation.7,8 Availability of an easily implemented, low-resource-demand team training program could benefit institutions unable to support more resource-intensive efforts. Computer-based training is frequently utilized in healthcare education due to its ease of distribution, low cost, and standardized content.9 The objective of this study was to evaluate the efficacy of a computer-based teamwork process training (cTPT) intervention on medical emergency teamwork behaviors (processes) and patient care performance during simulated patient resuscitations. We hypothesized that teams receiving cTPT would exhibit a greater number of appropriate teamwork and patient care behaviors as compared with teams receiving placebo training. Our study aims to advance patient safety by providing a sustainable approach to team training in acute care settings.
Methods: This study design used a randomized comparison design to evaluate the efficacy of a cTPT intervention on medical resuscitation teamwork and patient care during simulated patient resuscitations. The participants were fourth-year medical students and first, second, and third-year emergency medicine residents at Wayne State University. Each participant was assigned to a team of 4-6 members (Nteams=45). Demographic information, clinical experience, and simulation experience data were collected from all participants. Teams were randomly assigned to receive either the 25-minute cTPT intervention targeting appropriate resuscitation teamwork behaviors or placebo training. The cTPT was designed using evidence-based principles to teach knowledge and implementation of critical teamwork behaviors. The placebo training provided general information about different types of teams in a healthcare context but did not discuss specific teamwork behaviors. The simulation-based assessment included the following: Two resuscitation simulations were constructed using event-based scenario development techniques.10-12 Independent teamwork and patient care behavioral checklist measures were developed for each scenario using evidence-based guidelines.13,14 Scenario content, teamwork, and patient care measures were content-validated by teamwork (N=17) and clinical (N=10) subject matter experts external to the research team and primary institution.15 Following training, teams performed a single practice simulation that permitted exposure to the simulation environment. Each team then completed a second simulation that served as the training assessment. Instructor-led training and debriefing was held until the participants completed both scenarios. Scenario order was varied across teams to ensure the effects of the cTPT would generalize across different clinical content.16 Each simulation video recording was coded by 4 independent, trained raters (2 psychology research assistants coded teamwork behaviors and 2 emergency medicine physicians coded patient care behaviors) blinded to the condition assignments and study hypotheses. Inter-rater reliability was monitored throughout the data coding process. Because the format of teamwork and patient care behaviors differed slightly from one to the next (e.g., behavior counts versus ratings of quality), all items were standardized before averaging to create overall teamwork and patient care scores.
Results: Controlling for participants’ experiences, teams receiving the cTPT intervention engaged in a greater number of appropriate teamwork behaviors during the simulation (p<0.05), and performed better with regard to standards for patient care (p<0.05) than teams receiving the placebo training. See Figure:
Conclusion: Computer-based team training positively impacts teamwork and patient care during simulated patient resuscitations. This low-resource team training intervention may present a solution to the dissemination and sustainability issues present with larger, more costly team training programs.
1. Leonard M, Graham S, Bonacum D. The human factor: the critical importance of effective teamwork and communication in providing safe care. Quality & Safety in Health Care. Oct 2004;13:I85-I90.
2. Croskerry P, Wears RL, Binder LS. Setting the educational agenda and curriculum for error prevention in emergency medicine. Academic Emergency Medicine. Nov 2000;7(11):1194-1200.
3. Sundstrom E, Demeuse KP, Futrell D. Work Teams: Applications and effectiveness. American Psychologist. Feb 1990;45(2):120-133.
4. Edmondson AC. Speaking up in the operating room: How team leaders promote learning in interdisciplinary action teams. Journal of Management Studies. Sep 2003;40(6):1419-1452.
5. Schull MJ, Ferris LE, Tu JV, Hux JE, Redelmeier DA. Problems for clinical judgement: 3. Thinking clearly in an emergency. Canadian Medical Association Journal. Apr 2001;164(8):1170-1175.
6. Campbell SG, Croskerry P, Bond WF. Profiles in patient safety: A “Perfect Storm” in the emergency department. Academic Emergency Medicine. Aug 2007;14(8):743-749.
7. Alonso A, Baker DP, Holtzman AK, et al. Reducing medical error in the Military Health System: How can team training help? Human Resource Mangement Review. 2006;16:396-415.
8. Neily J, Mills PD, Young-Xu YN, et al. Association Between Implementation of a Medical Team Training Program and Surgical Mortality. Jama-Journal of the American Medical Association. Oct 2010;304(15):1693-1700.
9. Ruiz JG, Mintzer MJ, Leipzig RM. The impact of e-learning in medical education. Acad. Med. Mar 2006;81(3):207-212.
10. Fowlkes J, Dwyer DJ, Oser RL, Salas E. Event-based approach to training (EBAT). International Journal of Aviation Psychology. 1998;8(3):209-221.
11. American College of Surgeons Trauma Committee. Advanced Trauma Life Support for Doctors. Chicago, IL 2008.
12. Advanced Cardiovascular Life Support Provider Manual. Dallas, TX: American Heart Association;2006.
13. Rosen MA, Salas E, Wilson KA, et al. Measuring Team Performance in Simulation-Based Training: Adopting Best Practices for Healthcare. Simulation in Healthcare. 2008;3(1):33-41.
14. Salas E, Rosen MA, Held JD, Weissmuller JJ. Performance Measurement in Simulation-Based Training: A Review and Best Practices. Simulation & Gaming. 2009;40(3):328-376.
15. Barrett RS. Content validation form. Public Personnel Management. 1992;21:41-52.
16. Kim J, Neilipovitz D, Cardinal P, Chiu M, Clinch J. A pilot study using high-fidelity simulation to formally evaluate performance in the resuscitation of critically ill patients: The University of Ottawa Critical Care Medicine, High-Fidelity Simulation, and Crisis Resource Management I Study. Critical Care Medicine. Aug 2006;34(8):2167-2174.
Disclosures: Rosemarie Fernandez, MD, has received grant funding from the Agency for Healthcare Research and Quality (AHRQ). Steven WJ Kozlowski, PhD, has received grant funding from the Agency for Healthcare Research and Quality(AHRQ), but it did not fund this research, which is independent.
658 Delivery Room Communication Improves After Implementing Simulation-based Team Training
Rita Dadiz, DO4, Jan Schriefer, MSN, MBA, DrPH3, Joanne Weinschreider, RN, MS1, Christine Arnold, RN, MS2, Eva Pressman, MD2, and Ronnie Guillet, MD, PhD4
1NURSING, SAINT JOHN FISHER COLLEGE, ROCHESTER, NY, USA and 2OBSTETRICS AND GYNECOLOGY, UNIVERSITY OF ROCHESTER MEDICAL CENTER, ROCHESTER, NY, USA and 3PEDIATRICS, UNIVERSITY OF ROCHESTER MEDICAL CENTER, ROCHESTER, NY, USA and 4PEDIATRICS AND NEONATOLOGY, UNIVERSITY OF ROCHESTER MEDICAL CENTER, ROCHESTER, NY, USA
Introduction/Background: Providing safe medical care of pregnant women and their newborns during high-risk deliveries requires effective communication between obstetric (OB) and pediatric (Peds) teams. A standardized interdisciplinary curriculum using simulation-based training (SBT) was previously shown to improve communication during simulated deliveries.1 Our aim was to evaluate the effect of this curriculum on the communication skills of OB and Peds providers during actual high-risk deliveries.
Methods: During the 2008-2012 academic years, interprofessional teams of OB and Peds providers participated annually in SBT to improve the communication of vital information during high-risk deliveries between teams and between providers and families. Physicians, fellows, housestaff, mid-level providers and nurses involved in either maternal or newborn care during deliveries were eligible for participation (n=228). Standardized actors portrayed the pregnant woman and family member during simulations. Scenarios varied annually and included shoulder dystocia, post-partum hemorrhage, and maternal and newborn codes. Teams were debriefed after simulations. Communication between teams and between providers and families during actual high-risk vaginal and cesarean deliveries were videotaped during the 2009-2011 academic years. Videos were evaluated using a validated 20-item checklist previously demonstrated to have excellent inter-rater reliability. Checklist scores on delivery room communication in 2009-2010 (n = 22) and 2010-2011 (n = 22) were compared using the Wilcoxon test.
Results: During the 2008-2012 academic years, 127-158 (56-69%) eligible providers participated in annual SBT sessions. Checklist scores on communication during actual high-risk deliveries improved significantly from a median of 9 (IQR = 3) to 11 (IQR = 3), p = 0.006. This improvement was mainly due to the following improved communications: (1) when maternal and fetal information were relayed from the OB to Peds team and (2) when the Peds team spoke to the family about the newborn’s status.
Conclusion: Provider participation in interdisciplinary SBT can improve communication between the OB team, Peds team and families during high-risk deliveries. Future training can target areas for improvement, as revealed by the checklist. Standardizing communication with the checklist may promote patient safety and family-centered care.
1. Dadiz R, Weinschreider J, Schriefer J, Arnold C, Pressman E, Guillet R. Handoff communication improves with simulation-based interdisciplinary team training, SSIH #249, Simul Healthc 2011;6(6):386.
Disclosures: Joanne Weinschreider, RN, MS, is a consultant for TheraSim, Inc.
660 Effects of Computer-based Resuscitation Simulation on Nursing Students’ Mastery Learning, Self-efficacy, Post-code Stress, and Satisfaction
Young-Sook Roh, RN, PhD1
1RED CROSS COLLEGE OF NURSING, CHUNG-ANG UNIVERSITY, SEOUL, KOR
Introduction/Background: Computer-based simulation has intuitive appeal to both educators and learners with the flexibility of time, place, immediate feedback as well as self-paced and consistent curriculum.1 The purpose of the study was to verify the effects of precourse e-learning by comparing nursing students’ mastery learning, self-efficacy, post-code stress and satisfaction between precourse e-learning plus mannequin-based resuscitation training group (intervention group) and mannequin-based resuscitation training only group (control group).
Methods: This study was a nonequivalent control group posttest only design. A total of 213 second year nursing students were conveniently assigned to one of two groups before a week of simulation testing sessions: intervention (n=109) or control (n= 104) group. Both groups underwent 2-hour mannequin-based resuscitation training. The intervention group underwent the computer-based simulation as precourse selfdirected e-learning that provides structured training and feedback on resuscitation training until all students achieved the minimum passing score (70%). During each testing session, groups of four nursing students participated in mannequin-based simulation at a time using the human patient simulator. Three faculty members rated students’ mastery of learning in resuscitation skills using the Advanced Cardiac Life Support Skills Checklists2 designed to be used with a full-body human patient simulator after modification based on the 2010 American Heart Association guidelines. All participants completed the self-administered scales including the Self-efficacy in Resuscitation Scale,3 Post-Code Stress Scale,4 and Satisfaction with Simulation Experience Scale5 after completing the simulation testing session. Independent samples t and Chi-square tests were calculated to summarize the quantitative data.
Results: Homogeneity test for baseline characteristics between the two groups revealed that there were no significant differences in age (t=.838, p=.403), gender (χ2=2.748, p=.097), knowledge score (t=.155, p=.877), general self-efficacy (t=.072, p=.943), percentage of life support certificates holder (χ2=.068, p=.794), and frequency of encountered code events during clinical placement (t=-.984, p=.326). Percentage of correct resuscitation skills performer were significantly higher in the intervention group compared to the control group in terms of “Call for help” (χ2=10.977, p=.001), “Initiating chest compression” (χ2=8.544, p=.033), “Compression to breathing” ratio of 30:2’ (χ2=6.565, p=.010), “Assisting with equipment” (χ2=4.252, p=.039), and “Reporting” (χ2=6.664, p=.010). There were no significant differences in resuscitation-specific self-efficacy (t=.677, p=.499), post-code stress (t=-.342, p=.733) and satisfaction (t=1.277, p=.203) between the two groups.
Conclusion: Computer-based simulation as precourse e-learning with deliberate practice might be beneficial to build toward mastery in key resuscitation skills among nursing students. We suggest that adopting self-directed learning with computer-based simulation on the basis of learning objectives and level of engagement is needed as an adjunctive modality for resuscitation skills training for nursing students.
1. Finn J. E-learning in resuscitation training - students say they like it, but is there evidence that it works?. Resuscitation. 2010 Jul;81(7):790-1.
2. Wayne DB, Butter J, Siddall VJ, Fudala MJ, Linquist LA, Feinglass J, Wade LD, McGaghie WC. Simulation-Based Training of Internal Medicine Residents in Advanced Cardiac Life Support Protocols: A Randomized Trial. Teach Learn Med. 2005;17(3):202-208.
3. Roh YS, Issenberg SB, Chung HS,Kim SS, Lim TH. Building a simulation-based resuscitation training curriculum for nurses: results from a needs assessment survey in nurses. Poster sessionpresented at:The 12th Annual Meeting on Simulation in Healthcare; 2012 Jan28-Feb 1; San Diego, CA.
4. Cole FL, Slocumb EM, Muldoon Mastey J. A measure of critical care nurses’ post-code stress. J Adv Nurs. 2001 May;34(3):281-8.
5. Levett-Jones T, McCoy M, Lapkin S, Noble D, Hoffman K, Dempsey J, Arthur C, Roche J. The development and psychometric testing of the Satisfaction with Simulation Experience Scale.Nurse Educ Today. 2011 Oct;31(7):705-10.
666 Comparison of Anesthetists Team Communication Patterns in the Operating Theatre and in Simulation
Jennifer Weller, MD, MClinEd, MBBS, FANZCA, FRCA5, Craig Webster, PhD4, Jane Torrie, MBChB FANZCA2, Robert Henderson, MSc2, Kaylene Henderson1, Elaine Davies3, Boaz Shulruf, PhD6, and Alan Merry, MB, ChB, FANZCA3
1CLINICAL PLANNING, SIDRA MEDICAL AND RESEARCH CENTER, DOHA, QAT and 2ANAESTHESIOLOGY, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 3ANESTHESIOLOGY, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 4CENTRE FOR MEDICAL AND HEALTH SCIENCES EDUCATION, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 5UNIVERSITY OF AUCKLAND, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 6MEDICINE EDUCATION, UNIVERSITY OF NEW SOUTH WALES, SYDNEY, NSW, AUS
Introduction/Background: Simulation-based training is often used to develop the teamwork skills required for safe patient care. However, questions remain about the effect of simulation on the authenticity of participants’ behavioral responses. Manser et al1 compared anesthetists’ activity patterns related to task completion in simulation settings and the OT, and found them comparable. We wanted to know if anesthetists’ team communications were also comparable. Therefore we compared anesthetists’ communication patterns with their team in three conditions: 1) routine cases in an operating room (OR); 2) routine simulated cases (Sim R); and 3) non-routine, high complexity simulated cases (Sim C). Our hypotheses were that there would be no difference in anesthetists’ patterns of communication between routine OR cases and routine simulated cases and that there would be no difference between routine cases (OR or simulated) and high complexity simulated cases.
Methods: Video-recordings of anesthetic teams in the three conditions were coded against a pre-defined framework of team communications by two trained coders using video analysis software (Table 1). Proportions of the different communication types were compared across the three conditions using mixed-model ANOVA.
Results: We video recorded 20 anesthetists in the two simulations, and 17 across all three settings, identifying 114 comparable video segments for coding (2469.2 minutes). This generated 2501 coded communications. Patterns of team communication occurred in similar proportions in the OR and in the routine simulated case in every item except one: there was a significantly higher rate of communication with the circulating nurse in the simulation setting, probably reflecting the additional functions of this person in tasks specific to the simulations. Patterns of team communication also occurred in similar proportions in the complex simulated cases as in the routine cases in either environment, except for the following: 1) a similar increase in the rate of communication with the circulating nurse and; 2) significantly more communications related to sharing situational information in the complex simulation than in either the routine simulation or the OR. Small effect sizes (all <0.5) for comparisons that were not statistically significant would indicate our study was adequately powered to detect potentially relevant differences.
Conclusion: Anesthetists’ observed communication patterns were comparable in the OR and in a realistic simulation setting when managing routine cases, suggesting that the simulated environment elicited authentic communication responses in participants. Communication patterns related to situational assessment were significantly different when managing high complexity simulated cases, confirming previous work.2 We hypothesize that this would also happen in the OR. A better understanding of the effect simulation has on a range of behaviors across different contexts and with varying levels and will assist efforts to improve the validity and efficacy of the simulated environment for training and research.
1. Manser T, Dieckmann P, Wehner T, Rall M. Comparison of anaesthetists’ activity patterns in the operating room and during simulation. Ergonomics 2007; 50: 246-60.
2. Manser T, Harrison TK, Gaba DM, Howard SK. Coordination patterns related to high clinical performance in a simulated anesthetic crisis. Anesthesia & Analgesia 2009; 108: 1606-15.
Disclosures: Craig Webster, PhD, owns a small number of shares in a company called Safersleep Inc (Nashville, Tennessee) that attempts to improve the safety of medical systems. Alan Merry, MB, ChB, FANZCA, is a stockholder, ownder and partner of Safer Sleep LLC.
667 An Educational Video Intervention Improves Anaesthetists Call-out Scores in a Simulated Anaesthetic Emergency
Jennifer Weller, MD, MClinEd, MBBS, FANZCA, FRCA5, Jane Torrie, MBChB FANZCA3, Matt Boyd, PhD4, Sandy Garden, PhD, MBChB,FANZCA1, Robert Frengley, MBChB FANZCA, FCICM6, and Kaylene Henderson2
1ANAESTHESIA AND PAIN MANAGMENT, CAPITAL AND COAST HEALTH, WELLINGTON, NZL and 2CLINICAL PLANNING, SIDRA MEDICAL AND RESEARCH CENTER, DOHA, QAT and 3ANAESTHESIOLOGY, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 4CENTRE FOR MEDICAL AND HEALTH SCIENCES EDUCATION, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 5UNIVERSITY OF AUCKLAND, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 6INTENSIVE CARE, WAIKATO HOSPITAL, HAMILTON, NZL
Introduction/Background: The importance of effective teamwork is well established; sharing situational information is a key component, ensuring the team is working together towards a common goal.1 2 Various information-sharing strategies have been described, including recap of the situation, or call out. Our observations of teams in simulated events suggest this strategy is underutilized. We hypothesized that an educational intervention using videos to model a specified behaviour of call out would improve the quality of call out, and the sharing of information about the case between team members.
Methods: Forty-three anesthetists were randomly allocated to intervention or control groups then randomly assigned to one of the two simulated scenarios (A or B) for baseline assessment. Scenarios were set in a simulated post-anesthesia care unit (PACU) and crafted to ensure that there was diagnostic uncertainty likely to trigger a call-out. Anesthetists worked with a PACU nurse and an anesthetic technician, also naïve study participants. Prior to each simulation the nurse and technician received three unique and salient pieces of information about the case (information probes), to enable quantification of information sharing during the simulation. After scenarios, all participants independently answered multi-choice and written questions about the case, including information in the probes, and reported the treatment actions their team had instituted. The intervention or control followed the baseline simulation. The intervention video described a structured call out and then role modelled a simulated call out (20 mins duration). The control video was of similar duration and described and role modelled difficult airway management guidelines. At two weeks, all anesthetists received an email reminder on the main points in their video. Follow-up simulations took place at a minimum of four weeks later, anesthetists then doing the other study scenario (A or B), and working with different, naïve PACU nurses and anesthetic technicians. The number of information probes shared by each team was calculated and the treatment actions reported by participants were compared with a list of 28 expected actions as judged by a clinical expert to produce a “treatment actions taken” score. These were combined to give a composite score: team knowledge of the case. Trained raters scored the quality of the call outs in the video-recorded scenarios against a pre-defined scoring rubric on an 8-point scale. Data were analysed using ANOVA.
Results: The quality of the call out increased significantly more in the intervention group than the control group (p < 0.001). The score for the call out increased 2.3 points in the intervention group, and decreased 0.3 points in the control group (8-point scale). Paired data were obtained for 40 anesthetists in baseline and follow-up simulated scenarios. Team knowledge of the case, (a composite of information probes learned and the treatment actions taken) increased 2.4 points in the intervention group, and 0.2 points in the control group, which reached statistical significance (p < 0.05).
Conclusion: The quality of anesthetists’ call outs in a simulated case can be enhanced by using a video intervention and reminder email. Our results also lend some weight to the proposition that enhanced call out skills may have a positive influence on aspects of teamwork including information sharing, and shared understanding of the case.
1. Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiology Scandinavica 2009;53:143-51.
2. Salas E, Cooke NJ, Rosen MA. On teams, teamwork, and team performance: discoveries and developments. Human Factors 2008;50(3):540-7.
668 Reliability and Validity of an Instrument Used for Self-assessment of Teamwork in Critical Care
Jennifer Weller, MD, MClinEd, MBBS, FANZCA, FRCA3, Boaz Shulruf, PhD4, Jane Torrie, MBChB FANZCA2, Robert Frengley, MBChB FANZCA, FCICM5, and Kaylene Henderson1
1CLINICAL PLANNING, SIDRA MEDICAL AND RESEARCH CENTER, DOHA, QAT and 2ANAESTHESIOLOGY, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 3UNIVERSITY OF AUCKLAND, UNIVERSITY OF AUCKLAND, AUCKLAND, NZL and 4MEDICINE EDUCATION, UNIVERSITY OF NEW SOUTH WALES, SYDNEY, NSW, AUS and 5INTENSIVE CARE, WAIKATO HOSPITAL, HAMILTON, NZL
Introduction/Background: Teamwork is recognized as a key factor in patient safety, and there appear to be patterns of communication, coordination and leadership that support effective teamwork.1,2 Many team-training initiatives are off-site and tend to be resource intensive.3 Opportunities many also exist to reinforce or promote teamwork through reflection and learning from experience in the workplace.4 An accurate and validated teamwork measurement instrument could focus reflection on team function and identify areas for improvement, measure progress and improve team function. We have previously reported on the psychometric properties of a structured teamwork rating instrument used by three trained external assessors to rate teamwork of 40 Critical Care teams participating in four simulated emergencies set in a simulated Critical Care Unit.5 The instrument consists of 23 items, each describing a marker of team performance. This study measured the psychometric properties of this instrument when used by participants rating their own team performance in simulated emergency scenarios.
Methods: Forty Critical Care teams were recruited from nine different Critical Care Units in the region. Each team came from the same department, and consisted of one Critical Care Specialist or trainee, and three Critical Care nurses. The teams took part in four highly realistic simulated emergency scenarios set in the Critical Care Unit. Immediately after each scenario and prior to any discussion, each participant independently rated their team’s performance using the teamwork rating instrument. Teams had no prior experience with the teamwork rating instrument. This generated 640 team self-assessed rating forms from participants for analysis. The psychometric properties of the instrument when used for self-assessment were evaluated using Exploratory Factor Analysis (EFA) (maximum likelihood, oblimin rotation), and Cronbach’s α for internal consistency. Self-assessment data was then compared with the external assessor rating data from the same scenarios using Pearson’s correlation coefficient.
Results: EFA confirmed items loaded onto three Factors. These factors were leadership and team co-ordination; verbalising situational information; and mutual performance monitoring. Reliability coefficients for the three factors were .931, .957 and .888 respectively. Significantly increased scores over time supported construct validity. The EFA results were virtually identical to the EFA from trained external assessor data, with 17of the individual items loading onto the same factors. The correlation between factors from self-assessed scores and externally assessed scores was high (LTC .872, VSI .818, and MPM .642; all p<.0001). The stringency of the self-assessed scores was consistently lower than the external assessors.
Conclusion: Our results support the reliability and validity of the teamwork rating instrument when used for self-assessment. Participants were able to use the instrument without prior training. They were able to discriminate between different levels of team performance at a level of detail sufficient to identify specific elements of teamwork that could be improved. Validity is supported by the EFA. Participants consistently score their teams more lenient than external assessors, suggesting the need for calibration exercises. Millward and Jeffries6 suggest that interventions, such as full team discussions about teamwork following challenging clinical cases, could play a key role in health care team development. The present instrument would seem ideal for facilitating interdisciplinary team debriefing after critical events with the potential to improve team performance and patient safety.
1. Manser T. Teamwork and patient safety in dynamic domains of healthcare: a review of the literature. Acta Anaesthesiology Scandinavica 2009;53:143-51.
2. Salas E, DiazGranados D, Klein C, Burke CS, Stagl KC, Goodwin GF, et al. Does Team Training Improve Team Performance? A Meta-Analysis Human Factors: The Journal of the Human Factors and Ergonomics Society 2008;50:903-33.
3. McCulloch P, Rathbone J, Catchpole K. Interventions to improve teamwork and communications among healthcare staff. British Journal of Surgery 2011;98(4):469-79.
4. Boud D. Enhancing Learning Through Self Assessment, (Kogan Page, London, Philadelphia, 1995).
5. Weller J, Frengley R, Torrie J, Shulruf B, Jolly B, Hopley L, et al. Evaluation of an instrument to measure teamwork in multidisciplinary critical care teams BMJ Qual Saf Published Online First:. 2011;20:216-22.
6. Milward L, Jeffries N. The team survey: a tool for health care team development. Journal Of Advanced Nursing 2001;35(2):276-87.
674 Pediatric Resident Resuscitation Skills Retention after Mastery Learning: A Multi-center Randomized Trial
LoRanee Braun, MD6, Taylor Sawyer, DO, MEd3, Joseph Lopreiato, MD, MPH5, Kathleen Smith, MD, MPH6, Deborah Chan, PharmD4, Melinda Behrens, MD6, Angela Hsu, MD4, Jeffrey Hutchinson, MD2, Downing Lu, MD, MPH2, Raman Singh1, and Joel Reyes1
1USA and 2PEDIATRICS, NATIONAL CAPITAL CONSORTIUM, BETHESDA, MD, USA and 3MEDICAL SIMULATION CENTER, TRIPLER ARMY MEDICAL CENTER, HONOLULU, HI, USA and 4PEDIATRICS, TRIPLER ARMY MEDICAL CENTER, HONOLULU, HI, USA and 5FOREST GLEN SIMULATION CENTER, UNIFORMED SERVICES UNIVERSITY, BETHESDA, MD, USA and 6PEDIATRICS, UNITED STATES ARMY, MADIGAN ARMY MEDICAL CENTER, TACOMA, WA, USA
Introduction/Background: Life support courses such as Pediatric Advanced Life Support (PALS) and the Neonatal Resuscitation Program (NRP) do not provide sufficient training for pediatric residents to become competent in pediatric and neonatal resuscitation.1,2 Simulation training has been advocated as a method to assist pediatric residents in achieving competency in resuscitation.3-5 Using simulation-based mastery-learning, residents can be trained to achieve mastery (as defined by a minimal acceptable score) in cardiopulmonary resuscitation.6 However, after mastery is achieved some degree of skill degradation will occur over time. Thus, repeated, or maintenance, training is needed. Currently, it is unknown how long residents retain mastery level resuscitation skills after initial mastery training. We conducted the following study to determine the training intervals required to sustain mastery in pediatric and neonatal resuscitation. Our hypothesis was that resuscitation skills would decline as a function of time status post-mastery learning.
Methods: The study followed a prospective randomized design. To begin, each resident participated in a mastery learning session. During this session baseline resuscitation skills were evaluated on four standardized simulation scenarios, which included: respiratory arrest, septic shock, asystole, and meconium delivery. Performance on each scenario was scored on a specially developed scoring tool which showed good evidence for validity and reliability (IRR > 0.9). After determination of baseline skills each resident was provided immediate feedback on their performance. Residents then repeated each scenario until a score of ≥ 80% was achieved. For the study achievement of a score ≥80% was considered mastery. After completing the mastery learning session residents were randomized, by program and year group, into one of three retesting intervals: 2, 4, or 6 months. According to their group assignment (2, 4 or 6 months), each resident returned to the simulation lab and repeated each of the same four resuscitation simulations. Resident performance was again scored using the same scoring tool. No repeat testing was conducted at the retest session. Scores at retesting were used to determine the amount of resuscitation skill retention over time.
Results: Forty-two (42) residents from four military medical centers participated in the study (12 in 2 month group, 14 in 4 month group, 16 in 6 month group). At baseline, third year residents performed better than the 2nd and 1st year residents (3rd year 79.7% vs. 2nd year 72.4% vs. 1st year 69.3%; p=0.023). Baseline performance scores did not differ among the 2 month, 4 month, and 6 month groups. Overall performance on each scenario improved from baseline to retest (respiratory arrest: 81.1% ±15.4 vs. 87.3%± 10.5 [p =0.016], shock 69.2% ± 16 vs. 78.4%± 14.3 [p =0.003], asystole: 68.2% ± 14.7 vs. 74.5% ± 16.3 [p = 0.036], meconium: 81% ± 22.8 vs. 87.7% ± 14.4 [p= 0.041]). Overall scores at the time of retesting were lowest in the 6 month group (83.2% ± 7.7 at 2mo, 82.1% ± 5.8 at 4 mo, 80.5% ± 7.8 at 6 mo; p= 0.434). The percent of residents who maintained mastery (e.g. score ≥ 80%) was lowest in the 6 month group (92% at 2 mo, 71% at 4 mo, 56% at 6 mo; p=0.253).
Conclusion: After participation in a single simulation-based mastery learning encounter, residents in our study displayed improvements in resuscitation skills up to 6 months after the training session. Overall performance declined as a function of time after training, such that at 2 months after mastery learning > 90% of residents retained skills at a mastery level, but by 6 months only 56% retained skills at a mastery level. This data suggests that an optimal retraining interval for pediatric and neonatal resuscitation skills after mastery learning is less then 6 months.
1. Grant E, Marczinski C, Menon K. Using Pediatric Advanced Life Support in pediatric residency training: Does the curriculum need resuscitation? Pediatr Crit Care Med 2007; 8:433–9.
2. Nadel F, Lavelle J, Fein J, Giardino A, Decker J, Durbin D. Assessing pediatric senior residents’ training in resuscitation: Fund of knowledge, technical skills, and perception of confidence. Pediatric Emergency Care 2000; 16:73-6.
3. Nguyena H, Daniel-Underwooda L, Ginkeld C, Wonga M, Leed D, San Lucasc A, Palaganasc J, Bantab D, Denmarka T, Clema K. An educational course including medical simulation for early goal-directed therapy and the severe sepsis resuscitation bundle: An evaluation for medical student training.Resuscitation 2009;80:674-9.
4. Nadel F, Lavelle J, Fein J, Giardino A, Decker J, Durbin D. Teaching Resuscitation to Pediatric Residents. Arch Pediatr Adolesc Med 2000;154:1049.
5. Donoghue A, Durbin D, Nadel F, Stryjewski G, Kost S, Nadkarni V. Effect of High-Fidelity Simulation on Pediatric Advanced Life Support Training in Pediatric House Staff A Randomized Trial. Pediatr Emer Care 2009;25:139-44.
6. WayneDB, Butter J. Siddall VJ, Fudala MJ, Wade LD, Feinglass J, McGaghie WC. Mastery Learning of Advanced Cardiac Life Support Skills by Internal Medicine Residents Using Simulation Technology and Deliberate Practice. J Gen Intern Med. 2006; 21:251–256.
675 Interprofessional High-fidelity Simulation Training Improves Knowledge and Teamwork in Nursing and Medical Students During Internal Medicine Third Year Clerkship
Jason Morris, MD3, Dawn Taylor Peterson, PhD, EdS, MEd7, Penni Watts, MSN5, Chad Epps, MD2, Kevin Leon, MD6, Kathy Harrington, PhD, MPH4, and Caleb Pierce, BS1
1UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 2CLINICAL AND DIAGNOSTIC SCIENCE, UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 3INTERNAL MEDICINE, UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 4MEDICINE, UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 5NURSING, UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 6PULMONARY MEDICINE, UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 7PEDIATRIC SIMULATION CENTER, UNIVERSITY OF ALABAMA BIRMINGHAM, CHILDREN’S HOSPITAL OF ALABAMA, BIRMINGHAM, AL, USA
Introduction/Background: Previous studies have confirmed the effectiveness of high-fidelity simulation for improving nursing and medical students’ knowledge and communication. However, despite an increasingly interprofessional approach to medicine, most studies have demonstrated these effects in isolation. We enhanced an existing internal medicine curriculum with a series of interprofessional simulations. We hypothesized that simulation training will improve each profession’s medical knowledge, communication, and understanding of other profession’s roles in patient care.
Methods: Over 10 months, third year medical students (SOM) and senior nursing students (SON) participated in four “every-other-week,” one-hour simulation sessions during the medical student’s internal medicine clerkship. Simulations included: acute myocardial infarction, pancreatitis with hyperkalemia, an upper gastrointestinal bleed, and a severe chronic obstructive pulmonary disease exacerbation with allow natural death order. Knowledge was assessed with multiple-choice, pre- and post-tests. Attitudes toward professional roles and communication practices were assessed before and after the course using an attitude assessment with a 5-point Likert scale. After each simulation session, students were debriefed by a team of experts in medicine, nursing, simulation, and adult learning. After debriefing, all students completed an anonymous evaluation survey about the simulation and debriefing consisting of 9 questions on a 5-point Likert scale. The survey also included open-ended questions related to the simulation’s effectiveness and areas for improvement. Qualitative data was coded and analyzed for emergent themes. Quantitative statistical analysis was performed using SPSS (Chicago, IL) with paired t-tests and chi square tests.
Results: Seventy-eight third-year SOM and 30 SON students participated in all four high-fidelity internal medicine simulations. There was a significant improvement in correct responses for both SOM and SON students from pre- and post-tests on the knowledge questions (SOM 53±19% pre vs. 63± 20% post test p<0.0001; SON 32±15% pre vs. 43±16% post test p=0.05). Results of the post-simulation evaluations indicated that students felt the activity was applicable to their field (4.93/5 SOM, 4.99/5 SON) and a beneficial educational experience (4.90/5 SOM, 4.95/5 SON). Among the open-ended responses, the most frequent positive response for both groups was increased medical knowledge (37% of all SOM comments, 30% SON). An improved sense of teamwork and team communication were the second and third most common positive comments for both groups (17% SOM, 19% SON; 16% SOM, 15% SON, respectively). The most commonly recognized area for improvement among SOM was medical knowledge (24%). The most commonly cited area for improvement among SON was communication within the team (19%). The attitude assessment indicated several significant changes pre- and post-test. After the course, more students agreed “closed-loop communication is important even if it slows patient care” (p=0.004, all students) while fewer students agreed that the most senior physician has “the most important role” (p<0.0001, all students) or that “patient care is more important than patient safety” (p<0.0001, all students). The smaller number of nursing students made attitude changes statistically insignificant in isolation.
Conclusion: The results support our hypothesis that interprofessional simulation training for nursing and medical students can increase knowledge and improve communication and team role understanding as indicated by attitudes. Medical teams are increasingly crucial in quality care of complex patients and early team training for professional students may decrease adverse outcomes or inefficiency attributable to poor communication. By instituting a high fidelity simulation curriculum similar to the one used in this study, students can be exposed to other disciplines in a safe and realistic environment. Further research is needed to demonstrate the efficacy of interprofessional training in other fields and to evaluate effects of early interprofessional training on healthcare outcomes.
Disclosures: Penni Watts, MSN, offered a one-time consultation to Drake State Technical College on integrating simulation into the nursing curriculum.
Oral Presentation 667 Improving Communication and Team Performance Through Risk-informed In Situ Simulation
Jane Holl, MD, MPH3, Walter Eppich, MD, MEd1, Anna Nannicelli, BS3, Ranna Rozenfeld, MD2, Nicholas Seivert3, Olivia Ross, MPH, MBA3, and Donna Woods, EdM, PhD3
1EMERGENCY MEDICINE, NORTHWESTERN UNIVERSITY, ANN AND ROBERT H LURIE CHILDREN’S HOSPITAL, CHICAGO, IL, USA and 2PEDIATRIC CRITICAL CARE MEDICINE, NORTHWESTERN UNIVERSITY, ANN AND ROBERT H LURIE CHILDREN’S HOSPITAL, CHICAGO, IL, USA and 3INSTITUTE FOR HEALTHCARE STUDIES, NORTHWESTERN UNIVERSITY, FEINBERG SCHOOL OF MEDICINE, CHICAGO, IL, USA
Introduction/Background: Team training for healthcare professionals to improve patient safety is being promoted and simulation has been endorsed as a method to evaluate and advance team skills1. Conducted in the clinical operational environment, in-situ simulation offers a realistic representation of systems and processes encountered by healthcare workers. The “Risk Informed In-Situ Simulation for Pediatric Emergency Transfers” project (AHRQ R18-HS017912) evaluates the capacity to effectively expand team training beyond awareness to performance by: (1) integrating previously-identified clinician communication, care coordination, teamwork, and micro-system risks from prospective risk assessments into simulation scenarios; (2) conducting the training in the operational environment; and (3) incorporating AHRQ’s TeamSTEPPS™ roles and skills into the intervention. It is hypothesized that in-situ simulation is an effective means to assess healthcare professionals’ team performances and improve communication and care coordination in pediatric emergency transfer settings.
Methods: A multidisciplinary team of clinicians from five Chicago-area institutions developed four clinical, event-based2 scenarios representing two types of interfacility pediatric emergency transfers: (1) Emergency Department (ED) and (2) ED-pediatric intensive care unit (PICU). Participating institutions included two public hospitals, two suburban hospitals, and one free-standing children’s hospital, lending to the generalizability of this team training method. Common “triggers” that elicit teamwork behaviors were embedded throughout the scenarios to enhance realism. All simulations were video-recorded and used as a tool to facilitate the debriefing intervention focused on team skills. Videos were then randomly assigned amongst nine trained raters who used the Team Emergency Assessment Measure (TEAM)3 to evaluate the intervention’s effect on multiple teamwork-related competencies. Each simulation was rated by two coders independently using a web-based utility; for each video segment, raters also selected which participant they perceived to be the team leader. Additional outcomes include operational and process risks identified through systematic review of simulation video.
Results: In total, 352 clinicians and staff participated in 42 video-recorded in-situ simulations. Inter-rater reliability measures indicated high agreement (91% adjacent) for TEAM ratings. Using a regression, the relationship between the role of team leader, repeat participation in the simulations, and total TEAM score was investigated. Results reveal that overall TEAM score increased by 12 points (p=.00) when an attending physician who was a repeating participant was identified as the team leader. No similar increase in TEAM score was observed when resident or fellow physician participants were identified as the team leader, regardless of repeat participation. Review of the simulation videos also revealed system and process issues including: (1) teamwork and communication issues pertaining to: loss of essential information during patient handoffs, lack of designation of team roles, and inconsistent use of closed loop communication; and (2) systems failures including: delay in patient registration, problematic placement of computers for documentation, and inconsistent use of text and overhead paging to contact clinicians in emergent situations.
Conclusion: Initial results of the analysis indicate a strong positive relationship between the role of attending physician team leaders and improved teamwork scores. This finding could potentially have important implications for training of teamwork competencies. Future efforts should perhaps focus on providing specific teamwork training for attending physicians and examine the following: (1) the level of diffusion of skills and behaviors (e.g. does training attending physicians diffuse to rest of team?) and (2) decay of these competencies within clinical teams (e.g. how often do clinicians need to be retrained?). Future research should also assess the relationship between improved teamwork scores and improvements in actual patient safety and healthcare quality.
1. Gaba DM. The future vision of simulation in health care. Quality and Safety in Health Care 2004;13(suppl 1):i2-i10.
2. Rosen MA, Salas E, Wu TS, Silvestri S, Lazzara EH, Lyons R, et al. Promoting Teamwork: An Event-based Approach to Simulation-based Teamwork Training for Emergency Medicine Residents. Academic Emergency Medicine 2008;15(11):1190-98.
3. Cooper S, Cant R, Porter J, Sellick K, Somers G, Kinsman L, et al. Rating medical emergency teamwork performance: development of the Team Emergency Assessment Measure (TEAM). Resuscitation 2010;81(4):446-52.
678 Delays and Errors Among Pediatric Residents During Simulated Resuscitation Scenarios Using PALS Algorithms
Melanie Labrosse, MD, PhD2, Arielle Levy, MD, MEd, FRCPC2, Aaron Donoghue, MD, MSCE1, and Jocelyn Gravel, MD, MSc, FRCPC2
1EMERGENCY AND CRITICAL CARE MEDICINE, CHILDREN’S HOSPITAL OF PHILADELPHIA, PHILADELPHIA, PA, USA and 2PEDIATRIC EMERGENCY MEDICINE, UNIVERSITY OF MONTREAL, MONTREAL, QC, CAN
Introduction/Background: Recent data suggest alarming delays and deviations in major components of pediatric resuscitation during simulated scenarios by pediatric housestaff.1-3 However, there are still significant gaps in our understanding of which critical actions are most improperly performed by pediatric residents in different scenarios involving Basic Life Support (BLS) algorithms and/or Pediatric Advanced Life Support (PALS) algorithms. The objective of this study was to identify clinically significant delays and errors among pediatric residents on a series of specific actions during multiple simulated pediatric resuscitation scenarios.
Methods: This was a retrospective observational study conducted in an academic tertiary care hospital. Pediatric residents (PGY1 and PGY3) were videotaped performing a series of five pediatric resuscitation scenarios using a high-fidelity simulator (Simbaby, Laerdal): pulseless non-shockable arrest, pulseless shockable arrest, dysrhythmia, respiratory arrest and shock. Primary outcome was the presence of significant errors prospectively defined using a validated and reliable scoring instrument4 designed to assess sequence, timing and quality of specific actions during resuscitations based on the 2005 American Heart Association PALS guidelines. Residents’ clinical performances were measured by a single video reviewer. The primary analysis was the proportion of errors for each critical task for each scenario. We estimated that the evaluation of each resident would provide a confidence interval less than 0.20 for the proportion of errors.
Results: Twenty four of 25 residents completed the study. Across all scenarios, pulse check was delayed by more than 30 seconds in 56% (95%CI: 46%-66%). For non-shockable arrest scenarios, cardiopulmonary resuscitation (CPR) was started more than 30 seconds after recognizing arrest in 21% (95%CI: 7-42%) and inappropriate defibrillation was performed in 29% (95%CI: 13-51%). For shockable arrest scenarios, participants failed to identify the rhythm in 58% (95%CI: 37-78%); CPR was not performed in 25% (95%CI: 10-47%), while defibrillation was delayed by more than 90 seconds in 33% (95%CI: 16-51%) and not performed in one case. For dysrhythmia scenarios, cardioversion was inadequatelly performed in 17%; [lack of sychronization in 13% (95%CI: 3-32%), wrong dose in one case (95%CI: 0-21%)], and not performed at all in two cases. For respiratory arrest scenarios, residents did not assess airway and breathing in 38% and 13 % respectively, they failed to start bag-mask ventilation in 17% (95% CI: 5-37%) and had delayed ventillation by more than 30 seconds after recognizing apnea in 46% (95% CI: 28-65%). For shock scenarios, participants failed to ask for a dextrose check in 71% (95%CI: 51-86%), and it was delayed by more than 60 seconds for all others.
Conclusion: Our study revealed clinically significant delays and errors in all scenarios. We identified gaps in performances for specific actions considered as essential according to 2005 American Heart Association guidelines, which are mandatory to ensure good patient outcome. We found clinically significant delays and omissions in BLS algorithms, such as airway and breathing assessment, as well as in clinical assessments, especially delays in pulse check in all scenarios. We also noted delays in the initiation of critical resuscitation maneuvers, such as CPR omission and delaying defibrillation in shockable arrest scenarios, as well as inadequate cardioversion in dysrrhytmia scenarios. A major finding was also the omission of rapid dextrose check for all residents during shock scenarios. Future training in pediatric resuscitation should target these errors.
1. Hunt, E.A., Walker, A.R. Shaffner, D.A., Miller, M.R., & Pronovost, P.J. (2008) Simulation of In-Hospital Pediatric Medical Emergencies and Cardiopulmonary Arrests: Highlighting the Importance of the First 5 Minutes. Pediatrics, 121(1), e34-e43. doi: 10.1542/peds.2007-0029.
2. Hunt, E.A., Vera, K., Diener-West, M., et al. (2009) Delays and errors in cardiopulmonary resuscitation and defibrillation by pediatric residents during simulated cardiopulmonary arrests. Resuscitation, 80, 819-825. doi:10.1016/j.resuscitation.2009.03.020.
3. Shilkofski, N.A., Nelson, K.L., & Hunt, E. (2008) Recognition and Treatment of Unstable Supraventricular Tachycardia by Pediatric Residents in a Simulation Scenario. Simultation in Healthcare, 3, 4-9. doi: 10.1097/SIH.0b013e31815bfa4e.
4. Donoghue, A., Nishisaki, A., Sutton, R., Hales, R., & Boulet, J. (2010). Reliability and validity of a scoring instrument for clinical performance during Pediatric Advanced Life Support simulation scenarios. Resuscitation, 81(3), 331-336. doi: S0300-9572(09)00592-9 [pii].
682 Teaching Patient Handling with Simulators: The Role of Self-efficacy
Ketki Raina, PhD1, Rogers Joan, PhD1, John O’Donnell, CRNA, MSN, DrPH2, and Margo Holm, PhD1
1OCCUPATIONAL THERAPY, UNIVERSITY OF PITTSBURGH, PITTSBURGH, PA, USA and 2PETER M WINTER INSTITUTE FOR SIMULATION, EDUCATION AND RESEARCH (WISER), UNIVERSITY OF PITTSBURGH, PITTSBURGH, PA, USA
Introduction/Background: Although manual patient handling is a complex activity that can result in injury to the patient and professional, there is currently no widespread standard method to teach these handling (transfer) skills.1 Simulation is one method to educate students to safely transfer patients; however, research examining the use of simulation to teach and evaluate patient transfer skills is limited.2 Simulation provides a realistic learning environment with immediate feedback and is often used in medical education but not widely adopted in allied health, often due to cost concerns. Recent changes in education have emphasized the importance of the learning environment, participation, observation, and the use of instant feedback to improve performance and student self-efficacy.3 Self-efficacy has also been linked to learning success and prediction of achievement with some tasks; but not others, and its role in simulation warrants further examination.4 Cognitive learning theory was the basis for this study, in which different dosages of student participation and observation were manipulated to determine their effect upon student self-efficacy. Cognitive learning theory purports that successful task completion contributes to beliefs of self-efficacy and that these beliefs are thought to influence task performance.5,6 Self-efficacy has been widely used as a measure of performance, and elevated self-efficacy beliefs have been linked to effective clinical practice.7 Self-efficacy knowledge, skills and safety ratings were used to assess the effects of participation and observation as subjects performed a patient handling task that also required a response to an unplanned medical event. The aim of this study was to determine if learning out of bed transfers under various combinations of hands-on participation and observation influenced self-efficacy as strongly as repeated hands-on practice.
Methods: In this randomized clinical trial, 108 subjects first rated self-efficacy after completing transfers in a traditional classroom laboratory with peers. Second, subjects were randomly assigned to one of three dosage groups: 1). participation/2 observations; 2). participations/1 observation; 3). participations/0 observations), as they completed transfers in a simulated acute care environment using a high-technology simulator. The transfers were embedded in mock clinical scenarios that replicated critical events often encountered when treating a medically complex patient. Third, subjects returned to the simulation center to complete patient transfers that were embedded in different but equivalent critical event scenarios. All subjects engaged in the same number of participation experiences during this phase. Self-efficacy ratings were collected after each transfer experience. The objective of this study was to determine the effect of dosages of participation/observation experiences upon self-efficacy. It was hypothesized that: 1). there would be no difference in self-efficacy ratings among the different dosage groups; 2). self-efficacy ratings would decrease after the initial simulation experience, and 3). self-efficacy ratings would increase after the second simulation experience. Data were collected from assessments designed for this study and analyzed using SPSS 19.0. For Hypothesis 1, one-way ANOVAs were conducted with participation/observation dosage as the between subjects variable. Hypotheses 2 and 3 compared Knowledge, Skills and Safety self-efficacy ratings by time and teaching method. Three separate time-by-dosage analyses using ANOVA were conducted.
Results: Results indicated that subjects participating in every transfer experience did not differ significantly in self-efficacy ratings from subjects who engaged in a combination of participation and observation transfer experiences. Also, subjects rated self-efficacy in all three domains significantly lower after their first exposure to the simulated acute care environment, regardless of their participation/observation dosage, and significantly higher after their second exposure to the simulated environment.
Conclusion: Learning transfers in a combined dosage of hands-on participation and observation influenced student self-efficacy as strongly as repeated hands-on practice.
1. Nehring W, Lashley F, Ellis F. Critical incident nursing management using human patient simulators. Nurs Educ Perspect 2002; 23: 129-132.
2. O’Donnell JM, Goode JS, Henker RA, Kelsey S, Bircher N, Peele, P, Bradle J, Close J, Engberg R, Sutton-Tyrrell, K. Effect of a simulation education intervention on knowledge, attitude, and patient transfer skills from the simulation laboratory to the clinical setting. Simul Healthc, 2011; 84-93.
3. Lasater K. High-fidelity simulation and the development of clinical judgment: Students’ experiences. J Nurs Educ 2007; 269-276.PubMed PMID: 17580739.
4. Barnsley L, Lyon PM, Ralston SJ, Hibbert EJ, Cunningham I, Gordon FC, Field MJ. (2004). Clinical skills in junior medical officers: A comparison of self-reported confidence and observed competence. Med Educ. 2004; 38:358-67 PubMed PMID: 15025637.
5. Bandura A. Social foundations of thought and action: A Social Cognitive Theory. Englewood Cliffs, New Jersey: Prentice Hall; 1986.
6. Bandura, A. Perceived self-efficacy in cognitive development and functioning. Educ Psychol 1993; 28:117-148.
7. Lorenz R, Gregory RP, Davis DL. (2000). Utility of a brief self-efficacy scale in clinical training program evaluation. Eval Health Prof 2000; 23:182-193. PubMed PMID: 10947524.
690 The Effect of Simulation Training in Ultrasound Guided Central Venous Catheter Insertion Among Pediatric Residents: A Longitudinal Study
Scott Thomas, MD4, Wesley Burch2, Sarah Kuehnle1, Robert Flood, MD3, Anthony Scalzo, MD, BS, BA2, and James Gerard, MD3
1SAINT LOUIS UNIVERSITY SCHOOL OF MEDICINE, SAINT LOUIS, MO, USA and 2CURRICULAR AFFAIRS, SAINT LOUIS UNIVERSITY SCHOOL OF MEDICINE, SAINT LOUIS, MO, USA and 3PEDIATRIC EMERGENCY MEDICINE, SAINT LOUIS UNIVERSITY SCHOOL OF MEDICINE, SAINT LOUIS, MO, USA and 4PEDIATRIC EMERGENCY MEDICINE, WASHINGTON UNIVERSITY, SAINT LOUIS, MO, USA
Introduction/Background: Pediatric residents have limited opportunities to learn central venous catheter (CVC) insertion in the clinical setting. The use of simulation to augment this training among pediatric residents has not been previously reported. The primary objective of this study was to assess the psychomotor skills of pediatric residents performing US-guided femoral CVC insertion before, after, and at 3 months following simulation training. A secondary goal was to assess the effect of simulation training on residents’ self-confidence to perform this procedure. We hypothesized that psychomotor skills and self-confidence would improve immediately following simulation training and remain above baseline at 3 months post-training.
Methods: Institutional Review Board approval was obtained. Between September 2011 and January 2012, residents attended small group training sessions on US-guided CVC insertion conducted at our University’s Simulation Center. Data on previous US and CVC insertion experience were collected. After viewing a training video,1 residents were video recorded at baseline independently performing the procedure on a pediatric CVC task trainer. They then participated in a two-hour simulation training session with deliberate practice. Immediately following the training, residents were again video recorded performing the procedure. To assess skill retention, video recordings were repeated at three months post-training. Three expert, blinded raters independently scored the performances utilizing a 24-item checklist 2 and global assessment scale (GAS). The GAS is a 100 mm visual analog scale (VAS) anchored from 0 (poor) to 100 (excellent). Residents completed self-confidence questionnaires at each of the three time points using a 100 mm VAS anchored from 0 (not at all confident) to 100 (very confident).
Results: Twenty-six residents completed the study. Of these 23 (88.5%) had no prior experience performing CVC insertions, and only 2 (7.7%) had previously used US for CVC placement. The skills performances of residents at each of the time points are shown in the table. Checklist scores, GAS scores, and success rates improved immediately post-training (p < 0.05 for all comparisons). At 3 months post-training, however, there were no significant differences in performances when compared to baseline for any of the measures. Self-confidence rose from a pre-training median level of 8.0 to a post-training level of 52.0 (p < 0.01) and remained above baseline with a level of 61.0 at 3 months (p < 0.01).
Conclusion: Despite limited prior experience with US-guided CVC insertion, our study demonstrated that a two-hour, simulation-based training session significantly improved pediatric residents’ abilities to successfully perform US-guided femoral CVC insertion and improved their self-confidence. We found, however, that these skills were not retained at 3 months post-training. As skills deteriorated, residents appeared to be over-confident in their ability to perform the procedure at 3 months. Future studies are needed to determine the optimal strategy and frequency of training for this procedure among pediatric residents.
1. Tsui JY, Collins AB, White DW, Lai J, Tabas JA. Videos in clinical medicine. Placement of a femoral venous catheter. N Engl J Med 2008; 358:e30.
2. Barsuk JH, McGaghie WC, Cohen ER, Balachandran JS, Wayne DB. Use of simulation-based mastery learning to improve the quality of central venous catheter placement in a medical intensive care unit. J Hosp Med. 2009; 4(7):397-403.
Disclosures: Robert Flood, MD is partner-editor of the PEM Question Review Book (PEMQ LLC, GA), but this entity has no influence on producing healthcare or simulation-related goods or services.
691 Simulation-based Training as a Tool to Empower Primary Care Physicians in the Clinical Encounter in a Computerized Setting
Shmuel Reis, Prof1, Doron Sagi, MA2, Yossi Kuchnir, MD5, Varda Shalev, MD3, Joseph Azuri, MD4, Aviv Shachak, PhD6, and Amitai Ziv, MD, MHA2
1FACULTY DEVELOPMENT, BAR ILAN UNIVERSITY FACULTY OF MEDICINE IN THE GALILEE, SAFED, ISR and 2ISRAEL CENTER FOR MEDICAL SIMULATION (MSR), TEL HASHOMER, ISR and 3MACCABI HEALTH SERVICES, TEL AVIV, ISR and 4MACCABI HEALTHCARE SERVICES, TEL AVIV, ISR and 5FAMILY MEDICINE, MACCABI HEALTHCARE SERVICES, TEL AVIV, ISR and 6INSTITUTE OF HELATH POLICY, MANAGEMENT AND EVALUATION, UNIVERSITY OF TORONTO, TORONTO, ON, CAN
Introduction/Background: Electronic Medical Records (EMRs) are increasingly used in healthcare organizations.1 The use of EMRs has had a positive impact on medical care through better adherence to guidelines, clinical monitoring and medical error prevention. Nevertheless, EMR use causes new kinds of errors and changes the patient-doctor relationship. The computer is an active player in the medical encounter,2 so the traditional dyadic relationship has transformed into a triad: patient-doctor-computer.3 The presence of the computer affects the doctor-patient communication, usually hindering it. A fine balance exists between empowering the encounter by the use of EMR versus hindering it. There is a paucity of theoretical work in the Patient-Doctor-Computer communication (PDCC) field, as well as a scarcity of educational programs for implementing computer systems in the clinical setting. Simulation is considered ideal for instruction in the medical field,4 and thus we strove to test this methodology in this new field of EMR. The research compared simulation-based educational intervention with traditional lecture based instruction, in order to supply policy makers with an educational option to empower the medical encounter of primary care physicians. The main hypothesis was that simulation-based training would contribute to PDCC more than the traditional lecture based instruction.
Methods: Thirty-six primary care residents volunteered for this experiment. They were randomly assigned to test and control groups. Both groups attended a pre-test and post-test phase, composed of six simulated medical encounters each. The independent variable was the type of instruction; the test group participated in a day long simulation based workshop, and the control group received traditional lecture based instruction. The parameters measured were patient-doctor-communication and computer skills. These parameters were assessed by senior physicians according to an assessment form developed in a pilot.
Results: Both groups performed better at the post training encounters; no difference was found between control and experiment group.
Simulation based training was evaluated by the participants as more relevant and with higher contribution than lectures; 11 out of 12 scenarios were rated as “highly relevant” to physicians’ daily work. Seventy-eight percent of the test group rated the instruction contribution as high and no participant gave a low contribution score. In contrast, only 6%-18% rated the lectures contribution as high and 60% rated it as low or very low. All participants asked for more instruction in this field, while emphasizing the need for simulation based instruction rather than lecture based one.
Conclusion: Results did not support the main hypothesis. We assume that the mere exposure to very intensive simulated medical encounters in the research context supplied deliberate practice that caused learning and improved performance in the post intervention phase.5 Simulation based education has clear advantages in terms of motivation and compliance to future educational interventions. Future research in field conditions will be held in order to control for the Simulation Center effect.
1. O’MalleyAS. Tapping the Unmet Potential of Health Information Technology. N Engl J Med. 2011 Mar 24;364(12):1090-1.
2. Shachak A, Reis S. The impact of electronic medical records on patient-doctor communication during consultation: a narrative literature review. J Eval Clin Pract. 2009 Aug;15(4):641-9. Epub 2009 Jun 10.
3. Pearce C. Doctor, patient and computer–a framework for the new consultation. Int J Med Inform. 2009 Jan;78(1):32-8.
4. Beyth Y, Hardof D, Rom E, Ziv A. A Simulated-Patient-Based Program for Training Gynecologists in Communication with Adolescent Girls presenting with Gynecological Problems. J Pediatr Adolesc Gynecol. 2009 Apr;22(2):79-84.
5. Ericsson KA. Deliberate practice and the acquisition of expert performance in medicine and related domains. Acad Med. 2004 Oct;79(10 Suppl):S70-81.
Disclosures: Amitai Ziv, MD, MHA, receives grant support from the BIRD Foundation, is a consultant forSimbionix USA and the Michener Institute for Applied Health Professions, Canada, and is a stockholder of Semantic Medical Simulation (SMS) Ltd.
710 The Effectiveness of Transpacific Remote Co-facilitation in Simulation-based Pediatric Resuscitation Training
Hiroshi Kurosawa, MD1, Takanari Ikeyama, MD6, James Scott4, Scott Hayes3, Kunio Ohta, MD, PhD5, Vinay Nadkarni, MD, MS, FCCM2, and Akira Nishisaki, MD, MSCE2
1CENTER FOR SIMULATION, ADVANCED EDUCATION AND INNOVATION, CHILDREN’S HOSPITAL OF PHILADELPHIA, PHILADELPHIA, PA, USA and 2CRITICAL CARE MEDICINE, CHILDREN’S HOSPITAL OF PHILADELPHIA, PHILADELPHIA, PA, USA and 3INFORMATION SERVICES, CHILDREN’S HOSPITAL OF PHILADELPHIA, PHILADELPHIA, PA, USA and 4IS CLIENT SERVICES, CHILDREN’S HOSPITAL OF PHILADELPHIA, PHILADELPHIA, PA, USA and 5PEDIATRICS, SCHOOL OF MEDICINE, INSTITUTE OF MEDICAL, PHARMACEUTICAL AND HEALTH SCIENCES, KANAZAWA UNIVERSITY, KANAZAWA, JPN and 6PEDIATRIC EMERGENCY AND CRITICAL CARE MEDICINE, TOKYO METROPOLITAN CHILDREN’S MEDICAL CENTER, TOKYO, JPN
Introduction/Background: Recent evidence suggests a remotely facilitated simulation is feasible. However, the impact of training effectiveness is uncertain. We defined remote co-facilitation as: 1) remotely introducing simulation environment to students, 2) remotely running scenarios, and 3) remotely co-debriefing scenarios. We hypothesized that a remote co-facilitation in simulation-based pediatric resuscitation training for medical students is more effective compared to training facilitated by only an on-site facilitator.
Methods: We implemented a remote co-facilitation program for pediatric acute care training for 5th year medical students between Kanazawa, Japan and Philadelphia, USA. Students’ groups were allocated to either remote co-facilitation group (Remote) or on-site facilitation group (On-Site). All groups had identical one-hour simulation training sessions, including introduction to simulation and simulation environment (15 minutes), first scenario (15 minutes), debriefing (20 minutes), and second scenario (10 minutes) followed by a 5 minute debrief. Each session was video recorded with three cameras at identical acquisition angles. A trained rater (with documented high inter-rater reliability and calibration for these scenarios) scored each team performance using the Behavioral Assessment Tool (BAT). Students rated the value of the simulation training experience (Likert Scale: 1 = not satisfied, 7 = very satisfied). The primary outcome was the improvement of the BAT between the first scenario and the second scenario in Remote group compared to the improvement in On-Site group. A linear mixed effect modeling was used with the group analyzed for random effect. P value<0.05 as significant.
Results: Eight groups of On-site sessions (47 students) and 15 groups of Remote sessions (89 students) were analyzed, with no significant difference between group demographics. Mixed effect model shows the large variance of performance at first scenario. Group performance significantly improved in the Remote co-facilitated group (pre [8.5±4.2] vs. post [13.2±6.2], p=0.003), but not significantly for On-site facilitation (pre[6.9±4.1] vs. post[12.4±6.4], p=0.056). The degree of improvement in the BAT score was not significantly different between Remote co-facilitation and On-site facilitation (p=0.94).
Conclusion: Remote co-facilitation was feasible and significantly improved students’ behavioral team performances between the first and second scenarios. However, remote co-facilitation was not superior to on-site facilitation in improving overall behavioral team performance during a 1 hour simulation training session.
Disclosures: Vinay Nadkarni, MD, MS, FCCM, receives grant support from Laerdal Foundation for Acute Care Medicine, and was the recipient of a Laerdal Medical CorporationTravel reimbursement to a visiting professorship/keynote address. Akira Nishisaki, MD, MSCE receives grant support from Laerdal Foundation Center for Excellence.
713 Telemedicine for Training Remote Station Workers in Endotracheal Intubation Using Video Laryngoscope: A Pilot Study on Feasibility
David Musson, MD, PhD1, Thomas Doyle, PhD, PEng3, Bingxian Wang, MD, PhD2, and Greg Peachey, MD, FRCP, MHSc1
1ANESTHESIA, MCMASTER UNIVERSITY, HAMILTON, ON, CAN and 2CENTER FOR SIMULATION-BASED LEARNING, MCMASTER UNIVERSITY, HAMILTON, ON, CAN and 3ELECTRICAL AND COMPUTER ENGINEERING, MCMASTER UNIVERSITY, HAMILTON, ON, CAN
Introduction/Background: Securing a patient’s airway is vital in managing emergent conditions. Achieving this in remote settings, such as spaceflight or remote research stations, poses unique problems, including the absence of trained providers. Delays in medevac may require frontline personnel to provide advanced care. Just-in-time training has been proposed as a potential solution to such challenges.1 Video laryngoscopy has been shown to facilitate rapid acquisition of intubation skills.2,3 Simulation-based, video-guided intubation was chosen as a procedural skill to explore the potential of tele-instruction in a remote field setting. This pilot study was conducted to explore the feasibility of tele-training remote station personnel with no medical background in endotracheal intubation using a video laryngoscope, and to identify the key enabling and limiting factors in such efforts.
Methods: The institutional review board at McMaster University approved this study. Eureka, a small weather station on Ellesmere Island in the Canadian Arctic was chosen as the field site to conduct this study. Eureka is the second-northernmost permanent research community in the world. During the winter it has a population of approximately 10-15 people consisting of meteorologists, scientists, and support personnel. A SimMan®3G manikin (Laerdal Medical ®), brought to support a number of research activities, was used as the airway simulator. Conventional endotracheal tubes, a bag-valve mask, and video laryngoscope (Glidescope ®) were provided to participants. For the first part of study, each subject received one training session (15-20 minutes) on intubating the simulator using a video laryngoscope under the instruction of a anesthesiologist located at McMaster university (approximately 4200km to the south). Communication was facilitated by one-way video and two-way audio conferencing using a commonly available application (Skype ®) through a satellite (mediated Internet connection). The instructor was able to switch between room view and video feed from the laryngoscope. Each training session was followed by a debriefing session and was recorded on video. For the second part of the study, each subject independently performed an endotracheal intubation without supervision. Their performance was video recorded for subsequent analysis.
Results: Six subjects were trained and later (24hrs) evaluated. Time required for successful intubation (as defined by passing of the endotracheal tube through the vocal cords) was analyzed. All subjects were successful, though times to intubation ranged from 1 min. 7sec. to 10 min. (median: 3 min. 55 sec). Maintaining an Internet connection was a major challenge; weather and network traffic posed challenges. Video compression was variable and identified as problematic during several instructional episodes. Signal cut-out also occurred on several occasions, and highlighted the need for a consistent connection. Satellite transmission resulted in a 600ms signal delay, which was not identified as problematic. Subjects enjoyed the training; all felt that the training was essential should such a procedure need to be performed. Instructors identified a lack of pre-existing audiovisual aids as a major deficiency. The ability to switch between room and laryngoscope views was identified as being helpful.
Conclusion: This pilot study indicates that it is feasible to rapidly train remote workers in endotracheal intubation using a patient simulator and a video laryngoscope assisted by telemedicine. Securing a reliable Internet connection is vital. Solutions may include a dedicated communication channel, or assigning router priority on the communications network. Audiovisual aids were identified as a desirable adjunct for teaching; effort should be put towards identifying the procedures that may be required for a given environment and such resources prepared ahead of time. Laboratory studies to explore these concepts further, along with validating and informing field studies, are recommended.
1. NASA Human Research Program, http://www.nasa.gov/exploration/humanresearch/.
2. Berg BW et al. Stud Health Technol Inform. 2009; 142: 31-33.
3. Sakles JC, Mosier J, Hadeed G, Hudson M, Valenzuela T, Latifi R. Telemedicine and telepresence for prehospital and remote hospital tracheal intubation using a GlideScope™ videolaryngoscope: a model for tele-intubation. Telemed J E Health. 2011 Apr;17(3):185-8. Epub 2011 Mar 28. PMID: 21443441.
772 Assessment of Team ICARE Values during Simulated and Actual Deliveries
Paula Emrich, RNC, BSN1, Kristen Brown, NNP-BC, CNS1, and Rita Dadiz, DO2
1PEDIATRIC NURSING, UNIVERSITY OF ROCHESTER MEDICAL CENTER, ROCHESTER, NY, USA and 2PEDIATRICS AND NEONATOLOGY, UNIVERSITY OF ROCHESTER MEDICAL CENTER, ROCHESTER, NY, USA
Introduction/Background: Patient satisfaction and confidence in the healthcare provided to them in the hospital begins with their first interaction with medical and nursing providers. The University of Rochester Medical Center incorporates the ICARE values (Integrity, Compassion, Accountability, Respect and Excellence) as the standard for patient- and family-centered care. These values are tied to specific behaviors, including: introducing oneself to the patient and family, communicating with warmth, answering questions and addressing concerns, treating all colleagues and patients with dignity, and going above and beyond the expectations of patients and families. We performed a needs assessment to evaluate ICARE communication behaviors exhibited by providers from the neonatal intensive care unit (NICU) when speaking to families during simulated deliveries and actual neonatal care administered in the delivery room.
Methods: A retrospective review of communication by the NICU team with families during simulated deliveries (n= 11) and actual vaginal and cesarean deliveries that occurred in the operating room (n=11) was conducted. The NICU delivery room team, drawn from a total of 140 medical and nursing providers, was typically comprised of a resident, neonatal nurse practitioner, and delivery room nurse. For certain high-risk deliveries, the NICU fellow or attending may also be in attendance. Three trained individuals independently reviewed videos of the simulated and actual deliveries and assessed the NICU team for the following four ICARE communication behaviors: 1) introduce self and role, 2) communicate with warmth, 3) ask family if there are any questions, and 4) discuss initial plans for the baby. The proportion of teams who demonstrated the components of the ICARE behaviors was recorded. Team performance during simulated and actual deliveries was compared using the Student’s t-test.
Results: Only 1/22 (4%) of the neonatal teams performed all 4 components of ICARE. The majority of teams demonstrated either 2 or 3 behaviors (23 and 45% of teams, respectively). The two elements most often omitted were introduction of self and role (36% of teams) and asking the family whether they had any questions (90% of teams). Providers spent 1-8 minutes speaking with the mother and family, but there was no correlation between the time spent and the number of items performed. Team communication with families during simulated and actual deliveries was similar.
Conclusion: NICU teams providing newborn care during deliveries communicate with families using some of the ICARE behaviors, with room for improvement to meet institutional standards for patient and family-centered care. Observing teams during simulated and actual deliveries can identify opportunities to incorporate ICARE training into an interprofessional simulation-based curriculum focused on communication and family-centered care.
776 An e-Learning Curriculum Linked with an Immersive Simulation Game is Effective in Teaching Mass Casualty Triage in Disaster Response
Ivette Motola, MD, MPH, FACEP1, Stephanie Coe, BA, MD(cand)2, Robert Soto, EMT-P1, Angel Brotons, EMT-P1, and S Barry Issenberg, MD1
1GORDON CENTER FOR RESEARCH IN MEDICAL EDUCATION, UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE, MIAMI, FL, USA and 2MEDICAL SCHOOL, UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE, MIAMI, FL, USA
Introduction/Background: Prioritizing treatment of patients following natural disasters has an enormous impact on how many people survive. First responders must act quickly to coordinate care in order to maximize outcomes when the number of victims overwhelms available resources. Mass casualty triage training is usually conducted using face-to-face courses and mock disaster drills. This approach is resource intensive and not efficient for delivering standardized training to large numbers of providers who need to work together in a disaster response. To address the need for more efficient triage training, the University of Miami Gordon Center for Research in Medical Education (GCRME) developed a prehospital mass casualty triage training curriculum. This was developed in collaboration with the Florida Department of Health, and key stakeholders and content experts throughout the state. The training was targeted at all first responders in the state of Florida who may respond to a disaster involving large numbers of victims. The purpose of this study was to assess the efficacy of an e-learning course paired with an immersive simulation game in training pre-hospital providers throughout the state of Florida in triage during a mass casualty event. The null hypothesis was that there would be no difference in pre-course and post-course assessment scores.
Methods: The course consisted of a 2-hour e-learning didactic program that engages the learner using graphics, animations, and practice exercises. It contains 5 modules: fundamental concepts of triage, the START triage algorithm, the JumpSTART pediatric triage algorithm, triage operations, and special topics. Upon successful completion of the program, the participants had access to an immersive game where they triaged 35 casualties. The setting was a residential area that had been hit by a tornado. The casualties included pediatric and adult virtual patients, and participants had to determine the respiratory rate, pulse rate and mental status, initiate appropriate treatment, and triage the victim. At the completion of the game, learners reviewed an after-action report that provided feedback on their performance. We determined the effectiveness of the intervention by comparing the results of pre-course and post-course assessments that were comprised of 20 content-matched questions. The case-based assessments measured competency in key content areas covered during the course. The study was approved by the University of Miami Institutional Review Board. Paramedics throughout Florida were invited to participate in the training program. Those who completed all course components were included in the study. We performed statistical analysis of the assessments using a two-sample t-test for correlated samples in SPSS v.19.
Results: From October 7, 2008 through June 30, 2012, 2,963 first responders completed the curriculum.There was a statistically significant increase in the mean precourse-to-postcourse assessment scores: 14.14 (95% C.I. 13.99-14.28) or 70.7% for the precourse assessment versus 19.02 (95% C.I. 18.96-19.07) or 95.1% for the postcourse assessment (p<0.0001). The mean improvement was 4.88 (95% C.I. 4.74-5.02) or 24.4%.
Conclusion: Participation in a two-hour e-learning course with practice exercises and an immersive game significantly improved paramedics’ knowledge and problem-solving skills in mass casualty triage. At the completion of the educational intervention, providers were better able to rapidly triage patients using START and JumpSTART algorithms and were more familiar with triage goals and operations. This curriculum model allows for uniformity in training across a large geographic area and ability to reach rural providers.
779 A Simulation Course Focusing on Forensic Evidence Collection Improves Pediatric Knowledge and Standardizes Curriculum for Child Abuse
Chris Jolliffe, RN, SANE-A, SANE-P3, Amber Youngblood, BSN, RN4, Jerri Zinkan, MPH, RN5, Dawn Taylor Peterson, PhD, EdS, MEd5, Yuan Yih Ying (Eva), MS1, and David Bernard, MD2
1UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 2PEDIATRICS, UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 3UNIVERSITY OF ALABAMA BIRMINGHAM, CHILDREN’S HOSPITAL OF ALABAMA, BIRMINGHAM, AL, USA and 4NURSING EDUCATION, UNIVERSITY OF ALABAMA BIRMINGHAM, CHILDREN’S HOSPITAL OF ALABAMA, BIRMINGHAM, AL, USA and 5PEDIATRIC SIMULATION CENTER, UNIVERSITY OF ALABAMA BIRMINGHAM, CHILDREN’S HOSPITAL OF ALABAMA, BIRMINGHAM, AL, USA
Introduction/Background: According to the American Academy of Pediatrics and the Accreditation Council of Graduate Medical Education, pediatricians must be prepared to provide care for sexual abuse victims, including reporting abuse, assessing physical, emotional, and behavioral consequences, and providing care for the victims.1 Forensic evidence collection is often a necessary part of the workup for these patients.1 At our institution, pediatric-trained sexual assault nurse examiners (SANE) perform most forensic evidence collection, which unintentionally results in pediatric residents’ inadequate knowledge and experience with this skill. Studies have shown that chief residents and physicians have poor knowledge of prepubertal genital anatomy.2 Studies also indicate an overall lack of knowledge regarding child abuse among pediatric residents, general pediatricians, and pediatric emergency physicians.3 One national survey showed that specialties such as emergency medicine and family medicine, look to pediatricians for expertise in child abuse; therefore, pediatric programs must begin to develop this knowledge with their trainees.4 Our hypothesis was that pediatric residents and medical students who participated in a structured forensic evidence collection course, including video, simulation, and debriefing, would have improved knowledge of prepubertal evidence collection practices and pubertal genital anatomy.
Methods: The course curriculum included a 20-minute introductory forensic evidence collection video created by the SANE Program Director and the SANE Medical Director who are directly involved in the care of pediatric sexual abuse patients. A hybrid simulation setting was created using a Gaumard Pediatric HAL® Five year simulator (Gaumard Scientific, Miami, FL) and an anatomically correct female genitalia partial trainer. After watching the video, the participants, in groups of one or two, simulated forensic evidence collection utilizing state-approved forensic evidence collection kits while maintaining strict chain of evidence protocols. Prior to the course, evidence was staged on the simulator such as hair, a bite mark, and other debris that had to be collected during the simulation. The simulator also verbally expressed fear and embarrassment to the participants during the case, which required them to interact with the child simulator in a compassionate manner. All sessions were led by a pediatric SANE nurse. The participants completed a multiple-choice test and an eight item fill-in the blank anatomical diagram test before and after the sixty-minute course.
Results: Forty-two participants, including 22 (52%) pediatric residents, 18 (43%) medical students, and two (5%) of unknown status, completed the course. There was significant improvement in knowledge with the average pre-test score of 62 ± 20% and the average post-test score of 86 ± 9% (p<0.001). Anatomic labeling also improved; the average number of correctly identified parts on the pre-test was 5.2 + 1.3 and 5.8 + 1.1 post-test (p=0.008). Qualitative evaluations were overwhelmingly positive. Twenty-six (81%) participants stated that they gained much more knowledge on evidence collection process. Eight (25%) participants learned how to appropriately interact with abused patients. Seven (22%) participants liked the hands on nature of the experience and pointed to the benefits of walking through the exam. Three (9%) participants learned more about pubertal genital anatomy. Participants suggested that more instruction on anatomy would be helpful.
Conclusion: A simulation course focusing on child abuse and the forensic collection process improves knowledge of pubertal genital anatomy and management of patients for whom abuse is suspected. In addition, this course offers a standardized curriculum for pediatric residents and medical students allowing them to safely gain exposure to this important skill. Evaluation of the prepubertal child in the acute abuse setting is fraught with challenges to pediatricians. This study introduced the concept of using the simulated setting to improve medical students’ and residents’ knowledge of and experience with forensic evidence collection.
1. Kellogg N and The American Academy of Pediatrics Committee on Child Abuse and Neglect. “The Evaluation of Sexual Abuse in Children.” Pediatrics. 2005 Aug;116(2):506-12.
2. Dubow S et al. “Do pediatric chief residents recognize details of prepubertal female genital anatomy: a national survey.” Child Abuse Neglect. 2005 Feb;29(2):195-205.
3. Menoch M, Zimmerman S, Garcia-Filion P, Bulloch B. Child abuse education: an objective evaluation of resident and attending physician knowledge. Pediatr Emerg Care.2011 Oct;27(10):937-40. PubMed PMID: 21960095.
4. Starling SP, Heisler KW, Paulson JF, Youmans E. Child abuse training and knowledge: a national survey of emergency medicine, family medicine, and pediatric residents and program directors. Pediatrics. 2009 Apr;123(4):e595-602. PubMed PMID: 19273504.
790 Usability and Feasibility of Augmented Reality for Medical Student Procedural Skills Education
Ivette Motola, MD, MPH, FACEP2, Richard Rodriguez, MA2, Robert Levine, MD1, and S Barry Issenberg, MD2
1EMERGENCY MEDICINE, JACKSON MEMORIAL HOSPITAL AND ARCHIE MD, INC, MIAMI, FL, USA and 2GORDON CENTER FOR RESEARCH IN MEDICAL EDUCATION, UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE, MIAMI, FL, USA
Introduction/Background: Traditionally, clinical procedures in medical education have been taught on actual patients or in simulated environments using cadavers, animal models and, more recently, human patient simulators and task trainers. Patient safety and animal rights concerns, coupled with determining the optimal learning environment and training modalities have necessitated devising novel ways to teach and practice clinical and surgical procedures. Augmented reality (AR) technology is an innovation that merges digitally created images with real objects, such as simulation manikins to enhance visualization of anatomy during the learning process. It also provides realistic haptic feedback that purely virtual reality simulators lack. In this study, we evaluated the usability and feasibility of incorporating AR simulation into medical students’ procedural skills education in endotracheal intubation and central venous catheter insertion.
Methods: The population for this study consisted of fourth-year medical students from the University of Miami Miller School of Medicine currently enrolled in their emergency medicine clerkship. Students participated in a session where AR was used to teach endotracheal intubation and internal jugular venous access. The techniques were performed on simulation manikins outfitted to display virtual reality anatomy and device tracking, using technology developed by Archie MD, Inc. in Palm Beach, FL. For the endotracheal intubation procedure, the students were able to visualize the sagittal airway anatomy superimposed on the manikin, while the instructor located anatomical landmarks using a handheld laryngoscope and inserted an endotracheal tube. The view was changed to an antero-posterior perspective of the chest as the manikin was ventilated and tracheal and lung anatomy was visible on the screen. Three scenarios were presented: correct endotracheal intubation, right mainstem bronchus intubation, and esophageal intubation. The central venous access procedure included anatomic landmark identification on screen as the instructor explained the localization. The needle was marked and could be visualized as it was inserted into the vessel. Correct internal jugular insertion and incorrect carotid artery insertion were presented. After both procedures were completed, surveys were administered that gathered participants’ attitudes about the technology using Likert scales and open-ended questions. Statistical analysis of the responses was performed using SPSS v. 19. For open-ended questions, the responses were coded and categorized by emerging themes. The study was approved by the University of Miami Institutional Review Board.
Results: A total of 19 fourth-year medical students enrolled in the study and completed baseline computer use and post-intervention questionnaires. Questions included, “The anatomy was realistic and helped me identify landmarks;” “The technology helped improve my learning of this procedure;” and “I see this technology as a useful tool in my medical procedure skills training.” The survey revealed students had a positive reaction to the AR technology and its use in education. Ratings for the central venous cannulation procedure were slightly higher than for the endotracheal intubation, but almost all questions had a mode of 4 or 5 (agree or strongly agree). An ordinal logistic regression analysis did not reveal any significant association between computer use and reactions to the technology. Students thought AR was a useful tool in procedural skills training that aided them in learning the procedure.
Conclusion: Fourth-year medical students found augmented reality useful in learning two procedural skills: endotracheal intubation and central venous cannulation. They agreed incorporating the technology would be helpful in their training. They found most useful the ability to see anatomical structures and landmarks. The only usability difficulty identified was the lag time between voice commands and software response. The cost of acquiring enough models for use by many students was the most commonly raised concern.
Disclosures: Robert Levine, MD, is the President and Chief Medical Officer of ArchieMD, Inc. but was funded independently for this project by a grant from the National Institutes of Health. Dr. Levine was responsible for the development of the augmented reality application.
803 Simulation and Debriefing Promote Adult CPR Skills Retention
Judy Murphy, PHD, MSN1, Lan Jiang, BS, MS3, and Peter Friedmann, MD, MPH2
1PROVIDENCE VETERANS AFFAIRS MEDICAL CENTER, PROVIDENCE, RI, USA and 2MEDICINE, PROVIDENCE VETERANS AFFAIRS MEDICAL CENTER, PROVIDENCE, RI, USA and 3RESEARCH, PROVIDENCE VETERANS AFFAIRS MEDICAL CENTER, PROVIDENCE, RI, USA
Introduction/Background: Research has shown adult cardiopulmonary resuscitation (CPR) skills degrade by approximately 65% in ninety days.1 Although simulation is increasingly being used to teach skills, the effect on skills retention and the best method of debriefing are unknown. The main objective of this research was to study the effect of different debriefing methods on CPR skill retention. Traditionally, debriefing consists of a reflective discussion on the simulation, which may or may not include a video. Given the expense of video equipment, evidence as to the effect of the video as a useful debriefing option is needed. The hypothesis was that confidence, satisfaction and post intervention CPR skills performance and retention will be higher in the group that experienced both oral and video-assisted debriefing rather than the groups that experienced either method alone.
Methods: A mixed methods convenience sample employing a quasi-experimental design was used. Approximately 18% (n=72) of those invited participated. Each nurse dyad responded to a simulated cardiac arrest. The simulation was videotaped. On completion of the simulation they were randomly assigned to one of three debrief methods: oral-based video debrief (Group A), oral debrief alone (Group B), or video debrief only (Group C). Comments made by each participant during the debriefing were transcribed and analyzed for recurring themes. After the first simulation debrief, participants completed three self-assessment instruments: 1) Satisfaction in Learning from Simulation, 2) Self-Confidence in Learning from Simulation (National League for Nursing tools), and 3) a modified American Heart Association Basic Cardiac Life Support (BCLS) Skills Checklist. Two CPR experts, the researcher and a critical care clinical nurse specialist, rated the participants on their CPR skills by observation. Ratings were verified by watching the triple view video. Inter-rater agreement was determined to be satisfactory using the kappa statistic. Ninety days later participants returned for a second simulation requiring CPR and skills were viewed and rated by the expert observers.
Results: A t test comparison of CPR skill scores were compared for each dyad based on pre-post skills. There was no statistically significant difference between the three groups on skills performance, satisfaction, or self-confidence scores. However, there was a significant improvement in CPR skill scores from the first to the second simulation. Qualitative analysis revealed that participants in Group C (video debrief alone) had less to say than those with a more formalized oral debriefing (Group A and Group B). The themes that arose during the debriefing from all three groups included: skills performance, performance emotion, realism, teamwork, and communication. Although the group themes were similar, a higher percentage of the subjects in Group A and B reported a need to review their BCLS skills. Six percent of the total population stated that they need to practice CPR skills more than once every two years.
Conclusion: The quantitative and qualitative findings support the positive effect of simulation combined with debriefing on CPR skills retention. Unlike prior research where CPR skills degrade in ninety days, the CPR skill scores improved for 70.8 % of the participants. The qualitative data revealed that participants were interested in improving their performance. More rigorous research is needed to determine if debriefing using video is more effective than oral debriefing alone. The finding that the majority of participants in this study improved their CPR performance supports the value of simulation and debriefing regardless of the specific debriefing method.
1. Smith, K K, Gilcreast, D & Pierce, K Evaluation of staff’s retention of ACLS and BLS skills, Resuscitation 2008; 78: 59-65 PubMed PMID:18406037.
Disclosures: Peter Friedmanm, MD, MPH receives grant support from Alkermes.
804 A Randomized Clinical Trial of Interventions to Encourage Trainees to Speak-up When Witnessing Surgical Errors
Marco Barzallo, MD2, Jyothshna Bayya, MD2, Brian Gillett, MD1, Howard Minkoff, MD2, Jeremy Weedon, PhD4, Nadia Pletukhina, MD2, Pedram Bral, MD3, Helen Onoriode, MD2, and Nelli Fisher, MD2
1EMERGENCY MEDICINE, MAIMONIDES MEDICAL CENTER, BROOKLYN, NY, USA and 2OBSTETRICS and GYNECOLOGY, MAIMONIDES MEDICAL CENTER, BROOKLYN, NY, USA and 3OBSTETRICS and GYNECOLOGY, WOMEN’S HEALTH, MAIMONIDES MEDICAL CENTER, BROOKLYN, NY, USA and 4SCIENTIFIC COMPUTER CENTER, STATE UNIVERSITY OF NEW YORK, DOWNSTATE MEDICAL CENTER, BROOKLYN, NY, USA
Introduction/Background: Preventable adverse events are a leading cause of death in U.S. hospitals. Hesitancy to speak up has been reported to contribute to 23% of communication errors between junior and senior healthcare staff that lead to patient injury.1 The risk of error may be reduced by encouraging all members of the team to address any possible error without fear of reprimand.2 Our objective was to determine whether medical trainees who are encouraged to speak are more likely to question a senior surgeon when they witness a surgical mistake during simulated laparoscopic surgery. We hypothesized that the senior surgeon would play a key role in establishing a climate of safety in an operating room.
Methods: In this prospective randomized controlled study, we trained medical students (N=41) in the performance of basic tasks, including “burning” and “cutting”, on a laparoscopic simulator (LapSim). Specifically, students were trained that tissue must always be burned prior to cutting. A knowledge test was administered to confirm basic tasks sequence (burn, then cut) was learned. Students were then randomized to one of two groups: 1) “encouraged to speak up” (ES) and 2) “discouraged to speak up” (DS). The students subsequently assisted in a simulated laparoscopic salpingectomy within a week of their training. Just prior to starting the case, the senior surgeon used either an encouragement or a discouragement script. For example, a surgeon greeting students in the ES group said: “Speak up if something doesn’t look right to you”, while students in the DS group were told by the surgeon: “Just do what I say”. The surgery in both groups was conducted identically with the senior surgeon being silent except for the instructions to “burn” followed by “cut”. A predetermined surgical mistake was inserted into the case by the senior surgeon instructing students to cut without first asking them to burn. Student’s willingness to speak up was then assessed. Students were considered to have spoken up if they questioned the instruction and did not cut. To assess for potential personality bias between groups, students completed two validated personality tests prior to simulations: “Decision Making Scale” and “Independent vs. Interdependent Self-Construal Scale” to assess assertiveness and willingness to speak up. Students were blinded to the objective of the study and were unaware of their group assignment. The results of the personality tests as well as the intervention arm to which the student was assigned were correlated with the rate of speaking up. Data was processed with Mann-Whitney and Fisher exact tests. The study was approved by the Maimonides Medical Center IRB. Debriefing was performed to all subjects.
Results: Out of 41 students who participated in the study, 21 were in the ES group and 20 in the DS group. The students in the ES group were significantly more likely to speak up (N=16, 76.2% vs. N=5, 25%, p=0.002). There was no statistical difference between the personality traits in the two groups. There was no difference in medical student training level, (p=1.0) nor gender (p=0.53), between the participants in the two arms of the study. At debriefing, medical students commented favorably about the importance of the surgeon’s attitude in influencing their willingness to speak up during simulation.
Conclusion: During simulated surgery, an encouraging environment increases the willingness of junior trainees to speak up when a mistake is witnessed, whereas a discouraging environment decreases their willingness to speak up. The senior surgeon plays an important role in improving communication between junior and senior surgeons in the operating room and fosters a safer environment.
1. Descriptions of verbal communication errors between staff. An analysis of 84 root cause analysis-reports from Danish hospitals. Rabøl LI, Andersen ML, Ostergaard D, Bjorn B, Lilja B, Mogensen T. BMJ Qual Saf. 2011 Mar;20(3):268-74.
2. Patient Safety in the Surgical Environment. ACOG committee opinion. Number 464, September 2010.
806 Simulation-based Evaluation of Interprofessional Education Competencies
Michelle Russell, BSN student2, and Rita D’Aoust, PhD1
1COLLEGE OF NURSING, UNIVERSITY OF SOUTH FLORIDA, TAMPA, FL, USA and 2NURSING, UNIVERSITY OF SOUTH FLORIDA, TAMPA, FL, USA
Introduction/Background: According to the American Association of Colleges of Nursing (AACN) (2011), interprofessional education (IPE) is needed to improve patient and population outcomes, as well as prepare individuals to function in teams in the health care setting. The AACN (2011) has established four domains of competencies including ethics/ values, communication, teamwork/ collaboration, and roles/ responsibilities. Frequently IPE is needed involving clinical exposure to teach the importance of those competencies and provide an environment for healthcare individuals with different backgrounds to solve a patient scenario.
Methods: The purpose of this study was to examine 154 nurse practitioner students’ and third year medical students’ experiences in a complex high fidelity patient simulation, and analyze their written feedback. The simulation involved a patient who had an accidental overdose of a beta-blocking agent; however, the patient’s low heart rate and blood pressure did not respond to usual medications. Each team had a mixture of nurse practitioner and third year medical students to allow them the opportunity to problem solve together and appropriately diagnose and treat the patient. One minute concept papers were used to collect data on 156 de-identified student reflections on team communication, collaboration, and on their own performance. By using ATLAS-TI software, a qualitative study has been done creating core themes.
Results: By using ATLAS-TI software, a qualitative study has been done creating core themes such as: challenge, communication, confidence, experience, frustration, knowledge, roles, understanding, skills, helpful, learn, environment, and diagnosis. Each of these themes has been linked to de-identified individual quotes from the student reflections for further analysis and comparison. The themes identified from the qualitative analysis were compared to the Interprofessional Professional Education Consortium (IPEC) competencies to 1) identify if the themes were consistent with IPEC domains and competencies and, 2) understand student outcomes from one model of IPE (intradisciplinary education with interprofessional clinical practice) matched to IPEC competencies. This study revealed that students found it most challenging to identify team roles and responsibilities and how to communicate their ideas with other teammates when faced with a patient situation. Some nurse practitioner students felt inferior to the medical students, while some of the medical students felt they did not know enough about pharmacology compared to the nurse practitioner students. Students as a whole felt the simulation was a great opportunity to help them realize their incompetencies and collaborate with other students. Utilization of high fidelity simulation scenarios provides students from nursing and medicine the opportunity to enhance cognitive, psychomotor, and affective skills while improving communication skills needed to provide direct care to complex patients. Students report that high fidelity simulation provided the first opportunity to actually engage in a realistic setting to respond to a clinical scenario and work together as a team, as well as problem solve regarding the clinical scenario.
Conclusion: The ability of medical student and nurse practitioner students to actively engage in developing interprofessional team skills in clinical experiences is hampered by their student role in which they are not permitted the opportunity to communicate and collaborate directly with other team members. Typically, attending physicians, residents, nurse practitioners are in these roles, while students are observers. The scenario also allowed students to prepare themselves for the future when they will be problem-solving with other clinicians with different educational and professional backgrounds. By having IPE as part of the nursing and medical curriculum, students can be taught the core competencies needed, as well as how to work in a team with other clinicians to provide appropriate patient care. This study helped identify the IPE competencies supported and those not developed when using an intradisciplinary model of IPE education complemented by interprofessional practice. Second, high fidelity simulation offers the opportunity to provide students the opportunity to assume roles necessary for development of IPE competencies that are not supported in clinical learning by nature and realities of clinical practice.
1. Institute of Medicine. (2000). To err is human: Building a safer health system. Washington DC: National Academy Press.
2. Institute of Medicine. (2001). Crossing the quality chasm. Washington DC: National Academy Press.
3. Institute of Medicine. (2003). Health Professions Education: A Bridge to Quality. Washington, DC: The National Academies Press.
4. Interprofessional Education Collaborative Expert Panel. (2011). Core competencies for interprofessional collaborative practice: Report of an expert panel. Washington, D.C.: Interprofessional Education Collaborative.
810 Coaching Surgical Residents on Non-technical Skills Improves Performance During Simulated Operations
Steven Yule, MA, MSc, PhD3, Sarah Henrickson Parker, PhD4, Gillian Wilkinson, DipHE, BSc Hons, MSc6, Adrian Neill, MPhil, FRCSEd5, Jamie MacDonald, MB ChB1, Aileen Mckinley, MBChB FRCS2, and Tim McAdam, MbBchBao, DipMedEd2
1ANAESTHESIA AND INTENSIVE CARE, ABERDEEN ROYAL INFIRMARY, ABERDEEN, SCOTLAND, GBR and 2SURGERY, ABERDEEN ROYAL INFIRMARY, ABERDEEN, SCOTLAND, GBR and 3NEIL AND ELISE WALLACE STRATUS CENTER FOR MEDICAL SIMULATION, BRIGHAM AND WOMEN’S HOSPITAL, BOSTON, MA, USA and 4NATIONAL CENTER FOR HUMAN FACTORS ENGINEERING IN HEALTHCARE, MEDSTAR INSTITUTE FOR INNOVATION, WASHINGTON, DC, USA and 5SURGERY, SOUTHERN TRUST, CRAIGAVON, NORTHERN IRELAND, GBR and 6PSYCHOLOGY, UNIVERSITY OF ABERDEEN, ABERDEEN, SCOTLAND, GBR
Introduction/Background: Non-technical skills (NTS) are those critical cognitive and social skills which underpin high performing teams, yet are neither formally trained nor assessed. The use of NTS performance models in surgery is increasing; NOTSS (Non-technical skills for surgeons) is a behavior rating system, which allows trained assessors to observe, rate and provide feedback on these skills in the OR.1 Before embedding NOTSS in simulation-based training, it is important to identify the extent to which it can be used to improve performance. Models of team and individual coaching are used in organizational psychology,2 but in their infancy in healthcare.3 Early studies have shown promising results regarding the ability of coaching to improve technical skills.4 The present study used a randomized control group design to test the impact of NOTSS coaching on ability to deal with the pressure of unexpected bleeding during laparoscopic operations in a simulated OR. We hypothesized that residents who received coaching on non-technical skills would be faster to call for help when faced with unstoppable bleeding than those who did not receive coaching.
Methods: Sixteen senior surgical residents were recruited and randomly assigned to a NOTSS coaching or control group using an envelope selection technique. Each participant then performed part of a laparoscopic cholecystectomy (from exposing Calot’s triangle to dividing the cystic artery) on five different patient scenarios in a simulated OR, supported by an OR team who were confederates. The coaching group received individual coaching between scenarios from a trained attending surgeon/ coach who focused on the participants’ cognitive and social skills, using the NOTSS skills taxonomy. Participants in the control group were encouraged to self-reflect on their performance between scenarios. We created a simulated OR in a hospital teaching room. Central to this was a patient mannequin integrated with a Promis 2.5 Haptica Hybrid Laparoscopic Simulator and a gallbladder model, an anesthetic machine, diathermy and suction. The patient was draped and monitored with a pulse oximeter. During the scenarios, bleeding was programmed to occur at a standard time in line with the operative difficulty of the case. In scenarios 1-4 the participant could intervene to manage the bleeding; however in the fifth scenario, the bleeding could not be stopped. The primary endpoint was time taken to call for help in light of unstoppable bleeding in the fifth scenario. Secondary outcomes were time to stop bleeding in scenarios 1-4, time to complete each scenario, and path length of laparoscopic instruments.
Results: Using Analysis of Variance, we found a significant difference between the groups on time to call for help in scenario 5: F(1,15)=6.02, p=.028. Analyses of mean times showed that the NOTSS coaching group was significantly quicker to call for help (M=73 secs) than the control group (M=136 secs). There was also a significant difference between the groups in time control bleeding in scenario 1: F(1,15)=6.43, p=.024, but no differences in the other scenarios. There were no differences in overall completion time, or path length in scenarios.
Conclusion: A significant proportion of poor outcomes in surgery are predicated by breakdowns in non-technical or behavioral skills such as situation awareness, decision making, teamwork or leadership. These skills are not formally trained in medical school or surgical residency. One method of addressing these skills in the workplace is through coaching. The present study has shown that one-to-one coaching with surgical residents on their non-technical skills has improved performance in responding to unstoppable bleeding in the OR, a potentially life-threatening conditions for patients. Combining NOTSS and technical coaching may have an even more powerful effect on improving performance in the simulated and real OR.
1. Yule S, et al. Surgeons’ non-technical skills in the operating room: Reliability testing of the NOTSS behavior rating system. World Journal of Surgery 2008;32:548-556.
2. Hackman JR & Wageman R. A theory of team coaching. Academy of Management Review 2005;30:269-287.
3. Gawande A. Personal Best: Top athletes and singers have coaches. Should you? The New Yorker 2011, October 3rd.
4. Hu YY, et al. Postgame analysis: Using video-based coaching for continuous professional development. J Am Coll Surg 2012;214:115-124.
833 Team Training in the OR: Simulation Is More Effective Than Lecture in OR Fire-Safety Training
Jonathan Fraser, BA2, Greg Motuk, RN2, Josef Luba2, Gretchen Kolb2, Natalia Martinez Acero, MD3, Jon Morris, MD3, Noel Williams, MD1, Kristoffel Dumon, MD1, and Andrew Resnick, MD, MBA2
1DEPARTMENT OF SURGERY, HOSPITAL OF THE UNIVERSITY OF PENNSYLVANIA, PHILADELPHIA, PA, USA and 2PENN MEDICINE CLINICAL SIMULATION CENTER, HOSPITAL OF THE UNIVERSITY OF PENNSYLVANIA, PHILADELPHIA, PA, USA and 3SURGERY, HOSPITAL OF THE UNIVERSITY OF PENNSYLVANIA, PHILADELPHIA, PA, USA
Introduction/Background: Simulation has become increasingly integrated into graduate medical education. With simulation requirements in many specialties and a significant financial and time investment to create and run simulation programs, it is necessary to ensure that the investment generates improved learning. At the Penn Medicine Clinical Simulation Center, an Operating Room (OR) team training program has been in place since June 2010. Approximately 390 PeriOp nurses, staff and residents from OB/GYN, Anesthesia, Surgery, ENT, OMFS, and Orthopedics have participated in an OR Fire scenario with improvements seen in timeliness of response, as well as self-reported confidence and role recognition in the event of an OR fire. To demonstrate that, the effectiveness of this training is due to the simulation and debriefing, not the didactic component, we divided surgical interns undergoing OR fire training via a simulation-based versus a traditional lecture-based approach.
Methods: Participants were split into two groups, each containing four teams of residents, for a one-hour training on OR Fire Safety. Both groups completed a ten question written test to assess basic OR fire safety knowledge prior to training. Learners either received training via participation in a simulated OR fire followed by structured post-session debriefing (“simulation group”, n=22) or a PowerPoint lecture (“lecture group”, n=24). The same content was provided in both groups. Both training methodology groups participated in a final OR fire simulation with post-session analysis of their recorded response. Participants in the simulation were assigned team roles (scrub nurse, surgeon, anesthesiologist, etc.) and entered an OR equipped with a SimMan 3G. Following a time out, electrocautery cued a confederate to activate a smoke machine hidden beneath the OR table. Time from the appearance of smoke (t=0) to performance of four key response steps (call fire, call for help, remove drapes, shut off gases) was measured to gauge the efficiency of the team’s reaction. Finally both groups were given a self assessment (5 point Likert-type scale) of their knowledge of response steps, risk factors and team roles in the event of an OR Fire and completed the same ten question test administered at the start of the session.
Results: Incoming surgical interns (n=46) participated in OR Fire training during a surgical skills bootcamp at the Penn Medicine Clinical Simulation Center. This included all interns in the Department of Surgery and surgical subspecialties. Prior to training, interns scored similarly on a pre-training test (61% vs. 58% correct responses, NS). Following training, interns in the “simulation group” scored significantly higher than those in the “lecture group” on the same test (93% vs. 82% correct responses, P < 0.05). The “simulation group” improved in number of fire steps performed (4 vs. 2.5 steps, P < 0.05) and length of time to perform these steps (P < 0.05) from their initial to final simulation, similar to previously presented data. In their post session OR fire simulation, the “lecture group”, performed all steps but at a significantly slower pace than the “simulation group” (P < 0.05). Self-reported understanding of response steps, risk factors and role understanding in the event of an OR fire increased significantly for both groups following their training; however, the “simulation group” improved significantly more (P < 0.05).
Conclusion: This study demonstrates that OR fire safety training via a simulation followed by structured post-session debriefing, improves response in a simulated OR fire as well as fire safety knowledge, both objective and self-reported, better than a traditional lecture approach. While simulation-based learning requires significant resources, it provides a more effective learning experience than traditional lecture methods for topics such as OR fire safety.
876 A Tool for Assessing Behaviour in Simulated Neonatal Environment
Ranganath Ranganna, MBBS, MRCPCH(UK)3, Sarah Didier, MBBS, MRCPCH3, Asim Ahmed, Vrach, MRCPCH, PGC Med Ed2, Helen Moore, MBChB, MRCPCH4, and Makani Purva1
1MEDICAL EDUCATION, ANAESTHETICS, HULL AND EAST YORKSHIRE HOSPITALS NHS TRUST, HULL, GBR and 2PAEDIATRICS, HULL INSTITUTE OF LEARNING AND SIMULATION (HILS), HULL, GBR and 3PAEDIATRICS, HULL ROYAL INFIRMARY, HULL, GBR and 4PAEDIATRICS AND SIMULATION, HULL ROYAL INFIRMARY, HULL, GBR
Introduction/Background: Simulation training is currently being used for formative assessments in some specialities. The lack of a valid tool in neonatal simulation has limited its use for summative assessment. We initiated a research project aiming to create and validate a modified behaviour assessment tool (BAT) for use in a common neonatal scenario to assess trainees’ interactions with the environment. The BAT has already been used in other paediatric scenarios.1
Methods: Following ethics committee approval, we used a modified Delphi technique to achieve consensus among several experts in neonatal medicine to develop a scoring system to assess the behavioural performance. The BAT tool has nine categories under which there were several subtasks. The experts were requested to assign a score of 1 to 5 (1=not important, 5=most important), depending on the importance of the subtasks in the context of the scenario. The final weighted score was the mean of all the scores. The score given by the raters was tested for reliability and variability using intraclass coefficient analysis. We recruited eight trainees from year 1-3 (ST1-3) and 8 from year 4-8 (ST4-8) to participate in a recorded simulated neonatal scenario. Four independent raters are currently using the tool we developed to rate their performance, and scores obtained from them will be used to validate the tool.
Results: Scores (assigned for one subtask of the tool-knowledge of the environment) given by 6 consultants who participated in the Delphi process is shown in the table. The final weighted score for the response to the subtask is the mean of the scores. Using the Delphi process, after two rounds, consensus was achieved with a Kendall W Coefficient of 0.73 demonstrating good concordance. Similarly we have developed a weighted score for a further nine different subtasks. Refer to Table 1 (Knowledge of the environment).
Conclusion: Neonatal intensive care and resuscitation environment is a high-pressure environment and the stabilisation of the patient involves complicated interaction and behaviour among the team members. We believe that our tool, with a scoring system, will be a useful tool in Paediatrics to assess performance of doctors in a neonatal scenario. Further, as the tool is a relatively generic behavioural tool, we believe it has wider scope of being used with its scoring system for objective assessment of a doctor’s performance in a simulated scenario in various specialities.
1. Anderson JM, Yaeger KA: The Development of a Behavioral Scoring Tool for Neonatal Resuscitation. Society for Medical Simulation Meeting. San Diego, CA, January, 2006.
2. JoDee M. Anderson, MD, MEd, and Jamie B. Warren, MD Using Simulation to Enhance the Acquisition and Retention of Clinical Skills in Neonatology. Semin Perinatol 35:59-67.
922 Clinical Judgment Process in a Pediatric Nursing Simulation for Febrile Children
Hyunsook Shin, PhD1, and Eduardo Salas, PhD2
1COLLEGE OF NURSING SCIENCE, KYUNG HEE UNIVERSITY, SEOUL, KOR and 2INSTITUTE FOR SIMULATION AND TRAINING, UNIVERSITY OF CENTRAL FLORIDA, COLLEGE OF MEDICINE, ORLANDO, FL, USA
Introduction/Background: The essentials of practicum evaluation in nursing education are psychomotor, cognitive, and affective domains.The reliable and valid evaluation instruments in simulation education that encompass these three domains are needed (Kardong-Edgren & Fitzgerald, 2010). A Scenario-specific rubric can facilitate nursing faculties to evaluate specific tasks performed by nursing students more effectively (Clark, 2006). The purpose of the study was to present a tool to evaluate students’ clinical judgment processes using the Lasater Clinical Judgment Rubric (Lasater, 2007) and to compare the Lasater Clinical Judgment Rubric (LCJR) scores according to simulation designs.
Methods: Three raters viewed thirteen videotapes in a pediatric nursing simulation on febrile children and then evaluated the students’ performances independently using the LCJR, which consisted of four domains (Noticing, Interpreting, Responding, and Reflecting) and eleven sub-scales. Inter-rater reliability was calculated to be 0.78.
Results: The scenario-specific grading rubric was completed based on common performance observed in videotapes (Table 1). The range of total score was 15.67-40.00, the normative score of the LCJR is 11.00-44.00. The highest score was ‘Commitment to Improvement’ and the lowest score was ‘Making Sense of Data’ among the eleven sub-scales. The groups with standardized patients tended to have higher LCJR scores than with role play.
Conclusion: This is the first observational study to describe the clinical judgment process during pediatric nursing simulation in Korea. This scenario-specific rubric provides both the instructor and student with an objective and convenient evaluation criteria in simulation nursing education. Ultimately, this scenario-specific rubric can help students enhance their clinical judgment skills that are essential for clinical practice.
1. Kardong-Edgren S, & Fitzgerald C: A review of currently published evaluation instruments for human patient simulation. Clinical Simulation in Nursing 2010; 6: 25-35.
2. Clark M: Evaluating an obstetric trauma scenario. Clinical Simulation in Nursing Education 2006; 2: e75-77.
3. Lasater K: Clinical judgment development: Using simulation to create an assessment rubric. Journal of Nursing Education 2007; 46: 496-503.
925 Results of Resuscitation Training in Elementary School Children: Conducted by Teams
Taro Aoki, MA1
1JAPAN BASIC LIFE SUPPORT ASSOCIATION, YOKOHAMA, KANAGAWA, JPN
Introduction/Background: CPR training for lower grade children (elementary school 1st and 2nd year students) as rescuers has rarely been implemented in schools. Existing CPR training models are carried out in principle by a single rescuer, in the following sequence: discovery, evaluation, CPR, defibrillation. Mastery of all of these steps requires a highly perfected skill level, and is considered too difficult for lower-grade children. However, if lifesaving skills can be taught to younger children, lifesaving awareness can be nurtured more deeply, and the effective resuscitation rate can be improved in society as a whole. This goal can be accomplished in part by the development of effective, efficient and appealing CPR education for lower-grade children.
Methods: Our research involves the planning and development of a CPR training program incorporating elements of team dynamics. We will report the results of our trial program in 1st year and older elementary school students (hereafter referred to as Team CPR). “Team dynamics” refers to the concept of non-technical skills proposed by the advanced cardiovascular life support program (ACLS course) of the Education and Training Emergency Cardiovascular Treatment of the American Heart Association (AHA). This technique, where multiple rescuers cooperate to assist one patient, is considered a high-level technique that is chiefly implemented in hospitals by specialist healthcare professionals. We incorporated an element of team dynamics into Team CPR, even though this is a program for ordinary schoolchildren. If someone collapses at school, it is not realistic that one person will oversee their rescue from beginning to end. This is because it is natural that a large number of students can be found within a school ground. This study investigated the possibility of providing CPR training to lower-grade children that incorporated elements of the ACLS course into a basic life support program tailored to a school situation. Although the physical strength of lower grade elementary students makes it difficult for them to perform effective CPR individually, we hypothesized that it would be possible for them to assist in lifesaving as one member of a resuscitation team, and thus created this program.
Results: Conducting training under the observation of an instructor resulted in patients being able to receive adequate treatment, from evaluation to appeal for support, hands only CPR, and automated external defibrillator (AED). The 1st-year elementary students were able to discover the patient and call for help. They also learned that they needed to move away from the patient when AED was being used.
Conclusion: Basic Life Support by multiple rescuers is presumed to require high-level training, and to be limited to healthcare professionals. Most adults rarely have a chance to receive instruction in this technique, and it is even less available to students. However, through the Team CPR program, even elementary school students can participate with confidence in the resuscitation process and improve the odds for a favorable patient outcome.
926 Intrauterine Resuscitation by Maternal Hyperoxygenation Evaluated in a Simulation Model
Lauren Bullens, MD2, Beatrijs van der Hout, MSc1, Pieter van Runnard Heimel, MD, PhD2, and Guid Oei, MD, PhD2
1BIOMEDICAL ENGINEERING, EINDHOVEN UNIVERSITY OF TECHNOLOGY, EINDHOVEN, NLD and 2OBSTETRICS AND GYNECOLOGY, MAXIMA MEDICAL CENTER, VELDHOVEN, NLD
Introduction/Background: Recently, a mathematical model to simulate the cardiotocogram used to monitor fetal well-being during labor was developed and for use in obstetric simulation training.1,2 This model facilitates the simulation of various types of decelerations in fetal heart rate and provides insight in the parameters that influence fetal heart rate during labor. We investigated the possibilities of this model to simulate the effect of intrauterine resuscitation techniques on fetal heart rate. In case of fetal distress, when decelerations become visible on the cardiotocogram, several intrauterine resuscitation techniques can be used to improve fetal condition.3 One of these techniques is the administration of 100% oxygen to the mother by a non-rebreathing mask. Based on the evidence available from literature, we hypothesize that maternal hyperoxygenation will cause an increase in fetal pO2 and thus a decrease decelerations in the cardiotocogram. Literature describes an increase in maternal PaO2 up to 475 mmHg after 5 minutes of hyperoxygenation,4,5 and one study shows a decrease in deceleration depth.5
Methods: A mathematical model developed by our group for simulation of the cardiotocogram was used.1,2 The model is based on physiological parameters that influence fetal heart rate and oxygenation, both from the fetal and maternal circulation. These include maternal cardiac output, maternal saturation and oxygen pressure, uterine pressure and flow, oxygen diffusion capacity in the placenta, fetal cerebral blood flow, fetal oxygen consumption, baro- and chemoreceptor responses, vagal and sympathetic nerve response. During the simulation, maternal pO2 was increased from 100 mmHg to 475 mmHg, in line with the increase described in literature.6 We simulated umbilical cord compressions in a setting of maternal hyperoxygenation and focused on alteration of fetal heart rate, fetal venous and arterial pO-2, and pO2 in the microcirculation and intervillous space.
Results: The model shows a gradual increase in pO2 in the intervillous space from 44 to 55 mmHg, an increase in the umbilical vein from 31 to 41 mmHg and in the fetal arterial and microcirculation from 17 to 20 and from 15 to 17 mmHg respectively. In addition, the amplitude of the variable fetal heart rate (FHR) decelerations decreases when oxygen is administered to the mother (from -44 bpm to -25 bpm). FHR baseline is slightly increased (+3 bpm) in the model due to higher baseline pO2. See figure 1.
Conclusion: In our model, oxygen administration to the mother leads to increased oxygen pressures in the feto-placental circulation, and affects fetal response to umbilical cord compression-induced hypoxemia. Similar to the observations from,6 we see an improvement in fetal oxygenation status and a decrease in FHR deceleration depth, which may reflect a better fetal condition. The model is able to simulate intrauterine resuscitation of a fetus by the administration of oxygen to the mother.
1. Van der Hout-van der Jagt MB, Oei SG, Bovendeerd PHM. A mathematical model for simulation of early decelerations in the cardiotocogram during labor. Med Eng Phys. 2012 Jun:34(5);579-589.
2. Van der Hout-van der Jagt MB, Oei SG, Bovendeerd PHM. Simulation of reflex late decelerations in labor with a mathematical model. Early Human Dev. 2012 in press.
3. Simpson K, James D. Efficacy of Intrauterine Resuscitation Techniques in Improving Fetal Oxygen Status During Labor. Obstet Gynecol. 2005 Jun;105(6):1362-8.
4. Althabe O, Schwarcz R, Pose S, Escarcena L, Caldeyro-Barcia R. Effects on fetal heart rate and fetal pO2 of oxygen administration to the mother. Am J Obstet Gynecol. 1967 Jul;98(6):858-870.
5. D Young, R Popat, E Luther et al. Influence of maternal oxygen administration on the term fetus before labor. Am J Obstet Gynecol. 1980 Feb;136(3):321-4.
6. Vasicka A, Quilligan E, Aznar E, Lipsitz P, Bloor Ml. Oxygen tension in maternal and fetal blood, amniotic fluid, and cerebrospinal fluid of the mother and the baby. Am J Obstet Gynecol. 1960 Jun;79:1041-47.
Guid Oei, MD, PhD is the Medical Director of Medical Education and Simulation Center at Maxima Medical Center, Veldhoven, Netherlands.
936 Impact of Instructor Feedback on Performance in a Laparoscopic Virtual Reality Simulator at Six-month Follow-up
Flemming Bjerrum, MD1, Mathilde Maagaard, MD5, Jette Led Sorensen, MD, MMEd4, Christian Rifbjerg Larsen, MD, PhD2, Charlotte Ringsted, MD, PhD3, Bent Ottesen, MD, DMSc5, and Jeanett Oestergaard, MD5
1GYNECOLOGY, JULIANE MARIE CENTRE, RIGSHOSPITALET, COPENHAGEN, DNK and 2GYNECOLOGY, HILLEROED HOSPITAL, HILLEROED, DNK and 3CENTRE OF CLINICAL EDUCATION, RIGSHOSPITALET, COPENHAGEN UNIVERSITY HOSPITAL, COPENHAGEN, DNK and 4OBSTETRICS, JULIANE MARIE CENTRE, RIGSHOSPITALET, COPENHAGEN UNIVERSITY HOSPITAL, COPENHAGEN, DNK and 5OBSTETRICS, JULIANE MARIE CENTRE, RIGSHOSPITALET, UNIVERSITY OF COPENHAGEN, COPENHAGEN, DNK
Introduction/Background: The increasing use of minimal invasive surgery has changed surgical education. Both laparoscopic box trainers and virtual reality simulators have been shown to improve technical skills, which can be transferred to the operating room.1,2 However the optimal way to implement simulators and how training should be structured still needs to be determined.3 Feedback is an essential component of surgical education, but has also been shown to have significant impact on simulator training, by increasing efficiency, reducing errors and reducing the required time to reach a predefined proficiency level.4-6 However, the impact of instructor feedback on the retention of the skills needs to be determined. The objective of this study was to investigate the impact of instructor feedback versus no feedback on long-term retention after training a complex operation on a laparoscopic virtual reality simulator.
Methods: The methods included a follow-up study on a randomised trial.7 The setting was a skills centre in a university hospital. The participants were senior medical students on 4th to 6th year without prior laparoscopic experience. The Stratification variables used for randomisation were sex and computer gaming experience. All participants (n=99) initially trained a simulated operation (a laparoscopic salpingectomy) to a predefined proficiency level on a virtual reality simulator (LapSim®, Surgical Science, Göteborg). The main outcome measures were total time (minutes) and repetitions to reach the predefined proficiency level for the simulated operation.8 The intervention group (n=49) received standardized instructor feedback on operational technique and instrument handling a maximum of three times. The control group (n=50) did not receive instructor feedback. All participants received the automated feedback generated by the simulator. Participants who reached the proficiency level during the intervention period (n=91) were invited back after six months and were asked to train the same simulated operation and reach the same proficiency level (n=65). None of the participants received instructor feedback at follow-up. None of the participants received additional simulator training or performed laparoscopic procedures between the intervention and follow-up period.
Results: In the initial training period the intervention group used an average of 162 minutes and 29 repetitions to reach the proficiency level, where as the control group, in average, used 342 minutes and 65 repetitions. A significant difference (p<0,0005) was found regarding both repetitions and time spent. At six-month follow-up, the intervention group (n=36) used an average of 83 minutes and 20 repetitions to reach the same proficiency level, and the control group (n=29) in average used 73 minutes and 21 repetitions. Both the intervention and the control group showed at follow-up a significant reduction in time (p<0,0001) and repetitions (p<0,0001) compared to their initial training period, showing retention of skills. The reduction in time and repetitions were larger for the control group compared with the intervention group (p<0,0005), resulting in that the initial difference between the two groups had disappeared at six-month follow-up. Statistical analysis was carried out using the mixed model with repeated measures. Data were analyzed using SPSS (Chicago, IL), version 15.0.
Conclusion: Initially, instructor feedback results in reduced training time to reach a predefined proficiency level on a virtual operational task on a simulator; however, at six-month follow-up the difference between the intervention and control group was no longer significant. We recommend the use of instructor feedback to maximize the efficiency of simulator training when using proficiency-based training.
1. Orzech N, Palter VN, Reznick RK, Aggarwal R, Grantcharov TP: A comparison of 2 ex vivo training curricula for advanced laparoscopic skills: a randomized controlled trial. Ann. Surg. 2012, 255:833–839.
2. Larsen CR, Oestergaard J, Ottesen BS, Soerensen JL: The efficacy of virtual reality simulation training in laparoscopy: a systematic review of randomized trials. Acta Obstet Gynecol Scand 2012. DOI: 10.1111/j.1600-0412.2012.01482.x. [Epub ahead of print].
3. Stefanidis D, Arora S, Parrack DM, Hamad GG, Capella J, Grantcharov T, Urbach DR, Scott DJ, Jones DB, Association for Surgical Education Simulation Committee: Research priorities in surgical simulation for the 21st century. Am. J. Surg. 2012, 203:49–53.
4. Stefanidis D, Korndorffer JR, Heniford BT, Scott DJ: Limited feedback and video tutorials optimize learning and resource utilization during laparoscopic simulator training. Surgery 2007, 142:202–206.
5. Boyle E, O’Keeffe DA, Naughton PA, Hill ADK, McDonnell CO, Moneley D: The importance of expert feedback during endovascular simulator training. J. Vasc. Surg. 2011, 54:240–248.
6. Boyle E, Al-Akash M, Gallagher AG, Traynor O, Hill ADK, Neary PC: Optimising surgical training: use of feedback to reduce errors during a simulated surgical procedure. Postgrad Med J 2011, 87(1030):524-8. Epub 2011 Jun 3.
7. Oestergaard J, Bjerrum F, Maagaard M, Winkel P, Larsen CR, Ringsted C, Gluud C, Grantcharov T, Ottesen B, Sorensen JL: Instructor feedback versus no instructor feedback on performance in a laparoscopic virtual reality simulator: a randomized educational trial. BMC Med Educ 2012, 12:7.
8. Larsen CR, Grantcharov T, Aggarwal R, Tully A, Sørensen JL, Dalsgaard T, Ottesen B: Objective assessment of gynecologic laparoscopic skills using the LapSimGyn virtual reality simulator. Surg Endosc 2006, 20:1460–1466.
Disclosures: Charlotte Ringsted, MD, PhD receives grant support from Laerdal Foundation and TrygFonden. Dr. Ringsted is a consultant for Covidience.
945 A Course Response to Medical Communication Errors - Handoff Communication: Do not Drop the Ball!
Laura Daniel, PhD1, and Donamarie Wilfong, DNP, RN2
1INDEPENDENT CONSULTANT, PITTSBURGH, PA, USA and 2THE STAR CENTER, WEST PENN ALLEGHENY HEALTH SYSTEM, PITTSBURGH, PA, USA
Introduction/Background: Thousands of medical errors are made in hospitals every year, many due to poor communication. In fact, the Agency for Healthcare Research and Quality (AHRQ) reported that 37% of Intensive Care Unit sentinel events and 61% of Operating Room sentinel events were due to failures in communication. Such distressing statistics highlight the need for a re-evaluation and possible modification of the communication training that medical professionals receive. The Joint Commission even recognized this need and made improving communication a national patient safety goal in the years 2007 – 2011. In response, the Simulation, Teaching, and Academic Research (STAR) Center at West Penn Allegheny Healthcare System (WPAHS) hosted a handoff communication conference. Sorrel King, a patient safety advocate, was the main speaker who described her young daughter’s untimely death in a hospital due to communication errors. Following Ms. King, standardized patients portrayed five patient safety scenarios that incorporated a hybrid simulator/live actor combination. These scenarios were later used to develop an online training course, called “Handoff Communication: Don’t Drop the Ball” to continue the communication education initiative. This course was designed for all hospital staff members, with the goal of improving their communication knowledge and skills. The course incorporated instruction on aspects of proper communication from the TeamSTEPPS program, which was developed by the Department of Defense’s Patient Safety Program in collaboration with the Agency for Healthcare Research and Quality (AHRQ). Upon completion of the course, participants were expected to be knowledgeable on the proper techniques that can be used to enhance communication between interprofessional team members. Specifically, participants were expected to understand the concepts of situation awareness, situation monitoring, the SBAR communication tool, the shared mental model and the importance of integrating elements such as call-out and check-back. This research project is designed to evaluate the effectiveness of this hand-off communication course.
Methods: The sample (n = 43) consisted of self-motivated WPAHS employees. Participants may have been directly recruited from STAR course facilitators or the participants may have discovered this course on their own from STAR’s website. Students’ initial amounts of communication knowledge were assessed through a 10-item pretest that was comprised of both multiple choice questions and true/false questions. Learners then accessed a PowerPoint presentation of didactic communication content. Following the PowerPoint, participants watched five vignette videos that showcased proper and improper communication. These vignettes were modeled from the scenarios that were shown during the communication conference. Upon completing these course activities, participants were required to take a 10-item post-test to reassess their communication knowledge.
Results: When the conference attendees were asked on an evaluation how much they learned from the presentation, they answered on a 7-point Likert scale that ranged from “Nothing at all” to an “Extremely large amount.” The majority (87.9%) of respondents indicated that they learned a very large amount. Likewise, this handoff communication course, which was modeled from that conference, also had positive results. Results from a repeated measures t-test indicated that participants generally earned significantly higher scores on the post-test (M = 9.23, SD = 0.895) than they did on the pretest (M = 6.28, SD = 1.202), t(42) = -13.237, p <.01, d = 2.02.
Conclusion: Results from the pre/post-test analysis showed that participants gained communication knowledge, providing evidence that the course is effective. Participants who completed this online course had a more thorough understanding of communication skills. This knowledge is expected to translate to more effective teams in the workplace and eventually lead to an improved quality of care. Furthermore, employees’ refined communication skills are likely to be associated with decreased probabilities of adverse events.
947 No Effect of Multi-professional Simulation-based Skills Training in Management of Postpartum Bleeding Evaluated on Blood Transfusion and Time Delay to Surgical Intervention
Jette Led Sorensen, MD, MMEd2, Veronika Markova, Medical Student1, Charlotte Holm, MD2, Astrid Norgaard, MD, PhD3, and Jens Langhoff-Roos, MD, DMSci2
1HEALTH SCIENCES, COPENHAGEN UNIVERSITY, COPENHAGEN, DNK and 2OBSTETRICS, JULIANE MARIE CENTRE, RIGSHOSPITALET, COPENHAGEN UNIVERSITY HOSPITAL, COPENHAGEN, DNK and 3TRANSFUSION MEDICINE, CAPITAL REGION BLOOD BANK, RIGSHOSPITALET, COPENHAGEN UNIVERSITY HOSPITAL, COPENHAGEN, DNK
Introduction/Background: Multi-professional obstetric simulation-based skills training in the management of emergency obstetric situations such as postpartum hemorrhage (PPH), shoulder dystocia, basic neonatal resuscitation and severe preeclampsia was introduced in a high risk obstetric department in 2003. Midwives, obstetric nurses, auxiliary nurses, obstetric senior physicians and residents participated. The training was highly appreciated, and the participants reported improved collaboration and communication in obstetric emergencies. Training for the management of PPH resulted in a significant increase in self-assessed confidence among auxiliary nurses and residents, and a significant increase in the administration of uterotonics (i.e., medicine administered to contract the uterus in delivering women with postpartum bleeding).1 However, we did not know if the multi-professional obstetric skills training intervention resulted in improved health care for the women, and decided to compare the incidence of PPH indicated by red blood cell (RBC) transfusion and time delay in surgical interventions before, during, and after implementation of the simulation-based training program. Unnecessary blood transfusions expose women to adverse effects, and an endpoint of obstetric skills training can be hypothesized to be a reduction in cases of delivering women requiring blood transfusion; thus, this study provided evaluation on the patient level: the result level (i.e., level four according to Kirkpatrick).2 In general few educational studies obtain data which allow evaluating on the result level. The objective was to evaluate the effect of a multi-professional obstetric simulation-based skills training intervention on the incidence of PPH indicated by red blood cell (RBC) transfusion and time delay in surgical interventions before, during, and after implementation of a training program.
Methods: The research was conducted at the Obstetric Department, Rigshospitalet, Copenhagen University Hospital, Denmark using a database audit. The methodology used the linkage of the Danish Medical Birth Registry and the local transfusion database, followed by an audit of medical records. The main outcome measures were as follows: RBC transfusion and time delay to surgical intervention. The statistical analysis included chi-squared tests, which were used to analyze changes in rates between the three time periods. A trend-test was used to compare the amount of RBC units over time. Comparison of the non-categorical variables was done using ANOVA and Kruskal–Wallis tests.
Results: We identified 148 women with RBC transfusion for PPH in a total of 10461 deliveries and assessed for these cases the cause of PPH, surgical interventions and transfusion data. RBC transfusion rates for PPH were 1.5 % (2003), 1.6 % (2005) and 1.2 % (2007) (not statistically significant). The transfusion rates did not change after vaginal delivery but decreased after cesarean section (2.4 %, 2.1 % and 0.7 % (p < 0.01). Transfusion requirement and pre-transfusion hemoglobin values did not change. The median time from delivery to manual removal of the placenta increased insignificantly (64, 70 and 75 minutes). The median time from decision to manual removal of the placenta remained unchanged (30 minutes).
Conclusion: There was no effect of multi-professional obstetric skills training on the rate of RBC transfusion for PPH. To optimize management of PPH, obstetric skills training must focus on when to request surgery in order to minimize delay from delivery to surgical interventions. Changes must be made in order to improve early identification of the causes of PPH, especially retained placenta. Future obstetric skills training should (besides all staff in the labor ward) also include staff from the operation theater and the anesthesiology department.3
1. Sorensen JL, Lokkegaard E, Johansen M, Ringsted C, Kreiner S, McAleer S. The implementation and evaluation of a mandatory multi-professional obstetric skills training program. Acta Obstet Gynecol Scand 2009;17:1-11.
2. Kirkpatrick DL. Evaluating training programmes: the four levels. 2nd edition. Berret-Koehler Publishers, 1998.
3. Markova V, Sorensen JL, Holm C, Nørgaard A, Langhoff-Roos J. Evaluation of multi-professional obstetric skills training for postpartum hemorrhage. Acta Obstet Gynecol Scand. 2012;91:346-52.
949 Simulated Scenarios Induce Equivalent Stress Levels in Residents Compared to Real Emergency Care
Roger Dias, MD1, and Augusto Scalabrini-Neto, MD, PhD1EMERGENCY MEDICINE, UNIVERSITY OF SAO PAULO SCHOOL OF MEDICINE, SAO PAULO, SP, BRA
Introduction/Background: Clinical simulation-based learning is well established as effective in the acquisition of technical1 and non-technical skills.2 Only few studies have been published about the effects of clinical simulation on the stress response of the participants.3 The objective of this study was to evaluate the level of stress of internal medicine residents during a simulation-based training and compare it with the level of stress induced during real situations in an emergency room.
Methods: Thirty internal medicine residents participated after informed consent. In the emergency room,18 participants (each on different days) had the level of stress measured during the first emergency of the day. In simulated training, 12 participants (in groups of 4 each) had the level of stress measured during a simulated scenario of a medical emergency. Variables examined were heart rate, systemic blood pressure, salivary activity of alpha-amylase, salivary cortisol concentration and STAI (State-Trait Anxiety Inventory). The Student t test was used to test for differences between means, with statistical significance declared when P < 0,05.
Results: The mean duration for the care in emergency room was 24,6 ±6,8 minutes and in the simulated training was 25,7 ±2,0 minutes. Variables assessed in the pre and post care were not significantly different between groups (See Table 1).
Conclusion: Insufficient stress coping strategies can be correlated with poor performance as suggested by Hassan and Cols.4 In our study, we demonstrated that stress response to emergency care evaluated using physiological and hormonal variables, and the STAI-state questionnaire score, is similar when performed in a real situation or in a simulated scenario. These results suggest that simulation-based training is able to generate scenarios of psychological high-fidelity and can be used for training residents in emergencies.
1. Boulet JR, Murray D, Kras J, Woodhouse J, McAllister J, Ziv A: Reliability and validity of a simulation-based acute care skills assessment for medical students and residents. Anesthesiology 2003;99:1270–80.
2. Müller MP, Hänsel M, Stehr SN, et al: Six steps from head to hand: a simulator based transfer oriented psychological training to improve patient safety. Resuscitation 2007;73:137–43.
3. Müller MP, et al:Excellence in performance and stress reduction during two different full scale simulator training courses: a pilot study. Resuscitation.2009 Aug;80(8):919-24. Epub 2009 May 21.
4. Hassan I, Weyers P, Maschuw K, et al: Negative stress-coping strategies among novices in surgery correlate with poor virtual laparoscopic performance. Br J Surg 2006;93:1554–9.
Disclosures: Augusto Scalabrini, MD, PhD receives grant support from Laerdal Medical.
964 External Validation of Scoring Instruments for Evaluating Paediatric Resuscitation
Arielle Levy, MD, MEd, FRCPC3, Aaron Donoghue, MD, MSCE1, Benoit Bailey, MD., MSc, FRCPC2, and Jocelyn Gravel, MD, MSc, FRCPC4
1EMERGENCY AND CRITICAL CARE MEDICINE, CHILDREN’S HOSPITAL OF PHILADELPHIA, PHILADELPHIA, PA, USA and 2PEDIATRIC, CHU SAINTE-JUSTINE, MONTREAL, QC, CAN and 3PEDIATRIC EMERGENCY, SAINTE-JUSTINE HOSPITAL UNIVERSITY TEACHING CENTER, MONTREAL, QC, CAN and 4PEDIATRIC EMERGENCY MEDICINE, UNIVERSITY OF MONTREAL, MONTREAL, QC, CAN
Introduction/Background: Studies have reported that alarming delays and errors occur in major components of paediatric resuscitation,1 and that attention must be given to critically assess clinical performances during simulated paediatric mock codes. Many scoring instruments have been described to measure clinical performance during resuscitation; however, the validity of these tools has yet to be proven in paediatric resuscitation. A study by Donoghue et al.2 assessed the reliability and the validity of a new and simple scoring instrument designed to measure clinical performance during simulated resuscitations using PALS algorithms. Donoghue’s scoring instruments have not been evaluated in a setting different than the one where it was developed thus far. In addition, ability to detect a change over time after an intervention has not yet been established. The purpose of our study was to determine if Donoghue’s instrument is a valid measure of clinical performance during simulated resuscitations using PALS algorithms by examining changes in resident scores before and after taking a PALS course and by analyzing differences in scores between training levels. We also aimed to analyze inter-rater reliability of the instrument when used by multiple providers not involved in the development of the tool. It is mandatory to use a validated clinical performance assessment tool to evaluate clinical performances during simulated paediatric resuscitations as reflection and targeted feedback using key items from these tools will allow to adequately assess performance in actual resuscitations and ultimately lead to improving the quality of care delivered to a paediatric patient in extremis.
Methods: Using a prospective control group design, we proceeded with a field experiment in a simulation lab of a paediatric tertiary care academic facility. A total of 13 PGY 1s and 11 PGY 3s were videotaped during 5 paediatric resuscitation simulated scenarios. Raters scored the scenarios on resident performance before and after a PALS course using a standardized score sheet. Each video recording was viewed and scored independently by two different raters. The validity of the score was evaluated by measuring the variation of the score for participants following formal training in resuscitation (PALS course). A priori, it was determined that participants should improve their score after participating in the PALS course. It was also stipulated a-priori that the score should report better scores for senior residents in comparison to junior residents. Pre PALS and post PALS means for each level PGY1 and PGY3 were compared using an ANOVA. To investigate differences in the scores of the two groups over the 5 scenarios, a two factor ANOVA was used. Overall and individual inter-rater reliability was measured for the five scores using interclass correlation coefficient (ICC).
Results: Following the PALS course, the score improved by 8.6% (3.8-13.3) for the pulseless non-shockable arrest, 15.7% (8.6-22.7) for the pulseless shockable arrest, 6.3% (-1.8-14.3) for the dysrhythmias, 18.2% (9.3-27) for the respiratory scenario, 4.1% (-3.0-11.2) for the shock scenarios. There was no difference between PGY 1 and PGY 3 scores before or after the PALS course for any scenarios. The ICCs were 0.85 (0.74-0.92) for the pulseless non-shockable arrest, 0.98 (0.96-0.99) for the pulseless shockable arrest, 0.92 (0.87-0.96) for the dysrhythmias, 0.97 (0.95-0.98) for the respiratory scenario, 0.94 (0.90-0.97) for the shock scenarios.
Conclusion: Existing scoring instruments are a valid measure of clinical performance of junior and senior paediatric residents after PALS training. However it was not able to discriminate between PGY 1s and PGY 3s either before and/or after the PALS course for any scenarios. The scoring instrument showed very good inter-reliability for all scenarios. Paediatric resuscitation simulation training using a valid tool is mandatory to ensure critical assessment of performances and targeted feedback.
1. Donoghue, A., Nishisaki, A., Sutton, R., Hales, R., & Boulet, J. (2010). Reliability and validity of a scoring instrument for clinical performance during Pediatric Advanced Life Support simulation scenarios. Resuscitation, 81(3), 331-336. doi: S0300-9572(09)00592-9 [pii]10.1016/j.resuscitation.2009.11.011 [doi].
2. Hunt, E. A., Walker, A. R., Shaffner, D. H., Miller, M. R., & Pronovost, P. J. (2008). Simulation of in-hospital pediatric medical emergencies and cardiopulmonary arrests: Highlighting the importance of the first 5 minutes. Pediatrics, 121(1). doi: 10.1542/peds.2007-0029.
968 Simulation-based Workshop Improves Critical Care Fellow Communication Skills
Yue Dong, MD, MSCE1, Sherry Chesak, MS1, and Kianoush Kashani, MD2
1MULTIDISCIPLINARY SIMULATION CENTER, MAYO CLINIC, ROCHESTER, MN, USA and 2NEPHROLOGY/CRITICAL CARE, MAYO CLINIC, ROCHESTER, MN, USA
Introduction/Background: Effective communication between patient and provider is a critical skill required of all intensivists. In our institution this was taught using an apprenticeship model with supervised experience in the intensive care unit. We recently incorporated a scenario-based introductory communication workshop (CW) in the core curriculum of the fellowship program. This workshop is being offered to all the critical care fellows (fellows) as part of the Critical Care Boot Camp during their clinical orientation. The primary objective of our study is to investigate if fellows can improve their communication performance skill by attending CW.
Methods: A half-day scenario-based CW was attended by 29 first-year fellows during their fellowship orientation. Fellows participated in two standard patient scenarios to practice skills for relationship building (PEARLS)1 and delivery of unexpected news (SPIKES)2 in random order. Each scenario was followed by faculty guided video-assisted debriefing. Fellows’ performances were rated using a 14-item validated Communication Assessment Tool (CAT)3 by standard patient and their family member. Fellows also scored themselves after each scenario. The composite scores were calculated for total scores of 14 items. A paired t-test was conducted to compare fellows’ performances between these two scenarios.
Results: Fellows’ performance scores were improved as assessed by patient (51.0±6.4 vs. 54.7±6.8, p=0.04), and showed a trend toward improvement in the family member evaluation (51.8±10.4 vs. 56.9±10.8, p=0.07). However this improvement did not reach statistical significance when fellows rated themselves (52.4±1.3 vs. 53.2±1.3, p=0.65). The fellows seem to score their own performance lower than patient and family did.
Conclusion: These results suggest simulation based CW can improve fellows’ performances in a simulated ICU setting. It offers a unique experience for synthesis of knowledge, skill and attitude in a risk-free environment with the goal to improve patient safety and quality of the care. Future study is needed to evaluate the impact of the workshop on clinical outcome.
1. Milan FB, Parish SJ, Reichgott MJ. A model for educational feedback based on clinical communication skills strategies: beyond the “feedback sandwich.”. Teach Learn Med.2006;18(1):42–47.
2. Baile WF, Buckman R, Lenzi R, Glober G, Beale EA, Kudelka AP. SPIKES-A six-step protocol for delivering bad news: application to the patient with cancer. Oncologist. 2000;5(4):302–311.
3. Makoul G, Krupat E, Chang CH. Measuring patient views of physician communication skills: development and testing of the Communication Assessment Tool. Patient Educ Couns.2007;67:333–342.
993 Assessment of Doctor-Patient Communication, Interpersonal Skills and Professionalism Among Trainees in Post-graduate Medical Education
Jeanne Sandella, DO1, William Roberts, EdD3, and Erik Langenau, DO2
1NATIONAL BOARD OF OSTEOPATHIC MEDICAL EXAMINERS, CONSHOHOCKEN, PA, USA and 2CONTINUOUS PROFESSIONAL DEVELOPMENT AND INNOVATIONS, NATIONAL BOARD OF OSTEOPATHIC MEDICAL EXAMINERS, CONSHOHOCKEN, PA, USA and 3NATIONAL CENTER FOR CLINICAL SKILLS TESTING (NCCST) PSYCHOMETRICS AND RESEARCH, NATIONAL BOARD OF OSTEOPATHIC MEDICAL EXAMINERS, CONSHOHOCKEN, PA, USA
Introduction/Background: Assuring fundamental physician competence in humanistic skills (i.e., doctor-patient communication, interpersonal skills and professionalism) throughout the spectrum of medical education is gaining momentum.1-3 Yet, evidence of initiatives by the Accreditation Council for Graduate Medical Education (ACGME), the American Osteopathic Association (AOA) and others to assess these fundamental humanistic competencies is still emerging. The Global Patient Assessment ™ tool (GPA) was developed by the National Board of Osteopathic Medical Examiners (NBOME) as a clinical skills assessment for students taking the Comprehensive Osteopathic Medical Licensing Examination Level 2-Performance Evaluation (COMLEX-USA Level 2-PE).4,5 The GPA is a Likert-based tool that provides a scale for the evaluation of proficiency in these 6 dimensions: Eliciting Information, Listening, Giving Information, Respectfulness, Empathy, and Professionalism. Osteopathic medical students primarily take the COMLEX-USA Level 2-PE in the fourth year of medical school. Residents’ ratings for interpersonal and communication skills (based on program director ratings) have been shown to be positively correlated to their scores on the GPA from their COMLEX-USA Level 2 –PE performance.6 However, the specific GPA tool has never been applied to the assessment of residents. The purpose of this study is to evaluate the use of the GPA tool to assess residents in this domain.
Methods: Forty-two first and second-year osteopathic medical residents were recruited from residency programs proximal to the NBOME’s National Center for Clinical Skills testing to participate in the study. Residents were primarily from primary care resident tracks (family medicine, pediatrics, internal medicine), but others were represented (surgery, urology, orthopedics). As part of the assessment, residents rotated through six stations including triadic encounters, encounters with issues related to systems-based practice and practice-based learning and improvement, and encounters with partial-task simulators. Following the encounters, the residents completed various knowledge-based assessments, while the SPs completed the GPA for each encounter. In the encounters where there was more than one SP (i.e. patient and husband), each SP independently completed the evaluation. There were a total of 10 SP GPA ratings for each resident in the 6 stations. A set of independent SP raters also provided a second set of scores by reviewing digital recordings of the encounters. Inter- and intra-rater reliability analysis provided measures of agreement.
Results: Ten SPs provided ratings for each of the 42 residents in the 6 encounters. Inter-rater agreement of overall scores resulted in intraclass correlation coefficients ranging from .24 to .62. Weighted Kappa coefficients calculated across all cases for the six individual GPA dimensions ranged from .22 to .44.
Conclusion: Reliability indices indicate fair to moderate rater agreement when the GPA tool was used for evaluation of resident performance. Results show the GPA tool may be administered with some degree of reliability to residents in post-graduate medical education. This encourages further research with a larger number of residents across the residency disciplines. In addition, change in performance can be investigated by comparing a resident’s GPA scores measured in post-graduate education to their COMLEX-USA Level 2-PE GPA scores. The future of post-graduate education assessment to assure residents are competent in humanistic clinical skills could be advised by measurement methods presently used in medical licensure examinations.
1. Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. JAMA : the journal of the American Medical Association. Sep 5 2007;298(9):993-1001.
2. Van Zanten M, Boulet JR, McKinley D. Using standardized patients to assess the interpersonal skills of physicians: six years’ experience with a high-stakes certification examination. Health communication. 2007;22(3):195-205.
3. Van Zanten M, Boulet JR, McKinley DW, DeChamplain A, Jobe AC. Assessing the communication and interpersonal skills of graduates of international medical schools as part of the United States Medical Licensing Exam (USMLE) Step 2 Clinical Skills (CS) Exam. Academic medicine : journal of the Association of American Medical Colleges. Oct 2007;82(10 Suppl):S65-68.
4. Gimpel JR, Boulet DO, Errichetti AM. Evaluating the clinical skills of osteopathic medical students. The Journal of the American Osteopathic Association. Jun 2003;103(6):267-279.
5. Weidner AC, Gimpel JR, Boulet JR, Solomon M. Using standardized patients to assess the communication skills of graduating physicians for the comprehensive osteopathic medical licensing examination (COMLEX) level 2-performance evaluation (level 2-PE). Teaching and learning in medicine. Jan 2010;22(1):8-15.
6. Langenau EE, Pugliano G, Roberts WL. Relationships between high-stakes clinical skills exam scores and program director global competency ratings of first-year pediatric residents. Medical education online. 2011;16:7362.
1015 The Association of Team Leader Characteristics with Critical Events and Survival During Crisis Team Training
Hiroko Iwashita, MD6, Benjamin Berg, MD2, Paul Phrampus, MD, FACEP6, Nicolette Mininni, RN, MEd, CCRN3, Wendeline Grbach, MSN, RN, CCRN5, Michael DeVita, MD1, Takanori Hiroe, MSE7, Yoichi Kase, MD, PhD6, and Lillian Emlet, MD, MS, FACEP4
1INTERNAL MEDICINE, SAINT VINCENT’S MEDICAL CENTER, BRIDGEPORT, CT, USA and 2SIMTIKI SIMULATION CENTER, UNIVERSITY OF HAWAII, JOHN A BURNS SCHOOL OF MEDICINE, HONOLULU, HI, USA and 3CLINICAL ADMINISTRATION, UNIVERSITY OF PITTSBURGH, PITTSBURGH, PA, USA and 4CRITICAL CARE MEDICINE, UNIVERSITY OF PITTSBURGH, PITTSBURGH, PA, USA and 5NURSING, UNIVERSITY OF PITTSBURGH, PITTSBURGH, PA, USA and 6PETER M WINTER INSTITUTE FOR SIMULATION, EDUCATION AND RESEARCH (WISER), UNIVERSITY OF PITTSBURGH, PITTSBURGH, PA, USA and 7LABORATORY OF SOFTWARE ENGINEERING, WASEDA UNIVERSITY, TOKYO, JPN
Introduction/Background: One of the factors frequently considered in the outcomes of team performance is leadership. Currently there is no clear understanding of how team leader demographics affects team performance. Crisis Team Training (CTT) is an established course conducted for more than 8 years at the University of Pittsburgh, Winter Institute for Simulation Education and Research (WISER). CTT is designed to teach a flat interdisciplinary team hierarchy and emphasizes patient centered care over professional characteristics during simulated crisis response scenarios. This study was designed to evaluate and correlate individual experience, background and other demographic factors of participants who assumed the team leader (TL) role during CTT with task completion, critical events and patient outcomes.
Methods: Retrospective study data from 437 simulation sessions was collected during 105 CTT classes from February 2007 to April 2012. Participant demographic characteristics (DC) were recorded. CTT is comprised of 3 to 5 sequential scenarios, each with common elements:1) web-based pre-course curriculum, 2) brief didactic lecture reviewing key concepts of team performance, 3) video-recorded simulation scenarios for skills performance using Laerdal SimMan® (Stavanger Norway) and, 4) facilitated team de-briefing using a web-based interactive debriefing tool. Outcome variables include teamwork and clinical tasks recorded by checklist, critical events, and patient survival. Checklist completion was conducted by consensus of instructors and participants during debriefing. Tasks accomplished were recorded for each team member, including the TL. Critical events included absence of a treatment or a treatment that was harmful. First and last simulation sessions were analyzed to evaluate the effect of TL characteristics on simulation outcomes. First sessions were considered baseline and last sessions as post training. Sessions excluded from analysis: 1) no participant designated as the TL, 2) incorrect coding, and 3) missing data. The sessions were categorized based on: 1) TL profession (MD vs. non-MD), and 2) TL experience (<1 year vs. 1-3 years vs. 3-10 years vs. >10 years). The effects of team leader characteristics on outcomes were analyzed using multiple regression analysis (checklist completion) and logistic regression (critical events and patient survival). To account for differences by instructors for CTT, instructor was used as a covariate in both analyses.
Results: Forty-seven first sessions and 85 last sessions were analyzed. TL participants included nurses (63.3%), physicians (20.3%), respiratory therapists (11.0%) and others (5.2%). Experience: <1 year (31.2%), 1-3 years (28.1%), 3-10 years (19.4%), and >10 years (22.0%). First sessions and last sessions showed no statistically significant difference regarding leadership characteristics and overall task completion, occurrence of critical events and patient survival.
Conclusion: We did not find statistically significant differences in regard to leader demographic characteristics and experience on outcomes. These results may indicate that the role of TL is not independently related to outcomes, which may be a function of characteristics for all roles in the team. These results could also indicate that our checklist is not sensitive enough to provide an indication of leadership function or that team performance is independent of the background of the TL.
Disclosures: Paul Phrampus, MD, FACEP, is a consultant for Karl Storz Endoskopy, is a Principle for Pittsburgh Simulation Strategies, Inc, and is the Chief Medical Officer of SimMedical Inc. (a Company of UPMC). Benjamin Berg, MD is on the CME/CNE Speaker’s Bureau for Laerdal Medical. Michael DeVita, MD, receives royalties from the sales of the Rapid Response System training course, and receives other support from SimMedical and Laerdal.
1022 Validity of Serious Games for Medical Education and Surgical Skills Training
Maurits Graafland, MD1
1SURGERY, ACADEMIC MEDICAL CENTRE, AMSTERDAM, NLD
Introduction/Background: The application of digital games as a training modality for medical professionals is on the rise. These so-called serious games form training tools that provide a challenging simulated environment, ideal for future medical and surgical training. Ultimately, serious games are directed at reducing medical error and subsequent healthcare costs. If training and testing of health care professionals such as surgical trainees is to be carried out in digital game-based environments, strict requirements should be met. Use of these games and interpretation of underlying game-metrics must be reliable, valid and cause-specific to support their introduction in current residency teaching curricula. This systematic review aims to give an overview of current serious games for training medical professionals and to scrutinize validity testing. The hypothesis was that serious games directed at training medical professionals to this date did not undergo complete validation processes in accordance to current standards for novel educational instruments in healthcare.
Methods: A systematic literature review study was conducted in PubMed, Embase, The Cochrane Database of Systematic Reviews, PsychInfo and CINAHL databases and references for available studies up until April 2012. Studies were selected on predefined inclusion criteria, including the description of a serious game used to educate professionals in medicine. Data on the type of game and the validity study were extracted. The primary endpoint was achievement of steps in a formal validation process, according to current standards of validation.
Results: A total of 25 articles were found relevant, describing a total of 30 serious games. These were divided into two categories: games developed for educational purposes (n=17, Table 1) or games associated with, but not specifically developed for, improvement of skills relevant to medical personnel (n=13). Six serious games with a process of validation were identified. Of these six games, three games were developed for team training in critical care and triage, and three games were commercially available entertainment games applied to train laparoscopic psychomotor skills. None of the serious games completed a full validation process for the purpose of use. See Table 1 (Overview of serious games specifically developed for educational purpose, ranked according to purpose).
Conclusion: Blended and interactive learning by means of serious games may be applied to train both technical and non-technical skills relevant to the medical and surgical fields. Games developed for or used to train medical professionals need thorough scrutinizing on their validity, before integration into medical and surgical teaching curricula will be justifiable.
Disclosures: Maurits Graafland, MD, receives grant support, PID 101060 from Agentschap NL, funding agency of Dutch Ministry of Economic Affairs.
1027 Camera Navigation Training for Key User Groups in Laparoscopic Surgery: A Multi-center Prospective Cohort Study
Maurits Graafland, MD1
1SURGERY, ACADEMIC MEDICAL CENTRE, AMSTERDAM, NLD
Introduction/Background: Untrained laparoscopic camera-assistants in minimally invasive surgery (MIS) may cause decreased range of vision of the operating field, inducing risk for preventable error.1 Camera navigation is often performed by the least experienced member of the operating team, including inexperienced surgical residents, OR nurses and medical students. The latter two groups are currently not participating in structured laparoscopic training programs. A new virtual reality simulator module was specifically developed for laparoscopic camera navigation (LCN) among the above mentioned key user groups. The objective of this study was to assess face and construct validity of the LCN module. The first hypothesis was that expert laparoscopists and trainees would give a uniform and positive evaluation on the instruments’ resemblance to reality and usability, indicating face validity. The second hypothesis was that groups with a more laparoscopic experience would perform better on task time and movement proficiency than groups with less experience, indicating construct validity.
Methods: A multi-center prospective cohort study was conducted among 60 users from key user groups, consisting of surgical specialists, surgical residents, OR nurses and medical students. The first hypothesis was assessed through a questionnaire on resemblance to reality and perceived usability of the instrument among experts and trainees. The second hypothesis was assessed by comparing the scores of four groups with different levels of experience, including experts (>100 laparoscopic procedures as first operator), intermediate (<100 laparoscopic procedures as first operator), LCN experience only, and novice (no experience) during three repetitions of the LCN module. Scores were compared for the instruments’ outcome parameters time, camera stability, collisions and path length, using non-parametric statistical tests.
Results: Results show a uniform and positive evaluation of the LCN module among expert users and trainees, indicating face validity. Expert and intermediate groups performed significantly better in task time and camera stability during three repetitions, compared to less experienced user groups (P < 0.01, Figure 1). The differences between expert and intermediate groups on these outcome parameters were not significant, as were the differences in performances between all groups for the parameters collisions and path length. Comparison of learning curves showed significant improvement of proficiency in time and camera stability for all groups during three repetitions (P<0.02). See Figure 1. (Scores of the Novice, Camera Navigation only, Intermediate and Expert groups during three repetitions. (A) Task Time; (B) Camera stability (reversals); (C) Collisions; (D) Path length grasper).
Conclusion: Results from this study show face- and construct validity of the LCN module on the Simendo™ VR simulator. Results show that novice users and users with LCN experience only, are likely to benefit most from this training instrument. This module could therefore be implemented in training curricula for OR nurses and novice surgical trainees to improve team performance in MIS.
1. Gallagher AG, Al-Akash M, Seymour NE, Satava RM. An ergonomic analysis of the effects of camera rotation on laparoscopic performance. Surg Endosc 2009; 23(12):2684-2691.
Disclosures: Maurits Graafland, MD, receives grant support, PID 101060 from Agentschap NL, funding agency of Dutch Ministry of Economic Affairs.
1051 Continuing Education Curriculum Implementation of Dental Crisis Resource Management in Community Based Dental Practitioners
Niyati Mehta, DDS3, Shelley Miyasaki, DDS, PhD2, Richard Fidler, CRNA, CRNP, MSN, MBA1, and Mehran Hossaini, DMD2
1HEALTHCARE SIMULATION PROGRAMS, SAN FRANCISCO VETERANS AFFAIRS MEDICAL CENTER, SAN FRANCISCO, CA, USA and 2ORAL AND MAXILLOFACIAL SURGERY, UNIVERSITY OF CALIFORNIA SAN FRANCISCO, SAN FRANCISCO, CA, USA and 3DENTAL SERVICE, VETERANS AFFAIRS MEDICAL CENTER SAN FRANCISCO, SAN FRANCISCO, CA, USA
Introduction/Background: Nearly 44% of dentists encounter medical emergencies in community based office settings every year.1 The most common emergency encountered in a community based dental practice is syncope, followed by allergic reactions, angina/MI, cardiac arrest, postural hypotension, seizures, bronchospasm and diabetic emergencies. A majority of community based dental practitioners feel incompetent to diagnose the cause of an emergent event and half feel unable to manage the patient in the initial minutes of the event.2 To prepare for these situations, dentists must augment their knowledge and skills in dealing with medical emergencies.3 The focus of this study demonstrates incorporating simulation in a large scale, budget-minded continuing education course, to train community based dental practitioners in managing medical emergencies using a team approach.5 The objective was to identify perceptions and knowledge of dentists after utilizing a combined didactic and simulation based approach to medical crisis management in a community based dental practice; and furthermore, the aim was to determine the efficacy of simulation in community based dental practitioners implemented through a continuing education course.
Methods: Previous years’ workshop feedback served to guide curricular needs from a practicing dentist and refine the delivery format for information and simulation. The CE course implemented a curriculum shortening the morning didactic portion followed by the afternoon expanded hands-on training while incorporating a team approach to medical emergency management. In-situ simulation was conducted in the dental operatory in the UCSF School of Dentistry. Interactive hands-on task-training was conducted for basic airway management skills to be incorporated into the simulation scenarios. Medium and high fidelity manikins were used to simulate common medical emergencies that occur in the dental office. Course objectives focused on improving knowledge and skills with medical emergency drugs, basic life support and airway management. Pre and post-training evaluations were utilized to assess the reception of the hands on training, knowledge, attitudes, and perceived skills of the participants.
Results: The perceived confidence level with identification/assessment of several areas important to medical emergency training was collected. The data was analyzed using a Mann-Whitney U test comparing pre- and post-survey data with a sample size of n=73 from pre-survey data and n=55 from the post-survey data. Participants were 51% male, mean age was 47 years, and professions were as follows: 39% dental specialists, 38% general dentists, 12% dental assistants, and 11% dental hygienists.
Conclusion: Consistent with prior work from Fast in 1986, our results demonstrate a fundamental lack of knowledge and skill identifying and managing medical emergencies. Statistically significant increases in perceived confidence occurred after participation in a simulation-based approach to medical crisis management. This data clearly demonstrates the value of this CE curriculum in managing medical office emergencies for our community based dental professionals. The infusion of scenario and task training simulation into a previously didactic only course has been very well received by participants with recommendations to increase the hands-on simulation component further. We successfully utilized scenario simulation and task training to convey, reinforce our learning objectives, and foster deliberate practice through repeated simulations.
1. Haas, D. Management of Medical Emergencies in the Dental Office: Conditions in Each Country, the Extent of Treatment by the Dentist. JDSA, 53:20-24, 2006.
2. Girdler, NM, Smith, DG. Prevalence of emergency events in British dental practice and emergency management skills of British dentists. Resuscitation. 1999 Jul; 41(2):159-67.
3. T.B. Fast, et.al. Emergency preparedness: a survey of dental practitioners. JADA, 112:499-501, 1986.
4. Effectiveness of Continuing Medical Education. Evid Rep Technol Assess. 2007 Jan;(149):1-69.
5. The utility of simulation in medical education: What is the evidence? Mt Sinai J Med 2009 Aug; 76(4):330-43.
1052 The Effect of Model Fidelity on Learning Outcomes of a Simulation-based Education Program for Central Venous Catheter Insertion
Emily Diederich, MD3, Sally Rigler, MD, MPH4, Jonathan Mahnken, PhD1, Lei Dong, MS1, Timothy Williamson, MD2, and Matthew Sharpe, MD3
1BIOSTATISTICS, UNIVERSITY OF KANSAS MEDICAL CENTER, KANSAS CITY, KS, USA and 2INTERNAL MEDICINE, PULMONARY and CRITICAL CARE, UNIVERSITY OF KANSAS MEDICAL CENTER, KANSAS CITY, KS, USA and 3PULMONARY and CRITICAL CARE, UNIVERSITY OF KANSAS MEDICAL CENTER, KANSAS CITY, KS, USA and 4MEDICINE, UNIVERSITY OF KANSAS SCHOOL OF MEDICINE, KANSAS CITY, KS, USA
Introduction/Background: Simulation-based education for central venous catheter (CVC) insertion has been repeatedly documented to improve trainee competence and patient outcomes.1,2 The vast majority of studies published to date rely on either partial task trainers or cadaveric models,3 which maximize physical realism but also present barriers of cost and hygiene, respectively. The purpose of this study was to examine the effectiveness of simulation-based education for CVC insertion using a commercially available, high fidelity model compared with an identical curriculum relying on a simply constructed, low fidelity model.
Methods: We conducted a non-inferiority design, randomized controlled trial. Forty consecutive residents rotating through the medical intensive care unit of an academic medical center completed a simulation-based education program for CVC insertion. The curriculum was designed in accordance with the principles of deliberate practice and mastery learning. Each resident underwent baseline skills testing, and was then randomized to complete the educational intervention on either a commercially available, high fidelity CVC model (Simulab’s CentraLineMan) or a low fidelity model. The low fidelity model was assembled by allowing a gelatin mix to set in a rectangular container with two embedded surgical drains serving as blood vessels. The cost of materials for this model was $8.12, as compared to the cost of the commercial model, $1,340. Residents received one-on-one instruction until they were able to insert the CVC on their respective model while achieving a previously determined minimum passing score on a 26-item checklist. Upon completion of their ICU rotation, residents returned for repeat skills testing on the high fidelity model. The mean post-training checklist scores on the 26-item checklist were compared between low and high fidelity model groups.
Results: Post-graduate year, residency program and pre-training checklist scores were similar between low and high fidelity model groups. Mean post-training score on the 26-item checklist for the low fidelity model group was 23.8 ± 2.2 (91.5%), and was not inferior to the mean score of the high fidelity model group of 22.5 ± 2.6 (86.5%) (p<0.0001). Residents in both groups judged the training program to be highly useful despite perceiving a lesser degree of physical realism in the low fidelity model compared with the high fidelity model (p=0.05).
Conclusion: A simulation-based education program for CVC insertion utilizing an inexpensive, low fidelity model achieved learning outcomes comparable to those accomplished with a commercially available, high fidelity model. This research suggests that desirable learning outcomes can be achieved for CVC insertion using low fidelity simulation equipment provided robust educational principles are applied to curricular design.
1. Barsuk, JH, McGaghie, WC, Cohen, ER, O’Leary, KJ, Wayne, DB. Simulation-based mastery learning reduces complications during central venous catheter insertion in a medical intensive care unit. Crit Care Med 2009; 37: 2697-2701.
2. Evans LV, Dodge KL, Shah TD, Kaplan LJ, Siegel MD, Moore CL, Hamann CJ, Lin Z, D’Onofrio G. Simulation training in central venous catheter insertion: improved performance in clinical practice. Acad Med. 2010; 85: 1462-1469.
3. Ma IW, Brindle ME, Ronksley PE, Lorenzetti DL, Sauve RS, Ghali WA. Use of simulation-based education to improve outcomes of central venous catheterization: a systemic review and meta-analysis. Acad Med. 2011; 86: 1137-47.
Disclosures: Timothy Williamson, MD, receives grant support from United Therapeutics, Actelion, and Gilead (all pulmonary hypertension pharmaceutical companies).
1062 Identifying Performance Gaps in Child Abuse Assessments: A Pilot Study
Jim Anderst, MD, MSCI4, Chris Kennedy, MD1, Serkan Toy, PhD3, and Monica Nielsen-Parker, MSW2
1CHILDREN’S MERCY HOSPITAL AND CLINICS, KANSAS CITY, MO, USA and 2CHILD ABUSE AND NEGLECT, CHILDREN’S MERCY HOSPITAL and CLINICS, KANSAS CITY, MO, USA and 3MEDICAL EDUCATION, CHILDREN’S MERCY HOSPITAL and CLINICS, KANSAS CITY, MO, USA and 4PEDIATRICS, CHILDREN’S MERCY HOSPITAL and CLINICS, KANSAS CITY, MO, USA
Introduction/Background: Pediatric medical providers regularly evaluate cases of possible child abuse. Physicians may lack confidence and knowledge in identifying and managing child abuse.1-3 These shortcomings have been attributed to inadequate training4-7 and may result in misdiagnoses regarding abuse.8,9 Due to the complex, team-based nature of child abuse assessments, and the life-altering consequences of errors, a competency-based approach to training may be beneficial. High-fidelity simulated exercises with mannequins and standardized parent actors offer the opportunity to simultaneously teach and assess diagnostic ability.10 We hypothesized that practitioners would be more likely to make the correct diagnosis in cases of abuse versus accidental injury. We also hypothesized that a training program will result in gains in knowledge and attitudes of participants.
Methods: Physicians and nurses from a tertiary Pediatric Emergency Medicine Department volunteered for an IRB approved one-hour exercise using standardized parents actors and a high-fidelity simulated child. Participants were presented with one of four randomly assigned scenarios based on social risk factors for child abuse and nature of injury: low risk-abuse (LR-AB), low risk-accident (LR-AC), high risk-abuse (HR-AB), and high risk-accident (HR-AC). Scenarios included one of two child injuries that may occur via accidental or abusive means. All elements necessary to determine the correct medical diagnosis regarding abuse were available to participants. The primary outcome variable was proportion of correct medical diagnoses regarding abuse based on case type (accident vs. abuse). Additional assessments included pre-post changes in self-efficacy scales and multiple choice tests scores, and proportion of forensic injury history (FIH) tasks completed, appropriate tests ordered, and mean Likert scores (MLS) for professionalism, communication, and documentation. Regression analysis was used to evaluate the association between case type (AB or AC) and correct diagnosis and to identify predictors of correct diagnosis. Pre-post attitude and knowledge changes were evaluated using the Wilcoxon test.
Results: Thirty-three physician/nurse teams participated. A correct medical diagnosis regarding abuse was over 16 times more likely to occur in abuse cases than in accident cases (OR 16.5, 95% CI 2.68-101.33). Case type (AB or AC) (p=0.004) and provider pre-self assessment score (p=0.033) were the only variables found to be independently associated with correct diagnosis. FIH and proportion of correct tests ordered were not found to be associated with correct diagnosis. Secondary outcomes are in Tables 1 and 2:
Conclusion: A hybrid simulation program can identify gaps, characterize the overall quality of evaluation, and improve immediate attitudes and knowledge in the team based assessment of child abuse. A gap was identified in the recognition of true accidental injuries. The foundation of a forensic injury evaluation is the FIH and the ordering of appropriate medical tests.The lack of association between these elements and correct diagnosis suggests that physicians may not have appropriate diagnostic approaches in cases of possible abuse. The strong association between case type (AB or AC) and correct diagnosis indicates that pre-conceived notions about a case of possible abuse may be the driving factor in physician end diagnosis.
1. Bannon MJ, Carter YH. Pediatricians and child protection: the need for effective education and training. Arch Dis Child 2003;88: 560-2.
2. Shabde N. Child protection training for pediatricians. Arch Dis Child 2006; 91: 639-41.
3. Narayan AP, Socolar RS, St. Claire K. Pediatric residency training in child abuse and neglect in the United States. Pediatrics. 2006;117(6):2215-2221.
4. Dubowitz H. Child abuse programs and pediatric residency training. Pediatrics. 1988;82:477-480.
5. Ward MG, Bennett S, Plint AC, King WJ, Jabbour M, Gaboury I. Child protection: a neglected area of pediatric residency training. Child Abuse & Neglect. 2004;28:1113-1122.
6. Giardino AP, Brayden RM, Sugarman JM. Residency training in child sexual abuse evaluation. Child Abuse & Neglect. 1998;22:331-336.
7. Flaherty EG, Sege R, Hurley TP, Baker A. Strategies for saving and improving children’s lives. Pediatrics. 2008;122(Supplement 1):S18-S20.
8. Jenny C, Hymel KP, Ritzen A, Reinert SE, Hay TC. Analysis of missed cases of abusive head trauma. Journal of the American Medical Association. 1999;281:621-626.
9. Anderst JD, Kellogg N, Jung I. Is the diagnosis of child physical abuse changed when child protective services consults a child abuse pediatrics subspecialty group as a second opinion? Child Abuse & Neglect. 2009;33:481-489.
10. Ellaway RH, Kneebone R, Lachapelle K, Topps D. Practica continua: connecting and combining simulation modalities for integrated teaching, learning and assessment. Medical Teacher. 2009;31:725-731.
1065 Initial Validation of the Internal Medicine Simulated Practical Exam (IMSPE) for Third Year Medical Students
Moshe Feldman, PhD2, Gonzalo Bearman, MD, MPH, FACP1, Jeffrey Kushinka, MD3, Cheryl Bodamer, PhD, MPH, RN2, Ellen Brock, MD, MPH2, and Alan Dow, MD, MSHA1
1VIRGINIA COMMONWEALTH UNIVERSITY SCHOOL OF MEDICINE, RICHMOND, VA, USA and 2CENTER FOR HUMAN SIMULATION AND PATIENT SAFETY, VIRGINIA COMMONWEALTH UNIVERSITY SCHOOL OF MEDICINE, RICHMOND, VA, USA and 3INTERNAL MEDICINE, VIRGINIA COMMONWEALTH UNIVERSITY SCHOOL OF MEDICINE, RICHMOND, VA, USA
Introduction/Background: Medical student competency in Patient Care is generally evaluated by ratings from faculty and supervising housestaff.1 These ratings are often unreliable due to inconsistent assessment contexts and inadequate faculty training, even when using tools such as the mini-CEX.2 Simulation could provide a more standardized competency-based assessment of Patient Care by allowing students to demonstrate a level of competency through real-time management of standardized scenarios with consistent assessment processes and without threatening patient safety. To test this idea, we developed an Internal Medicine Simulated Practical Exam (IMSPE) to assess competency in Patient Care including medical knowledge, diagnostic skills, and clinical management skills. We hypothesize that the use of a robust simulation based exam tool will allow for a more reliable assessment of Patient Care that adds value to existing assessments.
Methods: A prospective study design was used to evaluate initial validity evidence for the IMSPE, which consists of a 52-item photo quiz, a 10-item evaluation using the Harvey cardiopulmonary simulator, and a 16-item evaluation using the iSTAN high fidelity simulator. The photo quiz required students to demonstrate medical knowledge and diagnostic skills by interpreting X-rays, EKGs and diagnosing common dermatologic and ophthalmologic conditions. The Harvey simulator component tested students’ auscultation skills for common cardiopulmonary conditions. The iSTAN simulator component provided a standardized, high fidelity, real-time assessment of common, acute medical conditions including asthma, sepsis, and myocardial infarction requiring participants to interpret clinical findings, determine diagnoses, and initiate an evaluation and management plan. All assessment items were multiple choice questions with responses entered electronically. Performance on the IMSPE was compared to student performance on other existing assessments including the written Shelf exam, a quiz that approximates the Shelf exam, Step 2 scores, clinical evaluations, OSCE scores, and overall clerkship grades. Students completed an evaluation of the learning experience after all exam responses were submitted.
Results: Seventy-nine 3rd year medical student volunteers participated. The IMSPE total mean score was 75.34 (SD = 7.37). IMSPE performance was positively correlated with written exam scores (r = .48; p < .01), text based equiz scores (r = .34; p < .01), and final clerkship grades (r = .37; p < .05). In addition, Harvey component scores were positively correlated with Step 2 scores (r = .61; p < .05). IMSPE scores were not significantly correlated with Ward scores or OSCE scores. A logistic regression analysis indicated that higher IMSPE scores were associated with an increased probability of an honors compared to a high pass or pass final grade (OR = .89; p < .05). Approximately 80% of participants agreed that simulation was better for assessing medical knowledge of cardiac murmurs, disease states, image recognition, and interpretation of physical exam findings compared to other assessments not using simulation. Ninety percent reported this was a fair exam.
Conclusion: Our results provided some validity evidence for the IMSPE as an additional assessment tool for evaluating competency in Patient Care. IMSPE scores were moderately correlated with other measures of medical knowledge, providing convergent validity evidence, and were uncorrelated with assessments related to interpersonal and communication skills such as the OSCE, providing discriminant validity evidence. In addition, correlations and logistic regression analysis supported the added value of IMSPE scores in estimating students’ final grades. Students perceived the simulated portion of the exam favorably for assessing skills related to competency in patient care. Comments reflected some technological limitations related to interpreting cardiopulmonary findings on the high fidelity mannequin and that greater time was needed per question. Future work will focus on refining the IMSPE and establishing additional evidence of the IMSPE’s utility as a valid and reliable summative assessment tool for competency in patient care.
1. Herbers, JE Jr; Noel, GL; Cooper, GS; Harvey, J; Pangaro, LN; Weaver, MJ. (1989) How accurate are faculty evaluations of clinical competence? Journal of General Internal Medicine 4(3):202-8.
2. Cook DA, Dupras DM, Beckman TJ, Thomas KG, Pankratz VS. Effect of rater training on reliability and accuracy of mini-CEX scores: A randomized, controlled trial. Journal of Internal Medicine. 2008; 24(1): 74-79.
Disclosures: Alan Dow, MD, MSHA, receives grant support from the Josiah H. Macy, Jr Foundation and the Donald W. Reynolds Foundation, and is a consultant for The Frontier Project, LLC.
1089 The Use of Web-based Observational Practice and Educational Networking Improves Simulation-based Education and Training of Central Venous Catheterization: A Pilot Study
Jeffrey Cheung, MSc (c)4, Jansen Koh, MD2, Kim MacKinnon, PhD5, Clare Brett, PhD1, Darius Bagli, MDCM3, and Adam Dubrowski, PhD4
1CURRICULUM, TEACHING, AND LEARNING, ONTARIO INSTITUTE FOR STUDIES IN EDUCATION, UNIVERSITY OF TORONTO, TORONTO, ON, CAN and 2SICKKIDS LEARNING INSTITUTE, UNIVERSITY OF TORONTO, HOSPITAL FOR SICK CHILDREN, TORONTO, ON, CAN and 3SURGERY, UNIVERSITY OF TORONTO, HOSPITAL FOR SICK CHILDREN, TORONTO, ON, CAN and 4THE WILSON CENTRE, SICKKIDS LEARNING INSTITUTE, UNIVERSITY OF TORONTO, HOSPITAL FOR SICK CHILDREN, TORONTO, ON, CAN and 5CURRICULUM, TEACHING, AND LEARNING, UNIVERSITY OF TORONTO, ONTARIO INSTITUTE FOR STUDIES IN EDUCATION, TORONTO, ON, CAN
Introduction/Background: Competency in Central Venous Catheterization (CVC) is an essential objective in several residency training programs. Simulation-based education and training (SBET) for CVC has been shown to improve performance.1 However, SBET for CVC can be a costly undertaking in terms of resources and faculty time. Exploring novel ways to improve the efficiency of SBET for CVC is important. Overcoming barriers of distance, time, and economies of scale, Web-based Learning (WBL) is a potential method of improving SBET for CVC. This pilot study sought to determine the feasibility of using preparatory Web-based Observational Practice and Education Networking (OPEN) to facilitate learning in SBET of CVC in novice learners. We hypothesized that preparatory WBL with OPEN would improve the efficiency of SBET for CVC.
Methods: Upon institutional ethical approval, fifteen medical students enrolled for SBET on CVC were randomized into 1 of the 3 treatment arms of the study, each differing in the type of preparatory material received on CVC: Traditional Reading Material (TM), Observational Practice (OP) and Observational Practice with Education Networking (OPEN). All materials were made available online through email, for the TM group, or a custom-built WBL course for the OP and OPEN groups (Figure 1). Students attended 2 sessions in our simulation laboratory, a preparatory session and a SBET workshop within a two-week interval. For the first session, all students were given their respective preparatory material, with a five minute session for the OP and OPEN groups on familiarization of the website. All students were subsequently left on their own to review the materials. The SBET workshop comprised of a video lesson on CVC and a live demonstration followed by deliberate practice trials and individualized feedback on a CVC part task trainer. Each trial was scored on a modified validated checklist2 (total achievable score of 28). Participants had to achieve a perfect score in 2 consecutive trials in order to complete the workshop. A blinded educator experienced in SBET provided feedback. The following data were collected for each participant: number of trials to achieve competency, checklist score on the first trial, total time required for deliberate practice, and the total time required for the entire workshop.
Results: The average number of trials to achieve competency in CVC for TM, OP and OPEN groups was 6, 4.3 and 4 trials respectively. The average checklist score for TM, OP and OPEN groups on their first trial was 14, 18 and 21 respectively. The average total time score for all trials for TM, OP and OPEN groups were 111, 77 and 48 minutes respectively. The average total time required for completing the workshop for each participant in the TM, OP and OPN groups was 90, 82 and 70 minutes respectively. Based on these averages and variability, sample size calculation showed that we will require at least 9 students in each group to reach statistical significance.
Conclusion: This study demonstrated a consistent trend: those with preparation through OPEN outperformed those in the OP group, who in turn outperformed those in the TM group. A reduced number of attempts to achieve competency, improved checklist scores on first trials, decreased times to reach competency through deliberate practice, and decreased overall SBET workshop time were associated with OPEN and OP WBL. Though underpowered, this trend merits a future RCT with at least 9 participants per group and refinement of the WBL intervention. Preparation through Web-based OPEN has the potential to improve the efficiency of SBET of CVC.
1. Use of Simulation-Based Education to Improve Outcomes of Central Venous Catheterization: A Systematic Review and Meta-Analysis. Irene W.Y.Ma, Mary E. Brindle, Paul E. Ronksley, Daine L. Lorenzetti, Reg S. Sauve, William A. Ghali. Acad Med. Sept 2011; 86(9): 1137-1147.
2. Use of Simulation-Based Mastery Learning to Improve the Quality of Central Venous Catheter Placement in a Medical Intensive Care Unit. Jeffrey H. Barsuk, William C. McGaghie, Elaine R. Cohen, Jayshankar S. Balachandran, Diana B. Wayne. J of Hosp Med. Sept 2009; 4(7): 397-403.
1097 Does a Female Patient Simulator Decrease the Quality of Resuscitation Care Delivered by Bystanders?
Chelsea Kramer, MA1, Nicole Percival, MSc, BSc3, Matthew Wilkins4, and Jeff Caird, PhD2
1HUMAN FACTORS, CAE PROFESSIONAL SERVICES, OTTAWA, ON, CAN and 2HUMAN FACTORS, UNIVERSITY OF CALGARY, CALGARY, AB, CAN and 3HUMAN FACTORS, RESEARCH AND INNOVATION CENTRE, UNIVERSITY OF CALGARY, CALGARY, AB, CAN and 4PSYCHOLOGY, UNIVERSITY OF CALGARY, CALGARY, AB, CAN
Introduction/Background: The purpose of this study was to determine the effect that a female patient had on the quality of resuscitation care during a sudden cardiac arrest (SCA). Previous studies have not examined the influence of patient sex on the ability of laypersons to effectively use an automated external defibrillator (AED). Female simulators, with the exception of birthing simulators, tend to be less widely available, although one-half of the population is female. A variety of social norms, including gender interaction, was expected to inhibit the removal of clothing by laypersons using an AED on a female patient in cardiac arrest.
Methods: A training version of a Medtronic LifePak CR+ AED was used on a METI iStan during a simulated SCAscenari.1 iStan was modified into a woman (i.e., iSam) using a wig, clothing, make-up and breasts (see Figure 1). Ceiling mounted directional microphones and video cameras recorded participant interactions with equipment during the SCA scenario. See Figure 1. The METI iStan was modified into an older woman (i.e., iSam). The common incorrect AED pad placement is shown with the bra partially removed. The Biomedical Ethics Committee certified the study and 30 participants volunteered. The 17 females and 13 males were undergraduate students (Mean age = 23.8 years). Ten participants had basic CPR-only training in the past five years. Participants were told that effective and timely use of the AED was necessary to save iSam’s life and to follow all directions provided by the AED. The experimenter left the room under the premise that they were calling for an ambulance.
Results: The video and audio from the SCA scenario was recorded by Noldus Recorder, coded with Noldus Observer XT (v. 9.0) and analyzed using Predictive Analytic Software (SPSS, v. 18). The Medtronic LifePak CR+ AED verbally prompts participants to remove all clothing from the patient’s chest. This step ensures direct pad application to bare skin, obstructions to pad placement can be addressed (i.e., medication patches, pacemakers) and issues resulting from clothing conductivity (i.e., underwire bras) can be eliminated. Only 40% of participants removed both the shirt and bra from the female patient’s chest. In comparison, previous studies with iStan reported 100% clothing removal.1 Of particular interest, only one male removed the bra from the chest of the female patient. When interviewed on their decision not to remove the bra, participants most often responded that they only needed to remove enough clothing to place the pads. Concerned for patient modesty, they did not want to remove more clothing than necessary. Underwire conductivity was not taken into account.
Conclusion: The use of a female patient simulator was shown to limit the removal of clothing from a patient during a SCA scenario. Observations of participant video and audio data indicate that the placement of the pads on the female patient are farther apart, which may limit the detection of ventricular-tachycardia and ventricular-fibrillation and the delivery of an effective shock. From a safety standpoint, the underwire in bras, if left on during shock, may cause burns to the patient. The results of this study may be optimistic. Bystander interaction with actual female patients during cardiac arrest may produce even less clothing removal and potentially longer shock delivery times. The assumption that only male patient simulators are necessary to train and understand the behavior of caregivers in a range of social contexts should be questioned. Current female simulator add-ons (e.g., chest plates, genitalia) lack realism. Particularly troubling is the expectation that parts can be interchanged on the male frame to adequately represent both genders. Simulator manufacturers should consider expanding their limited female mannequin products.
1. Percival N, Caird, JK. Layperson and nurse performance using two automated external defibrillators. Healthc Syst Ergon & Patient Saf.Boca Raton, FL: CRC Press; 2011.241–244 pp.
Disclosures: Chelsea Kramer, MA is a consultant for CAE Professional Services.
1099 Evaluation of Differences in Care Provided During a Novel, Thematically Paired Simulation Assessment Between Adult and Pediatric Populations
Yuemi An-Grogan, MD2, David Salzman, MD1, Umakanth Avula, MS, BS2, and John Vozenilek, MD, FACEP2
1EMERGENCY MEDICINE, NORTHWESTERN UNIVERSITY, FEINBERG SCHOOL OF MEDICINE, CHICAGO, IL, USA and 2SIMULATION TECHNOLOGY AND IMMERSIVE LEARNING, NORTHWESTERN UNIVERSITY, FEINBERG SCHOOL OF MEDICINE, CHICAGO, IL, USA
Introduction/Background: Pediatric emergency medicine (PEM) has grown through research and specialized curricula. However, despite ongoing efforts of post-graduate training programs and certification boards to educate providers, both trainees and board-certified physicians in EM often express discomfort and exposure deficits to pediatric emergency cases and pediatric critical care. According to Langhan et al, in a study surveying board certified EM physicians, participants reported that while 76% and 60% were completely prepared to handle adult CPR and adult trauma, respectively, only 24% and 27% were prepared to handle pediatric CPR and pediatric trauma resuscitations. However, the majority of pediatric patients are treated in general emergency departments. Pediatric critical cases are so rare that other methods of training, such as simulation, can be utilized to improve provider skills and comfort levels, and ultimately, patient outcomes. The purpose of our study was to evaluate for existing differences in delivery of care when an EM provider is presented with two similar simulated patient care encounters related to toxicological emergencies. We hypothesized that there are objective differences in care provided to adult and pediatric populations.
Methods: In this IRB approved, randomized, cross-over study occurring in our simulation center, all participants managed a series of cases individually. Half the participants were exposed to an adult case first (Digoxin overdose) while the other half engaged in a pediatric case first (Beta-blocker overdose). Both scenarios started with a nearly identical stem and followed a similar pre-determined course where the patient presented altered, with hypotension and bradycardia, and required similar management by the provider. Participants were blinded to the cases prior to the start of the session. Participants were evaluated using a checklist consisting of various tasks divided into major categories: history of present illness (HPI), physical exam, initial management, data acquisition, treatment and disposition. In addition, all participants completed pre and post-session questionnaires indicating self-assessments of skill and patient age group preference. Participant’s performance was evaluated using Sign rank and McNemar’s tests. Time-to-event analysis on critical actions was done using Kaplan-Meier principles. Lastly, subjective preference was compared to objective performance.
Results: The effect of case order on resident performance was analyzed and there was no significant difference found between groups. Subgroup analysis showed that HPI and disposition scores were better in the pediatric case, but treatment scores were better in the adult case. Five critical actions including ordering an EKG, administering atropine, voicing a diagnosis, checking a bedside glucose and ordering a chemistry panel showed time-to-performance differences and all five actions were performed earlier in the adult case (p<0.05). In both pre and post-session surveys, 75% and 96.9% (respectively) of the residents stated they were more comfortable in treating adult toxicity cases, but only about 34.3% actually performed better in the adult case (p<0.05). Only 9.4% of residents stated they were equally comfortable managing pediatric and adult toxicities.
Conclusion: This study identifies a discrepancy in management provided between pediatric and adult simulated toxicological scenarios by EM residents at a single institution. To our knowledge this is the first study directly evaluating for differences in emergency care provided between both populations. These initial findings merit further investigation with regards to generalizability to other institutions and practicing physicians. Additionally, this can serve as the first step in a needs assessment to identify gaps in curricular design and practice implementation for residency programs, as well as future maintenance of certification programs.
1. Bourgeois FT, Shannon MW. Emergency care for children in pediatric and general emergency departments. Pediatr Emerg Care. 2007; 23(2): 94-102.
2. Langhan M, Keshavarz R, Richardson LD. How comfortable are emergency physicians with pediatric patients? J Emerg Med. 2004; 26(4): 465-9.
1110 Do Role-specific Colored Vests on Code Team Members Improve CPR Performance During Simulated Emergencies?
Ruth Thomas, MSN4, Jill Sanko, MS, ARNP-BC1, Beatriz Valdes, MSN, MBA2, Guillermo Valdes, DNP, MSN-HCS, RN-BC2, Ilya Shekhter, MS, MBA5, and Lisa Rosen, MA3
1MIAMI, FL, USA and 2NURSING EDUCATION, JACKSON HEALTH SYSTEM, MIAMI, FL, USA and 3CENTER FOR PATIENT SAFETY, UNIVERSITY OF MIAMI, MIAMI, FL, USA and 4ANESTHESIOLOGY, UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE, MIAMI, FL, USA and 5CENTER FOR PATIENT SAFETY, UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE, MIAMI, FL, USA
Introduction/Background: Sudden cardiac arrest (SCA) is the leading cause of death among adults in the United States with approximately 325,000 fatalities each year.1 Although cardiopulmonary resuscitation (CPR) is a simple lifesaving technique to revive SCA victims, survival outcomes continue to remain low for in-hospital arrests. Correct CPR performance has been difficult to achieve and measure due to the emergent nature of the situation and chaos at the scene.2 However, the use of simulation technology to re-create emergency situations, such as SCA, allows investigators to identify and correct poor CPR performance measures without patient harm. Because survival rates decrease drastically with each passing minute after an SCA event, we have focused on the timeliness of resuscitation measures in simulated codes. We hypothesized that when providers wear colored vests (See Figure 1) identifying specific roles during the performance of CPR, time required to implement four primary emergency measures (ventilation, compressions, medication administration, and defibrillation) will be reduced and team performance in simulation improved.
Methods: Two-hundred and sixty nurses participated in a study conducted in a simulation setting using a SimMan or Resusci-Anne CPR-D (mannequin) over an eight-month period. Twenty groups of nurses participated in each arm (control/intervention), and each group consisted of five to eight team members. The intervention group wore different colored vests for role assignments (team leader, ventilation, compression, medication administration, defibrillation, IV nurse, recorder, and circulator), and the roles were pre-assigned. A five-minute scenario assessed the timeliness of the team’s performance of the following resuscitation measures: two to five cycles of compressions and ventilation, placement of the defibrillator pads and electrical shock, and the administration of the first round of resuscitation medications as recommended by the American Heart Association (AHA) 2010 guidelines. The Ottawa Checklist was used to measure team performance in the following five categories: problem solving, situational awareness, leadership, resource utilization, and communication.3
Results: A two-sample t-test was used to compare time to implement emergency measures and team performance scores among the two groups. Groups wearing colored vests were faster in starting compressions, ventilations, and defibrillation (See Table 1). There was no significant difference in the timeliness of medication administration between the groups. Also, the overall total scores in team performance were significantly better in the intervention group.
Conclusion: Our results suggest that delays in initiating CPR measures may be associated with the number of nurses available during a code and unclear role assignments for the various tasks. Physicians and respiratory therapists usually perform specific roles (team leader and airway manager) during a code while nurses have to assume multiple roles (give medications, set up and run the defibrillator, record code events, and start IVs). Since the colored vests are pre-assigned there are no delays in initiating specific tasks because nurses do not have to figure out their role. Therefore, a simple technique such as role assignment before a code may dramatically improve the coordination of the CPR team, provide faster start times, and lead to better in-hospital SCA survival outcomes. The use of colored vests by nurses during CPR may also help achieve AHA’s Emergency Cardiovascular Care 2020 goals of improving SCA survival outcomes.
1. American Heart Association Lean and Live: Heart Disease and Stroke statistics 2010 update at a glance American Heart Association.
2. Abella Bs, Sandbo N, Vassilatos P, Alvarado Jp, O’Hearn N, Wigder HN, Hoffman P, Tynus K, Vanden Hoek TL, Becker LB. Chest compression rates during cardiopulmonary resuscitation Are suboptimal a prospective study during in-hospital cardiac arrest Circulation 2005 Feb 1;111(4):428-34. UI: 15687130.
3. Kim J, Neilipovitz D, Cardinal P, and Chiu M. A comparison of global rating scale and checklist scores in the validation of an evaluation tool to assess performance in the resuscitation of critically ill patients during simulated emergencies (Abbreviated as “CRM Simulator Study IB”) Simulation in Healthcare: The Journal of The Society for Medical Simulation. Spring 2009 4(1):6-16. UI: 19212245.
1118 Using a Spatial Task to Measure Laparoscopic Mental Workload: Initial Results
Erik Prytz, MS4, Michael Montano, MS4, Rebecca Kennedy, MS4, Mark Scerbo, PhD4, Rebecca Britt, MD3, Stephen Davis, MD2, and Dimitrios Stefanidis, MD, PhD, FACS1
1SURGERY, CAROLINAS MEDICAL CENTER, CHARLOTTE, NC, USA and 2OBSTETRICS and GYNECOLOGY, EASTERN VIRGINIA MEDICAL SCHOOL, NORFOLK, VA, USA and 3SURGERY, EASTERN VIRGINIA MEDICAL SCHOOL, NORFOLK, VA, USA and 4PSYCHOLOGY, OLD DOMINION UNIVERSITY, NORFOLK, VA, USA
Introduction/Background: Performing laparoscopic surgery is more mentally demanding than traditional surgery in part because surgeons must operate in three-dimensional space while viewing a two-dimensional display. Consequently, laparoscopy places significant demands on visual attention and requires a great deal of practice to achieve proficiency. Presently, there is no standard method to measure the mental demands imposed by laparoscopic surgery. However, Stefanidis and his colleagues have used the secondary task technique to assess mental workload.1,2 According to Wickens’ multiple resource theory, pools of attentional resources are distinguished by three dimensions: processing stages (perceptual/cognitive and response), processing codes (verbal and spatial), and processing modalities (auditory and vision) with the visual processing modality separated into focal and peripheral channels.3 Two tasks that draw upon the same pool of resources can interfere with one another and increase mental workload. Thus, a secondary task that competes for the same resources as a primary task should be sensitive to differences in mental workload. A secondary spatial task was developed that requires the same visual processing needed for judging the position of objects on a laparoscopic display. Thus, our objective was to determine if the new secondary task could distinguish differences in mental workload associated with several different tasks in an FLS laparoscopic box trainer.
Methods: Sixteen undergraduate students with no prior laparoscopic experience were recruited to participate in this IRB approved study. They were asked to perform three primary tasks on a box trainer: 1) tracing the outlines of images on a computerized drawing tablet using a stylus attached to a laparoscopic instrument, 2) the FLS peg transfer task, and 3) the FLS cutting task. The secondary task presented observers with images of four balls in a simulated tunnel, superimposed at 50% transparency over the primary task display to ensure that both tasks were viewed with focal vision. The images were presented for 300 msec every 4 seconds. On half of the presentations, one ball changed its position and participants had to verbally identify those shifts in position. Participants performed the secondary task by itself and in conjunction with the primary tasks. The experiment used a within-subjects design with four task conditions: tracing task, peg transfer task, cutting task, and a baseline measure of the secondary task by itself.
Results: The percent correct responses on the secondary task were analyzed with a repeated measures ANOVA and showed a significant main effect, F(3, 45) = 41.285, p < .001, partial η2= .747. Bonferroni-corrected post hoc tests showed that percent correct scores on the secondary task were lower when performed in conjunction with the tracing task (M =80%, SE = 2.5%), p < .001, the peg transfer task (M = 72.4%, SE = 3.3%), p < .001, and the cutting task (M = 62%, SE = 3.3%), p < .001, compared to performing the secondary task alone (M = 92.7%, SE = 2.0%). Further, secondary task scores were significantly lower on cutting than the peg transfer task, p = .007.
Conclusion: The results show that the spatial secondary task was indeed sensitive to the mental workload associated with the laparoscopic tasks. Scores on the secondary task declined when performed in conjunction with each laparoscopic task. Further, secondary task performance with the cutting task was poorer than with the peg task, suggesting that the cutting task is much more mentally demanding. These initial results show that the secondary task can provide an objective index of mental workload that can complement traditional metrics of speed and accuracy on laparoscopic tasks. Future research will use this task to identify laparoscopic conditions that deprive surgeons of the spare attentional resources needed to multi-task effectively in the OR.
1. Stefanidis, D., Scerbo, M.W., Korndorffer, J. R. Jr., & Scott, D.J. (2007). Redefining simulator proficiency using automaticity theory. The American Journal of Surgery, 193, 502-506.
2. Stefanidis, D., Scerbo, M.W., Smith, W., Acker, C.E., & Montero, P.N. (2012). Simulator training to automaticity leads to improved skill transfer compared with traditional proficiency-based training: A randomized controlled trial. Annals of Surgery, 255, 30-37.
3. Wickens, C. D. (2002). Multiple resources and performance prediction. Theoretical Issues in Ergonomic Science, 3, 159-177.
Disclosures: Mark Scerbo, PhD, receives grant support from SimQuest, LLC. Dimitrios Stefanidis, MD, PhD, FACS receives grant support from Ethicon Endosurgery, and is a consultant for Ethicon, Bard, and Gore.
1122 Simulated Ventricular Fibrillation in an Anesthetized Pediatric Patient
Nancy Tofil, MD, MEd5, Collin King, MD, MPH2, Jennifer Dollar, MD3, Seth Jarrell, MS1, Jerri Zinkan, MPH, RN5, Amber Youngblood, BSN, RN4, Dawn Taylor Peterson, PhD, EdS, MEd5, and Marjorie Lee White, MD, MPPM, MEd5
1UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 2PEDIATRICS, UNIVERSITY OF ALABAMA BIRMINGHAM, BIRMINGHAM, AL, USA and 3ANESTHESIOLOGY, UNIVERSITY OF ALABAMA BIRMINGHAM, CHILDREN’S HOSPITAL OF ALABAMA, BIRMINGHAM, AL, USA and 4NURSING EDUCATION, UNIVERSITY OF ALABAMA BIRMINGHAM, CHILDREN’S HOSPITAL OF ALABAMA, BIRMINGHAM, AL, USA and 5PEDIATRIC SIMULATION CENTER, UNIVERSITY OF ALABAMA BIRMINGHAM, CHILDREN’S HOSPITAL OF ALABAMA, BIRMINGHAM, AL, USA
Introduction/Background: Pediatric emergencies in anesthesia are rare, and therefore exposure to these critical events depends on the residency program.1 In the operating room, anesthesia residents are designated as the leader when a pediatric patient goes into cardiac arrest.2 Literature shows that simulation of critical intraoperative events improves performance of anesthesia residents in the attainment of event specific skills.3 The purpose of this study was to evaluate time to recognition of cardiac arrest in a pediatric prone patient and to expose all learners to the difficulties of managing emergencies in prone patients.
Methods: Simulation sessions were designed for anesthesia residents and were conducted monthly for a total of 13 months. Each team, consisting of one or two anesthesia residents, participated in a scripted scenario in which a patient undergoing posterior spinal fusion surgery experiences decreased blood pressure, metabolic acidosis, hypothermia, and hyperkalemia from blood products. The patient was prone due to the nature of the procedure, and hyperkalemia resulted in ventricular fibrillation three minutes into the case. The simulation sessions were viewed by simulation staff, and times to critical events were recorded for each team. The teams were debriefed with a script and checklist of expected actions after each case to improve consistency. Evaluation questions were designed using a Likert scale and were filled out by each participant.
Results: Thirteen groups totaling 24 anesthesia residents participated in this study. Eleven of the 13 groups consisted of pairs and the remaining two groups consisted of individual participants. Table 1 shows the number of groups which completed each action and the time to events from the beginning of ventricular fibrillation.
Although the average time to recognize ventricular fibrillation was one minute and 15 seconds, defibrillation was not requested for another 40 seconds. Only five out of the 13 groups (38%) recognized hyperkalemia as the cause of the arrest. Ten out of the 13 groups (76%) ordered an arterial blood gas showing hyperkalemia and only 31% of these gave calcium. The learning themes most commonly mentioned by the participants involved debriefing (n=9), practice of rare but critical events (n=7), and teamwork (n=3). Participants’ combined course satisfaction scores were 4.96 out of 5 regarding the quality and content of the course. Individual participants noted that the course was a helpful learning experience, and that they would recommend this course to others. Participants’ combined scores regarding their ability to apply the concepts, knowledge, and skills learned compared to other clinical experiences were 4.92 out of 5. Recommendations for improvement included adding more simulation cases to the anesthesia curriculum.
Conclusion: Pediatric dysrhythmias are rare. Thus, anesthesia residents should consider hyperkalemia in an intraoperative arrest, and aggressive treatment must be initiated. Based on the results of our study, the average time to recognition of ventricular fibrillation was one minute and 15 seconds with only 38% of the groups recognizing hyperkalemia as the cause. Therefore time to defibrillation and recognition of hyperkalemia by anesthesia residents could be improved. Exposure to multiple critical events through simulation is an effective and safe means of providing anesthesia residents with the skills needed to respond effectively in these rare events.
1. Fehr JJ, Boulet JR, Waldrop WB. Simulation-based assessment of pediatric anesthesia skills. ANESTHESIOLOGY. 2011 Dec;115(6):1308-15. PubMed PMID: 22037637.
2. Howard-Quijano KJ, Stiegler MA, Huang YM. Anesthesiology residents’ performance of pediatric resuscitation during a simulated hyperkalemic cardiac arrest. ANESTHESIOLOGY. 2010 Apr;112(4):993-7. PubMed PMID: 20234308.
3. Park CS, Rochlen LR, Yaghmour E. Acquisition of critical intraoperative event management skills in novice anesthesiology residents by using high-fidelity simulation-based training. ANESTHESIOLOGY. 2010 Jan;112(1):202-11. PubMed PMID: 20010420.
1162 Interdisciplinary Team Training Leads to a Successful Management of a Rare Obstetric Emergency
David Marzano, MD1, and Pamela Andreatta, PhD2
1OBSETRICS and GYNECOLOGY, UNIVERSITY OF MICHIGAN, ANN ARBOR, MI, USA and 2OBSTETRICS and GYNECOLOGY, UNIVERSITY OF MICHIGAN, ANN ARBOR, MI, USA
Introduction/Background: Simulated interdisciplinary team training fulfills several goals: contextual aspects, repetition, and the ability to analyze performance. Several challenges to using simulation include: participant buy-in, time constraints that conflict from the different disciplines involved, and most importantly the lack of evidence of the transfer of abilities to applied practice. We present a case of a rare obstetric emergency, AIDP/Guillain Barre, that demonstrates the transfer of competencies from a simulation based training program that led to the successful interdisciplinary management of an actual clinical emergency.
Methods: Evaluate the transfer of abilities acquired through a prescribed interdisciplinary simulation-based training program targeting the management of Obstetric emergencies. We ran weekly, 2-hour, training sessions, drawing on 14 case scenarios comprised of emergency events with pregnant patient and neonate. Web portal was designed that included references, pre/post-tests. Participants included faculty, residents, nurses and ancillary staff (such as social work) from the four disciplines (Obstetrics, Emergency Medicine, Anesthesia, and Neonatology). Transfer of program acquired abilities to applied clinical performance assessed by tracking institutional clinical data using case logs and health system quality data. In identified cases, details collected included case specifics, clinical team members and whether they had participated in the OBEMAN training program. The training case was as follows: a 34 year old G2P1, 28 weeks, presents to ED, shortness of breath, stating that she “can’t get enough air,” worse with exertion. Her pregnancy uncomplicated, routine prenatal care, ROS negative The clinical objectives were as follows: acknowledges physiologic changes and challenges of pregnant patient; generates accurate DDx: Pulmonary embolism, Asthma, Pneumonia; initiates appropriate resuscitative effort. The team factors were as follows: demonstrates uses of the Birth Center Paging System, Team-leadership, Emergency communication skills (e.g., Call out/Check Back, Call for help when needed: Anesthesia, Obstetrics, Pediatrics).
Results: The clinical case presented as follows: A 25 year-old pregnant patient, admitted at 23+5-wks gestation, shortness of breath, numbness feet/hand/face; PMHx: negative; PSHx: negative; medications: prenatal vitamins; All: NKDA; POBHx: negative; PGYNHx: negative. The patient was diagnosed w/ acute inflammatory demyelinating polyneuropathy (AIDP/Guillain–Barré) and transferred to neurology service. The Obstetric team followed with close monitoring. She was discharged to acute rehabilitation floor at 26+4-weeks gestation. On day 28, she acutely aspirated. The code team was called to the unit. Attempts to re-establish an airway failed. Several hospitalists at bedside (who had not completed the program) determined the patient needed to be intubated. Following intubation, the patient experienced ventricular tachycardia and proceeded to full cardiopulmonary arrest. The second-year (PGY2) obstetric resident was paged, she correctly paged Obstetric & Neonatal emergency team (All OBEMAN trained) explicitly requested surgical instruments and neonatal resuscitation supplies. The Chief Obstetric resident and Neonatal staff arrived with instruments and supplies within 5 minutes. After 5 minutes with unsuccessful resuscitation, PGY2 performed emergency Caesarean delivery in patient’s room. The patient and her infant were successfully resuscitated, and subsequently discharged to home. The morbidity and mortality report indentified the following qualitative comments: “OBEMAN trained team members applied the following: Correct clinical intervention protocol, Activated both Birth Center & Code Team pagers, specified team members to bring required supplies, demonstrated situational awareness of resources on Rehabilitation floor, and delivered of infant after 5 minutes failed ACLS.” Specifically, both participating residents cited their OBEMAN training as giving them the confidence to proceed, despite un-trained supervising fellow unsure of required action. Untrained team members commented on quick action and calmness of the two obstetrical residents who led the code.
Conclusion: This AIDP/Guillain–Barré case recounts the results of a rare Obstetric incident where the transfer of competencies gained from simulation-based training led to successful interdisciplinary management of an actual clinical emergency. Although this was not a specific case practiced during program sessions, it demonstrates that participants were able to use lessons learned during their training and applied them to a different clinical situation: transfer of learning to applied clinical practice. Although there were favorable outcomes for the two patients, it could have gone the other way despite correct clinical and patient management.
1164 Breaking Down Barriers: A Simulation-based Course Designed to Improve Perioperative Handoff Communication
Erin Pukenas, MD1, Amanda Burden, MD1, Edward Deal, DO1, and Irwin Gratz, DO1
1ANESTHESIOLOGY, ROWAN UNIVERSITY, COOPER MEDICAL SCHOOL, CAMDEN, NJ, USA
Introduction/Background: The increased frequency of patient handoffs is an inevitable consequence of resident duty hour restrictions.1 Perioperative handoff communication is cited as a major source of medical error. Recent literature reveals that poor or inadequate patient handoffs were present in up to 43% of surgical malpractice cases involving communication failure.2 Several barriers to effective handoff communication have been identified among physicians including: physical environment, social and professional setting, hierarchy/status issues, cultural barriers, and communication medium limitations.3 Meta-analysis of current handoff literature identified patient complexity and lack of time as additional communication barriers.4 We instituted a one-day, simulation-based handoff course (SBHC) in an effort to identify and address these barriers. We described the results of this one-day SBHC at one year post-integration into an anesthesiology residency curriculum.
Methods: Ten anesthesiology residents (CA 1-3) were enrolled in this IRB-approved study. Each subject completed a pre-test, read a patient stem, and received a simulated hand-off. Another anesthesiologist arrived at the conclusion of the case to obtain report from the subject. Debriefing and didactic sessions followed the handoffs; a post-test and course evaluation were then completed by each subject. Key items addressed in the debriefing were limitations to effective handoff communication in the trainees’ daily practices. One year later, the residents returned to the simulation laboratory for assessment and participation in a simulated handoff. Debriefing focused on communication barriers and strategies employed to overcome them.
Results: Nearly 62% of subjects originally stated that others were not receptive to their handoff efforts and nearly 54% stated that there are too many time pressures to give effective handoffs. In the one-year follow-up debriefing session, 100% of residents who listed poor receivership as a barrier and 80% of those who listed time pressure indicated they had changed their handoff practice. Strategies to overcome receivership barriers included being more assertive, refusing to transfer care unless the receiver accepted a complete handoff, and concisely organizing their handoffs. Strategies to overcome time pressures included preparing the handoff in advance and having the anesthesia record available during the handoff.
Conclusion: Through this simulation-based handoff course, anesthesiology residents identified barriers to effective perioperative handoff communications processes. One year follow-up demonstrated practice changes among the course participants. Efforts to improve the current handoff communication culture within our perioperative environment are ongoing. This approach may be useful to other institutions in identifying strategies to improve perioperative handoff communication.
1. Greenburg, C.C. Patterns of Communication Breakdowns resulting in Injury to Surgical Patients. J Am Coll Surg 2007; 204(4):533-40 Pubmed PMID:17382211.
2. Kalkman, C. Handover in the perioperative care process.Curr Op Anaesth 2010; 23(6):749-753. Pubmed PMID: 21037475.
3. Solet DJ, Norvell JM, Rutan GH, et al. Lost in translation: challenges and opportunities in physician-to-physician communication during patient handoffs. Acad Med 2005; 80:1094 -1099 Pubmed PMID: 16306279.
4. Riesenberg, L, et al. Residents’ and Attending Physicians’ Handoffs: A Systematic Review of the Literature. Acad Med 2009; 84:1775-1787. Pubmed PMID: 19940588.
1168 Low-hanging Fruit: Using Clementines for Laparoscopic Surgery Training in Gynecological Oncology
Pamela Andreatta, PhD2, and David Marzano, MD1
1OBSETRICS AND GYNECOLOGY, UNIVERSITY OF MICHIGAN, ANN ARBOR, MI, USA and 2OBSTETRICS AND GYNECOLOGY, UNIVERSITY OF MICHIGAN, ANN ARBOR, MI, USA
Introduction/Background: Laparoscopic surgical skills require significant practice to master due to multiple inherent factors that complicate visualization and instrument control in the operative space.1-2 Indirect visualization of the three-dimensional operative field that is presented as a two-dimensional image on a monitor has been shown to have a steep learning curve that is more easily attained using simulation-based training.3-4 The long laparoscopic instruments introduce a fulcrum effect and limit tactility between the surgeon and tissue, both of which must also be conquered before any operative laparoscopic procedure can be accomplished.5,6 These factors are all the more critical for gynecologic oncology surgery, where delicacy and instrumental mastery must be achieved in order to perform the fine, precise dissection skills required for removing tumors and metastases from vital organs, vessels and nerves.7 Oncology procedures also require significant pre-operative planning and intra-operative critical decision-making to assure that as much of the cancerous tissue can be removed, with minimal damage to normal tissues.8 At present, most laparoscopic simulation models address less complicated procedures such as laparoscopic cholecystectomy, inguinal hernia repair and tubal ligation (among others). There currently are no models designed to address the challenges associated with gynecologic laparoscopic surgical oncology.
Methods: Ethics Review Committee approval was secured for this study. We asked residents/fellows to remove the peel of a Clementine in as few pieces as possible, separate and remove all pith from and between all fruit segments, and return the Clementine to as close to its natural state as possible with completely closed skin (sutured). Clinical decision-making included deciding when to complete the procedure “open” or when unacceptable segment damage would result by removing difficult to extract pith. The analogy corresponds to deciding when to leave cancerous lesions or metastases in place to be treated through other methods (radiation, chemo, etc.), rather than risking damage to the vital organs or other healthy tissues. Faculty, blinded to the training status of the subjects, assessed their video-recorded performance using a rating scale, in addition to noted objective performance measures.
Results: Scores Performed by Experience Levels
Factor Analysis using principal component analysis with no rotation and an Eigen value of 1 confirmed a single scoring construct (operative performance) with loadings from .714 to .951, and a 2-factor component matrix indicating two scoring dimensions (clinical reasoning and surgical skills). Factor loadings for surgical skills ranged from .835 to .936 accounting for 71.29% of score variance. The factor loading for clinical reasoning (.934) accounted for 10.63% of score variance. There were significant differences between the groups for the identified performance parameters of interest; F(2,39) = 59.18 p<0.001. MIS specialists and surgeons who operated at least one time per week performed significantly better than others on all parameters (p<.001). There were no significant differences between residents and gynecologists who operated less than one time per week, but both of those groups performed significantly better than novices (p<.001). These data are presented in Table 3. There were no statistical differences between the groups for completion time. These data are presented in Table 3.
Conclusion: A low-cost, easily facilitated simulation-based model for developing advanced laparoscopic surgical skills may advance the preparation of residents and fellows for gynecological oncology practice, providing a platform for development/maintenance of skills, critical thinking and clinical judgment. This model could also provide an option for laparoscopic skill development in low and limited resource environments globally.
1. Blavier A, Gaudissart Q, Cadiere G, Nyssen A. Comparison of learning curves and skill transfer between classical and robotic laparoscopy according to the viewing conditions: Implications for training. Am J Surg. 2007;194:115-21.
2. Van Sickle KR, Gallagher AG, Smith CD. The effect of escalating feedback on the acquisition of psychomotor skills for laparoscopy. Surg Endosc. 2007; 21(2): 220-224. DOI: 10.1007/s00464-005-0847-5.
3. Maithel SK, Villegas L, Stylopoulos N, Dawson S, Jones DB. Simulated laparoscopy using a head-mounted display vs traditional video monitor. Surg Endosc. 2005;19:406-11.
4. Emam TA, Hanna G, Cuschieri. Ergonomic principles of task alignment, visual display and direction of execution of laparoscopic bowel suturing. Surg Endosc. 2002;16:267-71].
5. Berguer R, Forkey DL, Smith WD. Ergonomic problems associated with laparoscopic surgery. Surg Endosc. 1999;13:466-8.
6. Bridges M, Diamond DL. The financial impact of teaching surgical residents in the operating room. Am J Surg. 1989;210:118-21.
7. Theororidis TD, Bontis JN. Laparoscopy and oncology: Where do we stand today? Annals of the New York Academy of Sciences. 2003; 997: 282–291. doi: 10.1196/annals.1290.032.
8. Kehoe SM, Ramirez PT, Abu-Rustum NR. Innovative laparoscopic surgery in gynecologic oncology. Current Oncology Reports. 2007; 9:472–477.
1171 Sticky Situations: Using Foam Stickers for Precision and Tissue Handling in Laparoscopic Training & Assessment
Pamela Andreatta, PhD2, and David Marzano, MD1
1OBSETRICS AND GYNECOLOGY, UNIVERSITY OF MICHIGAN, ANN ARBOR, MI, USA and 2OBSTETRICS AND GYNECOLOGY, UNIVERSITY OF MICHIGAN, ANN ARBOR, MI, USA
Introduction/Background: Low-cost laparoscopic surgery models that provide objective feedback about performance precision and tissue damage are not currently available. Current training models focus on factors such as time, haptecs, and completion of task. Other factors such as tissue handling, damage to tissue, and accuracy are as important if not more important than time. Our objective was to design a model for developing precise and respectful laparoscopic tissue handling skills with accompanying objective assessment measures.
Methods: Ethics Review Committee approval was secured for this study. We asked novices through expert laparoscopists to complete a series of exercises requiring fine precision and tissue handling in the placement of small foam stickers of varying size and shape (circles, squares, stars, triangles, hearts) on templates of varying density and complexity. The foam pieces were required to align exactly and undamaged within the template, and placed permanently on the template after removing the sticky-backing. All exercises were required to be completed within 20 minutes. Objective measures included the degree of offset for each piece on the template, the amount of damage apparent on each piece, the total time required to complete the exercise, and the total number of accurately placed pieces (at least one part of the piece within the correct template space). Three faculty, blinded to the identity of the subjects, scored all templates.
Results: There were significant performance differences between and across the levels of expertise on all performance parameters (p<0.05). Time: tissue/damage: accuracy ratios were the best indicators of skill; however experience performing laparoscopy was not the primary indicator of skill level. The Inter-scorer reliability between the three independent scorers was 0.98 indicating the consistency of the objective measures. Examples of two exercises are included in Table 1.
Conclusion: Low-cost, easily facilitated models using foam stickers for developing laparoscopic surgical precision and tissue handling skills may provide an affordable and portable training option without sacrificing objective performance measures, and may provide more objective evidence for assessment and evaluation than current methods of evaluation. These models also provide an option for laparoscopic skill development in low and limited resource environments globally.
1176 Simulation-augmented Education in the Rehabilitation Professions: A Scoping Review
Euson Yeung, BScPT, MEd, PhD (c)1, Adam Dubrowski, PhD2, and Heather Carnahan, PhD3
1PHYSICAL THERAPY, UNIVERSITY OF TORONTO, TORONTO, ON, CAN and 2THE WILSON CENTRE, SICKKIDS LEARNING INSTITUTE, UNIVERSITY OF TORONTO, HOSPITAL FOR SICK CHILDREN, TORONTO, ON, CAN and 3CENTRE FOR AMBULATORY CARE EDUCATION, WOMEN’S HEALTH HOSPITAL, TORONTO, ON, CAN
Introduction/Background: Simulation-augmented education is playing an increasingly important role in the development and assessment of health professionals.1 More and more, simulation-augmented strategies are shaping educational practice, policy and research in medicine and nursing,2-6 and its impact, in undergraduate training through to continuing education, has been reported in the literature.7-9 The benefits associated with simulation-augmented education are, however, context specific9; thus, the capacity for simulation to yield similar benefits in other health professions, such as the rehabilitation professions, remains unknown. It is, therefore, difficult to determine how these educational approaches can inform education practice, policy and research in the rehabilitation professions. The purpose of this review was to determine what is currently known about the use of simulation in the education of rehabilitation professionals and to identify research gaps.
Methods: We conducted a scoping review that included a literature review of published literature within relevant databases and the grey literature. Two authors independently reviewed the abstracts of the unique references against pre-determined inclusion criteria and met after reviewing a portion of these references to ensure that the criteria sufficiently met our study objectives. After the remainder of the abstracts was reviewed, the two authors met again to discuss any discrepancies in each of their included list of references until consensus was reached. Data from the included references were extracted and analyzed by one author. For each of the included articles we recorded information regarding author, year and location of study, intervention type and comparator (if any), duration of the intervention, study populations, aims of the study, methodology, outcome measures, important results, and key research priorities. We collated and summarized the data to describe the extent, range and nature of simulation-augmented education in rehabilitation as well as the gaps in the existing literature. A program evaluation framework was employed to structure data extraction and analysis.10
Results: Our search of relevant databases yielded 1429 abstracts of which 1229 were unique. Eighty-five of the unique abstracts fit the inclusion criteria and were retrieved for full review. Searching the reference lists of included articles and relevant conference proceedings produced an additional 7 and 3 abstracts respectively, while searching the grey literature did not yield any additional abstracts or reports. Upon full review, 35 references were found to address the purpose of this scoping review and underwent data extraction. We found that several forms of simulation are commonly used among rehabilitation professions for expertise development, formative and summative evaluations, and for enhancing course work and curricula. However, there is a lack of published literature concerned with the longitudinal impact of simulation-augmented educational strategies, the perspectives of accrediting bodies, and the processes that lead to successful or unsuccessful educational interventions.
Conclusion: This is the first synthesis of what is currently known about simulation-augmented education in the rehabilitation professions. To date, research has primarily focused on studies of effectiveness and on the role that simulation-augmented interventions play in assessment. While these are important investigations, more longitudinal and process-oriented studies are urgently needed so that research results can be generalized to other learner populations and be better linked to patient outcomes. This will assist stakeholders, including learners, program directors, policy makers, and potential funders to value simulation as a significant learning and assessment tool in rehabilitation education. For education program developers, the use of an evaluation model will be useful for framing future studies to examine how simulation can enhance learning experiences in the rehabilitation professions, particularly in regards to curricular processes. Licensing and accrediting bodies in the rehabilitation professions should consider publishing educational criteria or standards related to simulation-augmented interventions to inform future design of educational programs and research studies.
1. Reznick RK, MacRae H. Teaching surgical skills – changes in the wind. Med Educ. 2006; 355:2664-2669.
2. Bradley P. The history of simulation in medical education and possible future directions. Med Educ. 2006; 40:254-262.
3. Issenberg SB, McGaghie WC, Hart IR et al. Simulation technology for health care professional skills traning and assessment. J Amer Med Assoc. 1999;282(2):861-866.
4. Ravert P. An integrative review of computer-based simulation in the education process. Computers, Informatics, Nursing. 2002; 20(5):203-208.
5. Scalese RJ, ObesoVT, Issenberg SB. Simulation technology for skills training and competency assessment in medical education. J Gen Intern Med. 2007; 23(suppl.1):46-9.
6. Maran NJ and Glavin RJ. Low- to high-fidelity simulation – a continuum of medical education? Med Educ. 2003; 37(suppl.1):22-28.
7. Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL, Scalese RJ. Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Med Teach. 2005; 27(1):10-28.
8. Brydges R, Carnahan H, Rose D, Rose L, Dubrowski A. Coordinating progressive levels of simulation fidelity to maximize educational benefit. Acad Med. 2010; 85(5):806-812.
9. Lane JL, Slavin S, Ziv A. Simulation in medical education: a review. Simulat Gaming. 2001; 32(3):297-314.
10. Alkin, M.C.&Christie, C.A.(2004). The evaluation theory tree. In Alkin, M.C. (ed.), Evaluation Roots: Tracing Theorists’ Views and Influences. Chapter 2. London: Sage Publications.
1178 Is Simulation-Based Education Effective in Improving Nurse Preceptor Confidence in Providing Effective Feedback to Orientees? A Pilot Study
Sherry Chesak, MS2, Monica Farnsworth1
1MULTIDISCIPLINARY SIMULATION CENTER, MAYO CLINIC, ROCHESTER, MN, USA and 2SIMULATION CENTER, MAYO CLINIC, ROCHESTER, MN, USA
Introduction/Background: The ability to provide effective feedback is an essential skill for nurse preceptors.1 However, competence in this skill is not necessarily intuitive, and the level of expertise can vary greatly from one preceptor to another. Lack of providing effective feedback can lead to orientees’ repeated poor performances, which can have significant negative impacts on patient care as well as colleague interactions.1 It is not clear if simulation-based educational strategies are effective in improving nurse preceptors’ confidence in their ability to provide feedback to orientees, or what the most significant barriers are to implementing the skill successfully. The aims of this study were to: 1)compare critical care preceptors’ confidence levels in providing feedback before and after participating in a simulation-based education session using a five-item confidence scale, and 2) identify critical care preceptors’ perceived barriers to developing and implementing effective feedback skills. The hypothesis for the first aim is: critical care nurse preceptors’ confidence levels in providing feedback to orientees will significantly improve both immediately following a simulation-based educational intervention, and 4 months following the intervention. The results of this study will impact healthcare simulation by providing substantiation for the investment of institutional resources in this form of education for similar populations of healthcare providers, a domain in which there is currently limited knowledge. In addition, identifying barriers to developing and implementing effective feedback skills will assist in the development of future curriculum, aimed specifically at key obstacles.
Methods: This IRB approved study occurred at a large Midwestern educational healthcare institution.The intervention included 13 nurse preceptors who participated in intensive care based simulation scenarios with actors playing the roles of orientees who demonstrated challenging behaviors. The nurse learners provided feedback to the actors following the interaction. Subsequent to each scenario, the learners participated in a debriefing session, which was facilitated by masters prepared nurse educators. A pre-test/post-test design was employed to examine preceptors’ confidence levels in providing effective feedback. A post-test occurred immediately following the experience and a late post-test was administered four months following the intervention, at which point it was expected that the preceptors would have had the opportunity to employ feedback skills in the clinical setting. Changes in confidence levels from pre to post, from pre to late post, and from post to late post were evaluated using paired t-tests. A short answer questionnaire gathered qualitative information on the participants’ perceived barriers to developing and implementing effective feedback skills. Qualitative analysis was performed through identification of themes and categorization of results.
Results: Confidence levels increased significantly for questions 1 (p=.003), 3 (p=.017), 4 (p=.011), and 5 (p=.028) on the five-item scale from the pre to post assessments. Confidence levels for all questions increased significantly from the pre to the late post assessments (p=.002, .031, .026, .007, .031). There were additional statistically significant increases in confidence for questions 1 (p=.013) and 4 (p=.040) from the post to the late post assessments. Themes, which emerged regarding barriers to developing and implementing effective feedback skills, include lack of education, inadequate experience, fear of damaging the preceptor-orientee relationship, and not wanting to appear critical.
Conclusion: Simulation-based education was effective in improving critical care nurse preceptors’ confidence in providing feedback to orientees. This educational methodology should continue to be employed with this population, and larger scale studies are warranted in an attempt to replicate the results. Future curriculum content should focus on the themes identified by the preceptors as barriers to providing feedback to orientees.
1. Cantillon, P, Sargeant, J. Giving feedback in clinical settings. BMJ. 2008 Nov 10; 337:1292- 1294. doi: 10.1136/bmj.a1961.
1179 Comprehensive Program Evaluation of an Interdisciplinary Program for Managing Obstetric Emergencies
Pamela Andreatta, PhD2 and, David Marzano, MD1
1OBSETRICS AND GYNECOLOGY, UNIVERSITY OF MICHIGAN, ANN ARBOR, MI, USA and 2OBSTETRICS AND GYNECOLOGY, UNIVERSITY OF MICHIGAN, ANN ARBOR, MI, USA
Introduction/Background: Stress affects human performance and decision-making, and the degree of stress experienced by physicians during emergency situations has the potential to limit their effectiveness as both clinicians and team leaders. The purpose of this study was to assess the stress levels of clinical team members (leaders and non-leaders) during the management of Obstetric Emergencies.
Methods: A sample of 68 physicians, residents and nurses from Obstetrics, Emergency Medicine, Anesthesia and Neonatology participated in simulation-based drills to manage Obstetric Emergency cases as interdisciplinary teams. A total of 58 cases were included in the drills, with low, moderate or high variable difficulty ratings. Baseline, peak and mean heart rates were captured during the drills for each team member. Stress level was calculated for heart rate differentials and sustained elevations during the case. Case difficulty and team-leader status were considered in the analysis of variance. Direct measurement of clinical performance related to stress level was not measured for this study because we did not know whether or when stress responses were in effect.
Results: Mean heart rates across all cases were 81.01 (28.39) bpm for team leaders and 59.26 (18.77) bpm for non-leaders. There were significant differences between the stress levels of team leaders and non-team leaders for baseline-peak differential (p=.001), as well as sustained elevated heart rate (p=.001) across all cases. Both team leaders and non-team leaders had significant increased stress (p=.028) and sustained stress (p=.002) with increased case difficulty. Team leaders had significantly greater sustained stress (p=.004) compared to non-team leaders for all cases.
Conclusion: The results of this study demonstrated that it is possible to measure stress experienced by clinicians during simulation drills; and that although stress is experienced by all team members, team leaders have both significantly greater stress and sustained stress during the team management of Obstetric emergencies. It also confirms that stress reactions are in effect during simulated drills. These findings suggest the value of simulation-based training for developing stress inoculation behaviors to gain optimal performance for all team members, but especially for team leaders during crises. Future studies will address direct measurement of clinical performance with increased stress levels.
1184 The Impact of Transdisciplinary Simulation on Interprofessional Collaborative Practice in Women’s Health
Kymberlee Montgomery, DrNP, CRNP, BC1, Nina Multak, MPAS, PA-C3, Catherine Morse, MSN, CRNP, BC1, and Sharon Griswold-Theodorson, MD, MPH2
1NURSING, DREXEL UNIVERSITY, PHILADELPHIA, PA, USA and 2EMERGENCY MEDICINE, DREXEL UNIVERSITY COLLEGE OF MEDICINE, PHILADELPHIA, PA, USA and 3PHYSICIAN ASSISTANT, DREXEL UNIVERSITY COLLEGE OF MEDICINE, PHILADELPHIA, PA, USA
Introduction/Background: Education of health professionals is shifting to a model which supports collaboration, communication and a team-based approach to the provision of healthcare.1 This educational shift, although slow at onset, is moving forward with an increased pace. Studies suggest that promoting a collaborative team approach, incorporating joint responsibility for care, fostering clear delineation of roles, and encouraging effective communication improves patient outcomes and satisfaction and reduces medical error related to health care system failure.1-3 The goal of this inter-professional educational experience was to evaluate improvements in collaborative attitudes and behaviors among undergraduate nursing, nurse practitioner, physician assistant and medical students/residents. The implementation of simulation based education was utilized to evaluate healthcare collaboration among students in order to support the provision of safe and efficient care to patients.
Methods: This study used a pretest–post-test comparative design to measure changes in collaborative attitudes among 282 students before and after a transdisciplinary simulation experience. Collaborative attitudes were measured by the Team Attitudes Questionnaire (TAQ). Data analysis consisted of descriptive analysis, paired t-tests, and post hoc item analysis. Findings suggest significant increases in collaborative attitudes for mutual support, leadership and communication but no significant increases in attitudes for situation monitoring from pretest to post-test. Women’s health scenarios were utilized and provided students opportunities to participate in cases using standardized patients, high fidelity simulators as well as hybrid scenarios. Cases were designed by participating faculty in the Colleges of Medicine and Nursing and Health Professions who partnered for this study, modeling teamwork and communication skills for an inter-professional learning environment.
Results: A total of 282 healthcare providers and students participated in the simulation cases. Of the students who participated, 0.35% (1) were medical students, 12.8% residents (36), 44% (124) undergraduate nursing students, 26.6% (75) were nurse practitioner students (including a nurse anesthetist student), and 16.31% (46) were physician assistant students. The age range of the students was 20 years to 57 years, with 17.0% (48) 22 years old, and 13.1% 23 years (37). Results from the Team Attitude Questionnaire (TAQ), Teamwork perceptions questionnaire, Collaborative Practice Scale, and Collaborative Satisfaction with Decisions made were analyzed by a series of paired t-tests comparing pretest and post-test results. The non-parametric sign rank tests were also generated as a means of validating the paired t-test results. The primary analyses were conducted on the composite variables. When statistical significance was found, post-hoc exploratory analyses were run on the individual items to better understand what was driving the significance. The TAQ was used to compare 5 dimensions of attitude toward interdisciplinary collaboration including structure, leadership, situation monitoring, mutual support, and communication. Findings suggest that there was a significant increase in communication (24.90 to 25.60, p<.0001), in structure (23.57 to 26.59, p <.0001) and in leadership (21.95 to 27.29, p <.0001); near significant increase in situation monitoring (26.04 to 26.54, p=.0600); and near significant decrease in mutual support (24.72 to 24.28, p =.0764) from pre-test to post-test.
Conclusion: There was a significant increase in structure, leadership, and communication, but no significant increases in mutual support, and situation monitoring from pretest to post-test. Transdisciplinary simulation experiences among women’s health students may enhance mutual support and communication and promote better patient outcomes. Future research should focus on mechanisms to facilitate further improvements in interprofessional nontechnical skills to further identify how they affect improved patient care.
1. Frenk, J., Chen, L., Bhutta, Z.A., Cohen, J., Crisp, N., Evans, T., et al. (2010). Health professionals for a new century: transforming education to strengthen health systems in an interdependent world. The Lancet, 376 (9756), 1923-1958.
2. World Health Organization (WHO). (2010). Framework for action on interprofessional education & collaborative practice. Geneva: World Health Organization. Retrieved July 30, 2012 from http://whqlibdoc.who.int/hq/2010/WHO_HRH_HPN_10.3_eng.pdf.
3. King, H. B., Battles, J., Baker, D. P. Alonso, A., Salas, E., Webster, J., et al. (2008, July). TeamSTEPPS: Team Strategies and Tools to Enhance Performance and Patient Safety. In Advances in Patient Safety: New Directions and Alternative Approaches. Volume 3. Performance and tools. (Pp. 5-20). AHRQ.
1185 The Quality of Cardiopulmonary Resuscitation Using Supraglottic Airways and Intraosseous Devices: A Simulation Trial
Dena Reiter, MD2, Christopher Strother, MD2 and Scott Weingart, MD1
1EMERGENCY CRITICAL CARE, MOUNT SINAI SCHOOL OF MEDICINE, NEW YORK, NY, USA and 2EMERGENCY MEDICINE, MOUNT SINAI SCHOOL OF MEDICINE, NEW YORK, NY, USA
Introduction/Background: Our objective was to assess whether using interventions such as laryngeal mask airways (LMA) and intraosseous (IO) lines lead to improved resuscitation in a simulated cardiac arrest when compared to standard methods of endotracheal intubation (ETI) and central line placement.
Methods: Emergency Medicine residents at a single academic center were grouped into teams of four. Each team participated in two simulated ventricular fibrillation cardiac arrests using a high fidelity simulator. Peripheral IV access was unobtainable. Only ETI supplies and a central line kit were available in one case (control) and in the other case those supplies were replaced by an LMA and an EZ-IO drill kit (experimental). Groups were randomized to which set up they were given first. Data examined included time to airway placement, duration and success rate of airway placement, time to vascular access, time to defibrillation, and percent hands off time.
Results: Of the participants, 44 residents in 11 teams participated. Mean time to airway was shorter in the experimental group (122.8 seconds (s) vs. 265.6 s, p = 0.001). Mean duration of airway attempt was also shorter (7.6 s vs. 22.7 s, p = 0.002). Time to access was shorter in the experimental group (49.0 s vs. 194.6 s, p = <0.001). Time to defibrillation and percent hands off time did not significantly differ between the two groups.
Conclusion: Use of an LMA and an IO device led to significantly faster establishment of an airway and vascular access in a simulated cardiac arrest. The variation in devices did not affect time to defibrillation or percent hands off time.
1189 Influence of Simulation-based Practice on Emergency Care for Patients with Dyspnea on Learning Outcome in Nursing Students
Hea Kung Hur, PhD3, Hyang Ok Choi, MS2, Ji Soo Jung, BS1, Hye Won Kang, MS1, and Gi Yon Kim, PhD3
1DEPT OF NURSING, WONJU, PRK and 2NURSING, WONJU, PRK and 3NURSING, YONSEI UNIVERSITY, WONJU COLLEGE OF MEDICINE, WONJU, PRK
Introduction/Background: Clinical simulation has become a valuable tool in the Korean nursing education. A critical step in adopting simulation-focused pedagogy in nursing education programs is determining how to integrate simulations into courses and curricula. Planning for clinical simulation should begin with identifying learning outcomes that are necessary for safe and competent nursing practice (Gaberson & Oermann, 2010). This study was conducted to evaluate (1) knowledge, critical thinking, and problem solving processes as cognitive learning outcomes, (2) self-confidence and learning attitudes as affective learning outcomes, and (3) self-reported clinical performance ability as a psychomotor learning outcome of simulation-based practice for emergency care of patients with dyspnea in Korean nursing students.
Methods: A one-group, pre-post experimental design including 28 junior nursing students of Y University in 2011 was used. Sample size was calculated using G*Power 3.1.3. The students participated in simulation-based practice, including academic lectures, simulation lab exercises, and debriefing, for 4.5 hours. The scenario was developed based on the algorithm for emergency care of patients with dyspnea (Kang & Hur, 2010). Debriefing followed the guidelines of Schneider Sarver and Senczakowicz (2010). For measurement of learning outcomes, a knowledge questionnaire (13 items) for emergency nursing care of dyspnea patients (Hur & Park, 2012) was used. A critical thinking questionnaire (15 items, 5-point Likert scale) developed by Hur and Park (2012) and a problem solving process questionnaire (7 items, 5-point Likert scale) developed by Kim and Chang (2011) were used. A self-confidence questionnaire (10 items, 10-point Likert scale) and a learning attitudes questionnaire (6 items, 5-point Likert scale), both developed by Ko et al. (2010), were used to evaluate affective learning outcomes. A self-reported clinical performance ability questionnaire (Hur & Park, 2012) was used to evaluate psychomotor learning outcomes (21 items, 6-point Likert scale). High values in all variables indicated a high level of each learning outcome. The Kolmogorov-Smirnov test was tested to test the variable distributions for normality. The paired t-test was used for data analysis, which was carried out using PASW 18.0.
Results: After the completion of simulation-based practice, all variables of cognitive, affective, and psychomotor learning outcomes were significantly increased posttest as compared to pretest (Table 1).
Conclusion: Simulation-based practice is an effective educational method that can be used to improve the cognitive, affective, and psychomotor learning outcomes of nursing students. Development of an instrument to assess student performance during simulated clinical experience is needed in the future. Even though this study had some limitations due to the testing effect and a small sample size, the results regarding simulation-based practice found in this study could be applied to improving current limited emergency care training for nursing students and enhancing students’ competency in clinical situations.
1. Gaberson KB, Oermann MH: Clinical teaching strategies in nursing, 3rd edition. NY, Springer Publishers, 2010, pp10-11.
2. Kang H, Hur HK: Development of a simulation scenario on emergency nursing care of dyspnea patients. Journal of Korean Critical Nursing 2010; 3: 59-74.
3. Scheider Sarver PA, Senczakowicz EA: Development of simulation scenarios for an adolescent patient with diabetic ketoacidosis. Journal of Nursing Education 2010; 49: 578-586.
4. Hur HK, Park SM: Effects of simulation based education for emergency care of patients with dyspnea on knowledge and performance confidence of nursing students. Journal of Korean Academy Society Nursing Education 2012; 18: 111-119.
5. Kim YH, Chang KS: Effects of a simulation-based education on cardio-pulmonary emergency care knowledge, clinical performance ability and problem solving process in new nurses. Journal of Korean Academy Nursing 2011; 41: 245-255.
6. Ko IS, Kim HS, Kim IS, Kim SS, Oh EG, Kim EJ, Lee JH, and Kang SW: Development of a scenario and evaluation for simulation learning of care for patients with asthma in emergency units. Journal of Korean Academy Fundamental Nursing 2011; 17: 371-381.
1203 Snapshots of the Road to Reflective Practice: What Advanced Simulation Instructors Think During a Master Class
Michaela Kolbe, PhD1, and Jenny Rudolph, PhD2
1MANAGEMENT, TECHNOLOGY, ECONOMICS, ETH ZURICH, ZURICH, CHE and 2CENTER FOR MEDICAL SIMULATION (CMS), HARVARD UNIVERSITY, CAMBRIDGE, MA, USA
Introduction/Background: While there is an increasing array of recommendations for what constitutes basic simulation instructor skills,1,2 not much is known about how they are learned and sustained effectively. The learning needs of intermediate and advanced simulation faculty trying to improve their debriefing skills are largely unexplored in the simulation education literature. Understanding what captivates or concerns them could help simulation faculty development programs target instructor candidates’ needs more accurately. Whereas formal evaluation of instructor curriculum by post hoc measures can be done in a rigorous way, such evaluation is often prohibitively resource-intensive. More importantly, these measures may not capture instructor candidates’ subjective experiences and the nuances of skill development evolution. This qualitative study reports on an effort to capture the preoccupations of instructor candidates as they moved through an advanced instructor skills development course.
Methods: Using a method called experience sampling,3 we longitudinally “biopsied” learners’ subjective experiences throughout the four-day Institute for Medical Simulation’s Graduate Course (www.harvardmedsim.org) using repeated, identical, two-minute free writing tasks for each learner (for a total of 512 measures). Brief, written reflections after each session (called “headlines”) were built into the course to systematically initiate thinking about learning. The specific query was: “What is the headline for what is on your mind right now?”. These headlines were analyzed for the purpose of this study applying an inductive process4 to identify evident constructs. Participants included 25 experienced instructor candidates and 10 course facilitators.
Results: We found that learners described their learning with seven processes among which we identified a variety of themes (Table 1). These themes included monitoring one’s learning process with respect to performance gaps and what one needs to have (e.g., “need help”) or do (e.g., “need to learn about research and assessment”) to close them and what helps for doing so (e.g., “practice and watching others practice helps me”). Another central group of themes involved reflections about the actual learning content: participants reported insights into the importance of concepts such as psychological safety, honesty, and frames for high-quality debriefings. Surprisingly, participants also formulated behavior rules around these themes (e.g., “don’t hesitate to make my point of view known”), some of which even included if-then relations (e.g., “when I don’t know: explore”). These insights, rules, and meta-evaluations occurred more often after experience-based learning sessions than after lectures or informal discussion rounds.
Conclusion: When engaged in experienced-based learning, simulation instructor candidates process the learning content and setting in many shades that go beyond mere opinions of whether or not they liked a particular exercise or tool. They actively think about how they learn and how they can learn better. Particularly after having personally practiced and reflected on a learning content, participants reported insights and developed action-rules for future performance. As such, brief written reflections during courses provide meaningful insights into what and how simulation instructors learn.
1. Rudolph JW, Simon FB, Raemer DB, Eppich WJ. Debriefing as formative assessment: Closing performance gaps in medical education. Acad Emerg Med 2008;15:1010-6.
2. McGaghie WC, Issenberg BS, Petrusa ER, Scalese RJ. A critical review of simulation-based medical education research: 2003-2009. Medical Education 2010;44:50-63.
3. Larson R, Csikszentmihalyi M. The experience sampling method. New Directions for Methodology of Social & Behavioral Science 1983;15:41-56.
4. Miles MB, Huberman AM. Qualitative data analysis. Thousand Oaks, CA: Sage; 1994.
1220 Self-efficacy in an Interprofessional Postgraduate Education Programme: A Cross-sectional Study
Colm Watters, MB BCh BAO, MCEM3, Nicola Morgan, MBChB3, Libby Thomas, BMedSci, BMBS4, Rhodri Thomas, MB BS1, Alastair Ross, PhD2, Peter Jaye, BSc, MBBS, FCEM3, and Claire McHale, MB ChB3
1GENERAL INTERNAL MEDICINE, KING’S COLLEGE LONDON, KING’S HEALTH PARTNERS, LONDON, GBR and 2MANAGEMENT, KING’S COLLEGE LONDON, KING’S HEALTH PARTNERS, LONDON, GBR and 3SIMULATION AND INTERACTIVE LEARNING (SaIL) CENTRE, KING’S COLLEGE LONDON, KING’S HEALTH PARTNERS, LONDON, GBR and 4SIMULATION AND INTERACTIVE LEARNING (SAIL) CENTRE, SAINT THOMAS’ HOSPITAL, LONDON, UK, GBR
Introduction/Background: Interprofessionalism and collaborative practices have been climbing the healthcare agenda over the past fifty years and numerous organisations have heralded its coming, most notably WHO1 and CAIPE.2 In order to promote interprofessionalism within our current and future healthcare workforce, we should educate in a similar manner. 3–5 However robust evidence is lacking for Interprofessional Education (IPE) as highlighted by the Cochrane6 and BEME3 reviews. Recent reviews and publications have called for strengthening of the research agenda for IPE.7-9 Simulation is widely utilised to promote IPE in which learners from different professions are brought together to ‘learn with, from, and about each other’.2 To contribute to this agenda, differences between IPE and Uni-professional (UP) Education were examined to determine if there was something in the nature of the inter-professional interaction that enhances the learning for all involved.
Methods: We looked at an educational episode within the first and second years of doctors’ and nurses’ postgraduate experience. Each course was a high fidelity simulation day in which learners participated in five clinical scenarios and one communication scenario. 399 learners across 3 hospitals participated over 3 years. Cohorts consisted of either doctors only (n=94; foundation year (FY) one and two) and nurses only (n=177; first year postgraduate preceptorship nurses including midwives). The combined cohort (n=128) consisted of a mixture of these groups. After each simulation a facilitated debriefing took place in which the main focus of learning was on non-technical skills. Learners completed pre- and post- course questionnaires consisting of open and closed questions. Self-efficacy in the domains of management of emergency situations, leadership, inter-professional communication and team work were measured.
Results: Overall, IP training led to increased post-course self-efficacy when compared to UP. This difference was significant for one domain ‘communicate useful information effectively with colleagues,’ (82% vs 78%; t=2.7, p<0.01). A subset of 187 participants (70% Nursing and Midwifery, 30% FY doctors) were measured both before and after the course for evidence of improvements in self-efficacy. A significant positive shift (63%, SD 14.6 vs 77%, SD 12.3, p=<0.001) was revealed which was higher in the nursing/midwifery cohort (t=4.5, df 185, p<0.001). Overall self-efficacy was higher in the IP group (14% points) than UP group (12% points) but not significantly so (t=0.9, df 185,NS) following the course. An exploratory factor analysis of post course scores shows a two-factor solution (74% variance); namely leadership/management and communication/teamwork). There were significant improvements on both dimensions for both doctors and midwives, with the biggest shift being 21% for nurses on leadership/management. Thematic analysis of open responses aligned with three primary themes: communication, leadership and teamwork which triangulate with the quantitative findings.
Conclusion: Simulation training improved self-efficacy for all learners regardless of clinical background. IP education within a simulation setting provides a different focus of learning when compared to that delivered uniprofessionally, and suggests a richer experience for all learners involved. We conclude that IPE enhances the learning by providing a broader range of non-technical skills discussion, which we believe will foster greater interprofessionalism and collaborative practices and thus lead to improved patient safety and health outcomes. Based on our findings we suggest that if non-technical skills are to be the main focus of learning, then it should be delivered within an interprofessional setting.
1. World Health Organisation. Learning together to work together for health. Report of a WHO study group on Multiprofessional Education for health personnel: the Team Approach. Geneva: WHO; 1998 World Health Organisation Technical Report Series no 769.
2. Centre for Advancement of Interprofessional Education. The definition and principles of interprofessional education Internet. CAIPE UK; 2002 cited 10th April 2012. Available from http://www.caipe.org.uk/about-us/the-definition-and-principles-of-interprofessional-education/:
3. Hammick M, Freeth D, Koppel I, Reeves S, Barr H. A best evidence systematic review of interprofessional education. Med Teach. 2007;29:735-51.
4. Bristol Royal Infirmary Inquiry. Learning from Bristol: the report of the public inquiry into children’s heart surgery at the Bristol Royal Infirmary 1984-1995. Bristol Royal Infirmary 2001. Available from: http://www.bristol-inquiry.org.uk
5. General Medical Council. Tomorrows Doctors: Outcomes and Standards for Undergraduate Medical Education Online. GMC 2009. Available: http://www.gmc-uk.org/static/documents/content/TomorrowsDoctors_2009.pdf Accessed 17th April 2012.
6. Reeves S, Zwarenstein M, Goldman J, Barr H, Freeth D, Hammick M, Koppel I. Interprofessional education: effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews 2008, Issue 1. Art. No.: CD002213. DOI: 10.1002/14651858.CD002213.pub2.
7. Thistlethwaite J. Interprofessional education: a review of context, learning and the research agenda. Med Educ. 2012; 46(1):58-70.
8. Begley CM. Developing interprofessional learning: Tactics, teamwork and talk. Nurse Educ Today. 2009 Apr;29(3):276-83.
9. Zwarenstein M, Goldman J, Reeves S. Interprofessional collaboration: effects of practice-based interventions on professional practice and healthcare outcomes. Cochrane Database of Systematic Reviews 2009, Issue 3. Art. No.: CD000072. DOI: 10.1002/14651858.CD000072.pub2.
Disclosures: The Simulation and Interactive Learning (SaIL) receives grant support from Gaumard towards the support of research fellows and PhD students.
1230 Comparing No-Flow-Time During Endotracheal Intubation Versus Placement of a Laryngeal Mask Airway During a Simulated Cardiac Arrest Scenario
Vincent Miller, MD2, and Erin Flaherty, MD1
1UNIVERSITY OF VERMONT, BURLINGTON, VT, USA and 2ANESTHESIOLOGY, UNIVERSITY OF VERMONT, BURLINGTON, VT, USA
Introduction/Background: In 2010, the American Heart Association (AHA) released revisions of its Guidelines for Cardiopulmonary Resuscitation. The most notable AHA change is the basic life support sequence steps from “A-B-C” (Airway, Breathing, Chest Compressions) to “C-A-B” (Chest compressions, Airway, Breathing).1 Emphasis has been placed on adequate chest compressions and limiting the amount of no-flow-time (NFT) to 10 seconds or less. Regarding airway management, the guidelines stress that it is important that achievement of an advanced airway not significantly delay the administration of chest compressions or shocks. The guidelines endorse the use of supraglottic airway devices as an alternative to endotracheal intubation for advanced airway management during cardiopulmonary rescuscitation (CPR). Our study evaluates the efficacy of placing both laryngeal mask airways (LMA) and endotracheal tubes (ET) in the midst of ongoing, adequate chest compressions, to eliminate or minimize NFT, by respiratory therapists during a simulated cardiac arrest scenario.
Methods: Forty-one respiratory therapists at our academic hospital participated in this cross over trial. Following a standardized training and practice session covering endotracheal intubation via direct laryngoscopy and LMA placement, the airway managers were directed to place an advanced airway during a simulated cardiac arrest scenario. They were informed to attempt the airway management task during chest compressions and to ask for cessation of CPR only if absolutely necessary. Time to successful ventilation and no-flow times were measured.
Results: In the ET group, only one subject requested cessation of chest compressions during direct laryngoscopy for 2.3 seconds (p=0.175). There were no requested interruptions in chest compressions in the LMA group. The mean time for insertion of an ETT during adequate chest compressions was 49.2 seconds. The mean time for inserting an LMA during adequate chest compression was 31.6 seconds. Insertion of an LMA was significantly faster (p<0.001).
Conclusion: Respiratory therapists can successfully establish ventilation with an LMA and ETT without prolongng no-flow-times in a manikin during a simulated cardiopulmonary arrest. However, insertion of an LMA was significantly faster compared to endotracheal intubation.
1. Field JM, Hazinski MF, Sayre MR, et al. Part 1: executive summary: 2010 American Heart Association Guidelines for Cardiopulmonary Resuscitation and Emergency Cardiovascular Care. Circulation. 2010;122(suppl 3):S640-S656.
1236 Simulation-based Training: Do Residents Think about Their Learning of Procedural Skills Using Simulation?
Ryan Brydges, PhD4, Maria Mylopoulos, PhD3, David Shanks, MD1, and Rose Hatala, MD, MSc2
1INTERNAL MEDICINE, UNIVERSITY OF BRITISH COLUMBIA, VANCOUVER, BC, CAN and 2MEDICINE, UNIVERSITY OF BRITISH COLUMBIA, VANCOUVER, BC, CAN and 3PEDIATRICS, UNIVERSITY OF TORONTO, TORONTO, ON, CAN and 4THE WILSON CENTRE, UNIVERSITY OF TORONTO, MICHENER INSTITUTE MEDICAL RADIATION SCIENCES, TORONTO, ON, CAN
Introduction/Background: Simulation-based training is ubiquitous in medical education, yet we do not fully understand how trainees conceptualize the learning process in this environment. Without evidence for how trainees think about their learning (i.e., employ metacognition), some training techniques may be working against rather than supporting trainees’ learning strategies. Here, we explored junior residents’ conceptions of self-regulated learning (SRL) during simulation-based training of lumbar puncture (LP) skills.
Methods: We randomly assigned 21 post-graduate year 1 internal medicine residents to self-regulate their learning using two versions of the same LP simulator (easy and difficult). We conducted two semi-structured interviews: (1) when participants transitioned from the easy to the difficult version of the simulator, and (2) when they decided to complete a final posttest. The interview questions focused participants on the processes and strategies they used to monitor their own learning, and how they made important learning decisions (e.g., when to transition between models, when to stop learning). Three researchers conducted a thematic analysis of the interview transcripts and came to consensus on emergent themes during several team meetings. One researcher applied the final thematic structure to the entire data set using NVivo qualitative software and two researchers ensured that the final coding was consistent with the agreed upon thematic structure.
Results: Three major themes emerged from our analysis. First, participants showed an astute awareness of the learning context by discussing both the limitations (e.g., simulator not representative of all patients) and affordances (e.g., ability to make mistakes) of simulation training. Second, participants identified connections they made between performance and confidence/comfort, including the feel of instruments, not forgetting the procedural steps, and getting the cerebrospinal fluid. Third, participants discussed conceptions of learning that were centered on task-specific strategies (e.g., repetitive practice), rather than metacognitive strategies (e.g., self-monitoring or self-questioning). Further, participants tended to focus on outcome goals (e.g. obtaining CSF fluid) rather than the processes that permit a successful outcome and when they did discuss process, they emphasized memorizing the steps.
Conclusion: From the lens of SRL theory, the relative absence of metacognitive strategies (a key component of SRL) and residents’ preferences for outcome goals (vs. process goals) suggests their conceptions of simulation-based learning may need refinement. Further, the focus on memorizing the steps rather than maintaining awareness of performance processes “in the moment” suggest trainees are applying strategies useful in other domains (e.g., MCQ exams) to their learning of procedures. According to SRL theory, educators may need to explicitly instruct trainees to avoid prioritizing outcome goals and to more closely monitor learning decisions during procedural training. The implications of such instructional practices should be evaluated before being incorporated into simulation training practices.
Disclosures: Ryan Brydges, PhD, receives a Medical Education Research Grant from the Royal College of Physicians and Surgeons of Canada.
1242 Intrauterine Fetal Resuscitation during the Second Stage of Labor Evaluated in a Physiological Simulation Model
Lauren Bullens, MD2, Beatrijs van der Hout, MSc1, Pieter van Runnard Heimel, MD, PhD2, and Guid Oei, MD, PhD2
1BIOMEDICAL ENGINEERING, EINDHOVEN UNIVERSITY OF TECHNOLOGY, EINDHOVEN, NLD and 2OBSTETRICS AND GYNECOLOGY, MAXIMA MEDICAL CENTER, VELDHOVEN, NLD
Introduction/Background: Fetal well-being is monitored during labor by the use of the cardiotocogram (CTG), which is the continuous registration of fetal heart rate (FHR) and uterine pressure. The CTG serves as the only continuous and noninvasive indication of fetal oxygenation. In case of nonreassuring FHR patterns, it is important to improve fetal condition and prevent fetal hypoxemia, acidosis and asphyxia. Variable decelerations are caused by impaired blood flow toward the fetus by compression of the uterine vessels and umbilical cord. Both pressure of the uterine wall and abdominal wall contribute to the intrauterine pressure. Studies have shown that intrauterine pressure increases from 25 to 55 mmHg during contractions. While bearing down, the pressure increases up to 160 mmHg by the use of abdominal muscles.1 Increase of intrauterine pressures causes an intermittent decrease in blood flow.2,3 If this reaches abnormal levels the fetus may become hypoxic and changes in FHR will occur. Several intrauterine resuscitation techniques may be applied in case of suspected fetal distress.4,5 We hypothesize that to interrupt pushing leads to intrauterine resuscitation of the fetus by improving placental blood flow.
Methods: We developed a mathematical model that simulates the CTG based on maternal cardiac output, maternal saturation and oxygen pressure, uterine pressure and blood flow, oxygen diffusion capacity in the placenta, fetal cerebral blood flow, fetal oxygen consumption, baroreceptor, chemoreceptor, vagal and sympathetic nerve response.6,7 In this model we simulated variable decelerations caused by compression of uterine and cord vessels. We simulated contractions with a maximum strength of 160 mmHg (when bearing down), and with 70 mmHg (without pushing). We focused on alterations in fetal heart rate and pO2 in the feto-placental circulation.
Results: When intrauterine pressure is reduced, the model shows a decrease in the duration of a deceleration from 55 to 30 seconds. The depth of a deceleration is decreased from 73 to 63 beats per minute. Fetal arterial pO2 is increased from 7 to 11 mmHg. The pO2 in the fetal microcirculation increases from 8 to 9 mmHg. In the placenta, oxygen pressures do not increase as much as during bearing down: in the intervillous space pO2 decreases from 66 to 54 mmHg (baseline is 43 mmHg), while in the villous capillaries pO2 decreases from 63 to 49 mmHg (baseline is 30 mmHg).
Conclusion: In our model, the depth and duration of decelerations decrease when intrauterine pressure is reduced. Also, fetal oxygen status improves when blood flow towards the fetus is increased. In conclusion, the model shows that stopping to push during the second stage of labor results in improvement of the fetal condition and may be a useful technique for intrauterine resuscitation. The model has added value as a clinical training tool. Moreover, the model may be useful to evaluate hypothesis arised from clinical practice.
1. Woodbury RA, Hamilton WF, Torpin R. The relationship between abdominal, uterine and arterial pressures during labor. AJP. 1938;121;640-49.
2. Janbu T, Nesheim BI. Uterine artery blood velocities during contractions in pregnancy and labour related to intrauterine pressure. Br J Obstet Gynaecol. 1987 Dec;94(12):1150-55.
3. Janbu T, Koss KS, Nesheim BI, Wesche J. Blood velocities in the uterine artery in humans during labour. Acta Physiol Scan. 1985 Jun;124(2):153-61.
4. Simpson KR, James DC. Efficacy of intrauterine resuscitation techniques in improving fetal oxygen status during labor. Obstet Gynecol. 2005 Jun;105(6):1362-8.
5. Simpson KR, James DC. Effects of oxytocin-induced uterine hyperstimulation during labor on fetal oxygen status and fetal heart rate patterns. Am J Obstet Gynecol. 2008 Jul;199(1):34.e1-5.
6. van der Hout-van der Jagt MB, Oei SG, Bovendeerd PHM. A mathematical model for simulation of early decelerations in the cardiotocogram during labor. Med Eng Phys. 2012 Jun:34(5);579-89.
7. van der Hout-van der Jagt MB, Oei SG, Bovendeerd PHM. Simulation of reflex late decelerations in labor with a mathematical model. Early Human Dev. 2012 in press.
Guid Oei, MD, PhD is the Medical Director of Medical Education and Simulation Center at Maxima Medical Center, Veldhoven, Netherlands.
1243 Two Heads are Better Than One? Dyad Practice to Improve Simulation-based Procedural Skills Training
David Shanks, MD2, Ryan Brydges, PhD4, Wendie denBrok, MD2, Parvathy Nair, MD1, and Rose Hatala, MD, MSc3
1CARDIOLOGY, UNIVERSITY OF BRITISH COLUMBIA, VANCOUVER, BC, CAN and 2INTERNAL MEDICINE, UNIVERSITY OF BRITISH COLUMBIA, VANCOUVER, BC, CAN and 3MEDICINE, UNIVERSITY OF BRITISH COLUMBIA, VANCOUVER, BC, CAN and 4THE WILSON CENTRE, UNIVERSITY OF TORONTO, MICHENER INSTITUTE MEDICAL RADIATION SCIENCES, TORONTO, ON, CAN
Introduction/Background: Simulation has been widely adopted in the teaching of procedural skills to both medical and surgical trainees. Although it is resource intensive, simulation provides learners with the opportunity to develop and refine procedural skills without jeopardizing patient safety.1 While some features that enhance simulation-based learning of procedural skills have been established, one area that has received insufficient focus is the optimal group size. Research in motor learning with non-medical trainees suggests that dyad training, defined as observational training in pairs that is coupled with hands-on practice, may be as effective as individual training.2,3 Past research with non-medical trainees learning complex tasks has shown that dyad training results not only in greater immediate learning gains, but that these gains are maintained on retention testing a day later.4 Based on this body of research, we hypothesized that medical trainees will derive similar benefits from dyad training in a simulation-based procedural skills course. The objective of the present study is to compare the relative effectiveness and efficiency of dyad training versus individual learning of simulation-based lumbar puncture (LP) among first year internal medicine residents.
Methods: We conducted a two-group randomized equivalence trial. First year internal medicine residents (N=46) were randomly assigned to learn LP on a simulator as either dyad pairs or individual learners using a directed self-guided approach. Specifically, learners in both groups were given control over their practice time, including how they used two different versions of the simulator (i.e., an easy and difficult model), and an instructional video. Moreover, they could terminate the practice session whenever they saw fit. All participants were videotaped performing a simulated LP as a pre-test, an immediate post-test, and a 6-week delayed retention test. Two independent, blinded raters evaluated trainee performance using a 5-item global rating scale (GRS). We analyzed the average GRS score (mean of the 5 items) using a 2 group (dyad vs. individual) X 3 test (pre, post, retention) repeated measures analysis of variance.
Results: Our analyses showed no significant group differences (p=0.69) on pre-test, post-test, or retention test GRS scores between the dyad (2.39±0.57, 3.48±0.62, 3.12±0.85) and individual learners (2.67±0.57, 3.34±0.77, 3.21±0.79). A main effect of test showed that both groups improved significantly from pre-test to post-test (p<.001) and retained that performance following the 6-week delay. Notably, a significant interaction (p=0.02) revealed that the dyad learners experienced significantly greater pre-test to post-test gains than individual learners. Total practice time was 20.94 minutes for individuals, compared to 23.65 minutes for dyads (p=0.175).
Conclusion: The results indicate active learning in pairs, which combines both observational and hands-on practice, is as effective as individual self-regulated learning. Dyad training permits more efficient use of simulators with two learners using the same resources as an individual in solo practice. While this suggests dyad training does not create an efficiency/effectiveness trade-off, future work should test the limits of this trade-off with larger groups of learners (e.g., triads or quads).
1. Simulation-based medical education: an ethical imperative. Ziv A, Wolpe PR, Small SD, Glick S. Acad Med. 2003; 78(8): 783-8.
2. A dyadic protocol for training complex skills. Shebilske WL, Regian JW, Arthur W, Jordan JA. Hum Factors 1992;34:369–74.
3. Physical and observational practice afford unique learning opportunities. Shea CH, Wulf G, Whitacre C, Wright DL. J Mot Behav 2000;32:27–36.
4. Enhancing training efficiency and effectiveness through the use of dyad training. Shea CH, Wulf G, Whitacre C. J Mot Behav. 1999; 31:119-25.
Disclosures: Ryan Brydges, PhD, receives a Medical Education Research Grant from the Royal College of Physicians and Surgeons of Canada.
1259 Effectiveness of Simulation for Laparoscopic Surgery Training: A Systematic Review and Meta-analysis
Benjamin Zendejas, MD, MSc2, Ryan Brydges, PhD4, Stanley Hamstra, PhD3, and David Cook, MD, MHPE1
1GENERAL INTERNAL MEDICINE, MAYO CLINIC, ROCHESTER, MN, USA and 2SURGERY, MAYO CLINIC, ROCHESTER, MN, USA and 3ACADEMY FOR INNOVATION IN MEDICAL EDUCATION, UNIVERSITY OF OTTAWA, OTTAWA, ON, CAN and 4THE WILSON CENTRE, UNIVERSITY OF TORONTO, MICHENER INSTITUTE MEDICAL RADIATION SCIENCES, TORONTO, ON, CAN
Introduction/Background: The features that lead to effective simulation-based training of laparoscopic surgery have not been quantitatively synthesized in previous reviews. We sought to summarize the outcomes of simulation training for laparoscopic surgery among health professions learners.
Methods: We systematically searched MEDLINE, EMBASE, CINAHL, ERIC, PsychINFO, Scopus, key journals, and previous review bibliographies through May 2011 for original research in any language evaluating simulation, in comparison with no intervention or an active simulation-based or non-simulation training activity, for training health professionals in laparoscopic surgery. Reviewers working in duplicate evaluated study quality and abstracted study data. Outcomes were classified as reactions (learner satisfaction), skills (in a test setting) of time, process (e.g. performance rating), and product (e.g. knot strength) measures, and behaviors (when caring for patients). We pooled effect sizes using random effects, and effect sizes (ES) 0.5 to 0.79 were considered moderate and >0.8 large, and educationally relevant.
Results: From a pool of 10,903 articles, we identified 218 eligible studies enrolling 7115 trainees, including 91 (42%) randomized trials. For comparisons with no intervention (n=149 studies), pooled ES favored simulation for outcomes of knowledge (1.18 [95% CI, 0.76 to 1.62]; N=9 studies), skills-time (1.12 [95% CI, 0.98 to 1.25]; N=87), skills-process (1.23 [95% CI, 1.09 to 1.36]; N=111), skills-product (1.10 [95% CI, 0.56 to 1.64]; N=6), behavior-time (1.15 [95% CI, 0.72 to 1.58]; N=7), behavior-process (1.4 [95% CI, 0.89 to 1.54]; N=14), and patient effects (1.28 [95% CI, 0.08 to 2.48]; N=1), all p<0.05. When compared to other non-simulation instruction (n=3 studies each), results significantly favored simulation for outcomes of skills-time (ES 0.75 [95% CI, 0.24 to 1.26]) and skills-process (ES 0.54 [95% CI, 0.21 to 0.86]). Comparisons between different simulation interventions (n=79 studies) showed that while box trainers (BT) are moderately more effective than virtual reality (VR) simulators for outcomes of skills-time and satisfaction, they are no different for the remainder outcomes. Overall, there was no significant difference for training with or without computer enhanced haptic feedback. Interventions that allowed for repeated practice, incorporated features that resemble the clinical environment, and focused on clinically relevant tasks were associated with improved outcomes (See Table1).
Conclusion: Simulation-based laparoscopic surgery training of health professionals is largely effective when compared to no intervention and moderately effective when compared to non-simulation instructional modalities. The lack of clear benefit from VR over BT simulators calls into question the role of expensive VR simulators.
Disclosures: Ryan Brydges, PhD, receives a Medical Education Research Grant from the Royal College of Physicians and Surgeons of Canada.
1263 Does High Fidelity Hemorrhage Simulation Improve Intern Knowledge and Confidence?
Heather Straub, MD3, Ian Grable, MD, MPH5, Peggy Ochoa, RNC-OB, BS, MS1, Ernest Wang, MD2, Morris Kharasch, MD2, and Beth Plunkett, MD, MPH4
1CENTER FOR SIMULATION AND INNOVATION, NORTHSHORE UNIVERSITY HEALTHSYSTEM, EVANSTON, IL, USA and 2EMERGENCY MEDICINE, NORTHSHORE UNIVERSITY HEALTHSYSTEM, EVANSTON, IL, USA and 3OBSTETRICS AND GYNECOLOGY, NORTHSHORE UNIVERSITY HEALTHSYSTEM, CHICAGO, IL, USA and 4OBSTETRICS AND GYNECOLOGY, NORTHSHORE UNIVERSITY HEALTHSYSTEM, EVANSTON, IL, USA and 5OBSTETRICS AND GYNECOLOGY, WOMEN’S HEALTH, NORTHSHORE UNIVERSITY HEALTHSYSTEM, EVANSTON, IL, USA
Introduction/Background: Obstetric hemorrhage is the leading cause of maternal mortality world-wide. It is a rare and deadly complication. As such,simulation of obstetric hemorrhage has been used increasingly as an educational tool in hopes of improving clinical performance and outcome.1,2 Obstetrical hemorrhage simulations have been shown to improve resident confidence,1 which is an essential component for improvement in clinical performance.3 However, this improvement in confidence has not been linked to improvement in clinical knowledge. The purpose of this study was to determine if a high-fidelity simulation program improves residents’ knowledge and competence in the recognition and management of postpartum obstetric hemorrhage.
Methods: All incoming obstetric and gynecologic (OB) and family practice residents in 2011 received a state-mandated educational program which focused on the recognition and management of obstetric hemorrhage. The intervention entailed a 1.5 hour standardized lecture developed by the Illinois Department of Public Health, a blood estimation lab and a high-fidelity s simulation using the Noelle Simulator (Gaumard Scientific, Miami FL). After the simulation, a short debriefing session was held. The residents completed pre- and post-test including a 25 multiple-choice question standardized exam created by the Illinois Department of Public Health4 and a confidence survey. Residents rated their level of confidence in their ability to successfully perform delineated tasks on a 1-5 Likert scale (1=not confident, 5= very confident). Student’s t-test and Spearman’s correlation were used for statistical comparisons. A P-value of <0.05 was considered to be significant.
Results: Ten obstetric and four family practice residents participated in the simulation exercise. There were significant increases in both knowledge and confidence scores before and after the simulation (Refer to table). The confidence and test score appeared to be highly correlated with a Spearman’s coefficient of 0.651 (P<0.001).
Conclusion: A high fidelity simulation program during intern orientation significantly improves both resident confidence and knowledge. There appears to be a clear correlation between level of confidence and level of knowledge. Further research in resident education through simulation is needed to better characterize the relationship between confidence and knowledge.
1. Deering SH, Chinn M, Hodor J, Benedetti T, Mandel LS, Goff B. Use of a postpartum hemorrhage simulator for instruction and evaluation of residents. J Grad Med Ed.2009 Dec;1(2):260-3. PMID:21975989.
2. Pliego JF, Wehbe-Janek H, Rajab MH, Browning JL, Fothergill RE. Ob/gyn boot camp using high-fidelity human simulators: enhancing residents’ perceived competency, confidence in taking a leadership role, and stress hardiness. Simul Healthc.2008 Summer;3(2):82-9. PMID:19088646.
3. Maslovitz S, Barkai G, Lessing JB, Ziv A, Many A. Recurrent obstetric management mistakes identified by simulation. Obstet Gynecol. 2007 Jun; 109(6):1295-300. PMID: 175408004.
4. Obstetric hemorrhage education project workgroup. Maternal hemorrhage education project. Chicago, IL: Illinois Dept of Public Health; 2008. http://www.idph.state.il.us/.
1270 Impact of Instruction and Practice on Surgical Simulation Training
Angela Brunstein, PhD3, Joerg Brunstein, MSc1, David Sargsyan, MD, PhD2, Bakr Nour, MD, PhD,FACS4
1MANAMA, BHR and 2GENERAL SURGERY, HAMAD MEDICAL CORPORATION, DOHA, QAT and 3FACULTY OF HEALTH SCIENCES, ROYAL COLLEGE OF SURGEONS IN IRELAND, BAHRAIN, MUHARRAQ, BHR and 4SURGERY, WEILL CORNELL MEDICAL COLLEGE IN QATAR, DOHA, QAT
Introduction/Background: In surgical simulation training, time of mentor and trainees are the most scare resources. Time of the mentor impacts the availability of immediate feedback and guidance during training. Time of the trainee impacts the duration and timing of practice. Therefore, this research investigated the impact of immediate feedback and of sufficient practice on surgical skill acquisition for simulated laparoscopic cholecystectomy. For intelligent tutoring systems on algebra, we1 have found that self-directed learning and guiding instruction have different effects on skill acquisition. For laparoscopic surgical simulation training, research has demonstrated that trainees need expert assessment for at least some aspects of their performance,2 and that immediate feedback can reduce errors and smooth the learning curve.3 In addition, sufficient simulation training has been shown to result in automatization, expert-level performance, and transfer to the OR.4 Based on that research, we expected trainees with extensive practice to perform better during post-test and during transfer than trainees with minimum practice, improving especially in terms of speed. In addition, we expected mentored trainees with minimum practice to perform better during post-test and during transfer than participants with minimum practice, but without mentor, improving especially in terms of demonstrated skills.
Methods: In total, 26 third and fourth year medical students after surgical clerkship participated in this study for 5 individual sessions. During the first session, participants signed the consent form, performed case 1 on simulated laparoscopic cholecystectomy using a high fidelity laparoscopic simulator as a pre-test and were randomly assigned to 1 of 3 conditions: mentored, minimum practice, or extensive practice. During sessions 2 through 4, participants practiced the procedure for 30 minutes each (mentored and minimal practice) or for 60 minutes each (extensive practice). Mentored students were supervised by an experienced surgeon; all others relied on feedback of the simulation engine only. After mastering a case, participants continued with the next, more challenging case. During session 5, participants performed case 1 again as a post-test, and case 4 as a novel transfer task.
Results: Randomized and anonymized screen-recordings from pre/post and transfer tests were graded by 2 independent raters. Simulation engine protocols providing time on task, path length, proficiency, dexterity, and damage measures were analyzed for all completed cases. Both kinds of data were analyzed using MANOVAs with time of test as repeated measure for the three groups. Post-hoc analyses corresponding to Bonferroni were performed for identifying simple effects. All 3 groups improved performance significantly from pre-test to post-test. During post-test and transfer, mentored and extensive practice trainees received higher scores from faculty than minimum practice trainees. For the protocols, extensive practice trainees performed faster during post-test and transfer and received higher scores than minimal practice trainees. In contrast, mentored trainees did not perform faster during post-test than minimum practice trainees, but received higher scores. During transfer, mentored students received highest scores. When inspecting trainees’ protocols, we observed that extended practice trainees might receive perfect scores for a case in one trial and completely fail during the next trial.
Conclusion: Both, mentoring and sufficient practice were critical for surgical skill acquisition in this study. Mentoring seems to be essential for building safe practice and for preventing trainees from automatizing non-safe habits, like entering the cavity with widely open scissors. In contrast, sufficient practice is needed for automatizing skills and for speeding up performance. As a next step, we need to identify when exactly during training it is best to guide trainees’ learning by a mentor and when it is better for them to explore and practice on their own.
1. Brunstein A, Betts S, Anderson JR. Practice enables successful learning under minimal guidance.Journal of Education Psychology 2009 Nov; Vol. 101 (4), pp. 790-802.
2. Arora S, Miskovic D, Hull L, Moorthy K, Aggarwal R, Johannson H, Gautama S, Kneebone R, Sevdalis N. Self versus expert assessment of technical and non-technical skills in high fidelity simulation. American Journal of Surgery 2011 Oct; 202(4):500-6.
3. Boyle E, Al-Akash M, Gallagher AG, Traynor O, Hill AD, Neary PC. Optimising surgical training: Use of feedback to reduce errors during simulated surgical procedure. Postgraduate Medical Journal 2011 Aug; 87(1030): 524-8.
4. Stefanidis D, Scerbo MW, Montero PN, Acker CE, Smith WD. Simulator training to automaticity leads to improved skill transfer compared with traditional proficiency-based training: A randomized controlled trial. Annals of Surgery 2012 Jan; 255(1):30-7.
Disclosures: Angela Brunstein, PhD, received grant support from Qatar Foundation’s Undergraduate Research Experience Program, UREP 07-056-5-012.
1276 Estimated Blood Loss Assessment Simulation
Jyothshna Bayya, MD2, Ahmed Ahmed, MD, MSc2, Peter Homel, PhD1, Nelli Fisher, MD2
1MEDICINE, ALBERT EINSTEIN COLLEGE OF MEDICINE, MONTEFIORE MEDICAL CENTER, BRONX, NY, USA and 2OBSTETRICS AND GYNECOLOGY, MAIMONIDES MEDICAL CENTER, BROOKLYN, NY, USA
Introduction/Background: Hemorrhage remains one of the leading causes of maternal mortality in the United States.1 Peripartum blood loss is notoriously underestimated; this can lead to delay in treatment and cause significant maternal morbidity and mortality. Health-care providers tend to underestimate the volume of postpartum blood loss by about 30%. Error in estimating blood loss is dependent on actual blood loss volume. Visual estimated blood loss (EBL) can be inaccurate, overestimating at low volumes and underestimating at high volumes. Training staff to accurately assess estimated blood loss during postpartum hemorrhage will help timely initiation of management, thus preventing associated maternal morbidity and mortality.2 Medical students and experienced faculty demonstrate similar errors, and both can be improved significantly with limited instruction.3 Our objective was to evaluate if simulation-based teaching of blood loss assessment can improve accuracy of blood loss estimation.
Methods: We conducted a prospective cohort study approved by Maimonides Medical Center IRB. Two hundred twenty three participants (OB residents, interns, attending physicians, L and D Nurses, Post partum nurses, and nurse orientees) were introduced to 5 stations with known measured quantities of blood: under buttocks drape, clots, sanitary pad, speculum and floor spill. The participants completed visual estimation of these amounts as a pre-test. Teaching consisted of demonstration of 5 stations with commonly used containers like medicine cup (30ml), Dixie cup (100ml), and orange juice carton (250ml), ice cream pint (500ml), water bottle (1000ml). They were taught about the standard volumes of these items and how to correlate to the common surgical materials soaked with blood such as laparotomy sponge, 4 × 4 gauze, sanitary pad, chux, kidney basin, and placenta tray. The participants were reassessed on the 5 testing stations for blood loss assessment as a post-test. Paired t-test was used to assess the pre-test and post-test absolute differences between estimated blood volume and actual blood volume. Kruskal-Wallis test was used for comparisons of difference scores between staff groups.
Results: Participants (N=223) demonstrated statistically significant improvement in accuracy of blood loss estimation after EBL training. The absolute difference between mean pre-test estimate volume and correct blood volume (D1) was compared to the absolute difference between mean post-test estimate and correct blood volume (D2). (Table 1). Mean post-test blood volume estimates were significantly closer to correct blood volumes for all stations. No significant difference was found between difference in scores between the staff groups for any station except for the sanitary pad station (P=0.022). See Table (1): Comparison of pre-test and post-test absolute differences between estimated blood volume and actual blood volume
Conclusion: We demonstrate that simulation training of blood loss assessment improves providers accuracy in blood loss estimation. Simulation training may positively affect providers’ abilities to timely and accurately recognize obstetrical blood loss during an actual obstetrical hemorrhage, which in turn may lead to early initiation of hemorrhage management.
1. Berg C, Atrash K, Koonin M, Tucker M: Pregnancy-related mortality in the United States, 1987-1990. Obstet Gynecol 1996; 88:161-167.
2. Al Kadri H, Bedayah K, Al Anazi B, and Tamim H. Visual estimation versus gravimetric measurement of postpartum blood loss: a prospective cohort study. Arch Gynecol Obstet. 2011; 283:1207-1213.
3. Dildy G, Paine A, George N, and Velasco C. Estimating blood loss: can teaching significantly improve visual estimation? Obstet Gynecol. 2004; 104:601-606.
1344 For Your Ego or to Learn the Task? Qualitative vs. Quantitative Feedback in Endotracheal Intubation Training
Julian Manzone2, Luc Tremblay, PhD2, Devdatta Desai, MD, DHA1, Eric You-Ten, MD, PhD, FRCPC1, and Ryan Brydges, PhD3
1ANESTHESIA, UNIVERSITY OF TORONTO, TORONTO, ON, CAN and 2KINESIOLOGY AND PHYSICAL EDUCATION, UNIVERSITY OF TORONTO, TORONTO, ON, CAN and 3THE WILSON CENTRE, UNIVERSITY OF TORONTO, MICHENER INSTITUTE MEDICAL RADIATION SCIENCES, TORONTO, ON, CAN
Introduction/Background: When learning a novel task, individuals can focus on mastering the task (e.g., task orientation) and/or on how they are learning relative to others (i.e., ego orientation). Educational psychology researchers have shown that a task orientation is more conducive to learning than an ego-orientation. This benefit has traditionally been demonstrated in studies using qualitative (e.g., comments) and quantitative (e.g., grades) feedback, which researchers describe as task and ego orienting feedback respectively. Previous research has not examined whether the major driver of the learning benefit is if feedback is task or ego oriented or if it is quantitative versus qualitative. We assessed how feedback that differed on those dimensions affected medical students’ short- and long-term learning of endotracheal intubation skills. Further, we assessed how students perceived the quality and credibility of the feedback.
Methods: Using a 2 feedback format (quantitative vs. qualitative) × 2 feedback orientation (task vs. ego-orientation) orthogonal study design, we randomly assigned junior medical students to four groups: quantitative task (numeric hand motion feedback), quantitative ego (norm-referenced hand motion feedback), qualitative task (task-oriented expert feedback), or qualitative ego (norm-referenced expert feedback). Using a simulator, all participants (n=40) independently performed 20 trials of four clinical variations of endotracheal intubation: table-top, supine, ice pick, and lateral. We videotaped participants’ performances on a pre-test, immediate post-test, and 2-week delayed retention test. We assessed intubation skills using two measures: participants computer measured hand motion efficiency, and a global rating scale (GRS). Two blinded, trained anesthetists independently rated the videotaped performances on all tests using the GRS. Immediately after training, participants recorded their perceptions of feedback quality and credibility using 5-pt Likert scales. Due to high variability on pretest, we analyzed the hand motion data using 2 feedback format × 2 feedback orientation × 2 test repeated measures analyses of covariance, with participants’ pretest scores as the covariate.
Results: The group’s hand motion efficiency did not differ on the post-test or the retention test for the easier (e.g., table-top) and more difficult task variations (i.e., ice pick and lateral). For the moderately difficult supine variation, there was a significant main effect for format (p=.02) suggesting the quantitative groups used significantly fewer hand movements than the qualitative groups. Moreover, we observed a format by test interaction (p=.05) whereby the quantitative groups maintained performance from post-test (39.93±11.17) to retention test (38.90±4.77), whereas qualitative groups showed significant improvement (79.25±11.49 to 47.90±4.90). For the student perception data, we found significant orientation by format interactions. Specifically, the quantitative ego group rated feedback quality (3.20±.30 vs.1.80±0.30, p=.003) and credibility (3.78±.40 vs. 2.30±.38, p=.01) higher than the quantitative task group, whereas the two qualitative groups did not differ from either quantitative group.
Conclusion: Our findings suggest superior overall performance by groups that received quantitative feedback. That finding, along with the lack of any effect of whether feedback was task or ego oriented, suggests that medical students learning procedural skills with simulation respond in ways that do not correspond with previous education research. Despite coming from a machine rather than an instructor, students in the quantitative-ego group judged their feedback as higher quality and more credible than the other conditions. These findings suggest that medical students perform better and respond more positively to quantitative feedback that is provided with an external, ego-orienting criterion (e.g., scores achieved by residents and experts on the task). These results are particularly relevant for a framework called directed self-assessment, to which we add evidence of how students react to feedback in the simulation training context.
Disclosures: Ryan Brydges, PhD, receives a Medical Education Research Grant from the Royal College of Physicians and Surgeons of Canada.
1359 Using Virtual Reality Simulation to Assess Performance in Endobronchial Ultrasound
Lars Konge, MD, PhD2, and Charlotte Ringsted, MD, PhD1
1CENTRE OF CLINICAL EDUCATION, RIGSHOSPITALET, COPENHAGEN UNIVERSITY HOSPITAL, COPENHAGEN, DNK and 2CENTRE FOR CLINICAL EDUCATION, UNIVERSITY OF COPENHAGEN AND THE CAPITAL REGION OF DENMARK, COPENHAGEN, DNK
Introduction/Background: Mediastinal tissue staging is essential when deciding treatment for patients with non-small-cell lung cancer and should preferentially start with endosonographic procedures.1 By endobronchial ultrasound guided fine-needle aspiration (EBUS), it is possible to sample mediastinal lymph nodes, but the diagnostic yield is very influenced by the experience of the operator.2 According to clinical guidelines the important decision of when a physician is deemed competent should be based on assessing performance instead of demanding an arbitrary number of performed procedures.3 Virtual-reality (VR) simulators can provide unbiased performance data from a highly controlled, standardized measurement environment,4 but the clinical relevance of these simulator metrics needs to be explored. Earlier studies about VR-bronchoscopy simulators have found that simulator metrics could not discriminate between experts and novices who had practiced briefly on the simulator.5,6 The aims of this study were to identify simulator metrics with discriminative ability, test the reliability of these, set pass/fail standards, and use these to test the effect of EBUS simulator training. Our hypothesis was that only two hours of training would enable novices to perform like experienced clinicians on a VR simulator.
Methods: Sixteen trainees in respiratory medicine in Denmark and the Netherlands were randomized to either a novice group or a training group. The 8 physicians in the novice group and a convenience sample of 5 very experienced EBUS operators each performed two, standardized EBUS procedures on a VR-simulator. Simulator metrics were analyzed using two-way mixed design ANOVAs, and the data with discriminatory ability were combined to a single, aggregated score – the quality score (QS). The reliability of the QS was tested using generalizability theory in a G-study, and the effect of changing the number of procedures in the test were explored in a D-study. A pass/fail standard was set using the contrasting groups method. The 8 physicians in the training group were each trained individually on the EBUS simulator in a standardized fashion by the same instructor in both countries. Immediately following training each physician performed the same two EBUS procedures as the other groups, and their QS’s were compared to the pass/fail standard.
Results: Procedure time and successfully sampled lymph nodes were the only simulator metrics that showed statistically significant differences between the novices and the experienced operators, p=0.003 and p=0.039 respectively. Percentage of time with ultrasound visualization, number of biopsies made with suboptimal positioning of the scope, number of blood vessels accidentally punctured, number of hemodynamic complications, and amount of damage to the scope were all similar in the two groups. The resulting quality score (sampled lymph nodes per minute) showed acceptable reliability, generalisability coefficient=0.67. Reliability of 0.8 and 0.9 required for higher stakes examinations could be obtained by testing in 4 and 9 procedures respectively. The median QS of the novices was 0.098, range 0.04-0.21, and the median QS of the experienced operators was 0.24, range 0.21-0.26. The resulting pass/fail-standard was set at 0.19. The mean, effective simulator practice time of the physicians in the training group was 1 hour and 46 minutes, SD 17 minutes. Their median post-training QS was 0.11, range 0-0.17, meaning that none of them made the pass/fail-standard.
Conclusion: Great caution has to be applied when using simulator metrics to assess procedural competence. We only found two metrics with discriminative ability. However these data allowed for the setting of a credible pass/fail standard ensuring that a brief training session was not enough to pass the test. With proper and careful design of standardized tests, we believe that VR-simulators could be an important first line in credentialing procedural skills before proceeding to supervised performance on patients.
1. Tournoy KG, Keller SM, Annema JT. Mediastinal staging of lung cancer: novel concepts. Lancet Oncol 2012 May;13(5):e221-e229.
2. Kemp SV, El Batrawy SH, Harrison RN, Skwarski K, Munavvar M, Rosell A, Cusworth K, Shah PL. Learning curves for endobronchial ultrasound using cusum analysis. Thorax 2010 June;65(6):534-8.
3. Du Rand IA, Barber PV, Goldring J, Lewis RA, Mandal S, Munavvar M, Rintoul RC, Shah PL, Singh S, Slade MG, Woolley A. Summary of the British Thoracic Society guidelines for advanced diagnostic and therapeutic flexible bronchoscopy in adults. Thorax 2011 November;66(11):1014-5.
4. McGaghie WC, Issenberg SB. Simulations in Assessment. In: Downing SM, Yudkowsky R, editors. Assessment in Health Professions Education. 1 ed. New York: Routledge; 2009. p. 245-68.
5. Moorthy K, Smith S, Brown T, Bann S, Darzi A. Evaluation of virtual reality bronchoscopy as a learning and assessment tool. Respiration 2003 March;70(2):195-9.
6. Konge L, Arendrup H, von BC, Ringsted C. Using Performance in Multiple Simulated Scenarios to Assess Bronchoscopy Skills. Respiration 2012 March;81(6):483-90.
Disclosures: Charlotte Ringsted, MD, PhD receives grant support from Laerdal Foundation and TrygFonden. Dr. Ringsted is a consultant for Covidience.
1362 TeamSTEPPS Interprofessional Simulation Training of Pre-hospital and Hospital STEMI Teams
Susan Cristante, MSN, BSN, RN3, Shari Brand, MD5, Kim Ezrre, MBA, MSN, BSN, RN3, F David Fortuin, MD1, Bhavesh Patel, MD4, Steve Sabyan, RT (R) (CV)8, Kristen Slee, MSN, RN9, Jan Stepanek, MD7, Rebecca Wilson, RN, PhD2, and Richard Zimmerman, MD6
1MAYO CLINIC ARIZONA, PHOENIX, AZ, USA and 2ADMINISTRATIVE SERVICES EDUCATION, MAYO CLINIC ARIZONA, PHOENIX, AZ, USA and 3CARDIOLOGY, MAYO CLINIC ARIZONA, PHOENIX, AZ, USA and 4CRITICAL CARE, MAYO CLINIC ARIZONA, PHOENIX, AZ, USA and 5EMERGENCY MEDICINE, MAYO CLINIC ARIZONA, PHOENIX, AZ, USA and 6NEUROSURGERY, MAYO CLINIC ARIZONA, PHOENIX, AZ, USA and 7PREVENTIVE, OCCUPATIONAL AND AEROSPACE MEDICINE, MAYO CLINIC ARIZONA, PHOENIX, AZ, USA and 8CARDIAC CATH LAB, MAYO CLINIC HOSPITAL, PHOENIX, AZ, USA and 9EMERGENCY DEPARTMENT, MAYO CLINIC HOSPITAL, PHOENIX, AZ, USA
Introduction/Background: Acute myocardial infarction clinical guidelines recommend multidisciplinary meetings among pre-hospital (Emergency Medical Services) and hospital providers (Emergency Department (ED), Cardiology)1 in order to achieve rapid reperfusion by primary percutaneous coronary intervention (PCI) in patients presenting with ST elevation myocardial infarction (STEMI). This has become a top priority for hospitals in part, secondary to the Centers for Medicare and Medicaid ServicesHospital Quality Initiative. Processes for decreasing time to primary PCI have been effective, however quicker triage, transport, and balloon/device delivery may have the unintended consequences of creating potential threats to patient safety. For example, abbreviated communication between providers may compromise patient handoffs. These latent behavioral and environmental systems risks may not be captured by currently monitored processes and outcomes. Simulation-based team training facilitates exposure of latent risks as well as promotes the development of teamwork, which has been shown to reduce errors and improve outcomes in other settings.2,3 Team Strategies and Tools to Enhance Performance and Patient Safety (TeamSTEPPS) was designed by the Health and Human Services Agency for Healthcare Research and Quality and the Department of Defense Health Care Team Coordination Program.4 Emphasis is placed on defining team skills, identifying tools and strategies that can be used to overcome common barriers to achieve desired outcomes, and demonstrating proficiency in the skills.5 We hypothesized that interprofessional simulation training of STEMI contingency teams (pre-hospital and hospital-based) using the TeamSTEPPS® framework will provide improvements in teamwork and identify latent risks to patient safety.
Methods: This intervention study used a pre-training/post-training design. The participants included: Emergency Medical Technicians, Paramedics; Emergency Department physicians, registered nurses and technicians; and Cardiac Catheterization Lab (CCL) physicians, registered nurses and radiologic technologists. There were two educational interventions; a TeamSTEPPS® Fundamentals Course followed nine months later by simulation-based teamwork education. The simulation-based education consisted of two high fidelity STEMI scenarios. Patient care was initiated in a mock home setting, followed by patient transported to a mock ED and subsequently to a mock CCL, all within a dedicated simulation center. Scenarios incorporated planned errors and patient complications to meet the learning objectives of eliciting clear, concise communication with attention to detail and focusing on patient handoffs. The TeamSTEPPS Teamwork Attitudes Questionnaire (T-TAQ) is a useful, reliable, and validated tool for assessing individual attitudes related to the role of teamwork.6 The T-TAQ was administered prior to education, and then after simulation-based education. Unpaired student’s t-test was used to compare questionnaire results post training to pre-training, statistical significance defined as p < 0.05.
Results: T-TAQ was analyzed from 122 participants prior to training and 105 participants after training. Prior to training, mean (± standard deviation) T-TAQ responses for team structure, leadership, situation monitoring, mutual support, and communication were 4.5±0.63, 4.7±0.43, 4.5±0.57, 4.4±0.42, and 4.4±0.43, respectively (5 point Likert scale). There were no significant differences between disciplines or specialties. There were no changes in T-TAQ responses following training. The training was well received based on post training surveys. Other consequences of the training included the development of immediate feedback mechanisms to report patient findings and outcomes to pre-hospital providers. Also, latent risks were identified and to address some of these a STEMI checklist and worksheet have been implemented.
Conclusion: Interprofessional simulation training of contingency STEMI pre-hospital and hospital teams using the TeamSTEPPS framework was well received by attendees, resulted in tangible actionable items, fostered improved working relationships among disciplines and departments, and identified latent risks to patient safety. However this did not translate into improved T-TAQ scores which may be, at least in part, secondary to preexisting favorable attitudes toward teamwork.
1. Kushner, F. G., M. Hand, et al. (2009). “2009 Focused Updates: ACC/AHA Guidelines for the Management of Patients With ST-Elevation Myocardial Infarction (Updating the 2004 Guideline and 2007 Focused Update) and ACC/AHA/SCAI Guidelines on Percutaneous Coronary Intervention (Updating the 2005 Guideline and 2007 Focused Update): A Report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines.” Circulation 120(22): 2271-2306.
2. Mann, S., R. Marcus, and B. Sachs. Lessons from the Cockpit: How Team Training Can Reduce Errors on L&D. Contemporary OB/GYN. January, 2006, pp. 34-45.
3. Pronovost, P., S. Berenholtz, T. Dorman, et. al. Improving Communication in the ICU Using Daily Goals. Journal of Critical Care, 18:2, pgs. 71-75, (2003).
4. http://teamstepps.ahrq.gov/. Accessed January 28, 2011.
5. King, H., J. Battles, et al. (2008). TeamSTEPPS: Team Strategies and Tools to Enhance Performance and Patient Safety Performance and Tools). Advances in Patient Safety: New Directions and Alternative Approaches. Rockville (MD), Agency for Healthcare Research and Quality (US). 3: Performance and Tools.
6. TeamSTEPPS™ Teamwork Attitudes Questionnaire Manual Prepared for: U.S. Department of Defense, Tricare Management Activity, Subcontract agreement # SK-443-07-AIR-HCTCP, Prime Contract # W81XWH-06-F-0526Prepared by: David P. Baker, Ph.D., Kelley J. Krokos, Ph.D., Andrea M. Amodeo, M.S., American Institutes for Research, 1000 Thomas Jefferson Street, NW, Washington, DC 20007.
1383 How Do Participants of Multi-professional Obstetric Teams Evaluate Deliberate Practice?
Annemarie Fransen, MD1, and Guid Oei, MD, PhD1
1OBSTETRICS AND GYNECOLOGY, MAXIMA MEDICAL CENTER, VELDHOVEN, NLD
Introduction/Background: Expert performance is the end result of individuals’ prolonged efforts to improve their performance. The theoretical framework, which explains how different levels of performance are attained as a function of deliberate practice, was developed by Ericsson.1 Recently it was proven that simulation based medical education with deliberate practice is superior to traditional clinical medical education.2 However introducing deliberate practice into simulation training is not simple due to motivational and external constraints.3 In this study we investigate the opinion of trainees from different professions about simulation based medical team training courses, based on the principles of deliberate practice.
Methods: In a cluster randomized multi-center controlled trial, 24 obstetric departments were allocated to a one-day team training in a medical simulation center versus a similar training, but based on the principles of deliberate practice. The first “standard” group performed six different obstetric emergency situations, with emphasis on both medical skills and crew resource management. The second “deliberate practice” group trained only three of the six scenarios, which were repeated with intermittent feedback. Afterwards, the trainees had to fill in an evaluation form and rate the training on a scale of 1 to 10.
Results: The standard group, consisting of 440 trainees, rated the one-day training course significantly higher than the second group of 562 trainees(8.75 SD 0.629 versus 8.61 SD 0.651; unpaired t-test: p 0.001). Comparison between the two groups for the separate professions showed a significantly higher score among gynecologists and nurses in the standard group (8.75 SD 0.603 and 8.81 SD 0.647, respectively) compared to the deliberate practice group (8.49 SD 0.080 and 8.63 SD 0.653; p 0.016 and p 0.001 respectively). Within the standard group, there was a significantly lower rating among the midwives (8.58 SD 0.575) in comparison to the other professions (8.75, 8.66 and 8.81; ANOVA p 0.047). In the deliberate practice group there were no significant differences between the professions (Refer to Table 1).
Conclusion: This report demonstrates that a training based on deliberate practice is lower rated by trainees compared to a standard training. An explanation could be that trainees consider repetition of a scenario less interesting than participating in a new scenario, which can lead to a decrease in interest and motivation. As motivation of trainees is a fundamental part of deliberate practice, it is important to keep in mind that simulation courses based on deliberate practice should be challenging enough for all participants. However, looking at the separate score of the deliberate practice course, it is still a high score.
1. Ericsson KA. Deliberate practice and acquisition of expert performance: a general overview. Acad Emerg Med. 2008 Nov; 15(11): 988-94. Pubmed PMID: 18778378.
2. McGaghie WC, Issenberg SB, Cohen ER, Barsuk JH, Wayne DB. Does simulation-based medical education with deliberate practice yield better results than traditional clinical education? A meta-analytic comparative review of the evidence. Acad Med. 2011 June; 86(6):706-11. Pubmed PMID: 21512370.
3. Ericsson KA, Nandagopal K, Roring RW. Toward a science of exceptional achievement: attaining superior performance through deliberate practice. Longevity, Regeneration, and Optimal Health: Ann N Y Acad Sci. 2009 Aug; 1172:199-217. Pubmed PMID: 19743555.
Disclosures: Guid Oei, MD, PhD is the Medical Director of Medical Education and Simulation Center at Maxima Medical Center, Veldhoven, Netherlands.
1396 Simulation Training Improves Fundoscopic Examination Skills in Third Year Medical Students
Russell Levine, BA1, Arash Mozayan, MD5, Poulsen David, BA1, Monique Tanna, MD4, Manuela Calvo, MD4, Sharon Silberger2, Lewis Eisein, MD3, and Jamie Rosenberg, MD5
1OPHTHALMOLOGY, ALBERT EINSTEIN COLLEGE OF MEDICINE, MONTEFIORE MEDICAL CENTER, BRONX, NY, USA and 2MONTEFIORE MEDICAL CENTER, BRONX, NY, USA and 3INTERNAL MEDICINE, MONTEFIORE MEDICAL CENTER, BRONX, NY, USA and 4MEDICINE, MONTEFIORE MEDICAL CENTER, BRONX, NY, USA and 5OPHTHALMOLOGY, MONTEFIORE MEDICAL CENTER, BRONX, NY, USA
Introduction/Background: Direct ophthalmoscopy is an important skill for physicians, as it screens for numerous systemic and ocular problems. This project was designed to determine whether simulation training, followed immediately by clinical practice, would improve competency in direct ophthalmoscopy in third year medical students.
Methods: We administered a half-day course in direct ophthalmoscopy consisting of a didactic session, simulation training, and clinical practice on dilated patients. We conducted a pre-session survey and objective assessment, followed by a post-session survey and objective assessment. The survey measured students’ comfort and confidence in using the ophthalmoscope to visualize key structures. Students were tested with a mannequin head containing a fundoscopic simulator (Eye Examination Simulator 2, Kyoto Kagaku Co., Kyoto, Japan). As a novel method of objective assessment, we superimposed small-type letters behind the optic nerves of the simulators and prompted students to record the letter they saw. Each eye of the four simulators had a different letter on the optic nerve. Students used one simulator for the pre-session assessment and practice and another simulator for the post-session assessment. Statistical analysis of pre- and post-session survey and assessment results was conducted using the Wilcoxon signed rank-sum test, with p<0.05 considered statistically significant.
Results: The interventional group consisted of 9 third year medical students. Before the session, students’ median self-rated scores on a 5-point Likert scale where 1 is “strongly disagree” and 5 is “strongly agree” were 3.0 for comfort in using the direct ophthalmoscope; 2.0 and 4.0 for confidence in visualizing the optic nerve and retinal blood vessels; and 3.0 for likelihood that they would use a direct ophthalmoscope to examine patients in the future. Survey data showed an increase in student comfort to 4.0 (p=0.011); an increase in confidence in visualizing the optic nerve to 4.0 (p=0.011) and visualizing retinal blood vessels to 5.0 (p=0.012); and an increase in the likelihood they would use an ophthalmoscope to examine patients in the future to 4.0 (p=0.017). The students strongly agreed (median value 5.0) that ophthalmoscopy should be included in the curriculum during the clinical years. They felt that the didactic session (median value 4.0) and clinical session (median value 4.0) helped them gain proficiency with this skill. In addition, the assessment showed a statistically significant increase in students’ abilities to visualize the optic nerve. Before the session, 4 students were able to correctly identify the letter on the nerve; after the session, 8 students were successful (p=0.046).
Conclusion: Introduction of a half-day curriculum of direct ophthalmoscopy skills consisting of simulation training, followed by immediate clinical practice, is an effective way to increase student competency in this skill. Adoption of the curriculum will increase students’ confidence in the skill, as well as the likelihood they will use an ophthalmoscope when examining their patients.
1405 The Effect of Physical Stress on Simulated Operating Room Crisis Management
Vivian Lee, MD1, Niyati Mehta, DDS3, Richard Fidler, CRNA, CRNP, MSN, MBA2, and Jan Hirsch, MD, PhD1
1ANESTHESIOLOGY, SAN FRANCISCO VETERANS AFFAIRS MEDICAL CENTER, SAN FRANCISCO, CA, USA and 2HEALTHCARE SIMULATION PROGRAMS, SAN FRANCISCO VETERANS AFFAIRS MEDICAL CENTER, SAN FRANCISCO, CA, USA and 3DENTAL SERVICE, VETERANS AFFAIRS MEDICAL CENTER SAN FRANCISCO, SAN FRANCISCO, CA, USA
Introduction/Background: Anesthesia providers are required to manage emergency situations all over the hospital. Clinicians are generally notified of a crisis without prior warning and expected to arrive within minutes, to rapidly assess the situation and to act immediately and appropriately while still under physical stress from running to the site of the emergency. The necessary measures to respond to the emergency typically require short and long term memory, executive function and technical manual skills. There are conflicting reports in the literature if physical stress affects these skills, and if it does so in a positive or negative way.1-8 We therefore designed a study to test the hypothesis that physical stress alters the response of providers to an emergency.
Methods: Anesthesia providers were prospectively randomized to undergo either physical stress (PS) or no physical stress (NPS) prior to managing one of two randomly assigned simulated operating room crisis events. Both simulated crises incorporated measures of cognitive challenges (short and long term memory), executive function and technical tasks (intubation). At least two weeks later, the same clinicians were subjected to the other scenario (either physical stress or no physical stress) prior to managing the other simulated crisis event. Salivary cortisol levels were measured prior to and after each simulation as an objective and quantifiable measure of stress.
Results: Data analysis showed average heart rate / systolic blood pressure increases by 12% / 5% after the procedure without physical stress and by 52% / 24% after physical stress (n=8). Moreover, after physical stress the providers required 8% more time to perform the tasks and remembered 16% less diagnosis items from the patient’s history.
Conclusion: Our results indicate that both technical and cognitive performances are markedly reduced by physical stress. This could have significant implications for hospital planning and call schedules. We anticipate that further analysis will elucidate which technical skills and cognitive functions are most affected by physical stress.
1. Gothe N, Pontifex MB, Hillman C, McAuley E. J Phys Act Health. 2012. PMID: 22820158 [PubMed - as supplied by publisher.
2. Huertas F, Zahonero J, Sanabria D, Lupianez J. J Sport Exerc Psychol. 2011;33(5):649-65. PMID: 21984640 [PubMed - indexed for MEDLINE].
3. Lambourne K, Audiffren M, Tomporowski PD. Med Sci Sports Exerc. 2010;42(7):1396-402. PMID: 20019631 [PubMed - indexed for MEDLINE].
4. Audiffren M, Tomporowski PD, Zagrodnik J. Acta Psychol (Amst). 2009;132(1):85-95.
5. Audiffren M, Tomporowski PD, Zagrodnik J. Acta Psychol (Amst). 2008;129(3):410-9. PMID: 19632661 [PubMed - indexed for MEDLINE].
6. Coles K, Tomporowski PD. J Sports Sci. 2008;26(3):333-44. PMID: 18074301 [PubMed - indexed for MEDLINE].
7. Harvey A, Bandiera G, Nathens AB, Leblanc VR. J Trauma. 2011. PMID: 21808212 [PubMed - as supplied by publisher].
8. Lieberman HR, Tharion WJ, Shukitt-Hale B, Speckman KL, Tulley R. Sea-Air-Land. Psychopharmacology (Berl). 2002;164(3):250-61. PMID: 12424548 [PubMed - indexed for MEDLINE].
1418 Evaluating the Role of Medical Student Bias for Unintended Pregnancy Counseling Encounters in an OSCE
Shimena Li, BS1, Yurhee Lee1, Nikita Patel1, Shelby Marx, BS1, Brian Loveland, MPH1, Alice Chuang, MD2, and Donald Woodyard, BS1
1CLINICAL SKILLS AND PATIENT SIMULATION CENTER, UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL, CHAPEL HILL, NC, USA and 2OBSTETRICS AND GYNECOLOGY, UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL, CHAPEL HILL, NC, USA
Introduction/Background: The prevention and reduction of unintended pregnancy in the United States has been a national public health goal on the Healthy People agenda for over thirty years.1 Despite the urgency of this issue and the economic, social and health implications it ensues, it continues to be a problem for the US. Unintended pregnancies impose a significant financial strain on the nation, with an estimated $11.3 billion in U.S. taxpayer dollars being allocated to provide medical services for women with unintended pregnancies.2 The interaction between a healthcare provider and a patient provides a strong basis for the healthcare decisions that a patient makes. Those interactions can be affected by socioeconomic factors, domestic support systems and lifestyle choices of the patient, which in turn can influence the quality of care they receive. The purpose of this study is to analyze the influence that a patient’s lifestyle and household stability have on a student’s agenda during prenatal counseling for an unintended pregnancy.
Methods: Two versions of an unintended pregnancy Standardized Patient (SP) case were presented in the Clinical Performance Examination (CPX), a 14-station OSCE required of fourth year medical students. Half the students were randomly assigned to see an SP with higher socioeconomic status (SES) who is married with a stable job. The other half interacted with an SP who is of lower SES, unwed, unable to identify the father, is unemployed and relies on food stamps and Medicaid. All other factors were held constant between the two versions. Students were evaluated on a checklist to identify any differences in the overall tone of the consultation and the additional healthcare resources offered to the patient. A Chi Square analysis was performed on all dichotomous standardized patient checklist items. Checklist items that involved more than 2 choices were analyzed using independent sample t-tests.
Results: A total of 72 students saw the high SES case and 63 saw the low SES case (Total N=135). The number of students who discussed the patients’ employment status during the encounter was statistically higher (p = 0.001) for the SP with lower socioeconomic status. Students did focus more discussion on the option for terminating the pregnancy for the high SES vs low SES patient (p=.002), and students documented the SES into the progress note for low SES patients significantly more often (p<.001) than for high SES. However, the SPs did not report any greater emphasis by the students on other pregnancy options (keeping the pregnancy, or adoption) as there were no statically significant differences (p>.1), nor did the SP report any increased emphasis or pressure on their decision for any of the three options (p> .1).
Conclusion: While the students did recognize and tend to give more attention to the financial situation of the low SES patient, this did not significantly affect the students’ medical counseling during the pregnancy counseling encounters. Each patient left with an understanding of all the options available to her treatment plan. While some students did push a particular option, this was not statistically significant. We plan to investigate further to see if additional healthcare resources were more readily offered to one or the other.
1. Healthy people: the surgeon general’s report on health promotion and disease prevention. Department of Health, Education and Welfare. 1979: 8-2.
2. Wise investment: reducing the steep cost to medicaid of unintended pregnancy in the united states. Gold RB. Guttmacher Policy Review. 2011 Summer;14(3): 1.
1419 Human Simulators in Trauma Resuscitation Education for Junior Medical Staff
Geng-shiau Lin, MD, MBA, MPH, LLB1, and Shyr-Chyr Chen, MD1
1EMERGENCY MEDICINE, NATIONAL TAIWAN UNIVERSITY HOSPITAL, TAIPEI, TAIWAN, TWN
Introduction/Background: The objective was to design and implement a demonstration project to teach junior medical staff how to engage and participate in trauma resuscitation.
Methods: Scenarios with a human simulator were constructed to assess the performance of the junior medical staff for trauma patients in the emergency department. The emergency physicians gave the medical students, junior emergency residents, and student nurses general trauma resuscitation lectures based on advance trauma life support (ATLS). The junior medical staffs participated in trauma resuscitation scenarios with human simulators. A debriefing was given after every scenario by emergency physicians.
Results: All participants evaluated the medical simulation positively and a majority demonstrated improved knowledge and skills. The course modulators also learned lessons that will help better design future trauma education programs, including an emphasis on simulations over didactic lectures and the importance of group debriefing on trauma resuscitation. Drawbacks included the major time commitment for standard simulation scenarios design and implementation, sustainability, and the lack of resources to apply this curriculum for all the health care staff. The simulation-based trauma resuscitation program offers a positively evaluated possibility to enhance junior medical staffs’ performances in handling trauma patients.
Conclusion: Although it is difficult to teach the junior medical staff diagnosis and management of the trauma patient, specifically those who require resuscitation from different experts, simulation is an effective and safe model for them to learn how to engage and participate in trauma resuscitations with the concerns for patient safety. Human simulators play an important role in modern medical education, but their efficacy remains unclear. The correlation between simulator-based trauma resuscitation and real-world performance deserves objective evaluations in the future.
1. Simulation in resuscitation teaching and training, anevidence based practicereview. Sahu S, Lata I. J Emerg Trauma Shock. 2010 Oct;3(4):378-84. PMID: 21063561 [PubMed] PMCID: PMC296657.
2. Simulation-based learning: Just like the real thing. Lateef F. J Emerg Trauma Shock. 2010 Oct;3(4):348-52. PMID:21063557 [PubMed] PMCID:PMC2966567.
3. The development of a comprehensive school-wide simulation-based procedural skills curriculum for medical students. Sullivan M, Nyquist J, Etcheverry J, Nally M, Schaff P, Abbott A, Elliott D, Taylor C. J Surg Educ. 2010 Sep-Oct;67(5):309-15. PMID: 21035771 [PubMed - indexed for MEDLINE].
4. Use of simulator-based medical procedural curriculum: the learner’s perspectives. Shanks D, Wong RY, Roberts JM, Nair P, Ma IW. BMC Med Educ.2010 Nov 8;10:77. PMID: 21059253 [PubMed - indexed for MEDLINE] PMCID: PMC2988805.
5. Assessing medical decision making using human patient simulation. Fox BA. Fam Med.2010 Oct;42(9):661-3. PMID: 20927680 [PubMed - indexed for MEDLINE].
6. Does simulator-based clinical performance correlate with actual hospital behavior? The effect of extended work hours on patient care provided by medical interns. Gordon JA, Alexander EK, Lockley SW, Flynn-Evans E, Venkatan SK, Landrigan CP, Czeisler CA; Harvard Work Hours, Health, and Safety Group (Boston, Massachusetts). Acad Med. 2010 Oct;85(10):1583-8. PMID: 20881679 [PubMed - indexed for MEDLINE].
7. How to measure the quality of the OSCE: A review of metrics-AMEE guide no. 49. Pell G, Fuller R, Homer M, Roberts T; International Association for Medical Education. Med Teach. 2010;32(10):802-11. PMID:20854155 [PubMed - indexed for MEDLINE].
1435 Computerized Standardized Preoperative Patient Interview Simulator
Jason Epstein, MD1, Samuel DeMaria, Jr, MD1, and Adam Levine, MD1ANESTHESIOLOGY, MOUNT SINAI SCHOOL OF MEDICINE, NEW YORK, NY, USA
Introduction/Background: Performing a preoperative evaluation is a critical skill for anesthesiologists.1 Training and assessing the performance of preoperative evaluations with standardized patients is expensive, time consuming, resource intensive. In certain cases, virtual humans may provide more fidelity than standardized patients for certain kinds of pathophysiology.2 Residents are increasingly familiar and comfortable with virtual reality and this technology has the potential to offer a scalable, portable solution to such a problem. We created a virtual human preoperative patient interview simulator (Avatar) as a joint project with LogicJunction, Inc. (Cleveland, Ohio). The Avatar presents an immersive environment to match the sensory experience of interviewing a patient in the holding area of our hospital. Users may ask free text questions of the patient, as well as perform physical examination and order relevant laboratory studies. The AI model for the Avatar was created through an iterative test-build process. At the conclusion of the interview, the participant entered his/her assessment and anesthetic plan and was given performance based feedback based on a 23-item checklist generated by a panel of anesthesia experts via a modified Delphi process.
Methods: All 24 CA-1 residents in our program were recruited to participate in this study at four months into their training. As part of a standard performance assessment, residents were asked to perform a preoperative interview on an ASA 2 female patient presenting for emergent appendectomy. The residents were randomly assigned to interview the Avatar or a standardized patient. Interviews with a standardized patient were observed and adherence to feedback criteria was recorded. Performance of residents interviewing the Avatar was assessed by reviewing system logs.
Results: Group makeup as determined by characteristics and self-reported confidence in performing resident duties was comparable. Residents interviewing the Avatar spent 1.75 minutes longer asking questions (7.33 +/- 0.9 min vs 4.48 +/- 0.5 min; p=0.002); however, the total number of questions asked was comparable (26.92 +/- 2.3 vs 29.00 +/- 1.5; p=0.09). Residents interviewing the Avatar were more likely to perform a physical examination (83% vs 33%) and check vital signs (83% vs 8%). Overall performance as measured by total scores on feedback criteria were similar (16.25 +/- 0.6 vs 16.58 +/- 0.5; p=0.26).
Conclusion: Resident performance of preoperative interview on a virtual reality simulated patient and a standardized patient was comparable. While average interview time was increased, total number of questions, and performance on objective feedback criteria was similar between the two groups.
1. Pasternak LR. ASA practice guidelines for preanesthetic assessment. Int. Anesthesiol Clin. 2002 Spring;40(2):31-46. Review. PubMed PMID: 11897934.
2. Wendling AL, Halan S, Tighe P, Le L, Euliano T, Lok B. Virtual humans versus standardized patients: which lead residents to more correct diagnoses? Acad Med. 2011 Mar;86(3):384-8.
1464 Simulated Arterial Blood Pressure Feedback Improves Chest Compression Quality in a Single Rescuer Model
Samkon Gado, BS1, Martin Rieke, Dipl-Phys3, Paul Nietert, PhD2, Carlee Clark, MD1, Larry Field, MD1, Cory Furse, MD, MPH1, Matt McEvoy, MD1, and Horst Rieke, MD, PhD1
1ANESTHESIA AND PERIOPERATIVE MEDICINE, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 2MEDICINE (BIOSTATISTICS), MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 3PHYSICS, RUHR-UNIVERSITY BOCHUM, BOCHUM, NRW, DEU
Introduction/Background: High quality chest compressions (CC) are vital to success of cardio-pulmonary resuscitation (CPR), but have room for improvement.1 ACLS guidelines, designed as “one-size-fits-all” for lay rescuers, may fail to regard the variance of individual hemodynamic responses to CC. Individually adjusted CC may increase perfusion pressures, but would require real-time hemodynamic feedback, which is known to be effective.2 Such feedback could be established in the perioperative setting with an invasive arterial blood pressure (BP) display, which would enable the compressor to adjust CC. Thus, single specialized rescuers, going beyond ACLS guidelines, could possibly improve CPR results.3 Accordingly, this study tested the hypothesis that single-rescuer CC quality can be improved with real-time BP-display.
Methods: After IRB approval and informed consent, twenty 2nd year medical students in above average physical condition were enrolled. An endotracheally intubated, high-fidelity mannequin was equipped with additional hardware and software, comprised of a linear potientiometer/ microprocessor unit and a mathematical hemodynamic model. It allowed quantitative analysis of CC to calculate and display in real-time the achieved BP. The mathematical hemodynamic model represented a dampened harmonic oscillator coupled to a decaying reservoir. Compression rates of 100/min, depth of 5 cm, a 50% duty-cycle allowing for full recoil, and a delayed response requiring 5 effective CC for an optimized BP result were used as baseline parameters for computation of the BP-display, which ranged from 0 to 100 mmHg. The target parameter for data analysis was the area under the BP curve over time (AUC). Two CPR sessions were scheduled for each student over 2 days, one with and one without BP-display. During each session, students performed CC in five 2-minute-cycles with 10 second interruptions for simulated pulse checks. The study sessions were arranged such that possible improvements of CC due to learning effects would be taken into account. Data was analyzed and presented statistically as mean ± standard deviation for non-compared data and as mean with 95% confidence interval for comparisons. Significance was defined with p<0.05, and paired t-test was used.
Results: Five females and 15 males, who endorsed regular physical exercise of 5 hours/week were enrolled (age 25±2.7 years, weight 76.9±13.4 kg, and height 178±11.6 cm). Twenty CPR sessions with an average mean arterial pressure (MAP) of 43.3±8.7 mmHg for CC without BP-display and twenty CPR sessions with an anverage MAP of 50.9±6.1 mmHg for CC with BP-display were analyzed. The average AUC for CC without BP-display was 5201 units (95% CI of 4804-5597), and 6110 units (95% CI 5715-6507) (p<0.0001, see Figure) for CC with BP-display. The averaged increase of AUC from CC without to CC with BP-display for each student was 20.2%±17.4. Figure showing the average AUC with 95% confidence bar for CC without and CC with BP-display.
Conclusion: Our study confirms the hypothesis that a real-time BP-display during CPR allows for continuous adjustment of CC, thereby improving and maintaining the achieved BP quality of CC. Our data also confirms the legitimacy of employing measures beyond current ACLS guidelines for CPR in the high-tech environment of perioperative medicine. Regarding the learning effect due to BP-display, the common practice of rotating the compressors every two minutes could be reconsidered. Single, physically fit compressors could optimize CC over time, although being at risk of fatigue. Further simulation studies employing hemodynamic modeling and BP-display could disclose advantages of modified guidelines for high-tech environments.
1. Cardiocerebral Resuscitation: The New Cardiopulmonary Resuscitation. Ewy GA. Circulation. 2005; 111:2134-2142 PMID: 15851620 [PubMed - indexed for MEDLINE].
2. A counterbalance cross over study of the effects of visual, auditory and no feedback on performance measures in a simulated cardiopulmonary resuscitation. Cason CL, Trowbridge C, Baxley SM, Ricard MD. BMC Nursing, 2011. 10(1):15 PMID: 21810239 [PubMed].
3. Anesthesia Advanced Circulatory Life Support. Moitra VK, Gabrielli A, Maccioli GA, O’Conner MF. Can J Anesth/J Can Anesth. 2012. 59:586-603 PMID: 22528163 [PubMed - in process].
1469 Simulation-based Assessment and Retraining for the Anesthesiologist: A Case Series of Twenty Participants
Stefan Samuelson, MD1, Samuel DeMaria, Jr, MD1, Andrew Schwartz, MD1, Alan Sim, MD1, and Adam Levine, MD1ANESTHESIOLOGY, MOUNT SINAI SCHOOL OF MEDICINE, NEW YORK, NY, USA
Introduction/Background: Anesthesiologists are a heterogeneous group, and current models of determining and maintaining competency in physicians may be inadequate for those with greater needs. We present a case series of 20 participants in which the simulated OR environment provides an effective means by which to assess baseline competency in this specialized population, as well as a way to retrain physicians prior to return to clinical practice. Competency in anesthesia is a moving target throughout one’s career.1,2 The current model for assessment and maintenance, namely the ABA examination and MOCA programs, may not be adequate for all anesthesiologists; specifically, those whose needs are greater due to limited scope of practice, hiatuses from clinical duty, or medico-legal issues. While the use of simulation in anesthesia education has received increasing attention in recent years,3 at present few programs exist to address the needs of this specialized sub-population.4,5,6 Herein we describe a program developed in our institution using combined written and simulator-based education to help determine competency in licensed and previously-licensed anesthesiologists prior to return to clinical practice.
Methods: We have been using simulation as a tool for assessment and retraining at our institution since 2002. Physicians are referred or self-refer to our center with various needs: assessment, retraining, OR observation, or a combination of these. An initial needs assessment is followed by a feasibility assessment and finally an offer back to the participant or referring body regarding pricing, length, and a course of action specific to the participant’s needs. The participant then undergoes a four-hour simulator orientation, followed by an adaptable two-day formal simulation-based assessment conducted with two board-certified anesthesiologists observing the cases in real time and two other support staff who portray other OR personnel as indicated. A minimum of three cases are presented on each day with specific ACGME core competencies assessed. The assessment process is adjusted on an ongoing basis at the observer’s discretion to fully characterize areas of competency which need improvement. In addition, participants complete a standard CA-3 level Anesthesia Knowledge Test (AKT). After written and simulation-based assessment is completed, participants are debriefed extensively on their results. When indicated, a retraining regimen is designed, which is specific to their needs, but usually consists of a combination of simulation and observership with 3-4 simulated cases per day followed by 4 hours of OR time.
Results: Twenty anesthesiologists were referred to our institution between 2002-2012 for assessment. Thirteen of the 20 participants were available within one year of their program’s completion for a follow up survey. Eleven of these (55%) had successfully returned to practice. Five (25%), per our recommendations, were in supervised positions (i.e. residency, fellowship, or supervised clinical practice) and 8 (40%) were able to resume their old positions or took new positions without delay or need for supervision. Four of the 20 participants (20%) were not in practice as of one year after our program. The reasons were personal (1 participant) and medicolegal (3 participants). Two of the 14 physicians who were formally assessed in our program were deemed unlikely to improve adequately even with retraining or supervision. These physicians were unavailable for contact one year after assessment and their current employment statuses are unknown.
Conclusion: Anesthesiologists seeking to return to active clinical status are a heterogeneous group. The simulated environment provides an effective means by which to assess baseline competency and also a way to retrain physicians.
1. Gallagher CJ, Tan JM. The current status of simulation in the maintenance of certification in anesthesia. Int Anesthesiol Clin. 2010;48(3):83-99.
2. American Medical Association: Report 6 of the Council on Medical Education (A-08). Physician reentry Chicago; 2008 [http://www.ama-assn.org/ama1/pub/upload/mm/377/cmerpt_6a-08.pdf], accessed 7 April 2012.
3. Steadman RH. The American Society of Anesthesiologists’ national endorsement program for simulation centers. J Crit Care. 2008;23(2):203-6.
4. Varjavand N, Greco M, Novack DH, Schindler BA. Assessment of an innovative instructional program to return non-practicing physicians to the workforce. Med Teach. 2012; 34(4):285-291.
5. Larson CP, Steadman RH. An advanced specialty training program in anesthesiology: a special educational fellowship designed to return community anesthesiologists to clinical practice. Anesth Analg. 2006;103(1):126-30.
6. DeMaria S Jr, Levine AI, Bryson EO. The use of multi-modality simulation in the retraining of the physician for medical licensure. J Clin Anesth. 2010;22(4):294-9.
1471 Can 5 Minutes Really Make a Difference?
Patricia Ouellette, MSN4, Angelic Sorrell, MSN-ED2, Kristine Jose, ADN, PCCN1, and Patricia Denton, ADN, RN3
1INTERANL MEDICINE, VIDANT MEDICAL CENTER, GREENVILLE, NC, USA and 2CLINICAL SIMULATION CENTER, EAST CAROLINA UNIVERSITY BRODY SCHOOL OF MEDICINE, GREENVILLE, NC, USA and 3EMERGENCY RESPONSE TEAM, VIDANT MEDICAL CENTER, GREENVILLE, NC, USA and 4STAFF DEVELOPMENT, VIDANT MEDICAL CENTER, GREENVILLE, NC, USA
Introduction/Background: The nursing leadership at Vidant Medical Center, an academic medical center in eastern North Carolina, identified a staff development need related to quality outcomes and patient safety. With increases in cardiac monitored patient beds and patient acuity levels, a need for staff to respond quickly to changing patient status was apparent. Literature reviews support the need for the use of simulation training scenarios before these events occur in actual clinical care to assess resuscitation quality based on the performance of the team.1 A team of nursing educators, leadership and staff decided that the varying levels of staff experience, time and space constraints did not lend itself to a realistic goal for all nursing staff caring for cardiac monitored patients to be certified in Advanced Cardiac Life support. An American Heart Association (AHA) approved program, The First 5 Minutes® 2 (FFM), quickly became the topic of discussion as a viable option. The purpose of this research was to explore the impact of the FFM educational program on staff’s perception of competency and confidence and the factors that potentially impact patient care management during an emergent situation.
Methods: A descriptive design was utilized to evaluate the staff perception of competency and confidence in patient care management during a patient crisis situation. All of the unit’s Registered Nurses and Certified Nurse Assistants from the selected medical surgical pilot unit participated in the FFM education. Each participant was asked to complete a pre and post quiz. Unit staff was also asked to complete a Post Crisis Survey for each event that required Emergency Response Team (ERT) calls within the six month post education timeframe. Retrospective data from patient safety rounds, ERT quality data and calls was reviewed prior to and post implementation of the standardized educational offering of the FFM curriculum which served as the basis of comparison for analysis of the impact of program.
Results: Gold Standards help hospital teams save lives threatened by cardiopulmonary emergencies through consistent application of the most up-to-date scientific guidelines for in-hospital resuscitation.3 The ERT quality data revealed zero failures in deviations in the Gold Standards as measured by the AHA as compared to two failures in the prior twelve month timeframe. Quiz data indicate a trend-line in improvement of correct responses in all areas however the most significant improvement was in the areas of airway management and Automatic External Defibrillator (AED) utilization. The post crisis survey indicated that the staff “strongly agreed” with the ability to recognize a patient in crisis and active the ERT.
Conclusion: Despite the importance of teamwork and the delivery of quality care these skills are not typically included in healthcare profession curriculum.4 Findings of this pilot study support that providing education to Registered Nurses and Certified Nurses Assistants with the FFM curriculum improved preparedness of the patient care environment and increases staff perception of comfortable level with the crisis. Despite limitations to this research project such as: patient condition, staff mix, and lack of medical staff participation in the FFM curriculum, the data supports a positive impact on staff’s perception of competency and confidence with patient care management during an emergent situation.5 Data supports utilization of the FFM curriculum as a standard educational course within the organization and is currently being implemented with interprofessional teams that include RNs, CNAs and Respiratory Therapists (RTs). FFM is also offered to medical residents and attending physicians during residency conference sessions.
1. Wayne DB, Didwaina A, Fudala M, et. Al. Simulation –based education improves quality of care during advanced cardiac life support events: a case control study. Chest 2008;133:56-61.
2. Hunt Ea, Walker AR, Shaffner DH, Miller MR, Pronvost PJ. Simulation-of in-hospital pediatric medical emergencies and cardiopulmonary arrest: highlighting the importance of the fist 5 minutes. Pediatrics2008;121(1)e34-43.
3. American Heart Association. Get with the Guidelines Resuscitation. Site reviewed: June 10,2012. http://www.heart.org/HEARTORG/.
4. Institute of Medicine. Health Professions Education: A bridge to Quality. Washington, DC: National Academy Press; 2003.
5. Thomas EJ, Taggart B, Crandell S, et. al. Teaching teamwork during the Neonatal Resuscitation Program: a randomized trial. J Perinatal 2007;27:409-414.
Oral Presentation 1472 Non-parametric Resampling in Evaluating Reductions in Pneumothorax Rates and Hospital Charges of a Simulation-based Educational Intervention
Zachary Klein, PhD student3, Jesus Seda, BS1, Joshua Lenchus, DO, FACP, SFHM2
1ANESTHESIOLOGY, UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE, MIAMI, FL, USA and 2MEDICINE, UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE, MIAMI, FL, USA and 3ANESTHESIOLOGY (ECONOMICS), UNIVERSITY OF MIAMI MILLER SCHOOL OF MEDICINE, STATE UNIVERSITY OF NEW YORK, ALBANY, ALBANY, NY, USA
Introduction/Background: Patient safety has received considerable attention in recent years, particularly with respect to the ensuing costs of error.1,2 This literature makes clear the room for improvement, both in outcomes and the financial burden associated with current inefficiencies. To strengthen our institution’s alignment with these concerns we have implemented a simulation-based training program, teaching invasive bedside procedures to groups of medical residents. Our Procedure Team has demonstrated significant improvement over control-group performance regarding pneumothorax occurrence during thoracentesis. Our aim was to standardize sample demographics within the Team and control-group data, facilitating comparison and the incorporation of published figures on excess charges after pneumothorax.
Methods: Control data is obtained from 308 thoracenteses performed hospital-wide (excluding Procedure Team participants) between July 2007 and June 2008. The Procedure Team performed 95 thoracenteses during that period, and another 323 from July 2008 through June 2011. The one-year Team pneumothorax rate of 3/95 (3.16%) depicts an improvement in delivery of this procedure over the control rate of 17/308 (5.52%). The four-year Team rate of 8/418 (1.91%) showed statistically significant improvement over the control. However, these samples are not uniform in demographic composition, and speak little of the economic gains. Two studies, Shreve et al.3 and Zhan & Miller,4 convincingly assessed the costs of treating pneumothorax. Each calculated statistically significant excess charges of a large, control-matched group after pneumothorax. Further, they described specifically the gender-age distribution for their samples, allowing us to construct similarly-composed subsamples of our data (subsamples are of size 82, drawn with replacement). This effectively created standardized samples, facilitating comparison and incorporation of reported excess charges. Four pools of subsamples are required: both Team and hospital-wide samples each conforming to the demographic distribution of the two comparative studies. We attempted to mimic our single service-provider delivering roughly one thoracentesis per day for a year. Thus, we randomly drew 5 subsamples (410 procedures) from each data set, matched to each study (four sets of 410). The pneumothorax rate was recorded for each set of 410 procedures. This process was performed 100,000 times, producing a distribution of sample means; the pneumothorax rates we might encounter, insofar as our sample is representative. Finally, once our sample demographics agreed, excess hospital-charge estimates were applied to the analysis. The two studies reported figures of $27,570 (Shreve, et al.) and $17,312 (Zhan & Miller).
Results: Baseline comparisons (four-year Team rates to 2007-2008 hospital-wide) showed a 65% reduction in pneumothorax occurrence. However, the analysis indicated this reduction was closer to 73%, a figure consistent across demographic structures. Using the published excess charge rates, some $867-$1,188 in charges were avoided for every thoracentesis performed by our Team. In the four-year period under question, this amounted to between $362,544 and $496,695. If the Procedure Team performed all thoracenteses in our hospital during those four years, the reduction could have been between $1,431,096 and $1,960,640 (assuming 308*4 + 418 procedures).
Conclusion: Pneumothorax-rate estimates declined for the Procedure Team once our samples conformed to the demographic distributions of published studies. Conversely, rate estimates increased when the same procedure was applied to hospital-wide data (excluding Procedure Team participants). Such a process should be standard in reporting complication rates. Further, the calculation of unnecessary charge avoidance is staggering, both in magnitude and when considering that thoracentesis is one of five procedures performed by the Team.
1. Thomas EJ, Studdert DM, Newhouse JP, Zbar BI, Hoawrd KM, Williams EJ, et al. Costs of Medical Injuries in Utah and Colorado. Inquiry. 1999; 36:255–264.
2. Johnson WG, Brennan TA, Newhouse JP, Leape LL, Lawthers AG, Hiatt HH, et al. The Economic Consequences of Medical Injuries. JAMA. 1992 May 13;267(18):2487-2492.
3. Shreve J, Van Den Bos J, Gray T, Halford M, Rustagi K, Ziemkiewicz E. The Economic Measurement of Medical Errors. Schaumburg, IL: The Society of Actuaries, The Society of Actuaries Health Section; 2010 Jun.
4. Zhan C, Miller MR. Excess Length of Stay, Charges, and mortality Attributable to Medical Injuries During Hospitalization. N Engl J Med. 2003; 290:1868–1874.
1484 Are Movements of the Team Members in Obstetric Scenarios Related to Crew Resource Management Training in a Medical Simulation Center?
Annemarie Fransen, MD,2, Bas Oei,1 and Guid Oei, MD, PhD2
1AMSTERDAM, NLD and 2OBSTETRICS AND GYNECOLOGY, MAXIMA MEDICAL CENTER, VELDHOVEN, NLD
Introduction/Background: Crew resource management (CRM) courses teach healthcare teams about clear communication and leadership. The leader of the team should gather the available information, summarize, take decisions, and give instructions to the other team members. In this study the hypothesis is tested that the team leader moves less than other members of a medical team after they have received CRM training in a simulation center.
Methods: Twenty multi-professional obstetric teams who underwent a one-day course in a medical simulation center about CRM participated in this study that was part of a cluster randomized controlled trial. Before and after providing instructions about CRM ten different teams were videotaped during two different scenarios. The obstetric scenario before instruction was about shoulder dystocia and the scenario after instruction was about post partum hemorrhage. The video recordings were evaluated by an independent team of four observers who measured the time that different members moved during the scenario. Main outcome was the relative time: the absolute time the participant was moving divided by the absolute time the participant was in the room during the scenario.
Results: Eighteen gynecologists, 17 midwives, 3 residents, and 50 nurses participated in this trial. The average relative time that the gynecologist, midwife, and principal nurse moved during the scenario before the CRM course was given was resp. 9.34% (SD 3.49), 3.09% (SD 3.1), and 8.1% (SD 4.0) and after the CRM course resp. 6.27% (SD 3.43), 8.66% (SD 5.62), and 15.05% (SD 8.39). Eighteen scenarios could be used for analyses. In nine scenarios before and nine scenarios after the CRM course the relative time of the leading obstetrician could be compared to the other team members. In one of the scenarios before and in five of the scenarios after the CRM course the obstetrician moved less than the other team members (Fisher’s exact test p=0.06).
Conclusion: In an obstetric emergency the gynecologist should take the lead, followed by the resident, midwife, first nurse, second nurse, and third nurse. In this study the gynecologist did not move less than the other team members during an obstetric simulation scenario, but after an oral presentation and simulation training with focus on CRM the gynecologist moved less (not significant) than the other team members. There seemed to be a hierarchical order in which team members move. This is consistent with the principles of good leadership where one expects that the leader moves less than the other team members. Measurement of movement during obstetric emergency scenarios might therefore be an objective instrument to provide the team of feedback.
Disclosures: Guid Oei, MD, PhD is the Medical Director of Medical Education and Simulation Center at Maxima Medical Center, Veldhoven, Netherlands.
1492 Differences between Knowledge and Skills in ALS Among 1st Year Anesthesiology Residents in France
Grigoresco Benedicte, Resident1, Alexandre Mignon, MD3, and Antoine Tesniere, MD, PhD2
1ANESTHESIOLOGY AND INTENSIVE CARE, ASSISTANCE PUBLIQUE HOPITAUX DE PARIS, PARIS, FRC and 2INTENSIVE CARE, ILUMENS, APHP, PARIS, FRA and 3ANESTHESIOLOGY, UNIVERSITY PARIS DESCARTES, PARIS, FRA
Introduction/Background: The level of both knowledge and skills in performing ALS in anesthesia residents in France is poorly evaluated. During a regular simulation training for anesthesia residents, we tried to evaluate the knowledge and skills of 1st and 2nd year (respectively n=80 and n=65 students)) anesthesiology residents for performing CPR when facing an unexpected cardiac arrest situation. Among mandatory simulation training, we included a simulated CPR evaluation to adress this important question.
Methods: The assesment of theoritical knowledge and skills on ALS was first performed with a self evaluation form, that was given right before the simulated scenario, and that was sent after 5 months to test the retention of the knowledge. This form was based on the lastet ILCOR recommandations (2010), validated by two independent experts, and also included training informations concerning the residents. The residents were then asked to undergo a simulated scenario of CPR where they are required to take care of a 55 year old patient in pre hospital situation, first alone, and then with the hel p of a witness. The test was performed on a RescuciAnn Skill Station. Recorded parameters included the frequency of chest compressions, as well as depth of chest compression, and hands on time. After informed consent, all residents were video recorded during the scenario. The gold standard of clinical response in this scenario was defined by two experts and five anesthesiology professors, and was used to establish an objective criteria evaluation form. All recordings were reviewed off line by a work psychologist to assist the clinical evaluation performed by the examinator.) Statistical analysis was performed to compare the answers to the initial evaluation form with the observed clinical skills at day 0 and at 5 months. A p value < 0.05 was considered to be statistically significant.
Results: 145 residents were enrolled in the study. All responded to the initial evaluation form, but only 38% did fill the 5 months evaluation form. All 1st and 2nd year residents performed an acceptable level of theoritical knowledge on BLS and ALS, with no difference between the 1st and 2nd year residents. Most of the residents (94%) described appropriately the diagnosis signs of CPR, as well as the different steps for performing CPR (92%), the main features of chest compressions (95%), the use of epinephrine (83%), and the different algorithms for asystolia (92%) or venticular fibrillation (94%) in advanced life support. However, when asked to perform CPR in a specifically designed simulation scenario, skills were not correlated with theoritical knowledge. The diagnosis of asystolia, when present, was only performed in 62 % of the residents. Similarly, the main progonostic factors, and the benefits of hypothermia after CPR were poorly achieved (25% and 40 % respectively). Also, an insufficient number of residents performed an appropriate call for help (35%), or asked for an AED (50%). The data recored and analysed from the mannikin showed that the overall hands on time is only 45% +/_ 9%, and that the average chest compression rythm is 92+/_8 bpm. This acceptable knowledge was probably due to the recent participation of 78% of the residents to an ALS situation (either in anesthesia, intensive care, pre hospital teams or operating room rotations), observed on the initial evaluation form.
Conclusion: The knowledge of BLS and ALS in 1st and 2nd year anesthesiolgy and intensive care residents in France is rather good, but is in contrast with poor performance in skills for performing CPR. From the analysis of the data, we will identify the potential gaps in training and will propose appropriate simulation training to reduce the difference between knowledge and skills when performing CPR.
1528 An Application of a Clinical Map to E-mail Tutorials with a Quantitative Evaluation of Two-dimensional Knowledge Structure in an Emergency Department
Yasuhiko Ajimi, MD, PhD2, Tetsuya Sakamoto, MD, PhD2, Takehiro Nakamura, MD, PhD1, Benjamin Berg, MD3, and Hiroshi Okudera, PhD, MD4
1NEUROLOGICAL SURGERY, KAGWA UNIVERSITY, MIKI, KAGAWA, JPN and 2EMERGENCY MEDICINE, TEIKYO UNIVERSITY, SCHOOL OF MEDICINE, ITABASHI, TOKYO, JPN and 3SIMTIKI SIMULATION CENTER, UNIVERSITY OF HAWAII, JOHN A BURNS SCHOOL OF MEDICINE, HONOLULU, HI, USA and 4CRISIS MEDICINE, UNIVERSITY OF TOYAMA, TOYAMA, JPN
Introduction/Background: To know the true meaning of a medical word is to put the word into practice adequately in a medical situation. We must know the timing and the category to use the word more precisely when we put the word into practice in emergency. Both of the timing and the categories of the medical words form two-dimensional knowledge structure. We reported a new method to evaluate medical knowledge structure in emergency department using a clinical map (CM) on Research Abstract in the last IMSH. This time, we utilized a CM as a material of e-mail tutorials of emergency medicine for medical students.
Methods: Fifty four medical students in fifth-year participated in this study. A CM has a tabular structure like a clinical pathway and consists of a frame and elements. The horizontal axis of the frame means steps of an algorithm and vertical axis means the categories of medical practice consisting of physical and physiological examinations, blood/urinary examinations, imaging tests and treatments. The elements mean specific examinations, its results or treatments such as anisocoria, 12-lead ECG, atrial fibrillation, oxygen mask 6L/min, or extracellular fluid. We removed twenty four elements from a CM which expressed an initial management for a patient with severe ischemic stroke and designed a training to return those 24 elements to the exact places in the CM frame represented in a Excel file. We called this training a CM puzzle. Each student performed the CM puzzle before bed side learning. After the each puzzle game, we counted the number of erroneous, unused and correct elements. Erroneous elements were classified into three categories: step errors, practice errors and both errors.
Results: (1) Step errors: mean (SD), maximal and minimal numbers of 54 students were 6.9 (2.8), 14 and 2. (2) Practice errors: 0.7 (0.7), 3 and 0. (3) Both errors: 0.4 (0.7), 3 and 0. (4) Unused elements: 0.6 (1.5), 6 and 0. (5) Correct elements: 15.4 (3.0), 21 and 9.
Conclusion: The results showed their structure mistakes of an initial management for a patient with sever ischemic stroke. Step and practice errors mean lack of understanding timing to perform the practices and lack of understanding the meaning of the practices themselves, respectively. We could implement e-mail tutorials about true meaning of each medical word in the initial management for a stroke patient in the ED. E-mail tutorials for medical students using a CM puzzle game enable us to evaluate their two-dimensional medical structures in emergency medicine.
1. Ajimi Y, Okudera H et al. Knowledge management in ISLS course Introduction of a group work training using clinical maps for an initial treatment of stroke. Journal of Japanese Congress on Neurological Emergencies (2010) 22: 1-5.
2. Ajimi Y et al. Group Work with Clinical Maps: A new training method for constructing a medical knowledge structure of stroke in ER. WIP, 10th IMSH, 2010.
3. Ajimi Y et al. Utility of puzzle type of clinical maps as a training material for an initial treatment of stroke. WIP, 11th IMSH, 2011.
4. Ajimi Y et al. A new method for quantitative evaluation of medical knowledge structures in emergency room by a clinical map. Research Abstract, 12th IMSH, Jan. 30, 2012.
5. Ajimi Y et al. Handout of ISLS course. module D: Tabletop Exercise using Clinical Map Puzzles, Pre-course WS. 12th IMSH, Jan. 28, 2012.
Disclosures: Benjamin Berg, MD is on the CME/CNE Speaker’s Bureau for Laerdal Medical.
1540 Beyond the ADA: Standardized Patients with Physical Disabilities
Donald Woodyard, BS1, Brian Loveland, MPH1, and Pranay Prabhakar, BS1
1CLINICAL SKILLS AND PATIENT SIMULATION CENTER, UNIVERSITY OF NORTH CAROLINA AT CHAPEL HILL, CHAPEL HILL, NC, USA
Introduction/Background: In 2005, the US Surgeon General issued a call to action to “improve the health and wellness of persons with disabilities.” The report cited a failure of medical education programs to teach health professional students concepts of disability and how to treat the whole patient with a disability appropriately. While didactic lectures can provide an important framework for understanding ADA requirements and assessing, treating and managing patients with disabilities, learners must be afforded the opportunity to both practice and be assessed on their clinical skills with persons with disabilities.
Methods: We introduced an Observed Structured Clinical Examination (OSCE) case for a patient presenting with fatigue secondary to new onset diabetes. Two very similar versions of the case were presented. Both patients experienced a severe car accident seven years earlier that resulted in chronic pain and reduced physical activity; however, one patient suffered a Spinal Cord Injury (SCI) while the other had not. While the SCI patient did require slight modifications to the medical history, such as using a catheter, most relevant social and medical history items were kept constant between the two cases. Both patients presented with fatigue worsening over 6-weeks, increased thirst and urination, and recent weight loss. The SCI patient used a wheelchair. Fourth year medical students were randomly assigned to see only one version of the case during their Clinical Performance Examination (CPX). Students were instructed to take a focused history and provide a routine focused physical examination and were not informed of the disability status of the patient prior to the encounter. For the SCI patient, students were permitted to complete the physical examination in the wheelchair or to assist the patient to the exam table to complete the physical examination. Each encounter was allotted a maximum of fifteen minutes for the history and physical exam with the patient followed by ten minutes for ordering labs and completing a 20-item questionnaire using a 4-point Likert Scale (0-3). The SP completed a 28-item checklist on history questions asked and physical exam techniques performed and a 10-item relationship and communication instrument.
Results: Seventy-eight students saw the SCI case, while 89 saw the non-disability case (N=167). From the student post-encounter questionnaire, all students reported feeling comfortable around persons in wheelchairs with a mean of 2.91 following the SCI encounter compared to 2.84 from the non-disability case. Students reported feeling “sorry for” persons with disabilities significantly less following the SCI encounter with a mean of 1.76 compared to 2.27 on the second case. Students also reported feeling awkward having to examine patients during the CPX 32.5% of the time following the SCI compared to 25.9% in the other case. For a fatigue case with a significant history of increased urination, students elicited the information significantly more often (100% compared to 79.8%) during the SCI encounters. Students varied significantly on appropriate completion of all physical examination techniques most notably by doing physical examination techniques over the gown for the SCI patient.
Conclusion: The SCI case ran successfully, demonstrating that SP encounters with patients with physical disabilities is reasonable. This can be used as both a teaching and assessment tool. Students report feeling comfortable around persons in a wheelchair regardless of whether they had recently encountered one. However, students reported feeling sorry for patients with disabilities significantly less often when they had just seen the patient with a disability. Students approached the history and physical examination differently between the two patients. While that may be expected, students focused on more obvious external observations (emptying the catheter bag) and did not attempt to examine on bare skin for the patient with SCI. More opportunities to work with patients with disabilities should be provided to medical students.
1543 Comparative Human Factors Evaluation of Three Hospital Defibrillator User Interfaces
Richard Fidler, CRNA, CRNP, MSN, MBA1, and Meshell Johnson, MD2
1HEALTHCARE SIMULATION PROGRAMS, SAN FRANCISCO VETERANS AFFAIRS MEDICAL CENTER, SAN FRANCISCO, CA, USA and 2PULMONARY CRITICAL CARE, SAN FRANCISCO VETERANS AFFAIRS MEDICAL CENTER, SAN FRANCISCO, CA, USA
Introduction/Background: In-hospital cardiac arrest continues to be a major public health problem with much room for improvement in survival. Equipment-related issues have recently been cited as a significant contributor to the suboptimal outcomes of resuscitation management. A systematic evaluation of the human-device interface was undertaken to evaluate the intuitive nature of each competing defibrillator that met minimum equipment specifications.
Methods: A convenience sample of 74 multidisciplinary nurses, physicians, and biomedical engineers participated in this IRB-approved study at 5 different hospitals. All subjects’ performances were evaluated without any training or orientation to the devices being evaluated to assess the intuitive nature of the user interface. Head cameras and direct timed observations of performance were obtained by 2 raters in a structured, simulation-based usability evaluation, with follow up interviews on attitudes and recommendations that were conducted in private in clinical units. Each manufacturer supplied devices that met minimum specifications of AED and manual defibrillation modes, synchronized cardioversion, transcutaneous pacing, noninvasive blood pressure measurement, pulse oximetry, and waveform capnography. Devices tested were the Physio-Control LifePak 15, the Zoll R Series Plus, and the Philips MRx.
Results: There were significant differences in the usability of the devices: timed performance of critical steps for use of AED for ventricular fibrillation, manual defibrillation of VT, synchronized cardioversion of rapid atrial flutter, and transcutaneous pacing of complete heart block are shown in the figure. In this study, the Zoll was the fastest for AED shock, Philips for manual defibrillation, and Physio-Control for transcutaneous pacing. Participants also shared that the Philips was lightest and most portable; the Physio-Control was heaviest but most durable, and the Zoll was the simplest looking device.
Conclusion: Each device tested has positive and negative attributes for consideration in the context of cost, the amount of staff training, and size/weight for storage and transport when selecting new critical equipment. The characteristics of the end-user are often overlooked in selecting new equipment. Usability evaluations should also include determining the predominant uses of the devices: for example, if the most common use is defibrillation, then data regarding the top-performing device in defibrillation should be weighted more heavily than the best performing pacing function. Furthermore, usability evaluations can produce recommendations for equipment manufacturers to improve the user interface, such as adding color and flashing light to Philips sync button and differentiating energy settings from heart rate on the Zoll display by using different fonts or colors. This study was deliberately conducted without user training to test intuitiveness of usability, so it is expected the performance would improve with any education. Systematic comparison of defibrillator user interfaces allows contextually sensitive evaluation of the device usability prior to purchase demonstrating the utility of human factors evaluations prior to making expensive purchases.
1554 Simulation as an Assessment Tool for Chest Tube Placement by Junior Residents: Is There Need for Additional Training?
Raaj Ruparel, MD3, Shahzad Ali, MD1, David Farley, MD3, and William Dunn, MD2
1GENERAL SURGERY, MAYO CLINIC, ROCHESTER, MN, USA and 2MULTIDISCIPLINARY SIMULATION CENTER, MAYO CLINIC, ROCHESTER, MN, USA and 3SURGERY, MAYO CLINIC, ROCHESTER, MN, USA
Introduction/Background: Our 32 general surgery interns rotate though services that routinely insert chest tubes during their first year. Complication rates associated with tube thoracostomy are nearly 30% for all operators and higher among physicians in training.1 We sought to assess baseline proficiency of chest tube placement within a simulation environment.
Methods: Our 32 general surgery interns participated in a simulated scenario involving a clinical vignette and a model thorax with representations for skin, subcutaneous fat, intercostal muscle, ribs, parietal and visceral pleura. Participants were evaluated for prior training or experience and were assessed using a 7-point checklist based on the ATLS criteria for chest tube placement. A passing score was designated as 7/7.
Results: Of the 32 participants, 11 had received prior training in the form of an ATLS course. All 11 had done so within 4 weeks prior to the simulation. The mean score of the ATLS group was 5.5 (range 4-7, SD = 1.02) with 2/11 receiving a passing score. Of the 21 trainees without prior training, 0/21 achieved a passing score with a lower mean score of 3.6 (range 1-6, SD = 1.66, p= 0.0002). Common errors included failing to digitally examine the incision (n = 24), neglecting to site-prep (n = 17), and inserting the chest tube backwards (n = 5).
Conclusion: While participation in the ATLS course correlates positively with a higher score for chest tube placement, 81% of those with previous exposure did not achieve a passing score. This study exposes a very real need for chest tube training among residents with no prior experience and for recent ATLS course graduates. We are embarking on just such a dedicated, standardized experiential training and evaluation regimen.
1. Bailey RC. Complications of tube thoracostomy in trauma. J Accid Emerg Med. 2000 Mar;17(2):111-4. Pub Med PMID: 10718232.
1588 Implicit Theories to Analyze an Instructor Training Program for Undergraduate Clinical Simulation
Soledad Armijo, MD1, Matilde Pulgar Garcia, MD1, and Karen Vergara, EU1
1CENTRO DE SIMULACION CLINICA, UNIVERSIDAD DIEGO PORTALES, SANTIAGO, CHL
Introduction/Background: The insertion of clinical simulation in undergraduate curriculum not only involves training activities for students, but there is also a need for teachers to be prepared for simulation-based instruction. Clinical simulation assumes adherence to non-traditional teaching models, moving away from models spread among basic science and clinical teachers.1 In the context of developing curriculums, teachers usually come from the clinical setting, and already have multiple parallel academic tasks.2 To build a group of instructors who accept the methodology, and incorporate the principles of positive environment for learning could be a difficult road. The use of elements that can help to select the best instructors could be a valuable tool for simulation-based curriculum innovations. From the psychological perspective, the study of implicit theories is necessary to know the frameworks through the teachers perspective and process information and analyze information from their teaching.3 We thought that the knowledge of teachers’ implicits theories seems likely to contribute to the process of recruitment and the development of instructors in the context of undergraduate simulation. The objective of the study was to describe the results of a Clinical Simulation instructor training program applied to the second phase of curricular insertion at Facultad de Medicina, Universidad Diego Portales, and to analyze the association between the third level of Kirkpatrick and implicit theories of teachers at the time of initial instruction.
Methods: In the process of incorporating simulation into medical and nursing curriculum at Universidad Diego Portales, the teacher training program begins in parallel with the leaders formation as clinical simulation instructors. The contents of the internal teacher training program arose from the professor needs analysis between 2007 and 2010. The courses were implemented on January and July of 2011. At the first day of basic instructor training course, all participants completed an inventory of implicit theories of Javier Marrero. Data from this instrument were analyzed in the traditional way (average of questions associated with each theory and definition of the dominant theories using the highest average). The evaluation of teacher training program was based on the Kirkpatrick model. The tests to evaluate the program were used a Likert survey to assess program satisfaction (validated in 2008), semi-structured qualitative assessment survey and monitoring of curriculum integration projects, to evaluate the transference.
Results: There were 40 participants in the basic training program. On the third level of Kirkpatrick, there was a high transfer to the undergraduate programs (80% of all participants, 100% of teachers of nursing, 87% of teachers of medicine). 52% of the teachers involved in projects to be implemented during 2012; 20% of the teachers had been working only with low-fidelity simulation; 12% only with high fidelity simulation; and 68% with both. Those teachers who did not show a clearly dominant implicit theory formed the group that completed the transfer of basic education to its undergraduate and not working on integration projects in the short term. On the other hand, among those who established the transfer, there was a clear dominance of the profiles associated with interpretive theories among those involved in teaching novice learners, and coincidence of emancipatory theories and activity between developers teaching with students more advanced levels.
Conclusion: The success or failure of educational innovations depends largely on how the professors interpret, redefine, filter and shape changes proposed. The way these mediators operate processes depends on what the teachers think, feel and do and their views about different dimensions of education. A future task is to clarify whether implicit theories can be modified through the years of experience, and if they offer a tool for prediction of stability of the faculty.
1. The Conceptions of In-service and Prospective Primary School Teachers About the Teaching and Learning of Science. Raphael Porlánand Rosa Martín del Pozo. Journal of Science Teacher EducationVolume 15, Number 1(2004), 39-62.
2. Halting the revolving door of faculty turnover: recruiting and retaining clinician educators in an academic medical simulation center. Kim S, Ross B, Wright A, Wu M, Benedetti T, Leland F, Pellegrini C. Simul Healthc. 2011 Jun;6(3):168-75.
3. Marcelo García, C. (1987). El pensamiento del profesor. Barcelona: Ediciones GREAC.
1595 Effect of a Designated Reader and Cognitive Aid on the Resident Performance During Simulation of Perioperative Emergencies
Richard Sedlak2, Jeremy Smalley, MD4, Michael McEvoy1, Jonathan Smalley2, Carlee Clark, MD2, Larry Field, MD2, Cory Furse, MD, MPH2, Horst Rieke, MD, PhD2, Paul Nietert, PhD3, and Matt McEvoy, MD2
1ANESTHESIA, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 2ANESTHESIA AND PERIOPERATIVE MEDICINE, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 3MEDICINE (BIOSTATISTICS), MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 4ORTHOPAEDICS, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA
Introduction/Background: Use of cognitive aids by a team leader has improved performance during high-fidelity simulations of in-hospital cardiac arrest.1 One previous study reported that the use of a designated reader of patient management protocols may increase guideline adherence during perioperative emergencies.2 Accordingly, we conducted a randomized, controlled trial to test whether the addition of a Reader plus electronic cognitive aid (App) was superior for managing multiple perioperative emergencies when compared to management from memory alone.
Methods: After IRB approval and informed consent, 27 anesthesiology residents ranging from PGY1 to PGY4 were enrolled and randomized into two groups: Reader+App (N=14) and Control (N=13). Each resident managed four high-fidelity simulations of perioperative emergencies, which were video recorded for later scoring. The scenarios were Anaphylaxis, ST Elevation Myocardial Infarction (STEMI), Malignant Hyperthermia (MH), and Hyperkalemia (HYPERK). In all cases SimMan3G® and trained confederates were used to simulate the patient and clinical staff, respectively. At a standardized point in the scenarios, the Reader prompted the participant for a diagnosis, and the Reader offered to read protocol steps of that diagnosis for those in the Reader+App group. For the Control group, this person made themselves available to assist in any way needed. In the Reader+App Group, an anesthesia technician was trained and equipped with a novel iOS-based App containing management protocols for numerous perioperative emergencies. The Control group had an equivalent number of confederates available to assist with any task. Performance was assessed using a grading checklist that was developed through a modified Delphi technique and based upon published best-practice guidelines, with certain items deemed to be critical actions (e.g. beta-blockers in STEMI scenario). Performance was recorded as the percentage of overall and of critical items performed during the scenarios. Data is reported as Mean±SD and was analyzed via an ANOVA for all individual scenario comparisons and via a general linear mixed model to account for the effects of the scenario and the presence of the APP when comparing overall results.
Results: The Reader+App group scored significantly better in all four scenarios in overall performance (% of steps correctly done) and in 3 of the scenarios in percentage of critical steps performed (see Figure 1). The percentage of overall correct steps in Anaphylaxis increased from 62±15% in the Control group to 75±17% in the Reader+App group [p<0.04], in STEMI this increased from 50±8% in the Control group to 81±11% [p<0.0001], in MH this increased from 43±12% in the Control group to 61%±10% [p=0.0006], and in HYPERK this increased from 66±8% in the Control group to 79±8% [p=.0008].
Conclusion: Our results demonstrate that the implementation of a Reader with a novel decision support tool markedly improves adherence to published guidelines for patient management during a variety of simulated perioperative emergencies. Although participants in the Reader+App group performed remarkably better than the control group, the critical averages still fell below 100% compliance. Future studies are needed to elucidate: a) how to improve adherence to 100% of published guidelines, b) whether these findings are reproducible during in-situ simulation using real clinical teams, and c) who is the most appropriate personnel for the Reader role.
1. Low D, Clark N, Soar J, Padkin A, Stoneham A, Perkins GD, Nolan J. A randomised control trial to determine if use of the iResus© application on a smart phone improves the performance of an advanced life support provider in a simulated medical emergency. Anaesthesia 2011;66:255-62.
2. Burden AR, Carr ZJ, Staman GW, Littman JJ, Torjman MC: Does every code need a “reader?” improvement of rare event management with a cognitive aid “reader” during a simulated emergency: a pilot study. Simul Healthc 2012;7:1-9.
1616 Effect of a Novel Cognitive Aid on Adherence to Guidelines in the Management of Medically Unstable Patients
Louise Alexander, Jonathan Smalley2, Richard Sedlak2, Michael McEvoy1, Deborah DeWaay, MD4, Jeffrey Cluver, MD6, James Horne, NREMT-P3, Paul Nietert, PhD5, and Matt McEvoy, MD2
1ANESTHESIA, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 2ANESTHESIA AND PERIOPERATIVE MEDICINE, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 3CLINICAL EFFECTIVENESS AND PATIENT SAFETY CENTER, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 4INTERNAL MEDICINE, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 5MEDICINE (BIOSTATISTICS), MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA and 6PSYCHIATRY AND BEHAVIORAL SCIENCES, MEDICAL UNIVERSITY OF SOUTH CAROLINA, CHARLESTON, SC, USA
Introduction/Background: Each year in the United States, 500,000 people have a myocardial infarction (MI), 634,000 people are hospitalized for a Chronic Obstructive Pulmonary Disease (COPD) exacerbation, and 11-42% of medical in-patients have delirium.1,2,5 Each of these unstable conditions have best-practice guidelines that are published for proper patient assessment and management. Interns are frequently the first-line of assessment and care in these situations. However, little training is given for the assessment and management of unstable patients in most medical school curricula.3 One study has shown that cognitive aids can improve adherence to guidelines in simulations of cardiac arrest.4 However, this has not been evaluated in medically-unstable patients. As such, we performed a pilot study to test the hypothesis that the use of an iOS-based cognitive aid application (APP) will improve adherence to published guidelines for the management of a variety of unstable patient conditions by 4th year medical students and interns.
Methods: After IRB approval and informed consent, four interns and twelve 4th year medical students were enrolled and randomized into two groups: No APP (N=8) and APP (N=8). All participants had been given an orientation to the simulation setting prior to involvement with this study. Each participant managed 3 unstable patient scenarios: ST-Elevation MI (STEMI), COPD Exacerbation, and Delirium. Two actors trained as standardized patients portrayed a patient and nurse in each station. The APP group was given a brief orientation to the iOS-based APP containing management protocols. Performance was assessed through a grading checklist that was based upon published management guidelines and finalized through a modified Delphi technique. Overall scores and critical item scores were recorded as a percent (%) of total items completed. To compare the effect of APP v. No APP, data were analyzed via a general linear mixed model with random participant effects to account for multiple measurements recorded on individual participants. Data are reported as Mean±95%CI in the text and figure.
Results: Compared to no APP use, APP use was associated with significantly higher scores for each individual scenario and overall for both overall correct (%) and critical actions correct (%) (p<0.001 for all comparisons, see Figure 1). On average, the APP group performed 30.2 percentage points higher on the overall % correct than the No APP group, and 26.9 percentage points higher on the critical items % correct (p<0.0001 for both). Interaction terms involving scenario and APP use were also assessed, and they were not significant, meaning that the APP did not differentially help in one scenario versus another. This is true for overall scores and for critical scores.
Conclusion: This pilot study tested the effects of a novel electronic cognitive aid (APP) on the assessment and management of medically-unstable patients. Three important findings emerged. First, the APP significantly improved performance across scenarios involving cardiac, pulmonary, and neurologic/psychiatric instability. Second, senior medical students and interns managing these conditions from memory alone had a very poor adherence to published guidelines, adherence to which is associated with improved patient outcomes. Third, even with the APP, students failed to reach a score of 100%. Thus, future research needs to address the following: a) investigate the best pedagogical approach for training medical students and interns to manage medically-unstable patients and b) perform human factors analysis on the cognitive aid in order to determine the barriers to 100% adherence to published best-practice management guidelines for residents and practicing physicians.
1. Antman EM, Anbe DT, Armstrong PW, et al. ACC/AHA guidelines for the management of patients with ST-elevation myocardial infarction: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines (Committee to Revise the 1999 Guidelines for the Management of Patients with Acute Myocardial Infarction). Circulation. 2004 Aug 31;110(9).
2. Cole MG, Ciampi A, Belzile E, Zhong L. Persistent Delirium in Older Hospital Patients: a systematic review of frequency and prognosis. Age Ageing. 2009 Jan;38(1):19-26.
3. Hannon, FB. A national medical education needs’ assessment of interns and the development of an intern education and training programme. Med Educ.2000 Apr;34(4):275-84.
4. Low D, Clark N, Soar J, Padkin A, Stoneham A, Perkins GD, Nolan, J. A randomised control trial to determine if use of the iResus© application on a smart phone improves the performance of an advanced life support provider in a simulated medical emergency.Anaesthesia 2011;66:255-62.
5. Stein BD, Charbeneau JT, Lee TA, Schumock GT, Lindenauer PK, Bautista A, Lauderdale DS, Naureckas ET, Krishnan JA. Hospitalizations for Acute Exacerbations of Chronic Obstructive Pulmonary Disease: how you count matters. COPD. 2010 Jun;7(3):164-71.
1618 Medical Students Prefer Unguided Over Guided Communication for History-taking Rehearsal in a Virtual Patient Environment
Teresa Johnson, PhD2, Rebecca Lyons1, Regis Kopper, PhD5, Kyle Johnsen, PhD6, Benjamin Lok, PhD4, and Juan Cendan, MD3
1INSTITUTE FOR SIMULATION AND TRAINING, UNIVERSITY OF CENTRAL FLORIDA, COLLEGE OF MEDICINE, ORLANDO, FL, USA and 2MEDICAL EDUCATION, UNIVERSITY OF CENTRAL FLORIDA, COLLEGE OF MEDICINE, ORLANDO, FL, USA and 3MEDICAL EDUCATION AND SURGERY, UNIVERSITY OF CENTRAL FLORIDA, COLLEGE OF MEDICINE, ORLANDO, FL, USA and 4COMPUTER AND INFORMATION SCIENCE AND ENGINEERING, UNIVERSITY OF FLORIDA, GAINESVILLE, FL, USA and 5COMPUTER AND INFORMATION SCIENCES AND ENGINEERING, UNIVERSITY OF FLORIDA, GAINESVILLE, FL, USA and 6COLLEGE OF ENGINERING, UNIVERSITY OF GEORGIA, ATHENS, GA, USA
Introduction/Background: We created a communicative avatar platform (Neurologic Exam Rehearsal Virtual Environment: NERVE) which allows students to engage virtual patients (VPs) in a wide-ranging and unscripted dialogue for the development of history-taking skills and physical examination of select abnormal findings, including high-fidelity presentation of cranial nerve (CN) palsies rarely encountered through other educational opportunities. Two communication modes are available in NERVE: Chat mode allows users to type to VPs in a text window. VPs respond both verbally and with visual display of text (image). If VPs do not recognize text, a message informs users, “Try rephrasing the question.” Speech clarification (SC) mode functions similarly, except when VPs do not recognize text, NERVE presents up to three alternatives to guide the conversation. The purpose of this study was to compare students’ perceptions of history-taking experiences in NERVE by communication mode.
Methods: Seventy-eight of 80 first-year medical students (97.5%) consented to participate in this IRB-approved study. Participants ranged in age from 20-32 years (mean=24.6), and included 39 females (50.0%). Students were randomly assigned to communication mode treatments (chat=38; SC=40), wherein they practiced history-taking skills with VPs in four diplopia scenarios. Student perceptions were assessed by a 17-item survey. Independent samples t-tests were conducted to compare ratings by communication mode.
Results: Mean ratings of history-taking experience with VPs differed significantly by communication mode on two survey items (table; in bold). In chat mode, students agreed that they asked VPs the same questions that they would ask of real patients to a greater extent than in SC mode (p=0.03). Additionally, frequency of errors encountered with VPs was rated lower by students in chat mode as compared to SC mode (p=0.02).
Conclusion: Prior student feedback prompted us to explore alternative methods for communicating in NERVE. Initial reports suggested frustration with chat mode when faced with a question-to-answer mismatch. We, therefore, developed a sub-routine which guides students by presenting SCs. Current findings suggest that chat mode is preferable. Specifically, students in chat mode reported engaging with VPs in an authentic manner to a greater degree, and reported fewer errors. As educators, we value the open-ended, exploratory nature of chat mode without system-generated cues. This exchange mimics unscripted dialogue between physicians and patients, which may enhance skills transfer, and allows for development of critical thinking skills. We are compelled to improve the chat mode algorithm, rather than replace it with selection-based scaffolding.
Disclosures: Benjamin Lok, PhD is a stockholder/owner/partner in Shadow Health, Inc.
1621 How Much Force? A Possible Gap in Surgical Training
Timothy Kowalewski, PhD1, Lee White, PhC2, Thomas Lendvay, MD4, and Blake Hannaford, PhD3
1MECHANICAL ENGINEERING, UNIVERSITY OF MINNESOTA, MINNEAPOLIS, MN, USA and 2BIOENGINEERING, UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE, SEATTLE, WA, USA and 3ELECTRICAL ENGINEERING, UNIVERSITY OF WASHINGTON SCHOOL OF MEDICINE, SEATTLE, WA, USA and 4UROLOGY, UNIVERSITY OF WASHINGTON, SEATTLE CHILDREN’S HOSPITAL, SEATTLE, WA, USA
Introduction/Background: Simulation has become a valuable adjuvant to training laparoscopic technical skills, particularly in the widely-adopted and validated Fundamentals of Laparoscopic Skills (FLS) curriculum, which is used both for training and high-stakes evaluation.1,2 FLS scoring is based on task time and an error penalty intended to reward precision. However, these errors may not adequately address the levels of force exerted on the tissues by the tool, and thereby allow laparoscopic trainees to acquire poor habits of tissue handling. De et al.3 point out that excessive grasper-induced tissue stress injury, like crush injury, may cause “pathological scar tissue formation, bleeding, adhesions, and loss of bowel motility” and other groups report that laparoscopically manipulated organs “are susceptible to severe grasping injuries including perforation or hemorrhage.4,5 Even less severe immediate injury from grasping or manipulation may still result in clinically relevant consequences such as ileus (paralysis of the bowel), increased infection due to local breach of the bowel’s protective barrier and increased adhesion formation.”5,6 We hypothesize that the FLS scoring system does not adequately address the force levels trainees exert on tissues either in training or high-stakes evaluation.
Methods: We conducted an IRB-approved (ref), multi-institutional, cross-sectional study at three laparoscopic teaching hospitals (Seattle, Minneapolis, New Orleans) with surgical faculty, fellows, and residents as subjects. We employed the Electronic Data Generation and Evaluation (EDGE) platform (Simulab Corp, Seattle, WA) to measure tool motion and grasping force. Subjects were requested to perform the Block Transfer, Cutting, and Intracorporeal Suturing FLS tasks multiple times (3x, 2x, 2x respectively, in that order). FLS scores were computed for each task iteration along with peak grasping force and tool path distance.
Results: A total of 98 subjects participated, completing 447 FLS task iterations (193 Block Transfer, 165 Cutting, 89 Suturing). The correlations from FLS score to path length and peak force were measured via Pearson’s R for linearity and Spearman’s r for monotonicity. FLS score and path length correlated favorably for Block Transfer, Cutting, and Suturing with Pearson’s R = -0.92, -0.87, -0.95 (each p<0.0001 or less), respectively and Spearman’s rho=-0.89, -0.88, -0.93 (each p<0.001 or less), respectively. FLS score and Peak Grasp Force did not correlate, with Pearson’s R = -0.18(p<.01), -0.27 (p<.001), -0.16 (p <.13), Spearman’s rho =-0.08(p<.30), -0.27(p<.001), -0.26 (p <.01), again respectively. The attached scatter plots (Fig. 1) indicate a somewhat uniform dispersion of FLS scores vs. Peak Force, showing multiple instances where extremes of high peak forces occur in both the best and worst FLS score categories.
Conclusion: If two clinically-relevant metrics correlate perfectly, they are redundant; one provides no added information over the other. We observed a strong correlation, and therefore possible redundancy, between FLS score and path length but not for grasping force. This suggests grasp forces provide additional information not present in the FLS scores. If minimal forces or respect for tissue were training objectives, we would expect to see less spread in peak force among higher (better) FLS scores. Such phenomena are not present in our data, particularly for the Block Transfer and Suturing tasks. To be clinically relevant, tissue-specific levels of safe peak force must be established as errors and training targets. Since FLS instructions do not specify a target tissue to establish this level, we cannot conclude that any of our high-force subjects exhibited clinically deleterious tissue handling. Should such levels be established, the FLS scoring system would not provide a means to discriminate such errors. We conclude the FLS scoring system may not adequately address the force levels trainees exert on tissues either in its training or evaluation.
1. Peters J, Fried G, Swanstrom L, Soper N, Sillin L, Schirmer B, Hoffman K and the SAGES FLS Committee. Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surg 2004; 135: 21-27.
2. Fraser S, Klassen D, Feldman L, Ghitulescu G, Stanbridge D, Fried G. Evaluating laparoscopic skills; setting the pass/fail score for the MISTELS system. Surg Endosc 2003; 17(6): 964-967.
3. De S. The Grasper-Tissue Interface in Minimally Invasive Surgery: Stress and Acute Indicators of Injury [dissertation]. [Washington]: University of Washington 2008.
4. Heijnsdijk E, Dankelman J, Gouma D. Effectiveness of Grasping and Duration of Clamping Using Laparoscopic Graspers. Surg Endosc 2002;16: 1329-1331.
5. Marucci D, Shakeshaft A, Cartmill J, Cox M, Adams S, Martin C. Grasper Trauma During Laparoscopic Cholecystectomy. ANZ J Surg 2000;70:578-581.
6. Anup R, Balasubramanian K. Surgical Stress and the Gastrointestinal Tract. J Surg Res 2000; 92:291-300.
Disclosures: Timothy Kowalewski, PhD, is a consultant for Simulab Corporation. Thomas Lendvay, MD, is a co-founder in SPI Surgical, Inc. Blake Hannaford, PhD discloses that the Edge technology (described in this abstract) has been licensed from my laboratory at the University of Washington.
1656 Assessing High-fidelity Mannequin Facial Expressivity: A Preliminary Gap Analysis
Aaron Huus, B.Eng Biomedical Engineering, PhD Candidate1, and Laurel Riek, PhD1
1COMPUTER SCIENCE AND ENGINEERING, UNIVERSITY OF NOTRE DAME, NOTRE DAME, IN, USA
Introduction/Background: High-fidelity mannequin based medical simulators (HFMs) are limited by their lack of facial expressivity. Our preliminary work suggests that this lack of expressivity may negatively affect the realism of medical simulations, learner immersion, and patient safety.1,2,3 The objective of this study is to characterize how educators and clinicians perceive current HFM expressivity, and identify gaps. This gap analysis identifies the areas where HFM technology is unsatisfactory, and our preliminary work allows us to begin the initial stages toward technology improvement.
Methods: We created an online pilot survey to characterize this gap in HFM expressivity, and understand which limitations were most critical to learner education. To create the survey questions, we conducted three contextual inquiries with simulation center directors, each from a representative medical school, nursing school, and hospital educational setting. Based on these responses, we drafted and distributed a 70 question survey. The survey asked participants demographic questions to characterize their role within the simulation center, current HFM technology questions to characterize their existing HFM usage, and future HFM technology questions to characterize how they would envision the technological attributes of a new, facially expressive HFM. The current and future HFM technology questions were 5-point, discrete visual analogue scale questions (strong agree – strongly disagree), which addressed key technological issues and themes that emerged from the contextual inquiries. We randomly selected 65 members from the Society for Simulation in Healthcare (SSH) LinkedIn group who are employed in the fields of hospital and health care, higher education, and/or medical practice, and emailed them a link to our Survey Monkey instrument.
Results: The survey response rate was 33%, with 15 respondents finishing (23%). Survey participants consisted of mainly academic professionals, simulation program directors, and education specialists. Of the 15 respondents who completed the survey, 13 (87%) reported that they perform simulations on a high fidelity mannequin, 11 (73%) reported that they have worked with HFM technology for longer than six years, and 11 (73%) reported that over half of their simulations involve the HFM being awake and responsive. 11 respondents (73%) reported that when they perform simulations they required either themselves or someone else to verbally provide symptoms from a cue card for the HFM. 13 respondents (87%) reported that if an HFM could make eye contact with learners, as well as track a learner’s finger, it would result in greater learner immersion. 14 respondents (93%) reported that having a mannequin that had a more expressive face would improve the realism of simulations. Similarly, all 15 respondents (100%) reported that having a mannequin that could express pain or drowsiness would improve realism.
Conclusion: The results of this pilot study illustrate some limitations in existing HFM technology, such as the inability to synthesize facial expressions and requiring a cue card to express certain HFM symptoms. Our survey respondents desire HFMs to have more overall facial expressivity, and in particular be able to express pain and drowsiness, make eye contact, and track a learner’s finger. This work compliments our previous work that suggests expressive HFM technology may increase learner engagement, and create a more realistic training scenario. We will soon conduct this study on a much larger pool of SSH members to further define the HFM expressivity gap.
1. Huus, A. and Riek, L.D. 2012. An Expressive Robotic Patient to Improve Clinical Communication. In Proceedings of the HRI Pioneers Workshop at the 7th annual ACM/IEEE International Conference on Human-Robot Interaction (HRI), p. 52–54.
2. Martin, T.J., Rzepczynski, A.P. and Riek, L.D. 2012. Ask, inform, or act: communication with a robotic patient before haptic action. Proceedings of the 7th annual ACM/IEEE International Conference on Human-Robot Interaction (HRI), p. 221-222.
3. Rzepcynski, A.P., Martin, T.J. and Riek, L.D. 2012. Informed consent and haptic actions in interdisciplinary simulation training. Proceedings of the 140th annual meeting of the American Public Health Association (APHA). To appear.
1658 Innovations in Perfusion Education: The Use of Extracorporeal Circulation Simulator to Qualitative Measures as the Skills
Asako Tokumine, PhD1, and Yasuko Tomizawa, MD, PhD2
1BIOMEDICAL ENGINEERING, KINKI UNIVERSITY, KINOKAWA, WAKAYAMA, JPN and 2CARDIOVASCULAR SURGERY, TOKYO WOMEN’S MEDICAL UNIVERSITY, JPN
Introduction/Background: Application of the simulator for medical education is increasing. Recent years have seen the widespread introduction of simulation training involving extracorporeal circulation. Perfusionists need to be well-trained in both standard and emergency procedures, as complications during extracorporeal circulation can be potentially fatal.1 In light of increasing sophistication of medical simulators, the establishment of validated evaluation methods are likely to be a valuable addition to the educational program of perfusionists.2,3 This study was to explore whether a simulator system could evaluate extracorporeal circulation performances within a teaching environment. Furthermore, we performed quantitative evaluation of the basic skill and clinical skill training incorporating the simulation training procedure, which centered on changes in task accomplishment over repeated trials for perfusion education.
Methods: The perfusion simulator, CPB-Workshop (Senko, Tokyo, Japan) was used. The CPB-Workshop is controlled by the ECCSIM software, and simulated the condition of the patient at the time of an extracorporeal circulation. This software was used to record performances.2 A heart-lung machine, HAS-II (Senko, Tokyo, Japan) was used for the open circuit that incorporated venous reservoir, an oxygenator RX-25 and an arterial filter (Terumo, Tokyo, Japan). The circuit was primed with water, and gravity venous drainage was used. To evaluate performance in basic skills, a training session involving a simple training scenario was presented to several beginners. At the first session for the basic task training, the trainee was instructed to first reach arterial flow of 3.5 l/min within 30 seconds, maintain flow for 30 seconds, and to reduce flow within 30 seconds. And during all periods, he/she was instructed to maintain a reservoir level of 500 /- 50ml. And the second session for clinical skills, the trainee was required manipulation of the heart-lung machine to maintain simulated BP 80-89 mmHg. Each trainee was instructed on how to operate the simulator system from an instructor with experience in cardiopulmonary bypass procedures. After practicing freely for a period of 30 minutes, trainees completed a second trial. One hour later, a third trial was completed. One week later, all trainees performed the clinical skill training to maintain arterial pressure three times continuously. To fully compare accomplishment levels across training sessions, scores were recorded.
Results: On the 1st trial of the first training session, this participant (n=12) showed difficulty in controlling the arterial/venous flow and a sudden rise in the reservoir volume were observed. The reservoir level maintenance score became 75% (+8%) in the flow-up phase, 67% (+17%) in the flow-down phase at 3rd trial. On the second session held 1 week later, this skill level was maintained. However, the mission to maintain BP, at 1st trial, only one student achieved the mission. In the 3rd trial of this session, three persons (that include the former) achieved the mission (ex. mean BP 84.0 +/- 3.2 mmHg).
Conclusion: The evaluation of perfusion performances and/or techniques might be able can be behavioral and procedural scoring systems. Repetitive practice of protocols which develop key skills is effective in the training of new trainees.4 Effective training of perfusionists would benefit from the establishment of a standardized training regime, similar to the system adopted in clinical training. A fully trained perfusionist must be able to maintain a patient’s hemodynamic conditions adequately to accomplish the given surgical objective; and therefore, students have to perform evaluations of the basic skill and clinical skill training incorporating the simulation training procedure.
1. Palanzo D: Perfusion safety: past, present, and future. J Cardiothorac Vasc Anesth. 1997; 11: 383-390.
2. Ninomiya S, Tokumine A, Yasuda T and Tomizawa Y: Development of an educational simulator system, ECCSIM-Lite, for the acquisition of basic perfusion techniques and evaluation. J Artif Organs. 2007; 10: 201-205.
3. Morris R and Pybus D: “Orpheus” cardiopulmonary bypass simulation system. J Extra Corpor Technol. 2007; 39: 228-233.
4. Tokumine A, Ninomiya S, Tokaji M, Kurosaki T and Tomizawa Y: Evaluation of basic perfusion techniques, ECCSIM-Lite simulator. J Extra Corpor Technol. 2010; 42: 139-144.
1669 A Novel Mixed Reality Human Simulator Improves Medical Student Comfort and Competence in Digital Rectal Exam
David Lind, MD1
1SURGERY, DREXEL UNIVERSITY COLLEGE OF MEDICINE, PHILADELPHIA, PA, USA
Introduction/Background: While mannequins are used to teach intimate examinations, they lack the fidelity and feedback of a real patient. We have created a mixed reality human (MRH) simulator (interactive virtual patient (VP) and mannequin rectal/prostate exam simulator) designed to teach and assess students’ communication and digital rectal examination (DRE) skills. In this study, we piloted the MRH simulator with medical students at Drexel University College of Medicine.
Methods: Following IRB approval and informed consent, 26 third year medical students were randomized to two groups. Initially, both groups performed a baseline DRE on a mannequin simulator. Group 1 then performed another DRE without interacting with a VP, while group 2 conducted a medical interview with an interactive VP and then performed a DRE (Figure 1). Both groups received real time, color-coded feedback regarding their examination following their second DRE (Figure 1 inset). Following feedback, they performed one final DRE exam. During the simulation, students communicated with the VP on a LCD TV screen and performed DRE on a mannequin equipped with 12 pressure sensors covering the surface of the prostate. Students were surveyed pre and post MRH interaction regarding their confidence and anxiety in communication and DRE skills. DRE score calculated using pressure sensor data compared to an experienced clinician. YouTube video of a MRH simulator interaction at: