Effectiveness of a Simulation-Based Medical Student Course on Managing Life-Threatening Medical Conditions

Abstract

Objectives: 

To assess the competency and the comfort level of medical students in lifesaving skills after a simulation-based training session and then determine skill retention after 1 year.

Methods: 

Prospective observational before-after case series of medical students entering the third year. Each student participated in a half-day “How to Save a Life” course. The course consisted of a half-hour lecture on lifesaving skills followed by small group simulation-based skill sessions. Critical resuscitation actions were reviewed and demonstrated by the instructor using case-based scenarios and mannequins. The emergency medicine faculty and residents evaluated individual students’ performance of clinical skill using a standardized checklist at each skill station. Each checklist specified the critical actions necessary to perform the procedure properly. Outcome measures included global competency and level of comfort questionnaire for each skill, using a 5-point Likert scale, with 1 being “strongly disagree” and 5 “strongly agree.” Retention of lifesaving skills was assessed approximately 1.5 years later using a subset of the original group of the third year medical students. Without prior notification, students were assessed on the same skills using the same scenarios and outcome measures. Comparison of competency and level of comfort data between the initial group and the follow-up group were analyzed using descriptive statistics.

Results: 

One hundred fifteen third year students participated in the initial training program. Initial demographic information was available on 104 students and revealed that 96% of the students had previous experience with basic life support. After the initial training course, all students were rated as competent in all procedures. In the retention group, the proportion of students achieving competence in each procedure ranged from 47% to 100%. The level of comfort decreased during the interval period.

Conclusion: 

A short course in simulation-based life-saving clinical skills is an effective means to teach the third year medical students. We observed a decline in competency over time for recognition of ventricular fibrillation, defibrillation, airway management, and management of a choking child. Cardiopulmonary resuscitation and automatic external defibrillator competency did not decrease over time.

Organizations including the American Association of Medical Colleges (AAMC) and Liaison Committee on Medical Education have published guidelines and standards highlighting the importance of medical student mastery of certain lifesaving clinical skills.^1,2 A survey performed by the clerkship directors in internal medicine led to a recommendation that cardiopulmonary resuscitation (CPR) be taught during the internal medicine clerkship.³

Before students entering the clinical component of their medical school training, it is reasonable that they should have an understanding of and the ability to perform lifesaving clinical skills. Previous work has shown that students can be taught clinical procedural skills,^4,5 but without clinical reinforcement, there is poor retention.^6–8

Students can acquire the knowledge and skills to perform lifesaving clinical skills in a variety of venues such as life-support courses, lectures, and clinical experience. Previous work with residents has demonstrated that traditional educational models do not provide these requisite skills. The addition of simulation-based curriculums improves outcomes.^9–11 Despite the data supporting use of simulation in training residents in lifesaving skills, little work has been done in undergraduate medical education.^12,13 A review of the literature revealed that the Peter M. Winter Institute for Simulation Education and Research has developed a curriculum to teach medical students airway techniques but has not assessed its effectiveness.¹⁴ The purpose of this study was to develop and assess the effectiveness of a lifesaving clinical skills curriculum for medical students by measuring competency, level of comfort, and retention.

METHODS

Study Design

This study design was a prospective observational before-after case series, which received an exempt status from the institutional review board as an educational project. All students gave verbal informed consent. They understood that participation was voluntary and in no way would affect clerkship grades. The clerkship director was not directly involved in the assessments, and results did not include student identifiers. Because there were no student identifiers, the retention group could not be matched to the initial evaluation group.

Study Setting

At our institution, all the third year students receive a 1-week orientation before starting their clinical rotations. As part of the program, the students are required to participate in a half-day course called “How to Save a Life.” This course has been taught by the Department of Emergency Medicine (EM) since its inception in 1997.

Our curriculum was designed with the ultimate objective to teach lifesaving techniques that would support a critically ill patient for the first several minutes of a cardiac or airway emergency. Content validity was set by a group of educators at our institution. The specific objectives of the curriculum stated that students would be able to:

• Recognize ventricular fibrillation and defibrillate
• Perform basic airway management skills (open airway, use of airway adjuncts, apply a bag-valve mask for respiration)
• Perform adult cardiopulmonary resuscitation
• Use an automatic external defibrillator (AED)
• Manage a choking child

During the fourth year of medical school, all students are required to take a mandatory course in EM. The students participate in the EM clerkship after completion of internal medicine, surgery, obstetrics and gynecology, psychiatry, and pediatrics. It was during the EM clerkship that the students were reassessed to evaluate retention of the lifesaving clinical skills.

Population

All the third year medical students (N = 115) from the class of 2005 participated in the course during orientation to the third year of medical school. Approximately 18 months after the initial course over a period of 2 months, 34 students on the EM rotation were assessed individually for retention of the lifesaving clinical skills. This time period was chosen based on availability of the raters. We retested all the students during that 2-month period to avoid the possibility of self-selection.

Study Protocol

The “How to Save a Life” course included an introductory 30-minute lecture for all participants. The lecture was a case-based presentation that highlighted the key principles of lifesaving skills. This was followed by small group teaching and assessment at five skill stations.

Each skill station was supervised by either an EM faculty member or a senior EM resident. Critical resuscitation actions were reviewed and demonstrated by the instructor(s) using case-based scenarios, mannequins, and other essential medical equipment necessary to simulate the appropriate clinical scenario. Immediately after the training session, students were individually tested on their performance of the critical actions using a standard checklist. Each checklist specifies the critical actions necessary to properly perform the procedure (Fig. 1). In addition, the students completed a demographic form that included information about previous experiences and a level of comfort questionnaire.

FIGURE 1.: Ventricular fibrillation checklist. *Asked evaluator to add comments.

The subgroup of students reassessed were enrolled in the EM clerkship during a specified 2-month period in their fourth year. During the orientation to the EM clerkship, all the students received course material that included lifesaving clinical skills information. This information was part of the EM clerkship curriculum at the time of the reassessment. The students received no additional lifesaving clinical skill training during the rotation.

Each student was individually reassessed on the identical five life-saving clinical skills taught during the orientation, 18 months prior. The students were not provided any advanced warning of the reassessment. The students received the same clinical scenario and were assessed using the same evaluation form. The students completed a similar demographic form and level of comfort questionnaire.

Outcome Measures

The major outcome measure was a global assessment (pass/fail), which was evaluated as the percent of students that passed. Level of comfort was measured for each skill using a 5-point Likert scale, with 1 being “strongly disagree” and 5 “strongly agree” and evaluated as the proportion of students selecting a particular level of comfort. Demographic information included the proportion of students with previous courses and experiences with mannequins and patients. All proportions and confidence intervals were calculated using the one sample test for proportions in SAS ANALYST version 9.1 (SAS Institute Inc., Cary, NC).

RESULTS

One hundred fifteen third year students participated in the initial training program. Complete data sets were available on 104 of the 115 enrolled in the study. Initial demographic information revealed that 96% of the students passed a basic life-support course 1 year prior. A minority of students had previous training that included first aid, life guard training, advanced cardiac life support (ACLS), emergency medical technician, advanced pediatric life support, advanced trauma life support, and other training.

Although most of the students had some experience with the lifesaving clinical skills before our course, most of the students lacked clinical experience. The proportion of students with self-reported experience with lifesaving techniques on actual patients ranged from 3.8% to 14.4% (Table 1). In the interval before the reassessment of the students, they participated in a mandatory ACLS course. The subgroup who were reassessed (N = 34) had a self-reported experience with lifesaving techniques on actual patients that ranged from 6% to 79% (Table 1).

Table 1: Proportion of Students With Self Reported Previous Experience With Life Saving Techniques on Mannequins and Actual Patients

Competency after the initial training course was 100% for all the clinical skills. In the group of students who were reassessed, we noted that the ability to do CPR, use an AED, and manage an airway did not decline dramatically, 100%, 91.2%, and 88.2%, respectively. We noted a decline in competency for treatment of ventricular fibrillation by immediate defibrillation falling to 61.8%. Review of the data revealed that the cause of failed defibrillation was always due to a failure to recognize ventricular fibrillation on the monitor. Competence in the correct management of pediatric choking decreased to 47.1% (Table 2).

Table 2: Competency of Students Evaluated During Emergency Medicine Clerkship (N = 34)

Immediately after the lifesaving skills course, the proportion of students who felt comfortable with the lifesaving skills ranged from 80.2% to 100% (Table 3). Level of comfort decreased between the time of the initial training during the third year medical student orientation and the EM clerkship reassessment. The only technique in which the level of comfort did not seem to drop dramatically was performance of chest compressions (Table 3). Detailed examination of the level of comfort data demonstrated that the proportion of students who answered “strongly agreed” decreased at the time of the reassessment, whereas the number of “undecided,” “disagree,” and “agree” increased (Table 4).

Table 3: Proportion of Students Answering Yes on Level of Comfort Questions With Life Saving Clinical Skills

Table 4: Proportion of Students Answering Yes for Level of Comfort With Life Saving Clinical Skills

DISCUSSION

Medical students entering the clinical component of their medical school education have little clinical experience with lifesaving clinical skills. Although the importance of students having competency in lifesaving clinical skills is well stated by the Association of Professors of Medicine, AAMC, and Liaison Committee on Medical Education,^1–3 previous work has demonstrated that few schools have consistent skill training programs.¹⁵ One study that surveyed 122 medical students at seven medical schools noted that CPR was never performed by 72% of these students, once by 16%, and greater than 2 by 12%.¹⁶ Even when students present themselves to inpatient cardiac resuscitations, only 25% reported doing CPR.¹⁷ Our students had similar experiences during their clinical years reinforcing the importance of developing an educational program for lifesaving clinical skills.

Likely because of the lack of clinical experience, students report a lack of comfort with clinical skills. In one study, self-reported confidence was 2.6 on a 5-point Likert scale, with 1 being “not at all confident” and 5 “very confident.”¹⁶ In our study, we demonstrated high levels of comfort immediately after the initial training, and with the exception of performing chest compressions, this comfort decreased over time. This decrease in level of comfort may be due to lack of clinical experience and additional didactics.

The AAMC task force on the clinical skills education of medical students states that medical schools should define objectives, define a set of specific skills, and create a process to assess competency.¹ Our curriculum and study meet these needs for the lifesaving clinical skills by providing a curriculum and a method to assess competency. Data from our study confirm that our students have limited clinical exposure to many of the clinical skills that we defined as lifesaving. Our students developed a high level of competency and reported a high degree of comfort immediately after the initial training session, but this decreased at the time of the reassessment.

Events such as sudden cardiac death and a choking child are uncommon events, but delays or ineffective treatment are life threatening. Previous literature confirms that degradation in clinical skills is expected.^6–8 There is some evidence to suggest that skills can be maintained especially with the incorporation of refresher classes¹⁸ and simulation.^19–20 In our study, we noted that the ability to perform basic airway management, perform CPR, and use an AED were maintained, whereas the ability to recognize and defibrillate ventricular fibrillation and manage a choking child decreased over time. Our study and several nursing studies reached the same conclusion; AED competency is readily achieved and maintained.^21,22 However, continuous medical education may be necessary for students to maintain competency in more advanced lifesaving skills. Basic skills may not need to be reinforced, but the emphasis should be on advanced skills, where retention and confidence degrade over time. Training more advanced resuscitation skills will require supplemental didactic material, training with simulated experiences, and increase in clinical exposure.

The initial group of students was a homogenous group with little clinical experience, and a majority (96%) had passed a basic life-support course before our training. These experiences in combination with our training program likely explain the high-competency rates and level of comfort after our initial training. The students who were retested were all enrolled in the EM clerkship during a 2-month block, chosen based on the availability of the raters. The students represented a wide range of future specialties, and only one student had chosen EM as a career. Although we did not retest the entire initial group, we believe that the students in the retest cohort were representative of the initial group. We do not believe that there was any selection bias because we chose 2 months based on the raters’ schedules, students were not volunteers, and the students represented a wide range of academic interests. Despite potential differences in clinical experience based on our sampling, all the students (N = 104) received ACLS in the interim period. Likely because of the lack of clinical experience, skill retention, and level of comfort are significant concerns for recognition and treatment of ventricular fibrillation and management of the choking child. The maintenance of competence in the other skills tested was likely a combination of the ACLS course and clinical experience. Whether because of the lack of clinical exposure or other factors, targeted refresher courses and increased clinical exposure should be considered as an important addition to the undergraduate medical curriculum. Based on this preliminary work, future studies could examine the factors leading to degradation of skills, impact of clinical experience, and how a targeted curriculum including simulation will improve competency, confidence, and retention.

CONCLUSION

Procedural competency in basic lifesaving clinical skills can be achieved through an intensive hands-on small group simulation-based training model. This longitudinal study reveals retention of some skills but without interim practice or clinical experience competency in several potentially lifesaving clinical skills is lost. Supplemental didactics, training in simulated environments and increasing clinical exposure, are needed to improve skill acquisition, confidence, and retention.