Dehmer, Jeffrey J. MD; Amos, Keith D. MD; Farrell, Timothy M. MD; Meyer, Anthony A. MD, PhD; Newton, Warren P. MD, MPH; Meyers, Michael O. MD
The transition from preclinical course work to clerkships presents numerous challenges for medical students. Traditionally, the third and fourth years of medical school are devoted to rotations designed to provide comprehensive clinical exposure, reinforcing the scientific principles taught during the initial two years. There are a wide range of styles, techniques, and adjuncts available to augment the training received on the wards. In addition to mastery of the history and physical examination, students are introduced to procedural skills and are expected to have varying degrees of competence in these skills by graduation.
Performing procedures on live patients can be a stressful experience for any practitioner, particularly when the practitioner is a novice.1 The opportunity to attain and master these skills has also changed within the framework of increasingly complex accreditation processes for both students and postgraduate trainees. In response to this, medical schools employ sophisticated task-trainers and simulators designed to offer students a low-risk introduction to most aspects of patient care, including physical diagnosis, basic procedures, and even complex operations prior to live patient interactions. The effect of these changes on the skills that medical students acquire before completion of medical school, however, remains unclear.
In 1999, the Medical School Objectives Project (MSOP) of the Association of American Medical Colleges (AAMC) identified several key procedures that should be in the repertoire of any graduating student.2 These include venipuncture, intravenous (IV) catheter insertion, arterial puncture, thoracentesis, lumbar puncture (LP), nasogastric tube insertion (NGT), Foley catheter insertion (Foley), and suturing of lacerations (suturing).2 Subsequent recommendations have included intubation as an additional skill with which graduating medical students should be familiar.3 Despite these guidelines, recent data suggest that medical schools are falling short of their goal of preparing all students to perform these procedures independently.3–5 Furthermore, overall evaluation of student performance has traditionally relied on subjective evaluations rather than objective measures of performance, although this pattern is changing.6 Data indicate that preclinical participation in procedural skills can enhance student participation during clerkships.7 Strategies designed to improve exposure to procedural skills in the preclinical years exist, but their effects on the actual acquisition of these skills are not entirely clear, particularly how early exposure translates to clinical practice and long-term skill retention.8–11
Despite the evidence that early exposure to a particular skill can improve a student’s comfort level, offering early access to certain skills is difficult. Dedicated sessions require a time commitment by the students and the respective faculty separate from their existing clinical responsibilities, and specific procedure-focused large-group sessions may be temporally removed from opportunities to perform the procedures on actual patients.12 Further complicating this issue is the paradox in medical training that patients are willing to accept trainee involvement in nonprocedural care but, in general, are reluctant to allow medical students to perform procedures on them.13
To assess specific deficiencies in procedural competence and identify areas where intervention would be most beneficial, current data regarding graduating students’ levels of experience and comfort with procedures are needed. In this study, we evaluated the self-reported procedural competence of graduating medical students at a single institution and compared their self-reported competence with their desired competence for these skills.
With the approval of the University of North Carolina at Chapel Hill biomedical institutional review board, we sent an anonymous, Web-based questionnaire via e-mail to all 156 fourth-year medical students at the University of North Carolina at Chapel Hill in March 2011. Invitations to the survey were specific to each e-mail address, such that the survey could be taken only once from each link, thus allowing us to collect anonymous responses while ensuring that no individual could respond to the survey multiple times. We designed the survey materials using Qualtrics online survey software (Qualtrics, Provo, Utah) through the Odum Institute for Social Science Research at our institution. We obtained all responses anonymously and analyzed them in aggregate.
In addition to standard demographic questions, we queried students about prior experience with medical procedures, career intentions, primary method of instruction for the nine procedures outlined in the MSOP, the number of times they had performed each of the nine procedures on live patients as a medical student, individual confidence to perform the procedures without assistance, and their opinions regarding their actual level and desired level of competence with the procedures. Students indicated whether they performed each task at all and, if they had, whether they performed it one, two, or more than two times. Students rated their confidence in performing each procedure on a five-point Likert scale (1 = not at all confident, 2 = minimally confident, 3 = average confidence, 4 = above average confidence, 5 = very confident). Students rated their self-reported competence to perform each skill on a four-point Likert scale (1 = unable to perform skill, 2 = competent to perform with major assistance, 3 = competent to perform with minor assistance, 4 = competent to perform independently).
We performed statistical analysis using a mixed-model analysis of variance (ANOVA) with random subject effects. Fixed effects were followed by appropriate contrasts for procedure type, the competency level that students felt they should have as reported on the survey, and students’ self-reported levels of competence. We performed simple contrasts by ANOVA with post hoc Student t tests by group for each procedure. We performed all statistical analysis using SAS version 9.2 (SAS Institute, Cary, North Carolina).
One hundred thirty-four of 156 fourth-year medical students completed the survey for an 86% response rate with an even gender distribution (62; 46% male). The majority of students were ages 26 to 30 (84; 63%), and 124 respondents (93%) had no prior experience with the procedures queried in this survey before entering medical school. Seventy-two respondents (54%) indicated that they planned to enter a procedural specialty.
Figure 1 depicts the number of times students performed each of the nine procedures on a patient. Only suturing (116; 87%), Foley placement (85; 63%), and venipuncture (64; 48%) had been performed more than two times by approximately half of the students or more. All other procedures had been performed most commonly either zero times or once. Arterial puncture was divided rather evenly, with 64 respondents (48%) indicating that they had performed it two or more than two times and 70 (52%) indicating that they had performed it zero or one time. Although not every student responded for every procedure, results were striking in that greater than half of students who responded for each specific procedure had never performed an intubation (83; 62%) or a thoracentesis (104; 78%), and a significant number had never inserted an NGT (53; 40%) or IV (50; 37%) or performed an LP (53; 40%).
Students’ confidence in performing each of these skills is illustrated in Figure 2. Mean levels of confidence varied from 3.83 ± 1.05 for suturing to 1.75 ± 1.04 for thoracentesis. Only for suturing and Foley placement did more than half of students rate their confidence as at least above average. For venipuncture and IV insertion, 109 (83%) and 95 (72%) students, respectively, indicated an average level of confidence or higher. For intubation, LP, and thoracentesis, more than 60% of students reported either no or minimal confidence in performing these skills, and more than 40% expressed these levels of confidence for NGT placement and arterial puncture.
Students were asked to self-assess actual competence and desired competence for each skill. The four-point Likert scale for self-assessed competence ranged from 1 = “unable to perform the skill” to 4 = “competent to perform the skill independently.” Cumulative responses for self-assessment of actual competence at the level of independence or with minor assistance ranged from 11% (n = 14) for intubation to 93% (n = 123) for suturing. Along with suturing, the majority of students rated themselves as competent with minimal or no assistance in Foley placement (109; 81%), venipuncture (97; 72%), and IV placement (94; 70%). For intubation (118; 88%), LP (101; 75%), and thoracentesis (117; 87%), most students rated their competence as either requiring major assistance or unable to perform the skill at all. For all skills, students suggested that their desired level of competence was greater than their actual level. The gap between actual and desired level of competence was significant for all nine skills in the survey (P < .0001). Results are shown in Table 1.
To evaluate associations between competence performing a particular skill and our demographic data, we performed simple contrasts as described above. With respect to gender, males were more confident than females with venipuncture and IV insertion (P = .04 and .05, respectively). There was no relationship between confidence with any procedure and exposure to procedures before medical school or respondent age. To assess whether pursuit of a procedure-based specialty was associated with confidence, we eliminated the cohort that was undecided about their specialty choice (7; 5%). Individuals entering a procedural specialty were only more likely to report confidence at intubation (P = .03). For all skills except arterial puncture and suturing, there was a direct correlation between the number of times a procedure was performed in medical school and confidence as displayed in Table 2.
In this single-institution survey, we evaluated the self-assessed competence level of graduating medical students in performing basic procedural skills at a large, public medical school. Our data provide a snapshot of medical student procedural skills acquisition and provide several areas for reflection. We selected the nine skills evaluated in this study to mirror those identified by the AAMC in 1999 as skills that students should be able to perform on graduation.2 The level of competence that students participating in this survey reported having was highly variable. For several of these skills, such as venipuncture, Foley placement, and suturing, most students rated themselves as able to perform them without assistance or with minor assistance. Not surprisingly, for more advanced skills, such as intubation and thoracentesis, students were unlikely to report any level of competence for performing them.
Our data had mixed consistency compared with previous findings.4,5 For example, compared with Promes and colleagues’4 2009 survey of interns’ recollections of medical school, our students were more likely to report “zero experience” with venipuncture (28% in our study versus 3%) and IV placement (38% in our study versus 10%) but were similarly likely to report “zero experience” with more advanced procedures such as thoracentesis (78% in our study versus 66%). However, compared with a 2008 study of fourth-year medical students prior to graduation by Wu and colleagues,5 our students were similarly likely to report “zero experience” for basic procedures such as venipuncture (28% in our study versus 20%) and IV placement (38% in our study versus 23%) and more advanced procedures such as thoracentesis (78% in our study versus 81%). Neither of these prior studies specifically assessed intubation, which also was reported by most of our students as “zero exposure.”
None of these data are surprising, as a number of factors have conspired to limit the hands-on experience of medical students in the current era. Although data indicate that students find didactic training beneficial and that skills sessions improve specific task performance, we believe the simulated environment should facilitate performance of index procedures on actual patients.1,11,12,14 At our institution, it seems that student performance of procedures on live patients occurs randomly and with diminishing frequency. As graduate medical training and delivery of patient care change in parallel, the fragmentation of the medical student experience continues. With expanding involvement of midlevel providers and decreasing availability of residents due to duty hours restrictions, for example, students require a more structured system to ensure adequate exposure to live procedures.15
In conjunction with exposure, students must receive feedback, as some authors have suggested that merely performing skills without adequate feedback has no effect on long-term competence.16 Partnerships with ancillary service providers such as respiratory therapists, phlebotomists, and IV line teams may provide the structure for sufficient exposure and mentorship to allow students to master these skills in patients. Another forum for live patient experience is the operating room, where students may be supervised by anesthesiology, nursing, and surgical staff as they learn and perform the basic procedures common during preoperative preparation.
At the University of North Carolina, students do have several opportunities for performing procedures in a nonclinical setting. Students on their internal medicine rotation perform code simulations on electronic dummies that include defibrillation, cardiopulmonary resuscitation, and endotracheal intubation. They also practice placement of IV catheters on mannequins. Specifically within the department of surgery, there is a four-hour, faculty-led, inanimate suturing workshop using porcine feet, as well as dedicated sessions on venous and arterial cannulation and intubation.17 In addition, a formative suturing OSCE is required of students on the surgery clerkships, and they prepare for it with a suturing video in addition to the above training.
Although simulation plays a role in skills education, it should only serve as part of a curriculum that extends into the clinical environment and verifies students’ competence prior to emergence as independent care providers. Inanimate training should not replace actual patient encounters, or the system will almost certainly fail to deliver competent physicians. As medical advances occur, we hope to see similar improvements in simulation and its application to medical training. However, there will always be experiential nuances of dealing with live patients that are invaluable and essential in medical student education.
Advanced procedures such as endotracheal intubation, thoracentesis, and LP provide great challenges for medical student exposure. Such procedures were associated with low levels of perceived competence and independence in our study, as well as others, as outlined above.4,5 We believe that active learning for medical students in the clinical environment is being challenged at all levels. Computerized order entry and electronic medical records have affected basic integration of students into clinical teams at our institution. Similarly, duty hours restrictions, increases in ancillary staff and midlevel providers, andmore stringent guidelines for trainee supervision and coding have reduced the time residents and faculty spend on the patient wards. When procedures do occur, all of these factors tend to cause senior residents and attendings to prioritize patient safety and junior resident instruction over concerns of student learning. In addition, practice patterns may change as well, such that residents and teaching faculty may never develop competence to provide for student instruction or may fail to maintainit. For instance, at our institution, thoracentesis is most commonly performed by interventional radiology physicians and far less commonly by residents in other specialties. One must ask whether students in training, or even residents, should be expected to have any degree of competence or experience with more advanced procedural skills, or whether these objectives for broad-based training in our curricula should go the way of a rotating internship. With increasing specialization at ever-earlier levels of training, it is likely that some of these skills should fall into the domain of graduate medical education programs that apply only to the relevant subspecialist training.
One aspect of our study that has not been previously reported in the literature is our assessment of the students’ desired competence compared with their perceived levels of actual competence. Interestingly, of the “nonadvanced” skills—NGT, venipuncture, IV, and arterial puncture—all had gaps of >0.8 points on our Likert scale between students’ actual perceived competence and their desired levels of competence. We did find a strong correlation between the number of times a procedure was actually performed and self-assessed competence in most skills, with the exceptions of arterial puncture and suturing. The latter is surprising because suturing was the skill for which students reported the most self-assessed competence. At our institution, this result is not surprising because students rotating on our surgical services have the opportunity to perform this skill frequently. We find it encouraging that students have a desire to acquire a higher level of competence than they actually self-report. These data suggest that medical students remain motivated for skills development, and suggest that generational differences alone are unlikely to explain the variance in experiences and competencies achieved during medical school as compared with prior eras.
Our study does have limitations. The data reflect the experience and opinions of fourth-year medical students in one year at a single institution; thus, results may not be generalizable to all graduating medical students across the country. Individual student motivation represents a significant confounder. Students entering procedurally oriented fields may be more aggressive in the pursuit of opportunities to perform procedures than those entering pathology or diagnostic radiology, for instance. There may be some reporting bias, as we found no statistically significant correlation between career path and competence, except with intubation.
Education, performance, and acquisition of competence with basic medical procedures remain significant components of clinical undergraduate medical education, but there are a number of issues to address in the future. In 2003, the AAMC convened a Task Force on the Clinical Skills Education of Medical Students, the goal of which was to revisit the previously published MSOP in order to foster a national consensus on the design and implementation of clinical skills curricula during medical school. The report from this task force, entitled Recommendations for Clinical Skills Curricula for Undergraduate Medical Education,18 was published in 2005. The list of skills, including basic procedural skills, in this report is exhaustive and was intended to serve only as a resource for schools to use in determining which skills their students should learn. As noted in this report, general standards regarding which skills should be learned, and the degree of competence necessary, do not exist. Moreover, individual schools vary widely as to which skills they expect students to learn, the level of competence they expect for each skill, and whether they set specific standards at all. Although we believe curricular objectives should be comparable across schools, flexibility in delivery of content is important to allow medical schools with different missions and clinical profiles to achieve these objectives in the best manner possible.
Variations in curricula present a challenge, though, because students require not only a basic undergraduate medical education but also broad exposure as they make career choices and prepare for graduate training. As such, we are proponents of a set of basic procedural skills and a minimum level of competence, regardless of specialty, defined by validated national benchmarks. In addition, because expectations for some residency programs may be higher than these common basic competencies, we advocate for fourth-year curricular opportunities for advanced skills development for those planning a procedurally based career path.
As medical school curricula evolve, a number of questions remain regarding teaching procedural skills to medical students. Is it realistic to expect all medical students to be competent in the more advanced skills, such as thoracentesis, LP, and intubation by the time they finish medical school? Are basic procedural skills of any kind still something that every physician-in-training needs to learn? Should medical school learning objectives be scaled back on the basis of changes in the learning environment and shifting levels of competence, or should we strive to modify the delivery of education to prioritize the teaching of technical skills to medical students? On the basis of our results and those of prior authors,4,5 it seems that competence with the nine skills outlined in the MSOP from 1999 remains an unmet goal for today’s graduating students. Specifically, intubation and thoracentesis, and perhaps LP, are unlikely to be learned with any degree of competence by medical students as a whole. Moreover, the students themselves have modest expectations as to their desired levels of competence for these skills, a fact that likely mirrors the expectation set by the faculty and residents who are teaching them. The other six skills studied showed greater degrees of self-reported actual competence and, perhaps most important, greater levels of desired competence. Future work directed at a larger cross-section of medical students will be needed to inform these decisions. We have current plans to expand this survey to several medical schools and assess responses from students across several years. A broader picture will allow us to provide a more robust sample from which to draw conclusions and perhaps inform curriculum development nationally.
Acknowledgments: Many thanks to Chris Wiesen, PhD, of the University of North Carolina at Chapel Hill Odum Institute for Research in Social Science for his assistance with the statistical analysis.
Other disclosures: None.
Ethical approval: This study was approved by the University of North Carolina at Chapel Hill biomedical institutional review board, study number 11-0754.
Previous presentations: Portions of these data were presented at the Seventh Academic Surgical Congress, Las Vegas, Nevada, February 15, 2012.