Wu, Edward H.; Elnicki, D Michael; Alper, Eric J.; Bost, James E.; Corbett, Eugene C. Jr; Fagan, Mark J.; Mechaber, Alex; Ogden, Paul E.; Sebastian, James L.; Torre, Dario M.
The Association of American Medical Colleges (AAMC) created the Medical School Objectives Project (MSOP) to develop a consensus on graduating medical students’ skills and to establish learning objectives to help guide medical school curricula. Among the objectives of the MSOP Report I was that physicians be skillful and demonstrate prior to graduation “the ability to perform routine technical procedures, including at a minimum venipuncture, inserting an intravenous catheter, arterial puncture, thoracentesis, lumbar puncture, inserting a nasogastric tube, inserting a Foley catheter, and suturing lacerations,” as well as “the ability to interpret the results of commonly used diagnostic procedures.”1
A 1993 telephone survey of 60 medical schools regarding procedural skills found that 47 (78%) had no formal instructional courses other than an introduction to phlebotomy.2 A 2001 survey of medical school associate deans for academic affairs found that graduating students were perceived to be competent in most procedures listed in the MSOP Report I.3 However, students were deemed not proficient in thoracentesis and lumbar puncture, and it appeared that little had changed in the teaching and evaluation of technical skills in half of the medical schools since 1993. In a national survey, Clerkship Directors in Internal Medicine (CDIM) members reported that they felt that 20 procedural and interpretive skills should be learned in medical school, 17 of which should be learned by the end of the third year.4 The survey also found that 85% of clerkship directors would welcome national curricular materials on instruction of these skills.
Prior studies of students’ experiences have been limited in their scope or sample size. A single-institution study examined students’ exposure to, frequency of performance, and self-assessed competence in 52 procedures that were judged important by faculty and residents from various clinical departments.5 Another study found that third-year medical students in their medicine clerkship had variable experiences with procedures.6 A 2002 study tracked the number and type of procedures performed by third-year medical students at one institution, but was limited to 25 students over 9 months.7
If the skills in the MSOP Report I and the recent CDIM national survey are combined, there is a consensus that there are 21 procedural and interpretive skills that students should learn and perform during medical school. There are no prior studies of student experiences from more than one medical school, using the skills from the aforementioned reports. These skills may be targeted for the improvement of instruction and development of curricular materials. We wished to prioritize particular skills by conducting a needs assessment of medical students. We sought to uncover the estimated frequency with which students performed skills, and their self-confidence in and perceived importance of these skills. We also wished to learn whether students were presented with curricular materials, formally taught, or formally evaluated regarding these skills, and what effect these variables had on frequency of performance.
We developed a confidential, Internet-based survey and asked students to report their estimated frequency of performance of 21 procedural and interpretive skills since the beginning of medical school. We asked students to use a 5-point Likert scale to report their self-confidence in performing these skills unsupervised (1 = “not at all confident”; 5 = “very confident”) and their perceived importance of being competent in skill performance as a medical school graduate (1 = “not at all important”; 5 = “very important”). The skills were as follows: arterial blood sampling, phlebotomy, peripheral intravenous catheter insertion, blood culture performance, throat culture performance, nasogastric tube insertion, purified protein derivative placement, Papanicolaou smear performance, lumbar puncture, paracentesis, thoracentesis, urinalysis, urethral (Foley) catheter insertion separately in males and females, cardiopulmonary resuscitation, cardioversion, suturing a laceration, chest X-ray interpretation, electrocardiogram interpretation, Gram stain interpretation, peripheral blood smear interpretation, and spirometry interpretation. Prior to starting the survey, students encountered a screen with definitions of each procedure, developed from a consensus of the investigators; this information was accessible as a pop-up screen during the survey. Regarding performance, we instructed students to exclude times when a skill was performed for the purpose of evaluation, during a written exam, or on a model or mannequin. We also asked students to indicate if they had ever been presented with curricular materials, were formally taught, or were formally evaluated in each of these skills. We instructed students to exclude any experiences in which an attending physician, resident, or technician informally taught or evaluated them. To encourage participation, each responding student was eligible for a random drawing for one of four $100 gift cards.
We surveyed 171 students from the Class of 2006 at the end of their third year starting in June 2005 at Brown Medical School, the Medical College of Wisconsin, Texas A&M University System Health Science Center College of Medicine, the University of Miami Miller School of Medicine, the University of Massachusetts Medical School, the University of Pittsburgh School of Medicine, and the University of Virginia Health System School of Medicine. As a sample of convenience, we surveyed those students who completed the internal medicine clerkship in the last clerkship block for academic year 2004–2005. A coinvestigator at each institution or his representative e-mailed the Internet link to the survey to students, asking for their participation. Reminder e-mails were sent at 2 and 3 weeks after the initial message. All participants gave informed consent prior to participation, and provided their first and last names and 6-digit date of birth so that we could compare their data in a future survey. We obtained demographic information from each participant, including gender, age, prior career if applicable, whether or not a subinternship had been completed, and anticipated residency.
Means and standard deviations were calculated for continuous variables. Incidence rate ratios (IRR) were calculated to examine the effect of dichotomous variables on frequency of procedure performance. Relationships between frequency of performance, self-confidence, and perceived importance were assessed using the Spearman rank correlation. The study was performed with the expedited approval of the Institutional Review Board of the University of Pittsburgh and the IRBs at the other 6 participating institutions. All analysis was performed at the University of Pittsburgh using STATA 9 (Stata Corp., College Station, TX). Funding for the study was provided by the Shadyside Hospital Foundation (Pittsburgh, PA) and the Data Center at the Center for Research on Health Care at the University of Pittsburgh (Pittsburgh, PA).
The response rate was 71% (122/171). Individual schools’ students comprised 9% to 21% of the total respondents. Mean age was 27 years, and 60% were female. At least 90% of students completed clerkships in medicine, surgery, pediatrics, obstetrics/gynecology, psychiatry, and family medicine. Fewer than 8% of students completed a subinternship in medicine, surgery, or pediatrics. The most common prior health care-related careers included paramedic (8%) and phlebotomist (5%). The most common anticipated residencies included internal medicine (12%), family medicine (10%), pediatrics (9%), and obstetrics/gynecology (9%).
As illustrated in Table 1, for seven procedures, a majority of students reported never having performed them at all, and at least one-quarter of students had not performed procedures such as phlebotomy, arterial blood sampling, and peripheral IV catheter insertion. Students reported a mean self-confidence of less than 3.0 for 12 skills. Students’ perceived importance of being competent in skill performance was greater than 3.5 for all skills, and greater than 4.0 for 16 skills. Skills with high perceived importance and low self-confidence included cardioversion, lumbar puncture, thoracentesis, cardiopulmonary resuscitation, and paracentesis. An IRR > 1 in frequency of performance, indicating an increased number of times a skills was performed, was associated with formal teaching for 11 skills, with formal evaluation for 7 skills, and with the presence of curricular materials for 3 skills (all p < .05). Thus, for example, an IRR > 1 for lumbar puncture associated with formal teaching implies that there was an increased expected number of lumbar punctures performed when there was formal teaching of this procedure. Self-confidence and frequency of performance were positively correlated for all skills using the Spearman coefficient and ranged from 0.21 for electrocardiogram interpretation and 0.24 for chest X-ray interpretation, to 0.81 for arterial blood sampling and 0.85 for peripheral intravenous catheter insertion (all p < .05).
Despite the recommendations of the AAMC and the consensus of CDIM members, not all students are learning and performing skills, several of which are listed in the MSOP Report I. In 1994, Fincher and Lewis reported on the experiences of third-year students at one institution; of the 100 students responding, only 22% reported never having performed a lumbar puncture, and all students had performed phlebotomy and obtained blood cultures.8 Our study showed that, more than 10 years later, a greater percentage of third-year students reported never having performed these same procedures, and a majority are not performing important skills.
Students themselves appeared to value the importance of being competent in procedural and interpretive skills as a medical school graduate. Frequency of performance and self-confidence in skills were positively correlated. Hence, a key to improving students’ procedural skills is to enable their performance in the first place. It is tempting to simply recommend prioritizing curricular development for those skills that a majority of students have not performed. Our data showed that formal teaching, formal evaluation, and curricular materials increased the expected number of times a skill was performed for, at most, about half the skills studied. Therefore, it is important to consider factors preventing students from performing procedural skills.
The decreased frequency of procedure performance may be explained in several ways. Given the focus on patient safety, some procedures now may not be easily done by residents and students, whether for real or perceived risks. For example, at one institution in this study, a resident who has done fewer than 3 thoracenteses must locate an experienced resident or call a pulmonary consultation for supervision to do this procedure. It is often easier to have the thoracentesis performed by an interventional radiologist. Some institutions may have few opportunities to perform procedures such as thoracentesis and lumbar puncture, so residents may have priority in performing procedures, given that the Accreditation Council for Graduate Medical Education requires residency programs to document procedure competence for residents. There has been development of support services at hospitals for procedures considered “basic,” such as phlebotomy, peripheral IV catheter insertion, and arterial blood sampling, so that they are no longer done by residents or students but instead by phlebotomists, nurses, and respiratory therapists. It is conceivable that with the resident duty hours regulations, hospitals may be expanding support services to remove resident responsibility, and therefore student opportunity, for performing basic procedures.
Our study had several limitations. Only 7 schools were studied, and only subsets of medical students from each class were surveyed. Our list of skills is by no means exhaustive, and may have been biased toward internal medicine-related procedures; even so, we did not ask about arthrocentesis or skin biopsy, nor did we ask about procedures such as endotracheal intubation or splinting and casting. Students may have had recall bias in estimating the number of times a procedure was performed. Their perceived importance of procedures they have never done may not be accurate, as they would have been guessing at future significance. Students may also have misinterpreted the skills listed or not read the definitions of procedures as provided.
However, our study had several strengths that should be noted. This was a multi-institutional study combining both private and public medical schools, and the first study to the best of our knowledge asking students themselves about the skills listed in the AAMC MSOP Report I. We provided definitions of procedural and interpretive skills to the medical students to minimize their misunderstanding of a procedure’s details. By surveying students from the final internal medicine clerkship block in their third year, we hoped to reduce recall bias, as students may perform a greater number of procedures during the medicine clerkship.
In 2005, the AAMC Project on the Clinical Education of Medical Students published its Recommendations for Clinical Skills Curricula for Undergraduate Medical Education, which presented recommendations from national clerkship organizations, the Alliance for Clinical Education, and the American Academy on Physician and Patient regarding setting standards for clinical education.9 The skills recommendations, which included those in our study as well as others we did not examine, were meant as a resource for schools to use in determining what their students should learn. Medical educators will be faced with deciding which skills should be taught and how to do so. A question thus arises regarding the importance of students learning procedural skills that they may not perform in their future residency field or career. For example, should a student learn to place a urethral catheter if he/she will never do this as a resident or attending physician? Some may say that nurses are more proficient, and that having learned this skill does not make for a better doctor. Others may argue that learning the skill makes the student better understand what a patient experiences, even if the skill is not performed in the future. It is possible that the medical educators, who have recommended learning a wide variety of skills, trained at a time when ancillary services for procedures such as placing a urethral catheter or phlebotomy did not exist, and there was not as great a focus on patient safety; are their opinions applicable in 21st century medical education?
Still, our findings are worrisome in that through the third year, medical students do not appear to be performing skills that both they and medical educators at this time feel they should master. It is clear that formal teaching is important in helping students learn and perform skills. Given that opportunities for and policies regarding performance of skills may differ among institutions, we strongly suggest that clerkship directors and other medical educators investigate barriers to medical student procedure performance at their respective institutions as a first step in developing clinical skills curricula.
We gratefully acknowledge the generous support of the Shadyside Hospital Foundation (Pittsburgh, PA) and the Data Center at the Center for Research on Health Care at the University of Pittsburgh (Pittsburgh, PA). We also acknowledge the key assistance of Doris M. Rubio, PhD (Director, Data Center) and Jennifer D. Reisinger, MS (Systems Analyst).