Alcamo, Alicia M.; Davids, Abby R.; Way, David P.; Lynn, D. Joanne; Vandre, Dale D.
Background: Medical students use several strategies for United States Medical Licensing Examination (USMLE) Step 1 preparation. At Ohio State University College of Medicine, a yearlong, peer-designed and -led Step 1 review course is a new option for our second-year students. This study aims to ascertain the value of the peer-designed and -led Step 1 review course, to assess the difference in Step 1 scores between participants and nonparticipants, and to understand the course's role in improving preparation for Step 1 among participants.
Method: Eligible students completed a confidential survey. Scores between participants and nonparticipants were compared, controlling for preexisting differences between groups.
Results: Course participants had a higher average Step 1 score than nonparticipants (P = .005). The majority of participants felt the course was a valuable use of time and would recommend it to future students.
Conclusions: A Step 1 review course designed and led by near-peer senior medical students, those who had successfully completed the USMLE Step 1 exam within the previous year, was shown to be valuable to second-year medical students and improved Step 1 score outcomes.
At the Ohio State University (OSU) College of Medicine, the majority of medical students prepare for and take the United States Medical Licensing Exam (USMLE) Step 1 at the end of their second year. The USMLE Step 1 exam is the first of three examinations necessary for allopathic licensure, and exam performance is also influential for a student's placement within a residency program. Given the high-stakes nature of this exam, there are a number of different resources and methods for preparation available, including commercial review courses and books. However, several studies have shown that the commercial review courses do not improve performance on USMLE Step 1.1–4 Thus far, the only proven predictors for performance on USMLE Step 1 are academic performance in medical school and Medical College Admission Test (MCAT) scores.2,5–8
At our institution, most students develop self-study plans independent of commercial review courses. The college facilitates the dissemination of general preparatory ideas and advice from one class to the next. In the past, faculty-led Step 1 review sessions were an option; however, there has been no formal board preparation program offered since the academic year 2005-2006. To fill this need, two senior medical students developed and taught a new optional review course for second-year medical students.
Peer-assisted learning has been studied in other areas of medical education and has been shown to be beneficial for both the students and the teachers.9–11 However, its use for USMLE Step 1 preparation has not been examined. This investigation aimed to explore the role of supplemental peer instruction in preparation and performance on USMLE Step 1. The three major objectives of this study were to (1) assess the difference in USMLE Step 1 scores between participants and nonparticipants, (2) ascertain the value of supplemental peer education to participants, and (3) understand the course's role in improving preparation strategies and alleviating anxiety surrounding the exam.
At the beginning of the second year of medical school, all students were offered the opportunity to participate in an optional, year-long, weekly board review course that was both designed and taught by two senior students who recently completed the USMLE Step 1 examination and were on a leave of absence for a dual-degree program prior to beginning their clinical training (near-peers). The senior students served not only as near-peer teachers but also as resources for questions and strategic development.
The review course paralleled the Integrated Pathway (IP) curriculum at OSU, which is a classroom-based lecture discussion pathway organized around organ systems. While the majority of students are enrolled in the IP curriculum, OSU also offers an Independent Study Pathway focused on independent direction and learning that is organized in a more traditional physiologic/pathologic curricular sequence. The review course was intended to supplement the curriculum, but it was not considered an official component of the curriculum.
The course consisted of 19 classes held weekly over the period of September 2008 to April 2009. Five separate hour-long evening sessions were offered for each class, with enrollment in each session capped at no more than 30 students. Although initial interest exceeded capacity, all students interested in participating eventually were able to do so because of slight enrollment fluctuations throughout the year.
Each session followed the same format. Students were given a quiz with 10-12 USMLE-style questions to be completed independently within the first 10 minutes of class. Each question on the quiz covered a different subtopic reviewed during that session. The senior student leaders subsequently reviewed the questions with a supplemental presentation highlighting pertinent, high-yield aspects of the topic. Information was presented in an interactive, question-and-answer format to encourage student participation and discussion. Graphics and photos were used to enhance retention and provide clinical correlations. Each class ended with a review of corresponding rapid associations covering high-yield information. Students who attended the week's session received the corresponding course materials by e-mail for further self-review.
A program evaluation survey was designed by the two board review course instructors and reviewed by both a College of Medicine board review steering team and an individual with expertise in survey design. Survey items asked for student feedback on various board review strategies, including aspects of the board review course and its impact on their preparation for USMLE Step 1.
The survey was administered using an electronic survey tool. It was sent to all eligible medical students (n = 198) regardless of whether they attended the board review sessions or not. The timing of the survey administration was approximately eight weeks after students had taken the actual USMLE Step 1 examination and about five months after the conclusion of the board review course. This timing was chosen to ensure that students had received their Step 1 score reports before completing the survey.
Other variables used for this study included actual scores on the USMLE Step 1, attendance records for the board review course, and total MCAT scores. The study protocol was reviewed and approved by the OSU institutional review board for behavioral science research.
Survey data were analyzed and descriptive statistics were generated using SPSS Version 17.0. Individual students' attendance records for the board preparation course were matched to their USMLE Step 1 and total MCAT scores. Participant and nonparticipant groups were formed on the basis of attendance records, and students who attended more than half of the 19 sessions were considered participants. It should be noted that not all participants completed the survey (see results). A one-way analysis of covariance (ANCOVA) was used to compare the USMLE Step 1 scores between the two groups. Total MCAT score served as a covariate to control for potential preexisting differences in academic aptitude between the two groups.
The program evaluation survey return rate was 67.7% (134 of 198). Among the 134 survey respondents, 84 attended 50% or more of the board preparation sessions, and these students were defined as course participants. This group represents 62.7% of the survey return group and 42.4% of the total population of medical students in the class. The survey responses of course participants (Table 1) show that they agreed (as defined by survey results combining responses of agree and strongly agree) that it was helpful to have the preparation sessions run by their peers (85.5%; 71 of 83). Most participants also said that they would recommend this program for future medical students (69.1%; 58 of 84) and that they found the sessions to be useful even during the regular medical school year (71.4%; 60 of 84). Almost 64% (53 of 83) of the course participants said that they found the sessions to be a valuable use of their time. Further, the sessions were perceived as being a useful supplement to the regular curriculum in both addressing deficiencies in the curriculum as well as helping with their studies during and for the regular curriculum.
Interestingly, while most participants found the board preparation sessions to help them become more comfortable in analyzing and answering Step 1 style questions and identifying areas on which to focus their board preparation (72.3%; 60 of 83 and 63.85%; 53 of 83, respectively), the sessions did not seem to alleviate their anxiety or instill confidence in their preparation for the actual Step 1 examination (32.5%; 27 of 83 and 40.4%; 34 of 84, respectively). Participants also did not seem to agree that the preparation sessions helped them to improve their Step 1 scores (39%; 32 of 82). However, when the USMLE Step 1 scores were examined the ANCOVA revealed that the course participant group (including all students that attended more than 50% of the sessions regardless of whether they completed the survey tool or not) scored significantly higher than the nonparticipant group. Participant adjusted mean and standard error was 231.25 (1.81) on the USMLE Step 1 examination, while the nonparticipant adjusted mean was 223.2 (2.15) with P = .005; F = 8.17; df = 1,185; and the adjusted R2 for the model = 0.268.
Peer-assisted learning has been shown to be an effective method of improving academic performance in a number of clinical and health-related educational settings including gross anatomy, basic sciences, problem-based learning, clinical skills, and communication skills.9,12,13 This study examined the utility of a peer-designed and -led USMLE Step 1 review course as an optional adjunct to the traditional second-year medical school curriculum. We found that participants in the course not only scored higher on USMLE Step 1 than nonparticipants but also valued the time they spent in the course. To our knowledge, there have not been any other studies examining the effectiveness of a peer-designed and -led USMLE Step 1 review course. Therefore, the positive results we have shown in this study suggest that peer-assisted learning techniques are also an effective method to help medical students prepare for high-stakes national exams such as the USMLE Step 1.
The majority of students identified the peer-teaching format of the course to be of particular benefit. According to Cate and Durning,12 “near-peer teachers” (students separated by only a year or two of education but still part of the same program) create a positive learning environment in several ways. Peer teaching provides an avenue for students to learn from teachers on their same level of understanding, gain access to role models, and find additional motivation and techniques to study, all in an informal, nonthreatening environment.12 The value of near-peer teaching has been attributed in large part to both cognitive and social congruence in the relationship between the near-peer teachers and their junior student colleagues.14–16 The board review course in this study was designed by near-peers that had recently experienced and completed both the first two years of medical school and the USMLE Step 1 examination. As a result, the near-peer leaders in this study appeared to be at an optimal educational distance to provide the most effective teaching for the junior students. The results from this study, in addition to our experience in offering this course, reinforce these ideas. The majority of students identified their participation as a valuable use of time, felt their participation provided better familiarity with the exam, and acknowledged the course's role in helping to identify preparatory strategies. The fact that the majority of participants would recommend the course to future students also speaks to its value and effectiveness.
The majority of participants, however, did not feel that the course alleviated their anxiety toward the Step 1 exam, nor did it instill confidence in their performance on the exam. This finding may be due to the fact that the course exposed students to deficiencies in their knowledge and also reinforced the amount of work/study time that would be necessary to master the material in preparation for the Step 1 exam. The course, then, may have created additional anxiety surrounding the Step 1 exam. However, this additional perceived anxiety may have also been a powerful motivating factor for students enrolled in the course, and it may, in part, help explain why course participants had higher Step 1 scores than nonparticipants.
This study was limited by a number of factors. A single group of students at this institution were evaluated, therefore limiting the general applicability of our study. Replication of the experience in subsequent years and at different institutions may prove to be difficult, as the effects of the course may be limited by interactions between instructors and students and differences in class composition. Because the course was voluntary, more highly motivated students may have self-selected for participation. However, it is also possible that some of the self-selected students were those that felt they would do poorly on the exam and needed extra help. While differences in academic achievement were controlled for in this study, differences in motivation were not. Preliminary analysis of the attendance records and performance in the regular curriculum indicate that course participants seemed to be biased toward those performing around the mean in the regular curriculum, suggesting that poorly performing students could not effectively use the additional information, and highly performing students did not find the additional information useful. The present study was specifically designed to obtain student survey responses after having obtained their Step 1 scores, and thus individual students' feelings and attitudes about their Step 1 performance may have also influenced their responses.
Given these limitations and variables, it would be beneficial to study this concept both over time at our institution and at other institutions to determine if similar effects can be measured. The effectiveness of the peer-assisted learning sessions described here are in part the result of incorporating critical design features. Therefore, in implementing additional peer-assisted learning opportunities for USMLE Step 1 preparation, it will remain important to consider and incorporate effective design elements.17 In response to student enthusiasm for the course, the college created a formal board preparation team, which meets monthly to focus on comprehensive board review. The review course is in its second year with a new set of near-peer teachers engaged in the project. USMLE Step 1 remains a daunting and important step in medical education for which there is no proven pathway to success. However, the use of a peer-designed and -led review course may provide an effective and beneficial supplement to students' traditional preparation for the exam.
The authors would like to acknowledge Linda Stone, MD, for her devotion and support of this project from its earliest stages of development to its fruition.
The study protocol was reviewed and approved by the OSU institutional review board for behavioral science research.
1Werner LS, Bull BS. The effect of three commercial coaching courses on Step One USMLE performance. Med Educ. 2003;37:527–531.
2Thadani RA, Swanson DB, Galbraith RM. A preliminary analysis of different approaches to preparing for the USMLE Step 1. Acad Med. 2000;75(10 suppl):S40–S42.
3Scott LK, Scott CW, Palmisano PA, Cunningham RD, Nass JC, Brown S. The effects of commercial coaching for the NBME Part I examination. J Med Educ. 1980;55:733–742.
4Lewis LA, Kuske TT. Commercial national board review programs: A case study at the Medical College of Georgia. JAMA. 1978;240:754–755.
5Basco WT Jr, Way DP, Gilbert GE, Hudson A. Undergraduate institutional MCAT scores as predictors of USMLE Step 1 performance. Acad Med. 2002;77(10 suppl):S13–S16.
6Kleshinski J, Khuder SA, Shapiro JI, Gold JP. Impact of preadmission variables on USMLE step 1 and step 2 performance. Adv Health Sci Educ Theory Pract. 2009;14:69–78.
7Zhang C, Rauchwarger A, Toth C, O'Connell M. Student USMLE Step 1 preparation and performance. Adv Health Sci Educ Theory Pract. 2004;9:291–297.
8Way DP, Biagi B, Clausen K, Hudson A. The effects of basic science pathway on USMLE Step 1 scores. Acad Med. 1999;74(10 suppl):S7–S9.
9Secomb J. A systematic review of peer teaching and learning in clinical education. J Clin Nurs. 2008;17:703–716.
10Wong JG, Waldrep TD, Smith TG. Formal peer teaching in medical school improves academic performance: The MUSC supplemental instructor program. Teach Learn Med. 2007;19:216–220.
11Field M, Burke JM, McAllister D, Lloyd DM. Peer-assisted learning: A novel approach to clinical skills learning for medical students. Med Educ. 2007;41:411–418.
12Cate OT, Durning S. Peer teaching in medical education: Twelve reasons to move from theory to practice. Med Teach. 2007;29:591–599.
13Weyrich P, Celebi N, Schrauth M, Möltner A, Lammering-Köppel M, Nikendei C. Peer-assisted versus faculty staff-led skills laboratory training: A randomized controlled trial. Med Educ. 2009;43:113–120.
14Cate OT, Durning S. Dimensions and psychology of peer teaching in medical education. Med Teach. 2007;29:546–552.
15Bulte C, Betts A, Garner K, Durning S. Student teaching: Views of student near-peer teachers and learners. Med Teach. 2007;29:583–590.
16Lockspeiser TM, O'Sullivan P, Teherani A, Mulller J. Understanding the experience of being taught by peers: The value of social and cognitive congruence. Adv Health Sci Educ. 2008;13:361–372.
17Wadoodi A, Crosby JR. Twelve tips for peer-assisted learning: A classic concept revisited. Med Teach. 2002;24:241–244.