In a 2015 Academic Medicine article, Klamen1 proposed an entirely new model for third-year clerkships at Southern Illinois University School of Medicine (SIUSOM). The school is a small (~70 students/year), public, community-based, midwestern medical school. The envisioned clerkship leaned heavily on theories of deliberate practice2 and legitimate peripheral participation.3 It was created to address deficiencies found from research into the clinical year, including that students’ clinical reasoning ability did not increase at the rate expected, and students spent large amounts of time studying for standardized multiple-choice exams and listening to lectures rather than engaging in clinical work.4–8 A premise of the new model was that the goals and activities of today’s clinical settings are mismatched with students’ learning needs.9 As its creators, we also took issue with the overwhelming and unrealistic goals and objectives found in many traditional clerkships.10–15
In creating a new third-year curriculum, we aimed to create a learning environment that allowed for explicit practice in clinical reasoning as well as an opportunity for students to integrate into the center of medical work, rather than staying at the periphery. We wanted students to immerse in clinical work, not hide out to study for knowledge-based assessments (though we did expect students to do as well on their Step 2 Clinical Knowledge [CK] examinations as they have traditionally). Guiding principles of the new curriculum were:
- To more deeply engage students in experiential learning through clinical immersion.
- To pair individual faculty with individual students over longer periods of time, allowing trust to develop. Students could feel more comfortable exposing their weaknesses, and faculty could give more specific feedback and allow students more hands-on clinical activity.
- To achieve clinical reasoning goals, whereby students receive extensive and systematic clinical experience, provided over a longer period of time and under controlled clinical instructional conditions.
- To simplify goals and objectives for the core clerkships and align them with student learning needs. (Overall graduation objectives for students have not changed.)
- To follow core clerkships with individualized activities to help students explore areas of interest, choose their specialty, and improve areas of weakness in clinical work before the fourth year.
Since the 2015 article was published and as of our writing of this article, the new clerkship model has been developed and delivered through an entire year (2016–2017) and two-thirds of the next year (2017–2018). We offer this follow-up article to provide additional details about the envisioned curriculum as it has been delivered thus far and to provide initial evidence of its intended and unintended effects.
A year 3 program evaluation committee was formed in 2014 to plan the evaluation of the new curriculum. The program evaluation committee was charged with answering three questions: “What is the impact of the year 3 curriculum change on student performance, clinical skills, and diagnostic reasoning?” “How does the new year 3 curriculum change the clinical learning environment?” and “Do students have a more active role in clinical care in year 3?” The evaluation committee consisted of students, medical education faculty, clerkship directors, and nurse educators. One external evaluator (R.W.) worked with the committee to detail plans. The committee used a mix of quantitative and qualitative methods, including comparison groups, case studies, focus groups, performance exams, and surveys, in addition to existing outcome data. SIUSOM’s institutional review board gave this project a nonhuman subjects research determination.
The New Third-Year Curriculum, Enacted
The 2015 article1 divided the work of the third year into three components, which we discuss below.
- Critical clinical competencies (CCC) online, video-based curriculum (years 1–3)
- Core clinical clerkships (first 8 months of third year)
- Personalized educational plan (PEP) (last 15 weeks of third year)
With a generous grant from the Josiah Macy Jr. Foundation, the online, video-based CCC curriculum was implemented exactly as envisioned. Students learn to reason clinically by completing 12 chief complaints (e.g., headache and abdominal pain are 2) each year in a spiral manner, beginning in year 1, and ending two-thirds of the way through year 3 (see http://siucccdemo.org). This part of the curriculum enables deliberate practice and immediate feedback for 12 chief complaints (and 144 linked diagnoses) as well as the opportunity to address the problems of an unpredictable patient and experience a case mix associated with modern clinical care settings. It allows students to compare their clinical reasoning with that of practicing physicians from different specialties evaluating the same cases. These videos allow students to “see” the thinking of physicians, a goal that is frequently espoused in medical education theory but is rarely systematically accomplished in practice. It frees the clerkship year to be unapologetically idiosyncratic and opportunistic because all students systematically and deliberately encounter all the same 144 diagnoses during the CCCs.
To track performance in clinical reasoning, there are a total of 9 standardized patient encounters in the first year and 12 in the second. The comprehensive, uncued, 14-station summative clinical competency examination (SCCX), traditionally given at the beginning of the fourth year, was moved earlier by five to six months, immediately at the end of eight, 4-week clinical (core) clerkships.
Our hope was that performance on the SCCX would improve from an average of 65% scores on a standardized patient case,16 to averages of 85% or better. This did not happen in the first year of the curriculum (2016–2017), and the following class (clerkship rotations 2017–2018) performed similarly. It is notable, however, that both classes of 2018 and 2019 took the SCCX five to six months earlier than students in the traditional third-year curriculum, but scored equally well (see Table 1).
Core Clinical Clerkships
The core clinical clerkships were developed to provide clinical immersion to the specialties. Students rotate through eight rotations, each four weeks in length (internal medicine, psychiatry, pediatrics, surgery, family medicine, neurology, emergency medicine, and obstetrics–gynecology). In addition to further improving clinical reasoning, two goals were set for the eight-month period: that of socializing into medicine, and allowing students to find their medical specialty. (See the mobile “on-the-fly” form used for feedback on socializing into medicine, http://edaff.siumed.edu/Year3/Core/otf.asp.) Lectures and end-of-clerkship shelf examinations were eliminated. Students experience clinically immersive clerkships (5 to 7.5 hours/day in clinical practice, up from 3 hours/day in the traditional curriculum).8,17 Groups of students rotate through each clerkship paired with a single faculty supervisor over four weeks, rather than rotating through a series of very short experiences with multiple faculty or residents. This allows opportunities for deliberate practice2 in a coaching model. It also increases the likelihood that a relationship of trust will develop between the student and faculty member, resulting in more high-quality clinical opportunities for the student and more honest discussions about the student’s performance between student and faculty. Students may view this feedback 24/7.
The change for faculty from evaluator to coach, and for students from practicing impression management to deliberately exposing deficiencies and asking for feedback, required great amounts of faculty and student development. Mandatory faculty development sessions on coaching were instituted; over 90% of faculty completed them. Students in a medical education interest group, mentored by a faculty member, reviewed the extensive literature on coaching18–22 and the sparser literature on learner coachability23 and developed a curriculum on coachability for students. After this experience, as we learned from the evaluation questionnaire students completed after the coachability curriculum, third-year students felt more prepared to set learning goals, solicit feedback, and reflect and act on it.24
A longitudinal clerkship advisor is assigned to every student, and they meet monthly throughout the core rotations. The clerkship advisors help students review their on-the-fly feedback data and help their students develop and continually revise personal goals for each clerkship. Together, advisors and students select experiences for the following PEP period (described below). Poor performance is addressed directly. The clerkship advisors received mandatory faculty development sessions; 100% were trained.
Compared with graduates of our traditional curriculum, students were found to have very different attitudes toward what they defined as success in a clerkship in the new third-year approach.25 For example, when asked, “How would you know you are succeeding in the clerkships?” students from the traditional curriculum noted, “I am successful when I [achieve] honor[s in] the NBME shelf exam” and receive “positive feedback from my supervising doctors.” Contrast those statements with the predominant views from students in the new third-year curriculum: “I knew I was successful when I saw the patient leave the office/hospital satisfied,” and “I knew I was successful when I felt like a member of the team.” Students in the new curriculum were much more likely to use specific attendings’ names in their discussions of their clerkship experiences, whereas in the old curriculum, specific names were uncommonly mentioned, and the generic “my attendings” was used instead. We see these as hallmarks of clerkship success.
Students in the new curriculum reported performing more procedures. After curriculum reform, babies delivered by students rotating on the obstetrics–gynecology clerkship increased from 0.55 to 1.0/week, placentas delivered increased from 1.0 to 3.3/week, and pelvic exams performed increased from 2.2 to 4.9/week (for 68 students, 2016–2017).26 In the surgery clerkship, students scrubbed and participated more in surgery than in the traditional curriculum, from an average of 4 times per week to an average of 5.7 times per week (68 students, 2016–2017).27 Students formed longer-term relationships with faculty members who felt the students could be entrusted, and therefore allowed them to do more procedures (even though all clerkships are shorter than they were traditionally).
Overall, most stakeholders (students, clerkship directors, nurse educators, faculty, and residents) were positive about the movement toward a learning culture focused on clinical immersion and formative feedback. Faculty liked the opportunity to really get to know a student; 88.7% (92/104; 2016–2017) noted that they worked with the same student over multiple days.27 They liked not having to assign a summative grade. (All formative on-the-fly narrative data were compiled centrally in each department, and the clerkship director assigned a final grade.) Two faculty were not in favor of the new curriculum and wished to go back to longer clerkship times with exams and lectures: “I fear that it is inadequate in proper exposure to core areas.” Others did not like being asked by students for frequent feedback.28
Students overwhelmingly liked the ability to become a working part of the team, see more patients, and receive formative feedback. They also felt there was no wasted time in the clerkships, as had been seen in the traditional clerkships. Concerns raised included worries about future performance on Step 2 CK without mandated shelf exams, and feelings that an evaluation based merely on subjective data was not sufficiently standardized to evaluate students fairly. Students participating in the program evaluation focus groups, as well as the third-year student representative on the year 3 curriculum committee, suggested that the entire clerkship year become pass–fail, arguing that an honors designation was not in keeping with the goals and objectives of the new third year. Knowing that a designation of honors was possible pushed students back into “impression management” roles.
Although student performance data are still being analyzed, it is clear that students performed well in the new clerkships. Step 2 CK scores for the class of 2018 showed an overall pass rate of 97% (66/68 students). These numbers are well within the range of normal for SIUSOM students (91%–99% in the past 10 years29). We did not expect an increased pass rate, but we wanted to ensure that it was not diminished. One hundred percent of students in the class of 2018 (68 students) passed Step 2 Clinical Skills.
As a response to the need for a more individualized, flexible curriculum,30 a 15-week PEP period came next. Students met with their clerkship advisors and selected a PEP schedule that best met their personal learning needs. Some students who had selected a specialty (~60%) opted to do a “deeper dive” into it, including experiencing more subspecialties. Students who had not yet chosen a specialty (~35%), but having narrowed the options down, rotated through experiences within those specialties to help them make a choice. Students needing to remediate clerkships or showing deficiencies in clinical skills (~5%) did so during the PEP period.
Students and faculty both responded very positively to the PEP period. Students liked having a hand in designing their own curriculum, and faculty liked working with students who were specifically interested in their fields. There was considerable concern before implementation of the PEP that the logistics of scheduling would be impossible: “What if everyone chooses to do 15 weeks of medicine and no one chooses psychiatry?” These concerns never materialized, as students chose a wide variety of specialties and scheduling was not onerous.31
The graduating class of 2018 (the first to go through the new third-year curriculum) matched successfully to a wide variety of specialties; their match rate and match list distribution were essentially unchanged from those of the previous (traditional third-year) class.
Because the curricular change was significant, it may be helpful to discuss some of the critical issues for success as well as some barriers encountered, because these issues may arise in any medical school’s attempt at a large curricular revision.
Critical Issues for Successful Implementation
The formation of functional and interconnected subcommittees was crucial. In 2014, six subcommittees were formed in preparation for creation and implementation of the new clerkship year (core, PEP, assessment, faculty development, CCC, and program evaluation). Each of these subcommittees had representation from all clerkships, two medical students, an educational expert, and the senior associate dean for education and curriculum (D.L.K.). Everyone had a voice. Educational expertise to explain theories behind the change was available, and there was an administrative voice to keep everyone on track and moving.
The support of the dean from the beginning (2013) to the present has been essential. Not all faculty members wanted to make a change. It was critical that the dean was well informed about the change and the need for it, and took a public stance that work would move forward on the new curriculum structure.
Students at SIUSOM are known for their engagement.32 Without them, this change would not have been successful. Students worked tirelessly, spreading the word about the new curriculum and reassuring those who worried. Students were active on subcommittees. It was often a student’s voice that changed a faculty member’s perspective and manner of thinking about aspects of the new curriculum. Students developed and successfully implemented a coachability curriculum. Working on the new curriculum has resulted in a number of students being actively involved in medical education research and development.
Early adopters allowed the curriculum to gain momentum. Faculty hearing about the new curriculum’s theoretical base, and dissatisfied with the current curriculum, drove the implementation process faster. Nurse educators, embedded in the individual clerkships, bought in early, which also facilitated change.
Groundwork by the senior associate dean (D.L.K.) helped the process (2013–2016). She met with every clerkship director individually to describe the need, rationale, and proposed plan for the change. Questions and concerns were addressed. This allowed for a richer and more informed discussion at the curriculum meetings and prevented the group mind from polarizing against change before all the facts were known.
A generic resistance to change, as well as concern about the extra work required, were expected, and encountered frequently (2013–2017). Clerkship curriculum change as dramatic as this has rarely occurred. Change is scary, especially when changing to something entirely new, and when many were happy with the traditional clerkship. Faculty and staff were already working hard. Inertia was high at the beginning but slowly declined as faculty had a voice in the new curriculum and became dedicated to it. Celebrating small steps forward and watching how engaged the students became helped. The total amount of time spent by faculty with third-year medical students did not change in the new curriculum, which helped. Rather, faculty spent time with one student for a longer period rather than multiple students for one or two days each.
A particular argument that arose repeatedly was that SIUSOM should not adopt this radical change without knowing if it was going to work (2012–2013). This slowed movement significantly. Eventually, the committees realized that new innovations cannot be proven to work before they are enacted. This innovation came with a robust evidence base that predicted its success. Because an equally strong evidence base had not been available for numerous major innovations previously undertaken at SIUSOM (including the use of standardized patients and problem-based learning), this argument was eventually seen as specious.
Logistics were a barrier in several settings. There was concern that some specialties might be overwhelmed with students during the PEP period, while others would see no third-year students for four months (2015–2016). Although this did not occur, it was a major concern up front.
The other logistical issue was that of faculty rotating frequently in some clinical settings. Continuous monitoring of what was occurring at ground level in the clinical educational environment was beneficial in anticipating and troubleshooting this issue. Faculty might rotate off an inpatient service in the middle of a student’s four-week rotation. Smaller departments worried about scheduling a student with one faculty member, only to have that person go on vacation during the student’s rotation. These inevitabilities of clinical life led the clerkship directors to assign one to four faculty to a student during the four weeks, with an emphasis on having the longest connections between faculty and student as possible.
Not everyone bought into the new curriculum. For example, one clerkship assigned one faculty member for four weeks (the ideal situation) but another split its students between faculty in outpatient (two weeks) and inpatient (two weeks) settings (2016–2017). Faculty buy-in has improved but continues to need attention. Faculty from departments with less overall buy-in reported equally high satisfaction with the new curriculum, however.28
There was not enough time in the first implementation of this curriculum to provide faculty development for residents. It became obvious that there was a need, as many did not like being “bothered” by students asking for feedback, and some were worried by the students’ willingness to discuss their own deficiencies. Residents commented to students that “letting an attending in on your weaknesses is going to be really bad for your evaluation.”
There was early concern that the new curriculum would not meet Liaison Committee for Medical Education (LCME) standards (2014–2015). The LCME was notified two years before implementation of the new curriculum, and it saw no potential problems. SIUSOM had a site visit in 2015 by the LCME, and it received no citations, though the site visitors did request an additional meeting to discuss the upcoming change in more detail.
After the successful first year of the curriculum, we made several modest changes. The assessment committee dropped the honors designation for the core clerkships, leaving the core clerkships, as well as the PEP period, pass–fail only. (There was never an honors designation during the PEP period.) This was done to remove the ongoing “impression management” need, allowing students to be comfortable discussing their deficiencies. In the traditional curriculum and the first-year implementation of the new core curriculum, there had been a pass–fail–honors system. The student-run coachability curriculum is now offered in years 1, 2, and 3, per student request. Resident development is being created in podcast form, on topics of becoming a coach and using the on-the-fly feedback form.
The 2015 publication1 called for medical schools to rethink the third-year clerkship in a way that embraced the reality of today’s clinical settings, and maximized and optimized learning. It asked schools to stop pretending that “all is well” within the current traditional clerkships. In the new third year at SIUSOM, the addition of longitudinal, explicit, clinical reasoning experiences from years 1 through 3 enables deliberate practice in this crucial skill on 144 different diagnoses and helps overcome the inability to ensure systematic exposure to patients in clinical settings. Students were more active in the clinical setting and more engaged as part of the health care team. Faculty and students appreciated the closer relationships with one another and the new assessment system of formative feedback. This change met our original goals of increased clinical immersion and engagement for students and changed the clinical culture to a coaching model with trust built between students and faculty. It also met with both faculty and student satisfaction. We have, as the 2015 article requests, “gotten real.”
Thanks to Reed Williams, Kevin Dorsey, Jerry Kruse, Anna Cianciolo, Jeanne Koehler, Heeyoung Han, the year 3 program evaluation committee, and the medical student medical education interest group. Without you and the Southern Illinois University School of Medicine (SIUSOM), this curriculum would not have been realized.
1. Klamen DL. Getting real: Embracing the conditions of the third-year clerkship and reimagining the curriculum to enable deliberate practice. Acad Med. 2015;90:13141317.
2. Ericsson KA, Krampe RT, Tesch-Romer C. The role of deliberate practice in the acquisition of expert performance. Psychol Rev. 1993;100:363406.
3. Lave J, Wenger E. Situated Learning: Legitimate Peripheral Participation. 2003.Cambridge, UK: Cambridge University Press.
4. Williams RG, Klamen DL. Examining the diagnostic justification abilities of fourth-year medical students. Acad Med. 2012;87:10081014.
5. Williams RG, Klamen DL, Markwell SJ, Cianciolo AT, Colliver JA, Verhulst SJ. Variations in senior medical student diagnostic justification ability. Acad Med. 2014;89:790798.
6. Williams RG, Klamen DL, Hoffman RM. Medical student acquisition of clinical working knowledge. Teach Learn Med. 2008;20:510.
7. Williams RG, Klamen DL, White CB, et al. Tracking development of clinical reasoning ability across five medical schools using a progress test. Acad Med. 2011;86:11481154.
8. Han H, Roberts NK, Korte R. Learning in the real place: Medical students’ learning and socialization in clerkships at one medical school. Acad Med. 2015;90:231239.
9. Billett S. Constituting the workplace curriculum. J Curriculum Stud. 2006;38:3148.
10. Alliance for Academic Internal Medicine. Core medicine clerkship curriculum guide version 3.0. http://www.im.org/page/cdim-sgim-core-medicine-clerkship-curriculum
. Accessed August 10, 2018.
11. Curriculum Committee of the Association for Surgical Education. The manual of surgical objectives: A symptom and problem-based approach. 4th ed. http://www.surgicaleducation.com/assets/documents/surgicalobjectivesmanual4.pdf
. Accessed December 14, 2017. [No longer available.]
12. Association of Directors of Medical Student Education in Psychiatry. Learning goals and milestones in psychiatry. http://www.admsep.org/milestones.php?c=learning-goals
. Accessed August 10, 2018.
13. Olson AL, Woodhead J, Berkow R, Kaufman NM, Marshall SG. A national general pediatric clerkship curriculum: The process of development and implementation. Pediatrics. 2000;106(1 pt 2):216222.
14. Association of Professors of Gynecology and Obstetrics. Medical student educational objectives. http://www.youtube.com/playlist?list=PLy35JKgvOASnHHXni4mjXX9kwVA_YMDpq
. Accessed May 15, 2018.
15. Society of Teachers of Family Medicine Task Force. The family medicine clerkship curriculum. http://www.stfm.org/Resources/STFMNationalClerkshipCurriculum
. Accessed December 14, 2017. [No longer available.]
16. Petrusa ER. Taking standardized patient-based examinations to the next level. Teach Learn Med. 2004;16:98110.
17. Brock K, Beason A, Neumeister E. Oral presentation. Presented at: Association for Medical Educators in Europe Annual Conference; August 27, 2017; Helsinki, Finland.
18. Cianciolo AT, Kidd B, Murray S. Observational analysis of near-peer and faculty tutoring in problem-based learning groups. Med Educ. 2016;50:757767.
19. Régo P, Peterson R, Callaway L, Ward M, O’Brien C, Donald K. Using a structured clinical coaching program to improve clinical skills training and assessment, as well as teachers’ and students’ satisfaction. Med Teach. 2009;31:e586e595.
20. Deiorio NM, Carney PA, Kahl LE, Bonura EM, Juve AM. Coaching: A new model for academic and career achievement. Med Educ Online. 2016;21:33480.
21. Thorn PM, Raj JM. A culture of coaching: Achieving peak performance of individuals and teams in academic health centers. Acad Med. 2012;87:14821483.
22. Palamara K, Kauffman C, Stone VE, Bazari H, Donelan K. Promoting success: A professional development coaching program for interns in medicine. J Grad Med Educ. 2015;7:630637.
23. Ciuchta MP, Letwin CR, Stevenson RM, McMahon SR. Betting on the coachable entrepreneur: Introduction of a new construct and scale. Acad Manage Proc. 2015;(meeting abstract supplement):15065.
24. Ghareeb A. Coachability curriculum evaluation data. December 2017.Personal communication with D.L. Klamen.
25. Koehler J. Pre/post qualitative data on students’ views of clerkship success. December 2017.Personal communication with D.L. Klamen.
26. Dorsey K. Numbers of procedures done on the obstetrics–gynecology clerkship in the new third year versus the traditional third year clerkship. December 2017.Personal communication with D.L. Klamen.
27. Southern Illinois University School of Medicine (SIUSOM) Y3 Program Evaluation Committee report. Changes in surgery activity in the new third year versus the traditional third year clerkship. Presented to the Year 3 Curriculum Advisory Committee of SIUSOM; April 10, 2018; Springfield, Ill.
28. Koehler J. Faculty satisfaction data with regard to the new Year 3 curriculum, 2016–2017. December 2017.Personal communication with D.L. Klamen.
29. Robbs R. Pass rates of senior medical students from Southern Illinois University School of Medicine, 2007–2017. December 2017.Personal communication with D.L Klamen.
30. Irby DM, Cooke M, O’Brien BC. Calls for reform of medical education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Acad Med. 2010;85:220227.
31. Forsyth C. Personalized education plan scheduling counts during first year of the new clerkship. December 2017.Personal communication with D.L. Klamen.
32. Southern Indiana University School of Medicine (SIUSOM) winner of AMEE ASPIRE-to-excellence award in 2013. http://amee.org/awards-prizes/aspire-award/2013-winners
. Accessed August 10, 2018.