In Reply to Le et al

We thank Le and colleagues for furthering the dialogue on the timing of taking the United States Medical Licensing Examination Step 1. In our multiyear cross-institutional study, we demonstrated that moving Step 1 after core clerkships resulted in small increases in Step 1 scores and reduced failure rates. Increased scores are unlikely to represent meaningful knowledge gains. However, demonstration of noninferiority (maintained level of performance) allows schools to implement curricular reforms with less concern. Although the authors present their study as a contradictory result, the nonsignificant findings further support our claim that changing the timing of Step 1 is noninferior to current practices.

Our study has confounders. So, too, does Le and colleagues’ one-year, single-cohort study. First, although there was no significant change in average Step 1 scores, this was calculated after six months of clerkships with an unknown sample size that may be underpowered to detect effects. Most institutions have 9 to 12 months of core clerkships. Extrapolating a 0.35 increase in scores per week, small average score gains might be observed if students took Step 1 after completing all clerkships. Second, some struggling students were assigned schedules while others chose their clerkship order, potentially skewing results. Third, the authors mention differences based on clerkship order. This is not surprising if some learners took family or internal medicine (specialties with breadth) first. The authors did not address fail rates.

We, too, observed a tight relationship between preclinical grades and Step 1 scores. The strength of this association stayed relatively constant pre and post change. We did not observe significant learner distraction from patient care while on clerkships, and learners engaged more in the preclerkship curriculum, as they were less preoccupied by studying for Step 1.

During times of curriculum change, there are many cautionary tales. Thus, it is critically important that we share our findings and detail the unique circumstances in which they occur. We thank the authors for adding data to the dialogue, and encourage studies from other schools that have implemented rotating Step 1 schedules. We wholeheartedly agree there are challenges and unintended consequences to altering the timing of Step 1 (e.g., shelf exam performance weaknesses, increased anxiety), but there are also means to overcome them.¹

Michelle Daniel, MD, MHPE
Assistant dean for curriculum and associate professor of emergency medicine and learning health sciences, University of Michigan Medical School, Ann Arbor, Michigan; [email protected]; ORCID: http://orcid.org/0000-0001-8961-7119.

Daniel Jurich, PhD
Senior psychometrician, National Board of Medical Examiners, Philadelphia, Pennsylvania.

Sally A. Santen, MD, PhD
Senior associate dean of evaluation, assessment and scholarship and professor of emergency medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia.