Secondary Logo

Journal Logo

In Reply to Le et al

Daniel, Michelle MD, MHPE; Jurich, Daniel PhD; Santen, Sally A. MD, PhD

doi: 10.1097/ACM.0000000000002756
Letters to the Editor
Free

Assistant dean for curriculum and associate professor of emergency medicine and learning health sciences, University of Michigan Medical School, Ann Arbor, Michigan; micdan@umich.edu; ORCID: http://orcid.org/0000-0001-8961-7119.

Senior psychometrician, National Board of Medical Examiners, Philadelphia, Pennsylvania.

Senior associate dean of evaluation, assessment and scholarship and professor of emergency medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia.

Disclosures: The University of Michigan Medical School and Virginia Commonwealth University School of Medicine receive funding from Accelerating Change in Medical Education grants from the American Medical Association. D. Jurich works for the National Board of Medical Examiners, the organization that administers the United States Medical Licensing Examinations.

We thank Le and colleagues for furthering the dialogue on the timing of taking the United States Medical Licensing Examination Step 1. In our multiyear cross-institutional study, we demonstrated that moving Step 1 after core clerkships resulted in small increases in Step 1 scores and reduced failure rates. Increased scores are unlikely to represent meaningful knowledge gains. However, demonstration of noninferiority (maintained level of performance) allows schools to implement curricular reforms with less concern. Although the authors present their study as a contradictory result, the nonsignificant findings further support our claim that changing the timing of Step 1 is noninferior to current practices.

Our study has confounders. So, too, does Le and colleagues’ one-year, single-cohort study. First, although there was no significant change in average Step 1 scores, this was calculated after six months of clerkships with an unknown sample size that may be underpowered to detect effects. Most institutions have 9 to 12 months of core clerkships. Extrapolating a 0.35 increase in scores per week, small average score gains might be observed if students took Step 1 after completing all clerkships. Second, some struggling students were assigned schedules while others chose their clerkship order, potentially skewing results. Third, the authors mention differences based on clerkship order. This is not surprising if some learners took family or internal medicine (specialties with breadth) first. The authors did not address fail rates.

We, too, observed a tight relationship between preclinical grades and Step 1 scores. The strength of this association stayed relatively constant pre and post change. We did not observe significant learner distraction from patient care while on clerkships, and learners engaged more in the preclerkship curriculum, as they were less preoccupied by studying for Step 1.

During times of curriculum change, there are many cautionary tales. Thus, it is critically important that we share our findings and detail the unique circumstances in which they occur. We thank the authors for adding data to the dialogue, and encourage studies from other schools that have implemented rotating Step 1 schedules. We wholeheartedly agree there are challenges and unintended consequences to altering the timing of Step 1 (e.g., shelf exam performance weaknesses, increased anxiety), but there are also means to overcome them.1

Michelle Daniel, MD, MHPE

Assistant dean for curriculum and associate professor of emergency medicine and learning health sciences, University of Michigan Medical School, Ann Arbor, Michigan; micdan@umich.edu; ORCID: http://orcid.org/0000-0001-8961-7119.

Daniel Jurich, PhD

Senior psychometrician, National Board of Medical Examiners, Philadelphia, Pennsylvania.

Sally A. Santen, MD, PhD

Senior associate dean of evaluation, assessment and scholarship and professor of emergency medicine, Virginia Commonwealth University School of Medicine, Richmond, Virginia.

Back to Top | Article Outline

Reference

1. Pock A, Daniel M, Santen S, Swan-Sein A, Fleming A, Harnik V. Challenges associated with moving the United States Medical Licensing Examination (USMLE) Step 1 to after the core clerkships and how to approach them [published online ahead of print February 12, 2019]. Acad Med. doi:10.1097/ACM.0000000000002651
© 2019 by the Association of American Medical Colleges