Letters to the Editor
To the Editor:
In their Invited Commentary, Drs. Katsufrakis and Chaudhry1 asked, “Does Step 1 performance predict residency success?” and answered, “To our knowledge, no study has been done to answer this question.” In fact, many have. In a systematic review, Kenny and colleagues2 identified dozens of studies on the relationship between the United States Medical Licensing Examination (USMLE) Step 1 and measures of resident performance. Unsurprisingly, Step 1 scores were strongly associated with performance on in-training and specialty board certification exams. Importantly, Step 1 scores were also positively associated with subjective faculty evaluations of resident clinical performance, with effect sizes that were higher than those of subjective selection factors like interviews and reference letters.
Skeptics of the predictive utility of Step 1 three-digit scores will point out that success in residency is multifaceted, not limited to medical knowledge tests and faculty evaluations. As Kenny and colleagues2 noted, we need to evaluate whether Step 1 can also predict practice-based and long-term outcomes of residents. For example, one study found Step 1 performance to correlate inversely with major discordance rates of radiology residents’ on-call preliminary interpretations compared with attending final reads, a quality metric with direct patient care impact.3 In another study, Step 1 was predictive of core competency evaluations of neuroradiology fellows, who are usually at least seven years removed from that exam.4 Though far from definitive, these lines of research should grab the attention of all stakeholders affected by the secondary use of USMLE Step 1 scores for resident selection.
Francis Deng, MD
Radiology resident, Massachusetts General Hospital, Boston, Massachusetts; email@example.com; ORCID: https://orcid.org/0000-0003-3117-5076.
Austin Wesevich, MD, MPH
Internal medicine and pediatrics resident, Duke University Medical Center, Durham, North Carolina; ORCID: https://orcid.org/0000-0001-5202-1231.
1. Katsufrakis PJ, Chaudhry HJ. Improving residency selection requires close study and better understanding of stakeholder needs. Acad Med. 2019;94:305–308.
2. Kenny S, McInnes M, Singh V. Associations between residency selection strategies and doctor performance: A meta-analysis. Med Educ. 2013;47:790–800.
3. Agarwal V, Bump GM, Heller MT, et al. Do residency selection factors predict radiology resident performance? Acad Radiol. 2018;25:397–402.
4. Yousem IJ, Liu L, Aygun N, Yousem DM. United States Medical Licensing Examination Step 1 and 2 scores predict neuroradiology fellowship success. J Am Coll Radiol. 2016;13:438–444.e2.