Letters to the Editor
I thank Griffith and Weissman for their thoughtful contributions to the continued conversation about the United States Medical Licensing Examination Step 1. I agree with Griffith that the success of high performers—by whatever scale selected—cannot be used to judge whether medical educators are “good teachers.” Weissman answers Griffith’s question of why program directors (PDs) need to differentiate applicants: to select students to interview. PDs use Step 1 scores as a convenient shortcut, believing they predict residency performance despite evidence to the contrary.
We need better tools specifically designed to select residents that have validity evidence and demonstrated value in predicting resident success. One such tool may be the standardized video interview (SVI), which focuses on noncognitive competence used currently by emergency medicine applicants.1 Applicants have 30 seconds to reflect on each of 6 questions on interpersonal and communication skills and professionalism. As Griffith notes, these behavioral traits are the very domains that correlate more commonly with the “problem resident.” Six trained raters review and score the responses. The SVI contributes unique independent information about candidates.1 It changes, at least to some extent, who is invited to interview, most importantly for applicants at either extreme of the Step 1 score scale.2 Results from the first year of implementation indicate that PDs are understandably cautious yet open-minded.3 Similar tools, such as online situational judgment tests (SJTs), are increasingly required in medical school applications and could be used in residency selection as well. A recent study suggests SJTs are acceptable to general surgery applicants.4 Surgical PDs who used a program-specific SJT instead of Step 1 markedly diversified their interviewees and matched residents.5 Though none of these tools is “perfect,” they are examples of evidence-based predictive selection tests designed to show what PDs really want.
1. Egan DJ, Husain A, Bond MC, et al. Standardized video interviews do not correlate to United States Medical Licensing Examination Step 1 and Step 2 scores. West J Emerg Med. 2019;20:87–91.
2. Husain A, Li I, Ardolic B, et al. The standardized video interview: How does it affect the likelihood to invite for a residency interview? AEM Educ Train. 2019;3:226–232.
3. Gallahue FE, Hiller KM, Bird SB, et al. The AAMC standardized video interview: Reactions and use by residency programs during the 2018 application cycle [published online ahead of print March 19, 2019]. Acad Med. doi:10.1097/ACM.0000000000002714.
4. Gardner AK, Cavanaugh KJ, Willis RE, Dunkin BJ. If you build it, will they come? Candidate completion of preinterview screening assessments [published online ahead of print May 31, 2019]. J Surg Educ. doi:10.1016/j.jsurg.2019.05.006.
5. Gardner A. assistant dean of evaluation and research, Baylor College of Medicine. Personal communication with K.M. Andolsek, July 18, 2019.