To compare the performance characteristics of the electronic Standardized Letter of Evaluation (eSLOE), a widely used structured assessment of emergency medicine (EM) residency applicants, and the Association of American Medical Colleges (AAMC) Standardized Video Interview (SVI), a new tool designed by the AAMC to assess interpersonal and communication skills and professionalism knowledge.
The authors matched EM residency applicants with valid SVI total scores and completed eSLOEs in the 2018 Match application cycle. They examined correlations and group differences for both tools, United States Medical Licensing Examination (USMLE) Step exam scores, and honor society memberships.
The matched sample included 2,884 applicants. SVI score and eSLOE global assessment ratings demonstrated small positive correlations approaching r = 0.20. eSLOE ratings had higher correlations with measures of academic ability (USMLE scores, academic honor society membership) than did SVI scores. Group differences were minimal for the SVI, with scores slightly favoring women (d = −0.21) and US-MD applicants (d = 0.23–0.42). Group differences in eSLOE ratings were small, favoring women over men (approaching d = −0.20) and white applicants over black applicants (approaching d = 0.40).
Small positive correlations between SVI score and eSLOE global assessment ratings, alongside varying correlations with academic ability indicators, suggest these are complementary tools. Findings suggest the eSLOE is subject to similar sources and degrees of bias as other common assessments; these group differences were not observed with the SVI. Further examination of both tools is necessary to understand their ability to predict clinical performance.
L.R. Hopson is emergency medicine residency program director and associate professor, Department of Emergency Medicine, University of Michigan Medical School, Ann Arbor, Michigan; ORCID: https://orcid.org/0000-0002-1183-4751.
L. Regan is emergency medicine residency program director, vice chair for education, and associate professor, Department of Emergency Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland; ORCID: https://orcid.org/0000-0003-0390-4243.
M.C. Bond is emergency medicine residency program director and associate professor, Department of Emergency Medicine, University of Maryland School of Medicine, Baltimore, Maryland; ORCID: https://orcid.org/0000-0001-5527-6758.
J. Branzetti is emergency medicine residency program director, Department of Emergency Medicine, New York University School of Medicine, New York, New York; ORCID: http://orcid.org/0000-0002-2397-0566.
E.A. Samuels is assistant professor of emergency medicine, Warren Alpert Medical School of Brown University, Providence, Rhode Island; ORCID: https://orcid.org/0000-0003-2414-110X.
B. Naemi is manager, Admissions and Selection Research, Association of American Medical Colleges, Washington, DC.
D. Dunleavy is director, Admissions and Selection Research, Association of American Medical Colleges, Washington, DC.
M.A. Gisondi is associate professor and vice chair of education, Department of Emergency Medicine, Stanford University School of Medicine, Stanford, California; ORCID: https://orcid.org/0000-0002-6800-3932.
Supplemental digital content for this article is available at http://links.lww.com/ACADMED/A713.
Funding/Support: This project was supported by the Association of American Medical Colleges (AAMC) and CORD-EM (eSLOE database support) as a part of their routine operating budgets.
Other disclosures: B. Naemi and D. Dunleavy are employees of the AAMC.
Ethical approval: Institutional review board review was obtained through the American Institutes for Research (FWA00001666), which approved AAMC Standardized Video Interview research for this study on September 30, 2017, as a component of a larger SVI master research program.
Previous presentations: A subset of these data were reported at the following meetings: 2019 CORD-EM Academic Assembly, April 1, 2019, Seattle, Washington; and 22nd Annual Western Regional Meeting of Society of Academic Emergency Medicine, March 22, 2019, Napa, California.
Data: The eSLOE data were securely sent to the AAMC from CORD-EM on March 29, 2018. The SVI, demographic, and outcome data were obtained from the AAMC Data Warehouse on May 3, 2018. The manuscript was reviewed by CORD-EM, AAMC, and National Board of Medical Examiners leadership and was approved for publication by all parties.
Correspondence should be addressed to Bobby Naemi, Association of American Medical Colleges, 655 K St., N.W., Suite 100, Washington, DC 20001; email: email@example.com.