Share this article on:

What Aspects of Letters of Recommendation Predict Performance in Medical School? Findings From One Institution

DeZee, Kent J. MD, MPH; Magee, Charles D. MD, MPH; Rickards, Gretchen MD, MPH; Artino, Anthony R. Jr PhD; Gilliland, William R. MD; Dong, Ting PhD; McBee, Elexis DO, MPH; Paolino, Nathalie DO, MPH; Cruess, David F. PhD; Durning, Steven J. MD, PhD

doi: 10.1097/ACM.0000000000000425
Research Reports

Purpose: To study medical students’ letters of recommendation (LORs) from their applications to medical school to determine whether these predicted medical school performance, because many researchers have questioned LORs’ predictive validity.

Method: A retrospective cohort study of three consecutive graduating classes (2007–2009) at the Uniformed Services University of the Health Sciences was performed. In each class, the 27 students who had been elected into the Alpha Omega Alpha (AOA) Honor Medical Society were defined as top graduates, and the 27 students with the lowest cumulative grade point average (GPA) were designated as “bottom of the class” graduates. For each student, the first three LORs (if available) in the application packet were independently coded by two blinded investigators using a comprehensive list of 76 characteristics. Each characteristic was compared with graduation status (top or bottom of the class), and those with statistical significance related to graduation status were inserted into a logistic regression model, with undergraduate GPA and Medical College Admission Test score included as control variables.

Results: Four hundred thirty-seven LORs were included. Of 76 LOR characteristics, 7 were associated with graduation status (P ≤ .05), and 3 remained significant in the regression model. Being rated as “the best” among peers and having an employer or supervisor as the LOR author were associated with induction into AOA, whereas having nonpositive comments was associated with bottom of the class students.

Conclusions: LORs have limited value to admission committees, as very few LOR characteristics predict how students perform during medical school.

Dr. DeZee is associate professor of medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Magee is assistant professor of medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Rickards is assistant professor of medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Artino is associate professor of preventive medicine and biometrics, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Gilliland is professor of medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Dong is assistant professor of medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. McBee is assistant professor of medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Paolino is assistant professor of medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Cruess is professor of preventive medicine and biometrics, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Dr. Durning is professor of medicine and pathology, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Funding/Support: This study was supported by an intramural grant from the Dean’s Educational Endowment Fund, F. Edward Hébert School of Medicine, Uniformed Services University of the Health Sciences, Bethesda, Maryland.

Other disclosures: None reported.

Ethical approval: The authors obtained IRB approval from the Uniformed Services University.

Disclaimer: The views expressed in this paper are those of the authors and do necessarily represent the views of the Uniformed Services University, the Department of Defense, or other federal agencies.

Previous presentations: This research was presented as a poster at the annual meeting of the Society of General Internal Medicine, Denver, Colorado, April 25, 2013.

Correspondence should be addressed to Dr. DeZee, 4301 Jones Bridge Rd.–EDP, Bethesda, MD 20814; telephone: (301) 319-2369; fax: (301) 319-8240; e-mail: kent.dezee@usuhs.edu.

© 2014 by the Association of American Medical Colleges