Educating Physicians: Research Reports
A comparison of admission committee members' and premedical advisors' perceptions of recommendation letters revealed similar ratings of the usefulness of letters and impressions of applicants.
Dr. Johnson is associate professor, Department of Behavioral Science, and Dr. Elam is assistant dean for admissions, both at the University of Kentucky College of Medicine, Lexington.
Correspondence and requests for reprints should be addressed to Dr. Johnson, University of Kentucky, Behavioral Science Department, College of Medicine Office Building, Lexington, KY 40536-0086.
A national survey1 of admission directors showed that committee members use recommendation letters to confirm their perceptions of applicants, which play a critical role in selection decisions. Discrepancies or biases in the evaluations of recommendation letters or the impressions of applicants derived from them represent potential problems for applicants and admission committees. In an extension of a previous study,2 we compared how members of an admission committee and premedical advisors viewed the usefulness of letters of recommendation.
Fourteen of 15 admission committee members from one medical school and 42 of 48 premedical advisors representing 39 institutions in the medical school's state evaluated two sets of seven letters of recommendations. One set contained three letters for men applicants and four letters for women applicants; the duplicate set reversed the gender of each applicant. Applicants', referees', and institutional identities were masked. Participants evaluated the usefulness of the information (e.g., memorable, informative, redundant) using a ten-item instrument with five-point scales anchored by “not at all” and “extremely,” and they rated their impressions of the applicants with a single seven-point global rating scale anchored by “weak” and “strong.”
Premedical advisors were predominantly men (65%) faculty members (80%) with PhDs (83%), and their gender and professional positions were comparable to those of the admission committee members,2 although no premedical advisor was an MD. Analyses of variances indicated that, although both premedical advisors and committee members discriminated across the seven letters (p < .05), the two groups did not differ in their perceptions of usefulness and their global impressions of the applicants. Comparison of means across the gender versions of the letters showed that, for one of the seven letters, committee members and advisors had a better impression of the woman applicant than they did of the man. The letter, originally written about a woman, was considered descriptive of her compassion and personal qualities, whereas the gender-reversed version was considered too emotional and verbose. Both advisors and committee members thought that letters were most helpful when they were factual, descriptive, and cited examples of specific behaviors.
Premedical advisors and committee members were similar in their perceptions of the usefulness of recommendation letters and in the impressions of the applicants they derived from the letters. Gender bias may exist in evaluating some letters, particularly those dealing with emotional aspects such as compassion. In our study, no discrepancy between admission committee members' and premedical advisors' perceptions threatened the usefulness and relevance of recommendation letters. However, the important role of recommendation letters in the decision-making process justifies continued consideration of their composition and appraisals. Medical schools should continue to give feedback on the usefulness of letters and advisors should seek feedback to ensure that pertinent issues and potential sources of bias are addressed.
1. Chapman CP, Lane HC. Perceptions about the use of letters of recommendation. The Advisor. 1997;17:31–6.
2. Johnson M, Elam C, Edwards J, et al. School admission committee members' evaluations of and impressions from recommendation letters. Acad Med. 1998;73(10 suppl):S41–S43.