In Reply to Norman et al:
Norman et al (two of whom helped to develop the Multiple Mini-Interview [MMI]) make several comments we wish to address.
First, they claim that our findings about the association between extroversion and performance on the MMI suggest that the MMI “selects for individuals who are … interested in others, and skilled in the interpersonal domain.” Yet, extroversion is not uniquely associated with interest in people or demonstrated skill at communication.1 High extroverts do, however, tend to receive more favorable subjective communication ratings in brief exposure forums that may disadvantage the less extroverted (e.g., clerkship rounds).2 Even so, whether high extroverts are superior communicators in clinical practice is unstudied.
Second, they note that grade point average (GPA) is associated with conscientiousness and imply that we might also be concerned that using GPA in admissions screening may preferentially select highly conscientious students. But we do not view this as a concern, since conscientiousness (unlike extroversion) is strongly predictive of superior medical school academic and clinical performance.2–5
Third, are they implying that the association of MMI score with extroversion that we observed is statistical only and lacks practical significance? If so, we disagree. In our study, applicants in the top extroversion quartile scored almost 6 points higher on the MMI (total possible score = 30) than did the bottom-quartile participants—a sizeable difference. Furthermore, for our prediction model for base medical-school-acceptance offers, which excluded personality, the receiver operating characteristic curve C statistic was 62.6%. After adding personality variables to the model, the C statistic increased substantively to 70.6%. Finally, after adding MMI score to the model, extroversion was no longer associated with acceptance offers, indicating that the MMI “captured” high extroversion effects on acceptance offers. Full model details are provided in our report.
Fourth, for unclear reasons, they cite a book recounting the abhorrant practice of using personal interviews to discriminate against Jewish medical school applicants in the early 1900s. The book clearly does not address the utility of traditional interviews when appropriately employed.
Finally, they claim that the personal interview “has been repeatedly demonstrated to have no relation to important educational outcomes.” However, one of the two reviews they cite in support (by Albanese et al) concludes that “interview ratings are predictive of subjective clinical assessments, and low interview assessments are predictive of failure or withdrawal from medical school.” The other review, by Salvatori, concludes that “controversy remains regarding the reliability, predictive validity and cost effectiveness of the selection interview. The evidence is clear, however, that the use of a structured format as well as trained and experienced interviewers serves to enhance its psychometric properties.”
Anthony Jerant, MD
Professor, Department of Family and Community Medicine, University of California Davis School of Medicine, Sacramento, California; firstname.lastname@example.org.
Joshua Fenton, MD, MPH
Associate professor, Department of Family and Community Medicine, University of California Davis School of Medicine, Sacramento, California.
Peter Franks, MD
Professor, Department of Family and Community Medicine, University of California Davis School of Medicine, Sacramento, California.
1. Costa PT, McCrae RR Revised NEO Personality Inventory and NEO Five Factor Inventory: Professional Manual. 1992 Odessa, Fla Psychological Assessment
2. Lievens F, Ones DS, Dilchert S. Personality scale validities increase throughout medical school. J Appl Psychol. 2009;94:1514–1535
3. Doherty EM, Nugent E. Personality factors and medical training: A review of the literature. Med Educ. 2011;45:132–140
4. Enns MW, Cox BJ, Sareen J, Freeman P. Adaptive and maladaptive perfectionism in medical students: A longitudinal investigation. Med Educ. 2001;35:1034–1042
5. Ferguson E, Sanders A, O’Hehir F, James D. Predictive validity of personal statements and the role of the five-factor model of personality in relation to medical training. J Occup Organ Psychol. 2000;73:321–344