Secondary Logo

How Medical School Applicant Race, Ethnicity, and Socioeconomic Status Relate to Multiple Mini-Interview–Based Admissions Outcomes: Findings From One Medical School

Jerant, Anthony MD; Fancher, Tonya MD, MPH; Fenton, Joshua J. MD, MPH; Fiscella, Kevin MD, MPH; Sousa, Francis MD; Franks, Peter MD; Henderson, Mark MD

doi: 10.1097/ACM.0000000000000766
Research Reports
Free
SDC

Purpose To examine associations of medical school applicant underrepresented minority (URM) status and socioeconomic status (SES) with Multiple Mini-Interview (MMI) invitation and performance and acceptance recommendation.

Method The authors conducted a correlational study of applicants submitting secondary applications to the University of California, Davis, School of Medicine, 2011–2013. URM applicants were black, Southeast Asian, Native American, Pacific Islander, and/or Hispanic. SES from eight application variables was modeled (0–1 score, higher score = lower SES). Regression analyses examined associations of URM status and SES with MMI invitation (yes/no), MMI score (mean of 10 station ratings, range 0–3), and admission committee recommendation (accept versus not), adjusting for age, sex, and academic performance.

Results Of 7,964 secondary-application applicants, 19.7% were URM and 15.1% self-designated disadvantaged; 1,420 (17.8%) participated in the MMI and were evaluated for acceptance. URM status was not associated with MMI invitation (OR 1.14; 95% CI 0.98 to 1.33), MMI score (0.00-point difference, CI −0.08 to 0.08), or acceptance recommendation (OR 1.08; CI 0.69 to 1.68). Lower SES applicants were more likely to be invited to an MMI (OR 5.95; CI 4.76 to 7.44) and recommended for acceptance (OR 3.28; CI 1.79 to 6.00), but had lower MMI scores (−0.12 points, CI −0.23 to −0.01).

Conclusions MMI-based admissions did not disfavor URM applicants. Lower SES applicants had lower MMI scores but were more likely to be invited to an MMI and recommended for acceptance. Multischool collaborations should examine how MMI-based admissions affect URM and lower SES applicants.

A. Jerant is professor, Department of Family and Community Medicine, Center for Healthcare Policy and Research, University of California, Davis, School of Medicine, Sacramento, California.

T. Fancher is associate professor, Division of General Internal Medicine, Department of Internal Medicine, University of California, Davis, School of Medicine, Sacramento, California.

J.J. Fenton is associate professor, Department of Family and Community Medicine, Center for Healthcare Policy and Research, University of California, Davis, School of Medicine, Sacramento, California.

K. Fiscella is professor, Department of Family Medicine, University of Rochester School of Medicine and Dentistry, Rochester, New York.

F. Sousa is assistant dean, Admissions and Student Development, and volunteer clinical professor, Department of Internal Medicine, University of California, Davis, School of Medicine, Sacramento, California.

P. Franks is professor, Department of Family and Community Medicine, Center for Healthcare Policy and Research, University of California, Davis, School of Medicine, Sacramento, California.

M. Henderson is associate dean, Admissions and Outreach, and professor, Division of General Medicine, Department of Internal Medicine, University of California, Davis, School of Medicine, Sacramento, California.

Funding/Support: None reported.

Other disclosures: None reported.

Ethical approval: On April 18, 2014, the University of California Davis (UCD) institutional review board (IRB) reviewed the study protocol (UCD IRB reference number 599704-1) and concluded that it is not human subjects research, based on the use of completely anonymous data that were collected for a purpose other than the research, and is therefore exempted from IRB approval and oversight.

Correspondence should be addressed to Anthony Jerant, Department of Family and Community Medicine, University of California, Davis, School of Medicine, 4860 Y Street, Suite 2300, Sacramento, CA 95817; telephone: (916) 734-7081; e-mail: afjerant@ucdavis.edu.

The Multiple Mini-Interview (MMI) is replacing traditional interviews at medical schools.1–5 In the MMI, trained raters evaluate applicants in a series of brief, timed, structured stations, and station ratings are pooled to yield a summary score. Stations are designed to assess skills that are difficult to ascertain from medical school applications, such as interpersonal communication, teamwork, ability to handle stress, problem solving, and integrity/ethics. The MMI is well accepted by applicants, reasonably reliable, and predictive of medical school and subsequent performance.1–3

Little studied is how underrepresented racial/ethnic minority (URM) and lower socioeconomic status (SES) applicants may be affected by adoption of the MMI. This is a key issue given that U.S. medical schools admit disproportionately few URM and lower SES individuals.6–8 URM admissions declined at many schools in the wake of legislation restricting consideration of race and ethnicity in acceptance decisions,9,10 a trend that may accelerate following a Supreme Court decision upholding the constitutionality of such restrictions.11 Concurrently, medical education and health policy groups call for a more diverse physician workforce.12–16 In such a climate, it is necessary to evaluate how evolving admissions trends like MMI adoption may influence the diversity of medical school classes.

A long-recognized problem with traditional nonstructured interviews is vulnerability to interviewer biases triggered by various applicant characteristics.17–22 Implicit (i.e., unconscious) biases disfavoring racial/ethnic minority and lower SES persons are common in U.S. society,23 including among physicians.24 The effects of bias during interviews can be reduced by increasing structure (removing ambiguity and, therefore, the tendency to rely on stereotype-driven judgments) and pooling evaluations from multiple raters (potentially diluting or offsetting individual biases).20,25–27 In being structured and incorporating multiple raters’ perspectives, the MMI may thus be less susceptible to implicit bias effects than traditional medical school interviews.

Only three studies to our knowledge have explored the associations of medical school applicants’ racial/ethnic minority status or SES with MMI performance. In one study, involving six Canadian schools, aboriginal status was negatively correlated with MMI scores, whereas family income level was not significantly associated with MMI performance.28 The analyses included few aboriginal participants (< 3%) and did not explore other race/ethnicity categories. Further, a robust indicator of SES would consider factors beyond income, such as parental education.29 A single-school U.S. study using a limited dichotomous (yes/no) single-item self-report indicator of disadvantaged status found no evidence of an association between disadvantaged status and MMI performance.30 Similarly, a single-school United Kingdom study, using a geographic area-based measure of deprivation, found no association between applicants’ deprivation score and MMI performance.31 However, ecological measures such as geographic area deprivation scores have significant limitations.32

To our knowledge, no studies have examined whether applicants’ race/ethnicity influences acceptance following MMI participation, or whether race/ethnicity or SES influences the likelihood of being invited to an MMI. Such outcomes are likely to be more strongly influenced by parochial concerns (e.g., institutional mission focus) than the MMI process,12,33 which is relatively similar across schools.34,35 Nonetheless, it is important to consider MMI invitation and acceptance decisions to provide context for evaluating the MMI-based admissions process.

We examined the associations of applicants’ URM status and SES with MMI invitation, MMI performance, and post-MMI acceptance recommendation among applicants to the University of California, Davis (UCD), School of Medicine (SOM) in Sacramento, California, over three admission cycles (2011–2013), adjusting for other applicant demographic characteristics and postsecondary academic performance.

Back to Top | Article Outline

Method

We employed data collected as part of the routine admissions processes during the 2011, 2012, and 2013 application cycles. The admissions office provided relevant application data in an electronic spreadsheet with personal identifiers removed. The study was conducted from April 18, 2014, through the end of August 2014. The UCD institutional review board reviewed the protocol and determined it was exempt.

Back to Top | Article Outline

Application, screening, and MMI invitation and scheduling

Applicants initially applied to the UCD SOM via the American Medical College Application Service (AMCAS). Following initial screening based on cumulative grade point average (GPA) and Medical College Admission Test (MCAT) scores, admissions committee members reviewed all applications and invited a subset of applicants to submit a secondary application. Faculty evaluated secondary applications for invitation to an MMI based on cumulative GPA and MCAT scores, personal statements, extracurricular activities, recommendation letters, and other characteristics that could contribute to fulfilling the educational and service missions of the school. Invited applicants self-scheduled their MMI sessions via an online portal.

Back to Top | Article Outline

MMI process and scoring

The MMI consisted of 10 individual 10-minute stations. At each station, applicants had 2 minutes to read a brief set of instructions, and 8 minutes to address the assigned tasks on entering the room. Nine stations assessed skills in the following domains: integrity/ethics, professionalism, interpersonal communication, diversity/cultural awareness, teamwork, ability to handle stress, and problem solving. An additional station asked applicants to explain their choice to pursue a career in medicine. Most stations were adapted from content developed at McMaster University and marketed by ProFitHR.34

A single trained rater, blinded to participants’ AMCAS application information, attended each station. In some stations, raters interacted directly with applicants. At others, raters observed applicant interactions with actors or other applicants. There were 216 different raters during the study period; the mean number of MMI stations that each evaluated was 104 (standard deviation [SD] 61.9; range 8–276). Women made up 61% of raters. Rater professional backgrounds were as follows: physicians, 31%; medical students, 15%; other clinicians (e.g., nurses), 11%; basic science faculty, 6%; patients, 2%; and various nonclinician leaders (e.g., deans), professionals (e.g., lawyers), and high-level administrative staff (e.g., curriculum manager), 35%. The range of rater backgrounds reflected the conviction that diverse perspectives are helpful in selecting future physicians who will be able to work effectively with people from all walks of life. Mandatory rater training included a one-hour course reviewing the admissions process, rater roles and duties, and the need to avoid pursuing protected class issues (e.g., race/ethnicity, gender).36

At each station, raters scored overall applicant performance using an anchored four-point scale: 0, < 25th percentile performance (relative to other applicants); 1, 25th–50th percentile; 2, 51st–75th percentile; or 3, > 75th percentile. Raters were instructed to consider both the applicant’s communication abilities and the content (e.g., comprehensiveness) of their statements in assigning ratings. The total MMI score was the mean of each applicant’s individual station scores. Scale internal consistency (Cronbach alpha = 0.67) was comparable to that observed in other MMI studies.2,18,37–41

Back to Top | Article Outline

Acceptance recommendation

The admissions committee met weekly during the admission cycle to review each participant’s MMI performance, AMCAS application, and secondary application. Subsequently, the committee made one of the following recommendations: reject, low waitlist, high waitlist, or offer acceptance. For the current analyses, we dichotomized the recommendation (offer acceptance versus not).

Back to Top | Article Outline

URM status

We determined URM status (URM [black, Southeast Asian, Native American, or Pacific Islander race and/or Hispanic ethnicity] versus not [all other responses]) from self-reported race/ethnicity information in the AMCAS application. These groups remain underrepresented in the medical profession relative to the general population.42

Back to Top | Article Outline

Socioeconomic disadvantage

We developed a composite measure of SES using self-reported information in the AMCAS application, screening candidate indicators for inclusion using logistic regression analyses and based on their contribution to predicting applicants’ self-designated disadvantaged status. The following predictors (yes/no items except where indicated) were significant and maximized the area under the receiver operating characteristic curve (0.95): fee assistance received for medical school application (yes/no); childhood spent in an underserved area; family recipients of family assistance program; income level category of applicant’s family (< $25,000; $25,000 to < $50,000; $50,000 to < $75,000; or > $75,000); applicant contributed to family income; any financial-need-based scholarship(s) in paying for postsecondary education; percentage of postsecondary education costs contributed by the family; and parents’ highest level of educational attainment (< high school, high school graduate, some college, or college graduate). The model yielded a predicted probability of being self-designated disadvantaged, ranging continuously from 0 to 1.0 (higher predicted probability = lower SES). We employed the score, which correlated 0.91 with a factor-analytic-derived score, as the study measure of applicant SES. The SES score was preferable to alternatives29 because the continuous scale acknowledges that socioeconomic disadvantage is not a binary characteristic and reduces misclassification (false positives and negatives).

Back to Top | Article Outline

Other characteristics

The admissions office also provided information from AMCAS regarding applicant age, sex, cumulative postsecondary GPA, and total MCAT score.

Back to Top | Article Outline

Data analysis

We analyzed the data using Stata version 13.1 (Stata Corporation Inc., College Station, Texas). We modeled MMI invitation (yes/no) and medical school acceptance recommendation (accept versus not) using logistic regression (a separate regression for each dependent variable). We modeled total MMI score using linear regression. All models included the following characteristics: age category (< 22 [reference], 22, 23, or > 24 years); female gender (yes/no); URM status (versus not); cumulative GPA category (< 3.4, 3.4–3.6, > 3.6–3.8, or > 3.8 [reference]); total MCAT score category (19–26, 27–30, 31–32, 33–34, or > 34 [reference]); SES (0–1.0 continuous score), and application year (2011, 2012, or 2013). The acceptance recommendation model additionally included the total MMI score (0–3).

Back to Top | Article Outline

Results

During the three study application cycles, 15,844 people applied, and 8,933 (56.4%) were invited to submit secondary applications. Of the invitees, 7,964 (89.2%) submitted secondary applications, and 1,575 (19.8%) were invited to an MMI. Of the MMI invitees, 1,420 (90.2%) attended an MMI.

Back to Top | Article Outline

Applicant characteristics

Table 1 summarizes the characteristics of screened applicants. Compared with those not invited to an MMI, those who participated in an MMI were older and more likely to be female, from an URM group, disadvantaged (based on both self-designation and the continuous SES score), and had higher GPAs and MCAT scores.

Table 1

Table 1

Back to Top | Article Outline

MMI invitation

Although adjusted URM status was not associated with MMI invitation, lower SES was associated with receiving an MMI invitation, as were older age, female gender, and higher GPA and MCAT score (Table 2).

Table 2

Table 2

Back to Top | Article Outline

MMI score

Mean MMI score was 1.30 (SD 0.62). In adjusted analyses, URM status was not associated with MMI score, but lower SES was associated with lower MMI scores (Table 3). Older age, female gender, and lower GPA were also associated with MMI score (Table 3).

Table 3

Table 3

Back to Top | Article Outline

Acceptance recommendation

Of the 1,420 MMI participants, 334 (23.5%) were recommended for acceptance. Lower SES was associated with being recommended for acceptance, whereas URM status was not (Table 4). Of other factors examined, mean MMI score was the most strongly associated with being recommended for acceptance. Older age, female gender, and higher GPA and MCAT scores were also associated with being recommended for acceptance (Table 4).

Table 4

Table 4

Back to Top | Article Outline

Discussion

In a diverse sample of applicants over three admission cycles at UCD, superior MMI performance was strongly linked with being recommended for acceptance. Further, URM applicants were no less likely than non-URM applicants to receive an MMI invitation, performed similarly on the MMI, and were just as likely to be recommended for acceptance. In the only prior study to exploring this issue, MMI performance at six Canadian medical schools was worse for the < 3% of applicants with self-reported aboriginal status, whereas income was not significantly associated with MMI scores.28 Our findings provide some reassurance that adoption of the MMI-based admissions process at U.S. medical schools need not adversely affect admission prospects for URM applicants.

The similar MMI scores for URM and non-URM participants support the notion that structured interview processes that incorporate the perspectives of multiple evaluators like the MMI may be less vulnerable to the effects of individual evaluator implicit biases.20,25–27 Although we did not measure rater implicit biases regarding racial/ethnic minorities, such biases have been documented to be pervasive in U.S. society, including among physicians and other professionals,23,24 and can affect the outcomes of employment interviews in various fields including medicine.17,19–22 Thus, it is likely that implicit biases were present among our raters; however, they did not exert a significant net influence, given that mean MMI scores did not differ between URM and non-URM applicants. Because lack of URMs in medicine is a widely acknowledged problem,6,7,13,33,42–44 it is possible that biases against URM applicants were offset by ratings biased in favor of URM applicants, made by raters seeking to address limited racial/ethnic diversity in the physician workforce.

Our findings regarding MMI performance may have the broadest applicability, given the relatively high standardization of the MMI across institutions.34,35 By contrast, MMI invitation and acceptance decisions are shaped more by parochial concerns such as institutional mission and local workforce needs.12,33 In this context, our finding that lower SES applicants had worse adjusted MMI performance may be cause for concern. Although three prior studies reported no association of applicant SES with MMI performance, all relied on less robust measures of SES. Nonetheless, the decrement in MMI performance with decreasing SES in our study was small: The MMI score (scale of 0–3 points) declined by a mean of 0.12 points across the 0–1 range of the SES score. Further, the lower MMI scores among lower SES applicants were more than offset by their greater likelihood of being invited to an MMI and recommended for acceptance. These findings may reflect the ongoing shift from a purely metric-based applicant review process toward the more holistic process advocated by the Association of American Medical Colleges.12,15

Although the reasons for the lower MMI performance among lower SES applicants are unclear, poorer postsecondary academic preparation and performance are unlikely explanations because we adjusted for GPA and MCAT score. Lower SES applicants may have fewer life experiences bolstering skills assessed by the MMI. Similar reasoning has been suggested to explain the lower MCAT scores among such applicants.45 Although less affluent applicants are more likely to report paid employment during postsecondary education, their financial circumstances may require taking jobs that do not require MMI-type preemployment screening. Lack of prior experience with MMI-type screening may be a disadvantage in the medical school MMI because prior experience with a particular interview format is associated with better future performance with that format.46 Lower-level jobs also may not facilitate the higher-level communication, critical thinking, and problem-solving skills the MMI assesses, and the time required for such jobs may limit participation in pursuits that build such skills (e.g., scholarly presentations, volunteer clinic work).

Rater implicit bias could also help to explain the lower MMI scores among lower SES applicants. Raters were not provided any information about participants, and only one MMI station afforded applicants the opportunity to describe their backgrounds. Nonetheless, subtle information apparent to raters could have led to implicitly biased ratings, possibly triggered by the generally relatively large social distance between lower SES applicants and the typical rater.47,48 Most of our MMI raters were well educated and relatively affluent. Prior work indicates that applicant factors such as use of language unfamiliar to the typical rater could trigger a biased low rating.20,21 Applicants’ verbal skills have been shown to determine immediate interviewer impressions and, in turn, final appraisals.49 The issue of SES-based physician workforce disparities has received less attention than race/ethnicity-based disparities.6 Thus, it is less likely that raters consciously biased their evaluations in favor of lower SES applicants to address SES-based physician workforce disparities. If our findings are replicated, it would suggest the need to consider rater training to minimize the influence of SES-based biases.

We found that older applicants and women performed better than their younger and male counterparts both in the MMI (consistent with prior studies28,37) and throughout the admissions process. Older applicants are more likely to have had life experiences requiring effective communication. Women, more than men, have been noted to communicate in ways that quickly build rapport in novel social situations such as the MMI. Previously underrepresented in medicine, women now constitute well over half of all U.S. medical students.50 Given the increasing adoption of the MMI and strong influence of MMI performance on medical school acceptance, our findings suggest the potential for a disparity to develop disfavoring male applicants, and warrant monitoring.

Back to Top | Article Outline

Limitations

Our study was correlational, limiting causal inferences. Our data were derived from applicants at a single school. How the findings generalize to other schools is unclear. We lacked sociodemographic information regarding MMI raters and admissions committee personnel, precluding examination of how such characteristics may have influenced the study outcomes. We employed a novel composite measure of SES, albeit one with theoretical advantages over other indicators.29 Because schools vary widely in institutional mission,33 applicant pools, admissions personnel, and other attributes, multi-institution studies are needed to better gauge the impact of MMI-based admissions processes on URM and socioeconomically disadvantaged applicants. Studies ideally should account for the characteristics of admissions personnel and applicants, and examine the relative utility of different SES indicators. It is also unknown whether URM or lower SES applicants would have fared differently in one-on-one interview-based admissions processes. Ideally, randomized trials would compare traditional interview processes with MMI-based processes, examining their impact on the prospects of URM and lower SES applicants.

Back to Top | Article Outline

Conclusion

In conclusion, in analyses of data from one California medical school, an MMI-based admissions process did not disfavor racial/ethnic minority groups underrepresented in the physician workforce. Applicants from lower SES backgrounds, also underrepresented in medicine, had lower MMI scores but were more likely to receive an MMI invitation and be recommended for acceptance. Multischool collaborations are needed to further evaluate the impact of MMI-based medical school admissions processes on URM and lower SES applicants.

Back to Top | Article Outline

References

1. Eva KW, Reiter HI, Rosenfeld J, Norman GR. The ability of the multiple mini-interview to predict preclerkship performance in medical school. Acad Med. 2004;79(10 suppl):S40–S42
2. Eva KW, Reiter HI, Trinh K, Wasi P, Rosenfeld J, Norman GR. Predictive validity of the multiple mini-interview for selecting medical trainees. Med Educ. 2009;43:767–775
3. Reiter HI, Eva KW, Rosenfeld J, Norman GR. Multiple mini-interviews predict clerkship and licensing examination performance. Med Educ. 2007;41:378–384
4. Griffin B, Wilson I. Associations between the big five personality factors and multiple mini-interviews. Adv Health Sci Educ Theory Pract. 2012;17:377–388
5. Harris G. New for aspiring doctors, the people skills test. NY Times. 2011:A1 http://www.nytimes.com/2011/07/11/health/policy/11docs.html?pagewanted=al. Accessed April 4, 2015
6. Magnus SA, Mick SS. Medical schools, affirmative action, and the neglected role of social class. Am J Public Health. 2000;90:1197–1201
7. Saha S, Shipman SA. Race-neutral versus race-conscious workforce policy to improve access to care. Health Aff (Millwood). 2008;27:234–245
8. Association of American Medical Colleges. Minorities in Medical Education: Facts and Figures 2005. 2005 Washington, DC Association of American Medical Colleges
9. Steinecke A, Terrell C. After affirmative action: Diversity at California medical schools. AAMC Analysis in Brief. 2008;8(6) https://www.aamc.org/download/102358/data/aibvol8no6.pdf. Accessed April 4, 2015
10. Association of American Medical Colleges. Assessing Medical School Admissions Policies: Implications of the U.S. Supreme Court’s Affirmative-Action Decisions. 2003 Washington, DC Association of American Medical Colleges http://www.aacp.org/resources/studentaffairspersonnel/admissionsguidelines/Documents/AAMCassessmedschooladmissions.pdf. Accessed April 4, 2015
11. Liptak A. Supreme Court upholds Michigan’s affirmative action ban. NY Times. April 22, 2014. http://www.nytimes.com/2014/04/23/us/supreme-court-michigan-affirmative-action-ban.html?hp&_r=1. Accessed April 4, 2015
12. Association of American Medical Colleges. . Holistic review. https://www.aamc.org/initiatives/holisticreview/. Accessed April 4, 2015
13. Saha S, Guiton G, Wimmers PF, Wilkerson L. Student body racial and ethnic composition and diversity-related outcomes in US medical schools. JAMA. 2008;300:1135–1145
14. Walker KO, Moreno G, Grumbach K. The association among specialty, race, ethnicity, and practice location among California physicians in diverse specialties. J Natl Med Assoc. 2012;104:46–52
15. Addams AN, Bletzinger RB, Sondheimer HM, White SE, Johnson LM Roadmap to Diversity: Integrating Holistic Review Practices Into Medical School Admissions Processes. 2010 Washington, DC Association of American Medical Colleges https://members.aamc.org/eweb/upload/Roadmap%20to%20Diversity%20Integrating%20Holistic%20Review.pdf. Accessed April 4, 2015
16. Committee on Institutional and Policy-Level Strategies for Increasing the Diversity of the U.S. Health Care Workforce, Board on Health Sciences Policy. In the Nation’s Compelling Interest: Ensuring Diversity in the Health Care Workforce. 2004 Washington, DC National Academies Press
17. Salvatori P. Reliability and validity of admissions tools used to select students for the health professions. Adv Health Sci Educ Theory Pract. 2001;6:159–175
18. Eva KW, Rosenfeld J, Reiter HI, Norman GR. An admissions OSCE: The multiple mini-interview. Med Educ. 2004;38:314–326
19. Fiske ST, Markus HR Facing Social Class: How Societal Rank Influences Interaction. 2012 New York, NY Russell Sage Foundation
20. Macan T. The employment interview: A review of current studies and directions for future research. Hum Resour Manage R. 2009;19:203–218
21. Purkiss SLS, Perrewé PL, Gillespie TL, Mayes BT, Ferris GR. Implicit sources of bias in employment interview judgments and decisions. Organ Behav Hum Dec. 2006;101:152–167
22. Cuddy AJ, Fiske ST, Kwan VS, et al. Stereotype content model across cultures: Towards universal similarities and some differences. Br J Soc Psychol. 2009;48(pt 1):1–33
23. Greenwald AG, Poehlman TA, Uhlmann EL, Banaji MR. Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. J Pers Soc Psychol. 2009;97:17–41
24. Chapman EN, Kaatz A, Carnes M. Physicians and implicit bias: How doctors may unwittingly perpetuate health care disparities. J Gen Intern Med. 2013;28:1504–1510
25. Kutcher EJ, Bragger JD. Selection interviews of overweight job applicants: Can structure reduce the bias? J Appl Soc Psychol. 2004;34:1993–2022
26. Brecher E, Bragger J, Kutcher E. The structured interview: Reducing biases toward job applicants with physical disabilities. Employ Responsib Rights J. 2006;18:155–170
27. Levashina J, Hartwell CJ, Morgeson FP, Campion MA. The structured employment interview: Narrative and quantitative review of the research literature. Pers Psychol. 2014;67:241–293
28. Reiter HI, Lockyer J, Ziola B, Courneya CA, Eva KCanadian Multiple Mini-Interview Research Alliance (CaMMIRA). . Should efforts in favor of medical student diversity be focused during admissions or farther upstream? Acad Med. 2012;87:443–448
29. Grbic D, Jones DJ, Case ST Effective Practices for Using the AAMC Socioeconomic Status Indicators in Medical School Admissions. 2013 Washington, DC Association of American Medical Colleges
30. Uijtdehaage S, Doyle L, Parker N. Enhancing the reliability of the multiple mini-interview for selecting prospective health care leaders. Acad Med. 2011;86:1032–1039
31. Taylor CA, Green KE, Spruce A. Evaluation of the effect of socio-economic status on performance in a Multiple Mini Interview for admission to medical school [published online July 2, 2014]. Med Teach. 2015;37:59–63
32. Morgenstern H. Uses of ecologic analysis in epidemiologic research. Am J Public Health. 1982;72:1336–1344
33. Mullan F, Chen C, Petterson S, Kolsky G, Spagnola M. The social mission of medical education: Ranking the schools. Ann Intern Med. 2010;152:804–811
34. Advanced Psychometrics for Transitions Inc. . Welcome to ProFitHR. http://www.profithr.com/. Accessed April 4, 2015
35. Pau A, Jeevaratnam K, Chen YS, Fall AA, Khoo C, Nadarajah VD. The Multiple Mini-Interview (MMI) for student selection in health professions training—a systematic review. Med Teach. 2013;35:1027–1041
36. U.S. Equal Employment Opportunity Commission. . Title VII of the Civil Rights Act of 1964. http://www.eeoc.gov/laws/statutes/titlevii.cfm. Accessed April 4, 2015
37. Jerant A, Griffin E, Rainwater J, et al. Does applicant personality influence multiple mini-interview performance and medical school acceptance offers? Acad Med. 2012;87:1250–1259
38. Dowell J, Lynch B, Till H, Kumwenda B, Husbands A. The multiple mini-interview in the U.K. context: 3 years of experience at Dundee. Med Teach. 2012;34:297–304
39. Harris S, Owen C. Discerning quality: Using the multiple mini-interview in student selection for the Australian National University Medical School. Med Educ. 2007;41:234–241
40. Till H, Myford C, Dowell J. Improving student selection using multiple mini-interviews with multifaceted Rasch modeling. Acad Med. 2013;88:216–223
41. Sebok SS, Luu K, Klinger DA. Psychometric properties of the multiple mini-interview used for medical admissions: Findings from generalizability and Rasch analyses. Adv Health Sci Educ Theory Pract. 2014;19:71–84
42. Association of American Medical Colleges. . Underrepresented in medicine definition. https://www.aamc.org/initiatives/urm/. Accessed April 4, 2015
43. Ko M, Heslin KC, Edelstein RA, Grumbach K. The role of medical education in reducing health care disparities: The first ten years of the UCLA/Drew medical education program. J Gen Intern Med. 2007;22:625–631
44. Whitla DK, Orfield G, Silen W, Teperow C, Howard C, Reede J. Educational benefits of diversity in medical school: A survey of students. Acad Med. 2003;78:460–466
45. Davis D, Dorsey JK, Franks RD, Sackett PR, Searcy CA, Zhao X. Do racial and ethnic group differences in performance on the MCAT exam reflect test bias? Acad Med. 2013;88:593–602
46. Huffcutt AI, Van Iddekinge CH, Roth PH. Understanding applicant behavior in employment interviews: A theoretical model of interviewee performance. Hum Resour Manage R. 2011;21:353–367
47. Dohrenwend BS, Colombotos J, Dohrenwend BP. Social distance and interviewer effects. Milbank Q. 1969;41:213–226
48. Howard JL, Ferris GR. The employment interview context: Social and situational influences on interviewer decisions. J Appl Soc Psychol. 1996;26:112–136
49. Barrick MR, Dustin SL, Giluk TL, Stewart GL, Shaffer JA, Swider BW. Candidate characteristics driving initial impressions during rapport building: Implications for employment interview validity. J Occup Organ Psychol. 2012;85:330–352
50. Association of American Medical Colleges. . U.S. medical school applicants and students 1982–1983 to 2011–2012. https://www.aamc.org/download/153708/data/charts1982to2012.pdf. Accessed April 4, 2015
© 2015 by the Association of American Medical Colleges