To the Editor: Davis and colleagues1 concluded that their analyses of Medical College Admission Test (MCAT) performance across different racial/ethnic groups “do not … point to bias in the … predictive value of the MCAT exam.” To the contrary, I contend that the data show that the MCAT exam overpredicts the future performance of underrepresented minority (URM) students, as measured by Step 1 passage and timely graduation from medical school. Previous data similarly indicated that the MCAT exam overpredicts the preclinical grades of URM students.2
Rather than be satisfied that the MCAT exam is not biased against URMs, however, we should ask why there seems to be a predictive bias in favor of URMs. Most likely, the answer will not be as simple as construct-irrelevant test content but, instead, will suggest problems that extend beyond the exam itself.
The potential explanations are multiple. First, there may be differential validity in the measures of success used, such as bias within Step 1 of the United States Medical Licensing Examination or other assessments required for graduation.
Second, URMs may actually be underperforming in medical school, whether as a result of differences in socioeconomic status, institutional racism, stereotype threat, racial microaggressions, or other selectively negative influences and dispositions.
Third, prediction errors arise from selection procedures performed on groups with different mean predictor scores.3 These artifacts can magnify when selection occurs on variables not in the prediction model or when different cutoff scores are used for different groups.4,5 Specifically, schools may be using supplementary criteria that actually predict poorer likelihood of academic success to admit URMs preferentially (at rates higher than their MCAT scores would predict).
More research is needed to dissect the contributions of each of these factors. The results may hold important implications for policies in the admission and education of diverse groups of medical students.
Second-year medical student, Washington University School of Medicine, St. Louis, Missouri; [email protected]
1. Davis D, Dorsey JK, Franks RD, Sackett PR, Searcy CA, Zhao X. Do racial and ethnic group differences in performance on the MCAT exam reflect test bias? Acad Med. 2013;88:593–602
2. Koenig JA, Sireci SG, Wiley A. Evaluating the predictive validity of MCAT scores across diverse applicant groups. Acad Med. 1998;73:1095–1106
3. Linn RLWainer H, Messick S. Predictive bias as an artifact of selection procedures. Principals of Modern Psychological Measurement: A Festschrift for Frederic M. Lord. 1983 Hillsdale, NJ Lawrence Erlbaum Associates
4. Linn RL, Hastings CN. Group differentiated prediction. Appl Psych Meas. 1984;8:165–172
5. Borsboom D, Romeijn JW, Wicherts JM. Measurement invariance versus selection invariance: is fair selection possible? Psychol Methods. 2008;13:75–98