African Americans and other minorities remain underrepresented in medicine (URM), which is thought to exacerbate current racial health care disparities.1 Few studies have examined the possibility of unconscious or implicit racial bias in admissions as a contributor to the relative lack of diversity in medical school. The implicit association test (IAT) is widely used to determine biases outside of an individual’s conscious control.2 A result revealing a bias in favor of white people over black people on the black–white IAT is referred to as implicit white preference and indicates that the subject associates images of white people with positive words and images of black people with negative words. A report by Nosek and colleagues3 suggests that implicit white preference may be pervasive in modern society; between July 2000 and May 2006, approximately 70% of the more than two million visitors to a publicly available Web site providing the IAT demonstrated implicit white preference. Although this bias is unconscious, its presence has been associated with discrimination in the education,4 criminal justice,5 and health care systems.6 Indeed, a meta-analysis of 122 research studies found that IAT results more accurately predict discriminatory behavior than self-reported attitudes about race.7
A recent report showed that the majority of a large sample of physicians demonstrate implicit white preference,8 and several studies have linked physicians’ implicit white preference with clinical decisions that are detrimental to black patients.6 Little is known about unconscious bias in medical school admissions committees, which have strong physician representation but may also include nonphysician members. Implicit white preference in medical school admissions committees could disadvantage African American candidates and contribute to the paucity of URMs. As the gatekeepers to their institutions, medical school admissions committees wield a powerful influence over the health care of the nation and so have an ethical obligation to minimize their biases to the fullest extent possible. We sought to determine the presence and extent of unconscious racial bias on our admissions committee by having all members take the IAT. Later, we surveyed the members to determine their impressions of their individual results and the utility of the exercise. Finally, we sought to determine whether having our admissions committee take the IAT would have any impact on URM diversity in our next incoming class.
There were 140 members of the Ohio State University College of Medicine (OSUCOM) admissions committee in spring 2012. Of these, 43 (31%) were faculty (MDs and PhDs) and 97 (69%) were medical students. Sixty-seven (48%) members were women (50 students and 17 faculty), and 73 (52%) were men (47 students and 26 faculty). Twenty-one (15%) committee members were self-reported URMs. All admissions committee members participate in one-on-one interviews with and vote on candidates, but faculty members constitute the majority present during voting sessions.
In spring 2012 (after the 2011–2012 admissions cycle but before the 2012–2013 cycle), we contracted with Project Implicit to set up a password-protected, temporary Web site to collate the anonymous IAT results for the OSUCOM admissions committee members. The results were coded only to gender and student versus faculty status. The committee members were assigned deidentified codes that were used only to verify that all members completed the IAT. All 140 committee members completed the black–white IAT in late spring or early summer 2012. Individual results were only visible to the test taker and only at the time of the IAT. In August 2012, the aggregate results were presented to the committee by an implicit bias expert (A.G.G.), who discussed strategies to combat or reduce unconscious biases.
Implicit preference measure
The black–white IAT has previously been described in detail.2 Briefly, test takers quickly pair facial images and positive (e.g., joy, love, peace) or negative (e.g., agony, horrible, pain) words as they appear on a computer screen by pressing either a right or left computer key. Participants are first instructed to assign black faces and positive words to one key and white faces and negative words to the other key; then the task is reversed. The differences in response times indicate the relative strengths of the associations. For example, short response latencies for the association of a white face with positive words and a black face with negative words and longer response latencies for the association of black faces with positive words and white faces with negative words would indicate implicit white preference.
Explicit preference measure
For the explicit measure portion of the IAT, test takers were asked, “What best describes you?” with regard to these statements: “I strongly/moderately/slightly prefer white Americans to African Americans,” “I like white Americans and African Americans equally,” and “I strongly/moderately/slightly prefer African Americans to white Americans.” Similar methodology is routinely incorporated into IATs.
To measure explicit preference, we coded test takers’ answers to the explicit measure to a seven-point, Likert-type scale ranging from −3 to +3, with positive values indicating an explicit preference for white Americans over black Americans, negative values indicating an explicit preference for black Americans over white Americans, and zero indicating no relative preference. We calculated an explicit measure mean effect size for each group of test takers by dividing the mean (where a positive explicit measure mean indicates an explicit preference for white Americans over black Americans and a negative explicit measure mean indicates an explicit preference for black Americans over white Americans) on the seven-point Likert-type scale by its standard deviation.
To measure implicit preference, the IAT effect is calculated using a standard scoring algorithm applied to the response latencies, known as the D score, based on differences in the mean response latencies to the IAT’s two different combination task conditions. Our results are reported in standard deviation units—specifically, the D score divided by its standard deviation. This is known as Cohen’s d, a standardized effect size measure that we calculated for each group of test takers. Cohen’s d is interpreted as 0.20 = small effect, 0.50 = medium effect, and 0.80 = large effect, corresponding to 46%, 58%, and 69%, respectively, of respondents having statistically nontrivial bias.9
We compared means for the implicit and explicit measures for each group of test takers to zero or trivial, nonsignificant bias.
Admissions committee survey
In May 2013 (following the 2012–2013 admissions cycle), we distributed the annual, anonymous admissions committee survey, with additional questions specific to the IAT exercise, to all 140 committee members to record their impressions of the impact of the IAT on the admissions process. One hundred committee members completed the survey (71% response rate).
Explicit and implicit preferences
Self-reported explicit white preference was trivial and not significantly different from zero for all groups of test takers (mean effect size: faculty = 0; males = 0.080; students = 0.087; females = 0.042) (Table 1).
All groups displayed significant levels of implicit white preference (d: faculty = 0.820, P < .001; males = 0.697, P < .001; students = 0.379, P = .003; females = 0.321, P = .01) (Table 1). These findings are similar in magnitude to a sample of voluntary test takers in the United States and a group of medical doctors.3,8
Several observations about our findings and the composition of our admissions committee are noteworthy. First, whereas all groups demonstrated significant levels of unconscious bias in favor of whites, the faculty and males had the largest bias measures. Second, implicit white preference was lowest among females, consistent with previous reports.3 Regarding our medical students, the magnitude of their implicit white preference is less than half that of our faculty (d = 0.379 vs. 0.820). Finally, it has been shown that African Americans, both physicians and nonphysicians, tend to have no or minimal overall racial bias on IAT testing.3,8 Taken together, these observations argue that strong representation of medical students, women, and African Americans on admissions committees may help to reduce unconscious racial bias in the admissions process.
One hundred (71% response rate) committee members responded to the survey. Sixty-seven percent thought the IAT was valuable and might be helpful in reducing bias, 48% were conscious of their individual results when interviewing candidates in the next admissions cycle, and 21% reported that knowledge of their IAT results impacted their admissions decisions in the subsequent admissions cycle (responses of strongly agree or agree to multiple-choice questions 3–5, respectively, in Appendix 1). The survey ended with an open-ended question: “Please provide any comments about the implicit bias exercise and IAT testing for the admissions committee, including suggestions on how to make it more effective.” Full survey results, including representative responses to the open-ended question, are given in Appendix 1.
The class that matriculated following the IAT exercise was the most diverse in OSUCOM’s history at that time. In the 2011–2012 admissions cycle (the cycle immediately before the IAT exercise), 24/56 (43%) URMs who were offered acceptance matriculated at OSUCOM, ultimately accounting for 30/178 (17%) new entrants in 2012 (see Table 2). In the 2012–2013 admissions cycle (the cycle immediately after the IAT exercise), 31/57 (54%) URMs who were offered acceptance chose to enroll, increasing the URM percentage in the entering class (37/188 [20%]) (see Table 2). Although nearly the same number of URMs were offered acceptance in both years, the relative increase in yield (matriculants/offers × 100) is what accounts for the increase in matriculating URMs. This 26% relative increase in yield after the IAT exercise was not statistically significant. However, this trend may have a plausible explanation. Survey comments support the possibility that having been made aware of their implicit white preference, admissions committee members may have modified their behavior in the subsequent admissions cycle. Consequently, URM candidates in the 2012–2013 cycle may have perceived their interactions with our committee as particularly favorable, increasing their desire to matriculate to OSUCOM. This hypothesis should be tested in future studies.
In summary, admissions committee members at our large, public, Midwestern medical school displayed significant levels of implicit white preference on the IAT. Most committee members thought the exercise was valuable and might be helpful in reducing bias, and some reported that it impacted their admissions decisions in the next cycle. To build on this study, we plan to precede and follow IAT exercises for admissions committee members with more robust reflection and education on unconscious bias in the future. We anticipate that this effort will be threefold, starting with an independent learning requirement such as computer-based modules on unconscious bias to be completed before the annual orientation. The second phase will require members to take the IAT and reflect on their individual results before orientation. Finally, we will incorporate a workshop on strategies to reduce or neutralize unconscious biases into our orientation. We have multiple nonmedical colleagues (e.g., from our psychology department and law school) who are implicit bias experts at our institution, and we will consult these experts to inform the unconscious bias training for our admissions committee members. We recommend this multidisciplinary approach to others planning to replicate this exercise.
Our future efforts to correlate implicit bias training with an impact on enrollment will include surveying applicants who are offered acceptance regarding their perceptions of the climate at our medical school. We also plan to collaborate with peer medical schools to increase the number of subjects (admissions committee members) in future studies.
Our study has some limitations. First, because this is a single-center study, extrapolating our findings to schools in different regions of the country or with different support systems (private vs. public, sectarian vs. secular) should only be done cautiously. Second, while our committee members took the black–white IAT which uses pictures of black and white persons, we report a possible impact on total URM enrollment, which also includes Hispanics, Native Americans, Native Alaskan/Hawaiians, or Native Pacific Islanders. Although this is a limitation, it should be noted that the URMs applying to our medical school in the 2011–2012 and 2012–2013 admissions cycles were almost exclusively blacks and Hispanics with very few Native Americans, Native Alaskan/Hawaiians, or Native Pacific Islanders. Given that the Hispanic and Native American IATs reveal magnitudes of implicit white preference similar to the black–white IAT,3,10 our findings are not likely to be significantly affected by this consideration. However, in future studies we plan to categorize and report the specific races/ethnicities of our URMs with our findings.
The identification and elimination of unconscious racial bias in medical school admissions could ultimately help to reduce racial health care disparities, and we join others in calling for an examination of bias at all levels of academic medicine. Given the availability of a free, publicly available instrument to detect unconscious bias, we propose that all medical school admissions committee members be encouraged to take the IAT and review bias reduction strategies.
Acknowledgments: The authors wish to thank Morgan Johnson, MS, for her assistance with the preparation of this manuscript.
2. Measuring individual differences in implicit cognition: The implicit association test. J Pers Soc Psychol. 1998;74:1464–1480.
3. Pervasiveness and correlates of implicit attitudes and stereotypes. Eur Rev Soc Psychol. 2007;18:36–88.
4. Are teachers’ expectations different for racial minority than for European American students? A meta-analysis. J Educ Psychol. 2007;99:253–273.
5. Across the thin blue line: Police officers and racial bias in the decision to shoot. J Pers Soc Psychol. 2007;92:1006–1023.
6. Implicit bias among physicians and its prediction of thrombolysis decisions for black and white patients. J Gen Intern Med. 2007;22:1231–1238.
7. Understanding and using the implicit association test: III. Meta-analysis of predictive validity. J Pers Soc Psychol. 2009;97:17–41.
8. Physicians’ implicit and explicit attitudes about race by MD race, ethnicity, and gender. J Health Care Poor Underserved. 2009;20:896–913.
9. Statistical Power Analysis for the Behavioral Sciences. 1988.2nd ed. Hillsdale, NJ: Lawrence Erlbaum.
10. Do American Indian mascots = American Indian people? Examining implicit bias towards American Indian people and American Indian mascots. Am Indian Alsk Native Ment Health Res. 2011;18:42–62.
Survey Results From 100 of 140 (71% Response Rate) Admissions Committee Members on Questions Related to the IAT Exercise, OSUCOM, May 2013 (Following the 2012–2013 Admission Cycle) Results from multiple-choice survey questions (percentage of respondents selecting each answer is given in parentheses)
- 1. Regarding your individual results on the IAT:
- A. I was surprised by my results on the IAT—they were quite different from what I would have guessed. (15%)
- B. My individual results on the IAT were only slightly surprising—they were only slightly different from what I would have guessed. (57%)
- C. My individual results on the IAT were not surprising at all—they were exactly what I would have guessed. (23%)
- D. N/A. (5%)
- 2. The aggregate results of the entire OSUCOM admissions committee (faculty and students) were reviewed at the admissions retreat and indicate that the majority of committee members display implicit (unconscious) white preference. With regard to these aggregate results:
- A. I was very surprised—the results are far from what I would have expected. (9%)
- B. I was only slightly surprised—the results are not far from what I would have expected. (57%)
- C. I was not surprised at all—the results are what I would have expected. (34%)
- 3. The idea of a medical school admissions committee taking the IAT prior to beginning the interview season is worthwhile and might be expected to have a positive outcome on reducing bias in the admissions process.
- A. Strongly agree. (24%)
- B. Agree. (43%)
- C. Neutral. (24%)
- D. Disagree. (6%)
- E. Strongly disagree. (2%)
- 4. I am conscious of my individual IAT results when I interview medical school candidates.
- A. Strongly agree. (9%)
- B. Agree. (39%)
- C. Neutral. (30%)
- D. Disagree. (11%)
- E. Strongly disagree. (9%)
- F. N/A. (2%)
- 5. My knowledge of my individual IAT results impacted my evaluation and disposition of medical school candidates.
- A. Strongly agree. (1%)
- B. Agree. (20%)
- C. Neutral. (42%)
- D. Disagree. (21%)
- E. Strongly disagree. (13%)
- F. N/A. (3%)
- 6. The IAT/implicit bias exercise prior to this admissions cycle likely led to a reduction of bias in the evaluation of candidates.
- A. Strongly agree. (4%)
- B. Agree. (38%)
- C. Neutral. (44%)
- D. Disagree. (9%)
- E. Strongly disagree. (5%)
- 7. The IAT/implicit bias exercise is a worthwhile activity for the admissions committee that should be repeated annually or biannually.
- A. Strongly agree. (21%)
- B. Agree. (43%)
- C. Neutral. (26%)
- D. Disagree. (5%)
- E. Strongly disagree. (4%)
- 8. The IAT/implicit bias exercise should be accompanied by a workshop on strategies to neutralize unconscious bias.
- A. Strongly agree. (18%)
- B. Agree. (42%)
- C. Neutral. (27%)
- D. Disagree. (8%)
- E. Strongly disagree. (5%)
Representative comments from open-ended survey question
The survey ended with an open-ended question: “Please provide any comments about the implicit bias exercise and IAT testing for the admissions committee, including suggestions on how to make it more effective.” The authors have divided all narrative responses to this question into the three most common themes that emerged: (1) comments indicating a belief that taking the IAT will reduce bias in the admissions process (41%), (2) comments indicating a belief that the IAT exercise is not helpful to the admissions process (34%), and (3) comments indicating concerns that the IAT exercise will encourage the admission of underqualified minority candidates (24%). Representative comments from each category are shown below:
- 1. Taking the IAT will reduce bias in the admissions process:
“Made me more cognizant of my prejudices entering the interview season.”
“It’s an important topic to review prior to an admission season and this is probably the best way to initiate and address the topic.”
“It allows us to consciously be careful not to execute those underlying biases.”
“Continue the process and need workshops to help get rid of bias in [the admissions] process.”
“I have done similar exercises in the past and find them useful measures to assist in self-inventory.”
- 2. Taking the IAT exercise is not helpful to the admissions process:
“I think it’s good to know your biases, but I don’t know how they play into my interview/deliberation decisions.”
“… my IAT said I prefer blacks over whites, yet I feel it would be inappropriate for me to try to compensate for that.”
“A good exercise in self-awareness and self-evaluation but I’m not sure it is useful in this setting. However, I am of a younger generation and cannot speak for faculty who were raised in a different culture or era.”
“I don’t believe the results of this ‘test.’”
“Looking at pictures and clicking buttons does not equal how one interacts with a human being. I am not convinced of any validity of this test.”
- 3. The IAT exercise will encourage the admission of underqualified minority candidates:
“I thought it might cause people to overcompensate—for example, if I’m slightly biased against black people, I might look over deficiencies I would otherwise flag because I’m ‘trying’ not to be racist.”
“I think this is a good exercise in self-awareness, but I think that it could also have the effect of causing folks to overcompensate in the other direction for their biases.”
“It is more likely it creates bias in that it implicitly encourages admission of minority candidates rather than focusing on the content of their character.”
Abbreviations: IAT indicates implicit association test; OSUCOM, Ohio State University College of Medicine; N/A, not answered.