Secondary Logo

Journal Logo

A Framework for Integrating Implicit Bias Recognition Into Health Professions Education

Sukhera, Javeed, MD, DABPN, FRCPC; Watling, Chris, MD, PhD, FRCPC

doi: 10.1097/ACM.0000000000001819
Perspectives
Free

Existing literature on implicit bias is fragmented and comes from a variety of fields like cognitive psychology, business ethics, and higher education, but implicit-bias-informed educational approaches have been underexplored in health professions education and are difficult to evaluate using existing tools. Despite increasing attention to implicit bias recognition and management in health professions education, many programs struggle to meaningfully integrate these topics into curricula. The authors propose a six-point actionable framework for integrating implicit bias recognition and management into health professions education that draws on the work of previous researchers and includes practical tools to guide curriculum developers. The six key features of this framework are creating a safe and nonthreatening learning context, increasing knowledge about the science of implicit bias, emphasizing how implicit bias influences behaviors and patient outcomes, increasing self-awareness of existing implicit biases, improving conscious efforts to overcome implicit bias, and enhancing awareness of how implicit bias influences others. Important considerations for designing implicit-bias-informed curricula—such as individual and contextual variables, as well as formal and informal cultural influences—are discussed. The authors also outline assessment and evaluation approaches that consider outcomes at individual, organizational, community, and societal levels. The proposed framework may facilitate future research and exploration regarding the use of implicit bias in health professions education.

J. Sukhera is assistant professor of psychiatry and pediatrics, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada, and PhD candidate in health professions education, Maastricht University, Maastricht, the Netherlands; ORCID: http://orcid.org/0000-0001-8146-4947.

C. Watling is associate dean for postgraduate medical education and scientist, Centre for Education Research and Innovation, Schulich School of Medicine and Dentistry, Western University, London, Ontario, Canada.

Funding/Support: This work is supported by grants from the Academic Medical Organization of Southwestern Ontario, Children’s Health Foundation, and Associated Medical Services–Phoenix Fellowship.

Other disclosures: None reported.

Ethical approval: Reported as not applicable.

Previous presentations: An earlier version of this framework was discussed at the Association of American Medical Colleges Annual Professional Development Conference for the Council of Faculty and Academic Societies/Organization of Resident Representatives/Group on Diversity and Inclusion in Salt Lake City, Utah, March 4, 2016.

Correspondence should be addressed to Javeed Sukhera, London Health Sciences Center–Victoria Hospital, 800 Commissioners Rd. E, Suite B8-176, London, ON, Canada, N6A 5W9; telephone: (519) 685-8500, ext. 74968; e-mail: jsukhera@uwo.ca.

Biases, stereotypes, and associations that exist outside conscious awareness may adversely influence the health of minority, underserved, and stigmatized populations. Unconsciously held biases, also known as implicit biases, may lead to inaccurate or compromised clinical decisions1–3 and an erosion of trust between health professionals and patients due to poor interpersonal interactions and biased behaviors.4,5 Implicit bias in health care has been demonstrated in several studies. For example, among 202 first-year medical students at Johns Hopkins School of Medicine, 66% showed an implicit preference toward Caucasians, and 86% demonstrated a preference toward upper-class individuals.6 Among practicing physicians, another study found that, despite consciously expressing explicit egalitarian goals, physicians were less likely to recommend thrombolysis to African American patients, as compared with Caucasians with similar symptoms.1 Other work has highlighted that bias adversely impacts women,7 obese individuals,2 fathers,8 patients with chronic pain,2,9 and patients with chronic fatigue syndrome.10

Not all implicit bias is negative or destructive. In many settings, cognitive heuristics (including unconscious associations) are encouraged as a form of pattern recognition that is necessary for clinical decision making.11 In other situations, negative implicit associations may protect against workplace violence and promote safety.12 Further, positive implicit bias may lead to counterproductive outcomes related to conflicts of interest and professional misconduct.13 Rather than pursuing the goal of eliminating bias, any intervention that incorporates implicit bias into health professions education should emphasize how health professionals can mitigate the negative influence of bias on patient outcomes. Implicit-bias-informed curricula can, therefore, offer systematic educational strategies for addressing biases in health care by both promoting awareness of one’s own implicit biases and enhancing conscious efforts to overcome these biases.

The existing literature on implicit bias is fragmented and comes from a variety of fields like cognitive psychology, business ethics, and higher education. Implicit-bias-informed educational interventions have been underexplored in health professions education and are difficult to evaluate using existing tools. Despite increasing attention to implicit bias recognition and management in health professions education, many programs struggle to meaningfully integrate these topics into curricula because of difficulty recruiting faculty champions, perceived lack of relevance, and pressure to concentrate on medically focused curricular content.14,15 Despite some promising approaches, implicit bias recognition and management is often included in the form of brief interventions that are poorly integrated into curricula and lack cohesive assessment and evaluation strategies.16 Integrating existing research on educating health professions students on the impact of implicit bias and its relationship to patient-centered care into curricula, therefore, requires a cohesive and unifying framework.

While some authors have proposed such conceptual frameworks in the past,17–20 a recent surge in interest in implicit-bias-informed education21 has prompted a renewed call for a practical framework to integrate implicit bias recognition and management into health professions curricula. Further, attention to evaluation and assessment strategies that consider outcomes at individual, organizational, community, and societal levels is needed. In this Perspective, we propose a six-point actionable framework for integrating implicit bias recognition and management into health professions education that draws on the work of previous researchers and includes practical tools to guide curriculum developers. We then outline assessment and evaluation approaches to consider outcomes at the level of the individual, organization, community, and society.

Back to Top | Article Outline

A Six-Point Framework

Our framework includes six key features: creating a safe and nonthreatening learning context, increasing knowledge about the science of implicit bias, emphasizing how implicit bias influences behaviors and patient outcomes, increasing self-awareness of existing implicit biases, improving conscious efforts to overcome implicit bias, and enhancing awareness of how implicit bias influences others (Figure 1).

Figure 1

Figure 1

Back to Top | Article Outline

Creating a safe and nonthreatening learning context

Teaching about bias, stereotyping, and privilege can be risky; both learners and faculty may be challenged to confront attitudes that they may not feel necessary to address or willing to disclose. Directly challenging both negative and positive biases can also produce a negative “kick-back” that reinforces counterproductive biases.22 By emphasizing the pervasive qualities of implicit bias, instructors can reinforce that bias is everywhere and, therefore, guilt regarding individually held biases is a common human experience. Thus, a key distinction between traditional education about diversity or cultural competence and implicit-bias-informed curricula is a proactive shift away from guilt and toward responsibility.22 Several authors emphasize that, when teaching about bias, there should be explicit recognition that the removal of all bias is impossible.23–25

When designing interventions, educators should also recognize instructor and learner characteristics that support a climate of safety. Choosing instructors that are approachable, inclusive, nonthreatening, inspiring, open-minded, encouraging, and knowledgeable can best facilitate safe and nonjudgmental learning environments.21,26 Since students from a nondominant group are likely to approach discussions about privilege and bias with different personal experiences than students who belong to traditionally privileged groups, learning environments should proactively avoid reinforcing feelings of resentment or anxiety for learners. Instructors can promote a nonthreatening learning environment by openly addressing the discomfort that accompanies discussions about bias and privilege. Most organizations have existing codes of conduct that can be introduced into the intervention to emphasize the importance of core values, including respect, integrity, and confidentiality. Educators must ensure sufficient time so that learners do not feel rushed and power balances between teacher and learners, as well as between various health professions, are acknowledged.

Back to Top | Article Outline

Increasing knowledge about the science of implicit bias

Any intervention that incorporates implicit bias recognition into health professions education should include content regarding the psychological and neurobiological components of bias and provide an evidence-based framework for understanding the cognitive science that underlies implicit biases. Incorporating cognitive psychology and neuroscience into the intervention has the potential to increase relevance for learners, while illustrating that implicit bias is impossible to eradicate and common to patients, caregivers, and providers. Emphasizing that bias is a result of neurobiological mechanisms that can restrict an individual’s ability to be open to multiple perspectives14 helps activate prior knowledge about learners’ own biases and experiences with others’ biases. Teaching learners about psychological processes that are outside their conscious awareness is also important because health care providers may underestimate the extent to which biases influence their behaviors.27

Increasing knowledge about the nature of implicit bias should include a demonstration of different types of bias, including both positive and negative biases. Educators should ensure that there is a discussion about the constructive aspects of bias, clarifying that bias can be adaptive in some circumstances and destructive in others. Illustrating that bias may be a useful tool to promote safety and belonging, while providing clinically relevant examples of how bias influences patient outcomes, is important and highlights how implicit bias recognition and management is distinct from traditional approaches to diversity, cultural competence, and stigma reduction.

Back to Top | Article Outline

Emphasizing how implicit bias influences behaviors and patient outcomes

The influence of implicit bias on patient outcomes is a key component of our framework. By aligning interventions with existing curricula on health disparities, instructors can connect discussions about bias with the role of history and both visible and invisible sociocultural forces that impact health. Citing statistics and existing research on implicit bias in medical education, as well as discussing the impact of internalized bias on stereotypes, helps motivate learners to learn how to address and mitigate their biases. For certain health professions, specific reference to literature on clinical decision making and cognitive psychology, including certain types of bias, such as anchoring (relying too heavily on the first piece of information about a patient) or confirmation bias (the tendency to favor information in a manner that confirms preexisting beliefs), may lay the groundwork for learners to engage with ideas about how biases may adversely affect care.

Back to Top | Article Outline

Increasing self-awareness of existing implicit biases

The implicit association test (IAT) may provide a useful trigger for self-reflection, discussion, and awareness of one’s own existing biases.28,29 The IAT is a computer-based exercise that asks participants to associate words and pictures to assess automatic associations between concepts by measuring the time and latency of individuals’ responses. For example, the IAT can demonstrate an association between groups of people (blacks and whites) and stereotypes (good and bad).28 The IAT has typically demonstrated good internal consistency,30–33 insensitivity to procedural variation,28,34 high test–retest reliability,35 and less susceptibility to faking than explicit measures of bias, such as surveys.36 Criticism of the IAT suggests that, instead of reflecting negative attitudes, IAT scores may stem from associations such as victimization, maltreatment, and oppression.37–42 Given these critiques, we recommend using the IAT as a prompt for reflection and not as a metric for measuring implicit bias or evaluating curricula.

Other techniques that may help elicit awareness of existing implicit biases include facilitated discussions on how bias impacts care and reflection and identity exercises. Techniques that enhance reflective capacity may be useful, and curriculum designers are encouraged to make explicit connections between bias-related curricula and other curricular content that foster reflective practice. One simple potential identity exercise could pair up participants to discuss the dominant and nondominant cultures to which they belong. Individual reflection on identity and times that individual learners felt different or treated inequitably can also increase awareness of existing biases. There are examples in the health professions literature of potentially useful tools to facilitate reflection, such as the privilege and responsibility curricular exercise.43

Back to Top | Article Outline

Improving conscious efforts to overcome implicit bias

When learners become aware of their implicit biases, they often ask, “What should I do about it?”19 Since forming conscious egalitarian goals is simply not enough to reduce the impact of implicit bias on patient care,44 and research emphasizes the importance of conscious effort to avoid applying stereotypes,45,46 particular attention to training and tools that help learners understand their thinking, reasoning, and the influence of bias on their behaviors and decisions is an essential ingredient for implicit-bias-informed curricula.

Specific techniques for improving conscious efforts to overcome bias come from social psychology and include metacognitive strategies that facilitate the ability of learners to think about their thinking.22,47 Some authors have also advocated for improving self-regulation and self-monitoring.48,49 To this end, encouraging learners to set discrete goals and to reevaluate their success at longitudinal checkpoints over time facilitates increased monitoring and reflection on biases and attitudes.50 Another technique that holds promise is the role of mindfulness training, which can improve attention to biased judgments and behaviors.51

Back to Top | Article Outline

Enhancing awareness of how implicit bias influences others

In contrast to self-awareness, enhancing social perspective taking and empathy for others is another important component of implicit bias recognition and management.17,44 Explicit attention to the cognitive and affective components of empathy that unite patients and providers within a shared emotional context protects against stereotyping and discrimination that is rooted in implicit bias.52,53 More specifically, reframing patient–provider contact as an interaction between collaborating equals can shift one’s thinking of diverse individuals as outsiders to perceiving them as part of one’s own social groups by emphasizing a shared human identity.54–56

Cultivating empathy through implicit-bias-informed curricula can be achieved through strategies that enhance social perspective taking and facilitate positive emotions. Research demonstrates that when nurses were shown pictures of either white or black patients with expressions of pain and asked how much pain medication they recommended, those who were instructed to imagine how the patient felt recommended equal analgesic treatment, as compared with those who were told to use their best judgment, who recommended more pain medication for white patients.57 Techniques for enhancing perspective taking may include role-play, participatory theater, and social contact with matched patients or families with lived experience. For example, social-contact-based interventions may involve inviting patients who belong to marginalized groups that have experienced bias in clinical settings to speak to providers, share their story, and engage in discussion. Social contact has produced favorable outcomes in reducing implicit bias toward sexual minorities,58 patients with mental illness,59–61 and females.62

Social-contact-based interventions, however, may have unintended consequences. Research on mental illness stigma demonstrates that social contact will reduce stigmatizing attitudes and behaviors only when it is conducted in person and between individuals of equal power.59–61 Creating an authentic learning environment with an equalized power differential is challenging. Additionally, contact with patients with mental illness can lead to both positive and negative experiences for health care providers19,63 and should only be used with careful planning and caution because of the possibility that it may reinforce destructive stereotypes.

Back to Top | Article Outline

Designing Interventions

The first step in implicit-bias-related educational interventions is to determine whether educators will focus on a specific type of bias, such as gender, race, culture, etc., or bias in general. Once the focus of the intervention is determined, instructors should clarify whether they are teaching about bias related to specific or broad contexts. For example, will the curricula teach about gender bias in relation to cardiovascular health, to clinical outcomes in general, or to how it influences organizational issues and health policy? The broader the intervention, the more time and integration with curricula it will require. Attention to the influence and outcomes of bias at individual, organizational, community, and societal levels also requires deliberate attention from the early stages of instructional design and throughout the implementation process.

Educators should consider learners’ stage in their professional trajectory and clinical experience, as well as how oriented they are to implicit bias and its role in health care, when designing interventions. Considering clinical experience and context is crucial because expecting learners without clinical experience to engage with complex clinical scenarios may lead to feedback that the educational intervention lacks relevance. Dynamics related to interprofessional teams should also be considered because of their potential influence on discussions of privilege and power. For example, will the intervention target undergraduate preclinical students, clinical clerks, or postgraduate trainees? Are learners undifferentiated, or have they developed the identity of a specific specialty or subspecialty? Are learners from diverse health disciplines or members of one particular group? Using established methodologies to design curricula and write relevant and measurable learning objectives will help facilitate these decisions and develop the appropriate assessment and evaluation strategies.

Previous research emphasizes that the recognition of bias cannot be taught in a single session.16 Curriculum designers should also consider how they will sustain implicit bias recognition and management interventions within a learning environment or organization. Cultural and hidden curricular influences that may work against the intervention must be considered. Learners may internalize and perpetuate patterns of behavior that they are exposed to over time. In the landmark Medical Student Cognitive Habits and Growth Evaluation Study (CHANGES), van Ryn and colleagues63 found that, despite their efforts to reduce implicit race bias, hearing negative comments about black patients from physicians and residents worsened implicit racial biases over the course of undergraduate medical education.

Back to Top | Article Outline

Evaluation and Assessment Strategies

While traditional program evaluation and learner assessment strategies using a knowledge, skills, and attitudes framework are a good start,64 a purely outcomes-driven approach to program evaluation may be too narrow to capture the complexity related to implicit-bias-informed curricula.65 Evaluation of these curricula requires the broadening of existing frameworks to consider the influence of bias-related strategies on the individual, their organization, the community, and society at large. We recommend an evaluation approach that generates useful information to guide the adaptation of implicit-bias-informed curricula to rapidly evolving social and cultural contexts.66

We encourage curriculum designers to start with a logic model and establish their desired outcomes at organizational and societal levels. Is your curriculum designed to improve learner attitudes while promoting cultural change, diversity, and inclusion? Do you aim to improve health care quality and patient experience? Is there an overall equity-related goal in mind? Addressing these questions is of paramount importance when determining evaluation metrics. Organizational tools pertaining to achieving accreditation benchmarks related to diversity, inclusion, and cultural competence, such as those published by the Association of American Medical Colleges,13,67 may be useful in some circumstances, while population-based data might be useful in others. Patient-related outcomes, such as satisfaction surveys, may provide meaningful data to gauge the impact of curricula on real-world outcomes.18 Evaluation strategies must also consider that desired outcomes may take time. Therefore, longitudinal approaches and both quantitative and qualitative methods should be considered.

Learner-related outcomes are an essential component of any program evaluation. For these kinds of outcomes, we propose mapping assessment strategies to the points on our framework (Figure 2). For example, increasing knowledge about the science of implicit bias and emphasizing how implicit bias influences behaviors and patient outcomes can be assessed through pre and post knowledge tests.68 Improving conscious efforts to overcome implicit bias can be assessed through modified observed clinical evaluation methods, such as mini-clinical evaluation exercises and objective structured clinical examinations.69,70 Increasing self-awareness of existing implicit biases and enhancing awareness of how implicit bias influences others can be assessed through portfolios and multisource feedback, respectively. Portfolios are learner-compiled dossiers that include content such as work completed, reflection exercises, feedback received, and plans for improving competence.71,72 Portfolios may facilitate the assessment of reflective skills if there is adequate structure, coaching, and direction.73

Figure 2

Figure 2

There seems to be an emerging consensus that no single method of assessment or evaluation will be sufficient.74 Nonetheless, considering how evaluating and assessing implicit-bias-informed curricula relates to emerging models for assessing professional competence of learners merits further reflection, research, and exploration.

Back to Top | Article Outline

Conclusion

We believe that implicit bias recognition and management can be effectively integrated into health professions education by considering our six-point actionable framework. We suggest that educational interventions that are delivered in safe and nonthreatening environments and foster perspective taking and empathy through social contact are important. Interventions should also teach learners what implicit bias is, demonstrate how it influences clinical decisions and patient outcomes, increase awareness of existing implicit biases in learners, and enhance conscious efforts to overcome the adverse impact of implicit bias. Interventions should consider power dynamics between teachers, learners, and patients and consider the sociocultural context in which such dynamics are played out. Lastly, evaluation and assessment strategies should consider outcomes at the levels of the individual, organization, community, and society. Our proposed framework may facilitate future research and exploration regarding the use of implicit bias in health professions education.

Back to Top | Article Outline

References

1. Green AR, Carney DR, Pallin DJ, et al. Implicit bias among physicians and its prediction of thrombolysis decisions for black and white patients. J Gen Intern Med. 2007;22:1231–1238.
2. Sabin JA, Greenwald AG. The influence of implicit bias on treatment recommendations for 4 common pediatric conditions: Pain, urinary tract infection, attention deficit hyperactivity disorder, and asthma. Am J Public Health. 2012;102:988–995.
3. Chapman EN, Kaatz A, Carnes M. Physicians and implicit bias: How doctors may unwittingly perpetuate health care disparities. J Gen Intern Med. 2013;28:1504–1510.
4. Dovidio JF, Kawakami K, Gaertner SL. Implicit and explicit prejudice and interracial interaction. J Pers Soc Psychol. 2002;82:62–68.
5. Fazio RH, Jackson JR, Dunton BC, Williams CJ. Variability in automatic activation as an unobtrusive measure of racial attitudes: A bona fide pipeline? J Pers Soc Psychol. 1995;69:1013–1027.
6. Haider AH, Sexton J, Sriram N, et al. Association of unconscious race and social class bias with vignette-based clinical assessments by medical students. JAMA. 2011;306:942–951.
7. Verdonk P, Benschop YW, de Haes HC, Lagro-Janssen TL. From gender bias to gender awareness in medical education. Adv Health Sci Educ Theory Pract. 2009;14:135–152.
8. Stivers T, Majid A. Questioning children: Interactional evidence of implicit bias in medical interviews. Soc Psychol Q. 2007;70:424–441.
9. Weinstein SM, Laux LF, Thornby JI, et al. Physicians’ attitudes toward pain and the use of opioid analgesics: Results of a survey from the Texas Cancer Pain Initiative. South Med J. 2000;93:479–487.
10. Friedberg F, Sohl SJ, Halperin PJ. Teaching medical students about medically unexplained illnesses: A preliminary study. Med Teach. 2008;30:618–621.
11. Eva KW, Norman GR. Heuristics and biases—A biased perspective on clinical reasoning. Med Educ. 2005;39:870–872.
12. Sukhera J, Chahine S. Reducing mental illness stigma through unconscious bias-informed education. https://www.mededpublish.org/manuscripts/449/v1. Accessed April 27, 2017.
13. Association of American Medical Colleges. The Diversity Research Forum: “Tools for assessing and measuring diversity in medical education.” https://members.aamc.org/eweb/upload/The%20Diversity%20Research%20Forum%20Tools%20for%20Assessing.pdf. Published 2006 Accessed April 27, 2017.
14. Johnson I, Donovan D, Parboosingh J. Steps to improve the teaching of public health to undergraduate medical students in Canada. Acad Med. 2008;83:414–418.
15. Tyler IV, Hau M, Buxton JA, et al. Canadian medical students’ perceptions of public health education in the undergraduate medical curriculum. Acad Med. 2009;84:1307–1312.
16. Gonzalez CM, Kim MY, Marantz PR. Implicit bias and its relation to health disparities: A teaching program and survey of medical students. Teach Learn Med. 2014;26:64–71.
17. Burgess D, van Ryn M, Dovidio J, Saha S. Reducing racial bias among health care providers: Lessons from social-cognitive psychology. J Gen Intern Med. 2007;22:882–887.
18. Blair IV, Steiner JF, Havranek EP. Unconscious (implicit) bias and health disparities: Where do we go from here? Perm J. 2011;15:71–78.
19. Teal CR, Gill AC, Green AR, Crandall S. Helping medical learners recognise and manage unconscious bias toward certain patient groups. Med Educ. 2012;46:80–88.
20. Moss-Racusin CA, Dovidio JF, Brescoll VL, Graham MJ, Handelsman J. Science faculty’s subtle gender biases favor male students. Proc Natl Acad Sci U S A. 2012;109:16474–16479.
21. Boscardin CK. Reducing implicit bias through curricular interventions. J Gen Intern Med. 2015;30:1726–1728.
22. Ross HJ. Everyday Bias: Identifying and Navigating Unconscious Judgments in Our Daily Lives. 2014.Lanham, MD: Rowman & Littlefield.
23. Burgess DJ, Fu SS, van Ryn M. Why do providers contribute to disparities and what can be done about it? Gen Intern Med. 2004;19:1154–1159.
24. Czopp AM, Monteith MJ, Mark AY. Standing up for a change: Reducing bias through interpersonal confrontation. J Pers Soc Psychol. 2006;90:784–803.
25. Vorauer JD, Sasaki SJ. Helpful only in the abstract? Ironic effects of empathy in intergroup interaction. Psychol Sci. 2009;20:191–197.
26. Young M. What is learning and why does it matter? Eur J Educ. 2015;50:17–20.
27. van Ryn M, Fu SS. Paved with good intentions: Do public health and human service providers contribute to racial/ethnic disparities in health? Am J Public Health. 2003;93:248–255.
28. Greenwald AG, McGhee DE, Schwartz JLK. Measuring individual differences in implicit cognition: The implicit association test. J Pers Soc Psychol. 1998;74:1464–1480.
29. Teal CR, Shada RE, Gill AC, et al. When best intentions aren’t enough: Helping medical students develop strategies for managing bias about patients. J Gen Intern Med. 2010;25(suppl 2):115–118.
30. Bosson JK, Swann WB Jr, Pennebaker JW. Stalking the perfect measure of implicit self-esteem: The blind men and the elephant revisited? J Pers Soc Psychol. 2000;79:631–643.
31. Dasgupta N, Greenwald AG. On the malleability of automatic attitudes: Combating automatic prejudice with images of admired and disliked individuals. J Pers Soc Psychol. 2001;81:800–814.
32. Greenwald AG, Farnham SD. Using the implicit association test to measure self-esteem and self-concept. J Pers Soc Psychol. 2000;79:1022–1038.
33. Greenwald AG, Nosek BA. Health of the Implicit Association Test at age 3. Z Exp Psychol. 2001;48:85–93.
34. Nosek BA, Greenwald AG, Banaji MR. Understanding and using the Implicit Association Test: II. Method variables and construct validity. Pers Soc Psychol Bull. 2005;31:166–180.
35. Nosek BA, Greenwald AG, Banaji MR. Bargh JA. The implicit association test at age 7: A methodological and conceptual review. In: Automatic Processes in Social Thinking and Behavior. 2007:New York, NY: Pyschology Press; 265–292.
36. Steffens MC, Schulze König S. Predicting spontaneous big five behavior with implicit association tests. Eur J Psychol Assess. 2006;22:13–20.
37. Blanton H, Jaccard J, Christie C, Gonzales PM. Plausible assumptions, questionable assumptions and post hoc rationalizations: Will the real IAT, please stand up? J Exp Soc Psychol. 2007;43:399–409.
38. Gill MJ, Andreychik MR, Getty PD. More than a lack of control: External explanations can evoke compassion for outgroups by increasing perceptions of suffering (independent of perceived control). Pers Soc Psychol Bull. 2013;39:73–87.
39. Uhlmann EL, Brescoll VL, Paluck EL. Are members of low status groups perceived as bad, or badly off? Egalitarian negative associations and automatic prejudice. J Exp Soc Psychol. 2006;42:491–499.
40. Banaji MR, Nosek BA, Greenwald AG. No place for nostalgia in science: A response to Arkes and Tetlock. Psychol Inq. 2004;15:279–289.
41. Greenwald AG, Nosek BA, Sriram N. Consequential validity of the implicit association test: Comment on Blanton and Jaccard (2006). Am Psychol. 2006;61:56–61.
42. Greenwald AG, Poehlman TA, Uhlmann EL, Banaji MR. Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. J Pers Soc Psychol. 2009;97:17–41.
43. Holm AL, Rowe Gorosh M, Brady M, White-Perkins D. Recognizing privilege and bias: An interactive exercise to expand health care providers’ personal awareness. Acad Med. 2017;92:360–364.
44. Stone J, Moskowitz GB. Non-conscious bias in medical decision making: What can be done to reduce it? Med Educ. 2011;45:768–776.
45. Shelton D, Smith J, Purnell-Greene M. Design of a mental health promotion program for vulnerable African American youth. J Forensic Nurs. 2005;1:158–165.
46. Shelton JN. Interpersonal concerns in social encounters between majority and minority group members. Group Process Intergr Relat. 2003;6:171–185.
47. Schraw G, Crippen KJ, Hartley K. Promoting self-regulation in science education: Metacognition as part of a broader perspective on learning. Res Sci Educ. 2006;36:111–139.
48. Devine PG. Stereotypes and prejudice: Their automatic and controlled components. J Pers Soc Psychol. 1989;56:5–18.
49. Devine P, Monteith M. Mackie DM, Hamilton DL. The role of discrepancy-associated affect in prejudice reduction. In: Affect, Cognition, and Stereotyping: Interactive Processes in Group Perception. 1993:San Diego, CA: Academic Press; 317–344.
50. Sandars J, Cleary TJ. Self-regulation theory: Applications to medical education: AMEE guide no. 58. Med Teach. 2011;33:875–886.
51. Lueke A, Gibson B. Mindfulness meditation reduces implicit age and race bias: The role of reduced automaticity of responding. J Pers Soc Psychol. 2015;6:284–291.
52. Halpern J. What is clinical empathy? J Gen Intern Med. 2003;18:670–674.
53. Decety J, Cowell JM. Friends or foes: Is empathy necessary for moral behavior? Perspect Psychol Sci. 2014;9:525–537.
54. Dovidio JF, Gaertner SL. Aversive racism and selection decisions: 1989 and 1999. Psychol Sci. 2000;11:315–319.
55. Nier JA, Gaertner SL, Dovidio JF, Banker BS, Ward CM, Rust MC. Changing interracial evaluations and behavior: The effects of a common group identity. Group Process Intergr Relat. 2001;4:299–316.
56. Dovidio JF, Gaertner SL. Zanna MP. Aversive racism. In: Advances in Experimental Social Psychology. 2004:Amsterdam, Netherlands: Elsevier; 1–51.
57. Drwecki BB, Moore CF, Ward SE, Prkachin KM. Reducing racial disparities in pain treatment: The role of empathy and perspective-taking. Pain. 2011;152:1001–1006.
58. Burke SE, Dovidio JF, Przedworski JM, et al. Do contact and empathy mitigate bias against gay and lesbian people among heterosexual first-year medical students? A report from the Medical Student CHANGE study. Acad Med. 2015;90:645–651.
59. Corrigan P. Changing stigma through contact. Adv Schizophr Clin Psych. 2005;1:54–58.
60. Corrigan PW, Shapiro JR. Measuring the impact of programs that challenge the public stigma of mental illness. Clin Psychol Rev. 2010;30:907–922.
61. Corrigan PW. How clinical diagnosis might exacerbate the stigma of mental illness. Soc Work. 2007;52:31–39.
62. Carnes M, Devine PG, Baier Manwell L, et al. The effect of an intervention to break the gender bias habit for faculty at one institution: A cluster randomized, controlled trial. Acad Med. 2015;90:221–230.
63. van Ryn M, Hardeman R, Phelan SM, et al. Medical school experiences associated with change in implicit racial bias among 3547 students: A Medical Student CHANGES study report. J Gen Intern Med. 2015;30:1748–1756.
64. Kirkpatrick DL. Evaluating Training Programs: The Four Levels. 1998.San Francisco, CA: Berrett-Koehler Publishers.
65. Haji F, Morin MP, Parker K. Rethinking programme evaluation in health professions education: Beyond “did it work?” Med Educ. 2013;47:342–351.
66. Ginsburg S, McIlroy J, Oulanova O, Eva K, Regehr G. Toward authentic clinical evaluation: Pitfalls in the pursuit of competency. Acad Med. 2010;85:780–786.
67. Lie D, Boker J, Cleveland E. Using the tool for assessing cultural competence training (TACCT) to measure faculty and medical student perceptions of cultural competence instruction in the first three years of the curriculum. Acad Med. 2006;81:557–564.
68. Epstein RM, Hundert EM. Defining and assessing professional competence. JAMA. 2002;287:226–235.
69. Norcini JJ. Peer assessment of competence. Med Educ. 2003;37:539–543.
70. Norcini JJ, Blank LL, Arnold GK, Kimball HR. The mini-CEX (clinical evaluation exercise): A preliminary investigation. Ann Intern Med. 1995;123:795–799.
71. Van Tartwijk J, Driessen EW. Portfolios for assessment and learning: AMEE guide no. 45. Med Teach. 2009;31:790–801.
72. Driessen E, van Tartwijk J, van der Vleuten C, Wass V. Portfolios in medical education: Why do they meet with mixed success? A systematic review. Med Educ. 2007;41:1224–1233.
73. Driessen EW, van Tartwijk J, Overeem K, Vermunt JD, van der Vleuten CP. Conditions for successful reflective use of portfolios in undergraduate medical education. Med Educ. 2005;39:1230–1235.
74. van der Vleuten CP, Schuwirth LW, Scheele F, Driessen EW, Hodges B. The assessment of professional competence: Building blocks for theory development. Best Pract Res Clin Obstet Gynaecol. 2010;24:703–719.
© 2018 by the Association of American Medical Colleges