Recognizing Patients’ Emotions: Teaching Health Care Providers to Interpret Facial Expressions

Abstract

Purpose 

Facial expressions are an important clue to a patient’s emotions. The authors developed a 90-minute workshop in May 2011 to teach physicians and physicians-in-training to interpret facial expressions and to use that skill in the context of patient care.

Method 

The workshop included a didactic presentation in which facial expression features were taught using progressively more difficult examples, followed by three interactive exercises. The authors presented the workshop at six separate venues in the United States in 2011 and 2012. To test the effectiveness of this workshop, the authors designed a comprehensive pre- and postworkshop evaluation which assessed participants’ skill, knowledge, attitude (toward importance), and confidence.

Results 

A total of 156 health care providers participated in the workshop and completed pre- and postworkshop evaluations. Participants showed substantial improvement in skill and knowledge (fact- and case-based) scores, as well as modest improvement in importance and confidence ratings. Faculty and medical students demonstrated similar baseline measures of skill, knowledge, and importance, though faculty reported a slightly higher confidence in their skills. No correlation was found between baseline ratings of confidence in abilities and any baseline measure of performance.

Conclusions 

Given the similar baseline performance of faculty and medical students, the ability to interpret facial expressions does not appear to be learned through routine clinical practice, highlighting the need for dedicated facial expression training. The authors were able to demonstrate that physicians and physicians-in-training could effectively learn to recognize emotion by interpreting facial expressions through a short workshop.

Facial expressions are important clues to a person’s emotional state. Fortunately, these clues have some predictable features. Beginning with Charles Darwin¹ in the 19th century, anthropologists have shown that many facial expression features are not unique to specific cultures or populations but, rather, are universal.^2,3 One of the modern pioneers of this work is Dr. Paul Ekman, who has conducted research on facial expressions around the world (including famously with an isolated tribe in Papua New Guinea). His work describes the precise features of each facial expression, down to the detail of specific muscles, for seven cardinal emotions—happiness, sadness, anger, fear, surprise, disgust, and contempt—and shows that these features are consistent across cultures.^4,5 Based on this and related research, the skill of interpreting facial expressions is now used in a variety of different professions,⁶ from the U.S. Transportation Security Administration training agents to detect passengers with ill intent⁷ to professional sports teams hiring consultants to interpret whether prospective athletes would be the right emotional fit for the team.⁸

The practice of health care inevitably involves emotion. When physicians recognize emotion in their patients, they gain valuable information about their patients’ perspectives. The physician can then address that emotion directly or indirectly in communication with the patient to improve the patient’s experience. This is supported by studies which have shown improved diagnostic ability and patient satisfaction when providers attend to patients’ nonverbal signals.^9–11 These nonverbal signals can occur through many avenues, including body language, gestures, voice inflection, and facial expressions. While all these types of nonverbal communication could be useful to providers, we chose to focus on facial expressions, given the robust data about specific features and the evidence of their consistency across different cultures. Whereas prior studies of interventions to teach health care providers to interpret facial expressions have focused on medical students,^12–14 we are not aware of any published study which has evaluated facial expression training across a continuum of physicians and physicians-in-training.

To help meet this training need, we developed a 90-minute workshop in May 2011 to teach physicians and physicians-in-training to interpret facial expressions and to use that skill in the context of patient care. We hypothesized that this workshop would improve participants’ skill in correctly identifying the emotion in facial expression photographs, their knowledge of facial expression features, their attitude toward the importance of these skills, and their confidence in using these skills. We presented the workshop to faculty and medical students to see if both groups could effectively learn these skills through this format.

Method

Workshop description

The goals of our facial expression workshop were to teach participants (1) the facial expression features for seven cardinal emotions (see above) so that they could recognize subtle expressions of emotion in their patients and (2) to use this skill to address emotion when communicating with their patients. The first section of the workshop was a didactic presentation in which we taught facial expression features using progressively more difficult examples. This included photographs of people with strong emotional reactions from the news, models displaying emotions at varying intensities, and simplified cartoons of facial expression features (see Figure 1 for example images).

Figure 1: Examples of the images used in a workshop developed by the authors in May 2011 to teach physicians and physicians-in-training how to identify facial expression features and how to use that skill in the context of patient care. Panel A: Photograph of one of the authors (J.W.R.) showing facial expression features of anger, used in the didactic portion of the workshop. Panel B: Simplified cartoon showing facial expression features of anger, used in the didactic portion of the workshop.

After the didactic session, participants engaged in three interactive exercises. In the first exercise, they practiced facial expression features in a mirror as the workshop facilitators (J.W.R., C.S., R.V.D.) gave them feedback on how to perfect their expressions. We designed this exercise to promote a mental association between the features and their respective emotions, as research has shown that mimicking facial expressions can cause a person to experience that emotion.^15,16 In the second exercise, participants were paired with a partner and observed their partner’s reaction to various stimuli designed to provoke emotion (e.g., a disgusting image, a sad story). In the third exercise, small groups of participants discussed a series of hypothetical patient cases with accompanying facial photographs. Workshop facilitators guided the groups as they identified the emotion represented by each photograph and discussed techniques for addressing that emotion in the context of the case. Participants were then asked to role-play how their communication with the patient would change according to their identification of the patient’s emotion. Workshop facilitators guided the groups as they compared and contrasted different options for addressing the emotion in the context of each case.

Workshop participants

We presented the workshop at six separate venues in the United States in 2011 and 2012: (1) a national academic internal medicine meeting, (2) a multidisciplinary faculty development session at the Albert Einstein College of Medicine, (3) a general internal medicine faculty development session at the Louis Stokes Cleveland VA Medical Center, (4) an academic conference attended by general internal medicine faculty at the University of Pittsburgh School of Medicine (UPSOM), (5) an elective course for fourth-year medical students and residents at the UPSOM, and (6) a physical exam course for first-year medical students at the UPSOM. At each venue, participation in the workshop and in its evaluation was voluntary. Participants were informed of the study, and they provided consent via participation in the evaluation. They received no compensation for their participation.

Workshop evaluation

To test the effectiveness of this workshop, we designed a comprehensive pre- and postworkshop evaluation which encompassed multiple learning domains (i.e., skill and knowledge scores and importance and confidence ratings).

To assess participants’ skill in recognizing the emotions behind facial expression features, we created a test consisting of 10 facial expression photographs. One of us (J.W.R.) photographed a model mimicking the facial expression features for seven cardinal emotions as well as neutral expressions. He then selected images from this set which he deemed to be highly representative of each emotion and presented these images in a blinded, random fashion to two of us (C.S., R.V.D.), who independently labeled the emotion represented by each image. Any image on which the two of us disagreed or on which our labeled emotion differed from the original intent was removed. Using this refined set of images, we selected a final set of 10 photographs that included all seven cardinal emotions as well as both subtle and more overt expressions. We chose to duplicate some, but not all, of the seven emotions in the final set so that participants could not rely on the process of elimination to determine the correct answers. We presented this set of photographs to the participants in a random order at the beginning of each workshop (as part of the preworkshop evaluation) and again in a different order at the conclusion of each workshop (as part of the postworkshop evaluation). For each test, the photographs were shown for five seconds each, and participants selected the emotion demonstrated in each photograph on a response sheet. None of these photographs were used at all in the training portion of the workshop.

To assess knowledge, importance, and confidence, we asked participants to complete paper-based pre- and postworkshop evaluations (see Supplemental Digital Appendix 1 at https://links.lww.com/ACADMED/A332). We chose to assess participants’ knowledge with two different types of questions. We assessed participants’ knowledge of facial expression features using four fact-based, multiple-choice questions (e.g., If a patient has lowered eyebrows and a raised upper lip, it would be most consistent with which emotion?), and participants’ knowledge of facial expression features in the context of patient care using four case-based, multiple-choice questions (e.g., A patient in the hospital gets sicker because he is given the wrong medicine. When you inform him of the error, you note his lips are stretched horizontally and his eyebrows are raised and drawn together. What emotion is he most likely experiencing?). We assessed participants’ attitudes about importance using two questions in which participants rated the perceived importance of the skills taught on a five-point Likert-type scale, where 1 = not at all important and 5 = extremely important. We assessed participants’ confidence using three questions in which participants rated their confidence in the skills taught on a five-point Likert-type scale, where 1 = not at all confident and 5 = extremely confident. On the postevaluation, participants rated how much they agreed (from strongly disagree to strongly agree) that the workshop would help them identify emotions in their patients and that it would help them manage difficult patient encounters.

The postevaluation also included demographic questions, including a question about prior facial expression training. The only information on the evaluations identifying the participants was the last five digits of their social security number, which was used to pair pre- and postworkshop evaluations. After the pre- and postevaluations were paired (by J.W.R.), this number was replaced with a participant identification number.

Approval to conduct this evaluation was obtained from the University of Pittsburgh institutional review board.

Analysis

We used summary statistics to describe the participants’ demographic characteristics. We calculated the mean percentage correct for the 10-photograph skill test, the four fact-based knowledge questions, and the four case-based knowledge questions, and compared pre- and postevaluation means using a paired Student t test. We created a composite rating for the two importance questions and a composite rating for the three confidence questions, and compared pre- and postevaluation means for these ratings using a paired Student t test. In the case of nonparametric data, we calculated the median with pre- and postevaluation comparisons using a Wilcoxon signed-rank test.

To determine the optimal groups to receive this workshop, we conducted a subgroup analysis to assess for baseline differences in ability and for differential effectiveness of the workshop. For baseline differences, we used Student t test and Wilcoxon signed-rank test to compare preevaluation scores according to gender, current role (i.e., faculty, resident, medical student), and year of medical school training. For differential effectiveness, we used Student t test and Wilcoxon signed-rank test to compare improvement from pre- to postevaluation scores according to gender, current role, and prior facial expression training. To assess whether confidence in the skills taught was correlated with measured performance, we used Spearman rank correlations to assess for an association between preworkshop ratings of confidence and preworkshop performance on measures of skill, fact-based knowledge, and case-based knowledge.

We used SAS 9.3 (SAS Institute Inc., Cary, North Carolina) to perform the statistical analysis.

Results

A total of 156 health care providers participated in the workshop and had both pre- and postworkshop data available for analysis. Of these participants, 64 (41.0%) were faculty (with a median of 11.0 years in practice), 4 (2.6%) were residents, and 85 (54.5%) were medical students; 3 (1.9%) did not specify their role (Table 1). Additional characteristics are listed in Table 1.

Table 1: Demographic Characteristics of Facial Expression Workshop Participantsa

Participants’ skill in correctly identifying the emotion portrayed in photographs increased substantially from 54.7% correct before the workshop to 79.6% correct after (P < .001) (Figure 2). For knowledge questions, participant scores increased substantially on fact-based questions, from 30.4% before the workshop to 69.9% after (P < .001), as well as on case-based questions, from 36.5% before the workshop to 73.1% after (P < .001). Participants’ importance ratings increased modestly from 4.5 before the workshop to 5.0 after (P = .02). Their confidence ratings also increased modestly from 3.1 before the workshop to 3.6 after (P < .001).

Figure 2: Multiple learning domain measures of effectiveness (shown via pre and post scores and ratings) of a workshop developed by the authors in May 2011 to teach physicians and physicians-in-training how to identify facial expression features and how to use that skill in the context of patient care. Skill was measured with a 10-photograph skill test, and fact- and case-based knowledge were measured using multiple-choice questions. Importance and confidence were measured by self-reported ratings (1 = not at all important or confident to 5 = extremely important or confident). A single asterisk indicates P < .001, and double asterisks indicate P = .02.

At the conclusion of the workshop, 149 (95.5%) participants agreed or strongly agreed that the training would help them identify emotions in their patients, and 119 (76.3%) agreed or strongly agreed the training would help them manage difficult patient encounters.

Using scores from the preevaluation, we compared baseline measures by subgroup. We found no differences between women and men in baseline measures of skill, knowledge, or confidence (data not shown). Women rated the importance of the skills taught slightly higher than men did (median 5.0 vs. 4.5, P = .04). We found no differences between faculty and medical students in baseline measures of skill, knowledge, or importance (data not shown). Faculty reported a slightly higher confidence in their skills at baseline compared with medical students (mean 3.3 vs. 3.0, P = .002). We found no difference between first-year and fourth-year medical students in any measure at baseline (data not shown). Residents were included in the overall analysis but were not analyzed as a separate group given their low numbers.

Using the difference between pre- and postworkshop evaluation scores, we compared the change in measures by subgroup. Men and women improved similarly in skill, knowledge, importance, and confidence (data not shown). When faculty were compared with medical students, medical students’ skill (improvement of 28.2% vs. 21.7%, P = .04) and confidence (improvement of 0.5 vs. 0.4, P = .046) improved more than faculty’s, but both groups improved similarly in knowledge and importance (data not shown). When we compared scores according to prior training in facial expressions, those without prior training improved more in both measures of knowledge (fact based: improvement of 41.3% vs. 22.1%, P = .01; case based: improvement of 38.0% vs. 22.1%; P = .02). Participants with and without prior training had similar improvement in skill, importance, and confidence (data not shown).

Using the preworkshop evaluation scores, we found no correlation between baseline ratings of confidence in these skills and baseline measures of skill (ρ = 0.11, not significant), fact-based knowledge (ρ = −0.04, not significant), or case-based knowledge (ρ = 0.09, not significant) (data not shown).

Discussion

By evaluating this workshop using multiple measures (i.e., skill, knowledge, importance, and confidence), we have shown that this training was effective in multiple learning domains. Participants improved considerably in the skill of recognizing the emotion contained in facial expression photographs and in their knowledge (both fact based and case based) of individual facial expression features. We found only modest improvement in ratings of importance of the skills taught, likely due to high preworkshop ratings. The high value placed on the skills at baseline may have reflected the bias of the venues in which the workshop was presented and/or the self-selection of the participants. We also found only modest improvement in ratings of confidence despite considerable changes in skill and knowledge. This may result from the challenge of self-assessment, a high expectation that participants had for their abilities in order to feel confident, and/or a lack of opportunity to practice these skills over time and build confidence through repetition. Although we did not attempt to measure the workshop’s effects on actual doctor–patient communication, we believe the skills will likely improve such communication based on participants’ considerable improvement in multiple measures combined with their belief that these skills will help them manage difficult patient encounters.

Baseline ratings of confidence did not correlate with any baseline measure of ability. These data are not surprising given that multiple studies have shown that physicians’ self-assessment of abilities does not correlate well with actual performance.¹⁷ Similarly, our data support that confidence in the ability to interpret facial expressions is not a good indicator of actual ability. This finding could be useful in convincing providers who feel confident in their skills that they could still benefit from facial expression training. We also found that participants who had received prior training in facial expressions improved in skill just as much as those without prior training (though they did improve less in both measures of knowledge). Taken together, our data support that this type of training is effective even for confident, experienced providers.

When comparing men and women, we found no evidence that either group was more in need of this type of training or benefited more from it. We also found no difference between faculty and medical students in any measure of baseline ability, indicating that these skills may not be learned in a robust way during routine clinical practice. This was true despite faculty participants having a median of 11 years in practice. On the basis of these data, dedicated training such as this appears to be necessary if providers are going to become adept at recognizing emotion in their patients’ faces. We did not present the workshop to enough residents to permit direct comparison with that group, but we have no reason to believe they would perform differently than medical students or faculty as their level of training falls between these two groups.

Currently, there are limited opportunities for health care providers to learn how to interpret facial expressions,^18,19 and most of the available opportunities are time consuming and/or expensive. We designed this workshop to provide a free opportunity for faculty and students to learn these skills in a relatively short period of time. We chose to rigorously evaluate the workshop to see whether this format could be equally effective for multiple subgroups of providers. One strength of the study is that it included physicians and physicians-in-training representing a range of ages and clinical experience, which supports the generalizability of our findings. Although effects on patients were not measured, we evaluated the training using multiple learning domains, all with consistent results. The most relevant measure, the ability to recognize the emotion represented in facial expression photographs, was assessed using images which were rigorously selected and not used elsewhere in the workshop to avoid biasing the results.

One limitation of this study is that we used a convenience sample of participants from the various venues in which the workshop was presented. Because participation was voluntary, participants may have already had an interest in this area. Another limitation is that a majority of the faculty were from internal medicine, so it is not known how faculty in other disciplines would perform. In addition, the study did not look at actual effects on doctor–patient communication or patient satisfaction. Lastly, an assessment for retention of these skills over time was not performed because of the nature of the venues in which the workshop was presented.

We were able to demonstrate that physicians and physicians-in-training could effectively learn to recognize emotion by interpreting facial expressions through a short (90-minute) workshop. This training could be applied to multiple medical education arenas, such as training medical students to be aware of emotional responses as they interview patients or in faculty development to help experienced providers identify subtle emotional responses. Recognizing emotion in patients is an important first step toward being able to address that emotion in the context of clinical care. In future efforts, it will be important to study whether participants can perform similarly with standardized patients and whether they can recognize and respond to facial expressions in the patients they are caring for. Though interpreting facial expressions is an important component of nonverbal communication, future endeavors should examine this ability in the context of other types of nonverbal communication (e.g., the interpretation of body language or verbal tone) to see if providers might benefit from one type of nonverbal communication training more than another or from multiple types in concert. This report outlines a novel way to teach providers how to interpret the facial expressions of their patients. This is one important way in which nonverbal communication skills could be used to provide care in a more empathic and humanistic way.

Acknowledgments: The authors would like to thank the individuals who helped arrange these workshops: Felise Milan, MD, and Dan Myers, MSW, at the Albert Einstein College of Medicine; Megan McNamara, MD, MSc, at the Louis Stokes Cleveland VA Medical Center; Melissa McNeil, MD, MPH, at the University of Pittsburgh School of Medicine; and the program planning committee for the Association of Program Directors in Internal Medicine. The authors would also like to thank Jim Bost, PhD, who helped with the statistical planning, and Patrice Gibbs, MS, who helped with the statistical analysis.