Communication skills are often not taught explicitly in residency training.1,2 Although the impact of lapses in communication is high, the obstacles to teaching communication skills remain significant.3 There have been efforts to develop formal teaching programs in communication skills, but the need to teach these skills remains largely unaddressed in procedural specialties such as anesthesiology.1,4,5 A prominent theme in the literature suggests including multisource feedback, particularly patient feedback, in programs aiming to enhance these skills.6–8
Our long-term goal is to enhance anesthesia residents’ communication skills during their training. We therefore adapted the Four Habits Coding Scheme (4HCS), developed by Krupat et al.,9 into a patient survey to rate anesthesiology residents’ communication skills. We used this survey to develop a curriculum in communication for anesthesia residents and to detect any changes in patient feedback after the curricular intervention.
Consent for Publication
We obtained IRB approval for exempt status for this project from the Committee on Clinical Investigations at Beth Israel Deaconess Medical Center (BIDMC). The Committee on Clinical Investigations approved of a waiver of informed consent from patients and a verbal consent script for residents. All residents consented to the publication of this case report. All data for this project were deidentified.
We adapted the 4HCS into an ambulatory surgical patient survey for patients to provide feedback on anesthesia residents’ communication skills.a,9,10 After administering the survey for 4 months, we designed and implemented a simulation and web-based curricular intervention in early 2013 (Fig. 1). After the intervention, we administered the survey for 3 months and analyzed the survey data. We mailed surveys (one for each resident–patient encounter) 1 to 2 weeks after patient discharge from the hospital.
All ambulatory surgical patients who received care from an anesthesia resident at BIDMC during the survey data collection periods (920 patients in the preintervention period and 689 patients in the postintervention period) were invited to complete the survey. All 55 anesthesia residents at BIDMC were invited to complete the curriculum. Residents who cared for ambulatory surgical patients during both the preintervention and postintervention data collection periods (38 residents) were invited to be included in the data analysis if they completed the curriculum; they were informed that their standings and evaluations in the residency program would not be affected.
We modified the 4HCS into our patient survey using an iterative process (Table 1). First, we identified a group of experts in perioperative education with experience in feedback and communication skills as demonstrated by at least 3 of the following criteria:
- Five years or more of teaching experience;
- Active in organizing of didactics/teaching (>5 sessions per year);
- Participation and presentation in national educational conferences;
- Active involvement in educational research;
- Successful completion (or teaching) of an intensive feedback training course and at least 1 live or online training module in providing feedback on communication skills; and/or
- Participation in an advanced-level educational training course or an advanced degree in an education-related field.
Using a modified Delphi technique, the experts reduced the 23 items from the 4HCS to 20 questions related to an ambulatory surgical patient’s encounter with his or her anesthesia resident. After cognitive interviewing of patients, we mailed the first version of the survey, which consisted of these 20 questions, to 1158 ambulatory surgical patients at BIDMC who received care from anesthesia residents from February 2012 to April 2012.
Because of the low response rate (17.8%), we refined the survey after statistical consultation and expert reassessment into a second version, for which the experts reduced the 20 questions to 10 questions and reworded 1 question (question 12 in the first version [“Did the anesthesia resident encourage you to express any emotions that you felt?”] was changed to question 5 in the second version [“Did the anesthesia resident provide ample opportunity for you to express your emotions and was he or she receptive to them if you did?”]). We then mailed the second version of the survey to 192 ambulatory surgical patients at BIDMC who received care from anesthesia residents in July 2012. Our response rate increased to 30.7%, and responses to 9 of the 10 questions in version 2 were not significantly different from the responses to the same respective questions in version 1. The question that had significantly different responses in the 2 versions was the question that was reworded for version 2. On the basis of expert review, we reworded the question to “Did the anesthesia resident provide ample opportunity for you to express your emotions?” for the final version of the survey. This final version was used for data collection in this pilot study.
The Four Habits structure was maintained throughout all versions of the survey. The survey questions asked patients to rate aspects of residents’ communication skills as “definitely yes,” “somewhat yes,” “somewhat no,” and “definitely no” (Appendix 1, Survey to assess anesthesia resident communication skills).
After collecting survey data for 4 months, we analyzed the results to identify areas for improvement in residents’ communication skills. We selected 3 areas as the focus for curricular development: patients’ emotions, shared decision-making, and setting of expectations for the recovery room.
We designed a curriculum consisting of 3 simulated preoperative interviews between the resident and patient. The 3 scenarios, which reinforced the tenets of the Four Habits Model, reflected common issues in communication (Table 2). We presented the scenarios during 2 1-hour live teaching and discussion sessions. Residents at these sessions took part in or observed the simulated scenarios and participated in a large group discussion facilitated by experts. The scenarios were recorded (Supplemental Digital Content, Supplemental Video, which is a scene where the resident explains a nerve block to the patient, http://links.lww.com/AACR/A55) and were the basis for an online module. The online module consisted of videos of the recorded scenarios and a series of reflective open-ended questions covering the same topics as the discussion session for each scenario (Appendix 2A, Screenshots of online module for scenario 3).
The 55 residents in our program were invited to complete the curriculum by attending the live sessions, completing the online module, or both. After reviewing responses to the questions in the online module, we selected 8 sets of “best practice” responses for overall quality and shared them via e-mail with all residents to further enhance awareness and reflection on communication (Appendix 2B, Sample of best practice responses for scenario 3 from 2 residents).
The response rates for the preintervention and postintervention surveys were 233 of 920 (25%) and 236 of 689 (34%), respectively. Of the 38 residents who had survey data in both data collection periods, 30 completed the curriculum and were included in the data analysis. After removing blank surveys, we had 159 preintervention surveys (Cronbach α: 0.94, 95% one-sided confidence interval [CI], 0.93) and 152 postintervention surveys (Cronbach α: 0.95, 95% one-sided CI, 0.93) in total for these residents. We analyzed the deidentified data as follows:
- We coded responses on the surveys with the following rating scale: definitely yes = 4, somewhat yes = 3, somewhat no = 2, and definitely no = 1.
- We calculated an overall survey rating for each survey by averaging the responses on the survey.
- For each resident, we calculated his or her mean overall survey rating in each (preintervention and postintervention) data collection period.
- We compared the residents’ preintervention mean overall survey ratings with their postintervention ratings using the Mann-Whitney U test and Wilcoxon-Mann-Whitney odds (WMWodds) (the WMWodds was calculated using the method described by Divine et al.).11
Figure 2 shows the distribution of the residents’ preintervention mean overall survey ratings (median, 3.64; interquartile range, 3.42–3.83) in comparison with their postintervention ratings (median, 3.80; interquartile range, 3.68–3.90). The residents’ postintervention ratings were higher than their preintervention ratings (P = 0.03; WMWodds: 1.97; 95% CI, 1.05–4.35).b The WMWodds were attributable principally to questions 2 and 6 on the survey (Table 3).
Despite the high ratings of resident performance before the curricular intervention, residents’ mean overall survey ratings modestly increased. The probability that a resident in the postintervention period had a higher mean overall survey rating than a resident in the preintervention period was 66%.
Providing a resident with feedback on his or her performance is critical for achieving progression through the milestones related to communication. Furthermore, under the Next Accreditation System paradigm, concrete data are needed to document the progression of competencies.12 Patient feedback on the quality of care is a source of input that is becoming increasingly valued in health care and medical education.13,14
Some use has been made on patient satisfaction surveys to evaluate resident communication skills based on patient feedback but largely in an outpatient clinic setting and not for procedural specialties.6,8 Our survey proved useful in the ambulatory surgical setting in assessing residents’ communication skills from the patients’ perspectives. Furthermore, it helped to identify areas of improvement that guided curriculum development.
Simulation and web-based modules have both been used to teach communication skills, but outcome measures are seldom related directly to patient care.15–17 We implemented a curriculum using both simulation and a web-based module and used a patient survey to detect any changes in patient care after the curriculum was implemented.
In this pilot study, we note the following limitations, which have led to opportunities for improvements in future studies:
- Patient demographics: We designed an anonymous survey with no demographic questions because we wanted to analyze overall patient satisfaction for each resident and did not aim to categorize patients based on medical records or demographics in our analysis. However, in doing so, we were not able to investigate any potential bias in patient responses.
- Sample size and selection bias: Our small sample size limited the power of our analysis and our ability to generalize the results. Resident schedules determined how many residents did ambulatory surgical cases during the study and therefore limited the number of residents included in the analysis. We did not collect demographic data on the residents and therefore were not able to formally compare residents who participated in the curriculum with those who did not. However, because schedules of residents were the primary driver for participation, we do not suspect that selection bias was an important influence on our findings. Future studies should focus on collecting resident demographics, ensuring that all residents complete the intervention, and working with schedulers to allow more residents to have ambulatory surgical cases during the study period.
- Response rates: Because of our low response rate, we cannot ensure that the respondent population represented the general ambulatory surgical population. As such, these results, although suggestive of an important relationship, should still be interpreted with caution. In addition, our patient response rates (25%–35%) were low but comparable with other mailed healthcare surveys such as the Hospital Consumer Assessment of Healthcare Providers and Systems survey.18
- Teaching modalities: Because of the different schedules and preferences of residents, not all residents attended the live simulation sessions for the curriculum. As an alternative, we allowed them to complete the curriculum online by watching the simulations and responding to the reflective questions. Although the reflective questions covered the same topics as the discussions from the live sessions, the learning experience from the live sessions was likely not exactly the same as the learning experience from the online module. Future studies should focus on making the learning experience more consistent across modalities for residents.
Overall patient satisfaction with residents’ communication skills improved after our curricular intervention. Although other factors may have contributed to this change, this pilot study paves the way for future research in assessing the curriculum’s effectiveness and the reliability of our results and in exploring the generalizability of the survey and curriculum to other procedural specialties seeking to improve the quality of resident interactions with patients during short episodes of care.
APPENDIX 1: SURVEY TO ASSESS ANESTHESIA RESIDENT COMMUNICATION SKILLS
APPENDIX 2: SCRENSHOTS AND “BEST PRACTICE” RESPONSES FROM ONLINE MODULE
The authors thank Emily Shafer, PhD, Ariel Mueller, MA, and Jennifer Massa, PhD, for their assistance in the data analysis; Amy Sullivan, EdD, Richard Schwartzstein, MD, and the Center for Education at Beth Israel Deaconess Medical Center (BIDMC) for their support and guidance; Brendan Lutz, BS, for his assistance in the preparation of the manuscript; and the BIDMC Department of Anesthesia, Critical Care and Pain Medicine Center for Anesthesia Research Excellence (CARE), residents, and leadership for their participation in and support of the study.
a Developed by Krupat et al.9 in 2006, the 4HCS uses a model of optimal communication between physicians and patients known as the Four Habits Model. The Four Habits as developed are (1) invest in the beginning [of an encounter]; (2) elicit the patient’s perspective; (3) demonstrate empathy; and (4) invest in the end [closure]. The 4HCS is composed of 23 items derived from these 4 core skills. Krupat et al. validated the 4HCS as an instrument to rate a physician’s communication with patients in an outpatient primary care population.9,10
b The odds that a resident randomly selected from the postintervention period had a higher mean overall survey rating than a resident randomly selected from the preintervention period were 1.97 to 1 (probability of 0.66). The lower limit of the 95% confidence interval of the WMWodds was 1.05 to 1 (probability of 0.51); the upper limit was 4.35 to 1 (probability of 0.81).
1. Hulsman RL, Ros WJ, Winnubst JA, Bensing JM. Teaching clinically experienced physicians communication skills. A review of evaluation studies. Med Educ. 1999;33:655–68
2. Makoul G. MSJAMA. Communication skills education in medical school and beyond. JAMA. 2003;289:93
3. Rhoton MF, Barnes A, Flashburg M, Ronai A, Springman S. Influence of anesthesiology residents’ noncognitive skills on the occurrence of critical incidents and the residents’ overall clinical performances. Acad Med. 1991;66:359–61
4. Hagemeier NE, Hess R Jr, Hagen KS, Sorah EL. Impact of an interprofessional communication course on nursing, medical, and pharmacy students’ communication skill self-efficacy beliefs. Am J Pharm Educ. 2014;78:186
5. Waisel DB, Lamiani G, Sandrock NJ, Pascucci R, Truog RD, Meyer EC. Anesthesiology trainees face ethical, practical, and relational challenges in obtaining informed consent. Anesthesiology. 2009;110:480–6
6. Brinkman WB, Geraghty SR, Lanphear BP, Khoury JC, Gonzalez del Rey JA, Dewitt TG, Britto MT. Effect of multisource feedback on resident communication skills and professionalism: a randomized controlled trial. Arch Pediatr Adolesc Med. 2007;161:44–9
7. Liu GC, Harris MA, Keyton SA, Frankel RM. Use of unstructured parent narratives to evaluate medical student competencies in communication and professionalism. Ambul Pediatr. 2007;7:207–13
8. Brinkman WB, Geraghty SR, Lanphear BP, Khoury JC, Gonzalez del Rey JA, DeWitt TG, Britto MT. Evaluation of resident communication skills and professionalism: a matter of perspective? Pediatrics. 2006;118:1371–9
9. Krupat E, Frankel R, Stein T, Irish J. The Four Habits Coding Scheme: validation of an instrument to assess clinicians’ communication behavior. Patient Educ Couns. 2006;62:38–45
10. Fossli Jensen B, Gulbrandsen P, Dahl FA, Krupat E, Frankel RM, Finset A. Effectiveness of a short course in clinical communication skills for hospital doctors: results of a crossover randomized controlled trial (ISRCTN22153332). Patient Educ Couns. 2011;84:163–9
11. Divine G, Norton HJ, Hunt R, Dienemann J. Statistical grand rounds: a review of analysis and sample size calculation considerations for Wilcoxon tests. Anesth Analg. 2013;117:699–710
12. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system–rationale and benefits. N Engl J Med. 2012;366:1051–6
13. Elwyn G, Buetow S, Hibbard J, Wensing M. Measuring quality through performance. Respecting the subjective: quality measurement from the patient’s perspective. BMJ. 2007;335:1021–2
14. Gran SF, Braend AM, Lindbaek M. Triangulation of written assessments from patients, teachers and students: useful for students and teachers? Med Teach. 2010;32:e552–8
15. Blum RH, Raemer DB, Carroll JS, Dufresne RL, Cooper JB. A method for measuring the effectiveness of simulation-based team training for improving communication skills. Anesth Analg. 2005;100:1375–80
16. Ponton-Carss A, Hutchison C, Violato C. Assessment of communication, professionalism, and surgical skills in an objective structured performance-related examination (OSPRE): a psychometric study. Am J Surg. 2011:1–8
17. Oliven A, Nave R, Gilad D, Barch A. Implementation of a web-based interactive virtual patient case simulation as a training and assessment tool for medical students. Stud Health Technol Inform. 2011;169:233–7
Supplemental Digital Content
© 2016 International Anesthesia Research Society