Interprofessional teamwork is recognized as essential for quality patient care,1 a notion which has led to the formulation of interprofessional competencies2 and widespread implementation of interprofessional education (IPE) programs.3 One competency domain is interprofessional communication, which includes “giving timely, sensitive, instructive feedback to others about their performance on the team, [and] responding respectfully as a team member to feedback from others.”2 Studies have shown that intrateam feedback improves team performance,4–6 which, in interprofessional settings, requires team members to be receptive to feedback from other professionals. Little is known about acceptability of interprofessional feedback between health care professionals, and how to introduce giving and receiving such feedback into IPE curricula.
Social identity theory conceptualizes human behavior as having both interpersonal and intergroup aspects, with social identity defined as a person’s self-concept derived from perceived membership in a particular group.7 From this framework emerges a prediction that tensions between different professional groups may create unique challenges for interprofessional feedback. Contributing factors include hierarchy, professional identity, stereotypes, societal expectations, and attitudes toward collaboration.8,9 Indeed, the implementation of team training programs in aviation and the military that incorporate intrateam feedback has hit barriers due to power differences among team members.10 Nonetheless, such programs have been adapted for health care teams, where similar barriers exist.11,12 In several countries, feedback from other professions has become an integral part of physicians’ (continuing) education, embedded in multisource feedback programs.13,14 Such programs are generally well received, but most reports do not separate findings regarding feedback within and outside the profession. A few studies indicate that not all physicians are receptive to feedback from nurses or act on such feedback.15,16 To our knowledge, whether the reverse is true and how receptiveness to feedback plays out among other health care professionals and at varying levels of training has received limited attention thus far.
Thus, much remains to be clarified about the acceptability of interprofessional feedback in the health professions before it can be effectively integrated into interprofessional teamwork and education. We undertook the current study as a first step in clarifying these issues by examining health professions students’ perceptions of learner-to-learner feedback after an interprofessional team exercise. In particular, we analyzed whether students found it challenging to give students from other professional backgrounds feedback, whether they perceived feedback they received from different professional groups as more or less useful or positive, and whether there was a relationship between how they perceived the feedback and whom they guessed the feedback was from. We hypothesized that students would demonstrate bias in their ratings of feedback from other students dependent on which health professions school they believed the feedback provider to be from.
This prospective descriptive cohort study examined students’ perceptions of feedback they received from team members following an interprofessional team exercise.
Participants and setting
Students from seven health professions education programs at two institutions in San Francisco participated in an Interprofessional Standardized Patient Exercise (ISPE) early in their health professional education (during year 1 or 2 of clinical training). Participants included students from five professional schools at the University of California San Francisco (UCSF) (medicine, pharmacy, nursing, dentistry, and physical therapy) and from the San Francisco State University dietetics and social work programs. All 355 students who participated in the ISPE during the fall/winter of 2012 were eligible for the study.
Context of feedback comments
In the ISPE, described in detail elsewhere,17 students worked in small teams during a half-day session to plan, execute, and summarize an encounter with a standardized patient who has a complex medical history. The exercise started with a group discussion to divide tasks and create a plan for each team member’s responsibilities in the patient encounter. Team members then took turns conducting their part of the patient interview and examination, observed by other team members. In the final part of the exercise, team members collaborated to generate a written assessment and plan. At the start of the exercise, all students received instructions about providing anonymous feedback to all team members, which they were told should be specific, be targeted at behaviors, and contain suggestions for improvement. After the exercise, students wrote separate feedback comments on interviewing skills and teamwork skills for each team member. All feedback was collected anonymously (based on ISPE faculty leadership preference) through an online survey instrument (Qualtrix). The health professions school of each feedback provider could be tracked through a unique identifier associated with each survey. Of note, the study investigator who distributed the feedback and associated surveys was not associated with the ISPE or any other curricular activity for any of the students in the study.
Immediately after students completed all feedback, a separate online survey question asked them to rate their agreement with the statement “Giving feedback to the students on my team from other professions was challenging” (4-point scale; 1 = strongly disagree to 4 = strongly agree).
Next, we created a new survey for students to rate the feedback they received from their team members. We collated deidentified comments from team members for each student from the Qualtrix survey and imported these into a different online survey instrument (SurveyMonkey). In this survey, each comment was followed by three questions: “Please rate your agreement with the statement: This feedback will help me improve my performance (1 = strongly disagree to 5 = strongly agree)”; “How do you perceive this feedback? (1 = very negative to 5 = very positive)”; and “Whom do you think this feedback is from: give us your best guess?” (checkbox options provided to represent participating students’ professions). In creating this survey, we grouped comments about interviewing skills separately from those about teamwork skills and presented interviewing skills comments first. We presented comments from different team members in random order.
We sent students an e-mail link to the feedback comments two to four weeks after the ISPE, followed by weekly reminders until they had accessed the feedback or until four reminders had been sent. The UCSF committee on human research approved the study. Participation in the study was voluntary, and students were told that they could access the feedback without answering any survey questions. No incentives were provided.
To compare access and completion rates between groups of students from different schools, we performed chi-square analyses. To compare the perceived challenge of providing interprofessional feedback, we performed a one-way between-subjects analysis of variance (ANOVA) with feedback provider profession as the independent variable and “challenge” ratings as the dependent variable.
To analyze recipient perceptions of the feedback, we performed two sets of analyses, one using the teamwork skills feedback and a second using the interviewing skills feedback. For each of these sets of analyses, we first calculated the percentage of feedback comments for which students correctly identified the source. Then for each of the dependent variables (usefulness and positivity), we performed mixed-design two-way ANOVAs using recipient profession as a between-subjects variable and, in separate analyses, either actual or guessed provider profession as a within-subjects variable. For statistically significant findings, we identified differences between individual groups using S-N-K and Tukey post hoc analyses. Finally, we performed Pearson correlations to examine relationships between ratings of usefulness and positivity of feedback. The small numbers of dietetics and social work students limited our ability to perform statistical analyses on those groups, so we excluded them from all analyses. We used SPSS statistical software, version 21 (IBM SPSS Inc., Armonk, New York) for all data analysis. Level of significance was set at P = .05.
Participants and survey completion
The 355 students who participated in the ISPE did so in 103 groups of 3 or 4 students each. Group composition was variable depending on the ISPE schedule. Sixteen groups had 2 medicine students and three had 2 pharmacy students; all other groups had at most 1 representative per profession. Because of a technical error with the survey instrument, students with 2 medicine or 2 pharmacy students on their team could only leave feedback comments for 1 of the 2. Table 1 summarizes study participation by school. A total of 295 (83%) students provided feedback comments on both interviewing and teamwork skills for all their team members. The 60 students who did not leave feedback for any (n = 8) or some (n = 52) of their team members were evenly distributed across different professions (χ24 = 6.28; P = .18). Two of the students who participated in the ISPE did not receive any feedback comments from their peers and were thus ineligible for the next study phase. Ten students (4.5% of those who completed the survey) did not attempt to guess the source of one or more feedback comments.
Feedback providers’ perceptions about giving interprofessional feedback
Students rated providing feedback to team members as moderately challenging, with an average rating of 2.35 (standard deviation = 0.59) out of 4 (2.5 is the midpoint on the scale). The one-way ANOVA revealed no significant differences between professions (F6,346 = 0.27; P = .951).
Feedback recipients’ perceptions of usefulness and positivity of feedback
Students identified the feedback source correctly for 255 of 547 (46.6%) interviewing skill comments and for 248 of 511 (48.5%) teamwork skill comments. It is uncertain how this compares to chance performance because some students had two other team members (where chance would be 50%) and others had three other team members (where chance would be 33%). We therefore did not perform any statistical analysis on these data, but we can estimate that students identified the correct provider profession at only slightly higher than chance levels.
Figure 1 summarizes the mean ratings of usefulness and positivity of feedback in the interviewing and teamwork domains by profession of feedback recipient (Panel A), by actual profession of the feedback provider (Panel B), and by guessed profession of feedback provider (Panel C). Supplemental Digital Tables 1 and 2, http://links.lww.com/ACADMED/A312, provide detailed information with descriptive statistics for each group. In general, students saw the feedback as useful and positive (all averages exceeding 3.66 out of 5). Two-way ANOVAs using the actual profession of the feedback provider as a within-subjects factor revealed no significant interactions between provider profession and recipient profession for any of the dependent variables (see Supplemental Digital Table 3 at http://links.lww.com/ACADMED/A312). There were statistically significant main effects of recipient profession with respect to their perceptions of the positivity of the feedback they received on both their interviewing and teamwork skills. Post hoc tests indicated that physical therapy students rated the feedback as significantly more positive than medicine, nursing, and dentistry students. Dental students rated the feedback as significantly less positive than medicine, pharmacy, and physical therapy students. No other pairwise comparisons differed statistically. We also found a statistically significant main effect of recipient profession regarding perceptions of usefulness of feedback on teamwork skills. On post hoc analysis, pharmacy and physical therapy students rated the usefulness of feedback on teamwork skills significantly higher than did medicine and nursing students.
The pattern of data was similar when using the guessed rather than the actual profession of the feedback provider (Supplemental Digital Table 4 at http://links.lww.com/ACADMED/A312). That is, the two-way ANOVAs revealed a significant main effect of feedback recipient but no significant main effect of guessed profession of feedback provider and no significant interactions between guessed provider profession and recipient profession. Further, feedback ratings were not significantly different for cases in which students guessed the source correctly and those in which they guessed incorrectly (Table 2).
Students’ usefulness ratings of feedback correlated with their perception of sign, indicating that more positive feedback was seen as more useful (r = 0.33 for interviewing skills; r = 0.62 for teamwork skills; P < .001).
Our study of interprofessional feedback among health professions students suggests that students generally have a positive attitude toward receiving feedback from other team members even though they find it somewhat challenging to give such feedback. Overall, the students in our study perceived the feedback from peers as useful and positive, with only small differences in perceptions between students from different schools. Our data did not reveal evidence for bias against the feedback received from any group; students saw feedback as useful and positive regardless of who provided it (or who they guessed provided it).
These findings contradict our hypothesis, grounded in social identity theory, that students would favor feedback from students of one professional background over others. It is possible that the students in our study simply did not differentiate between the feedback provided by different groups, consistent with the observation that students had trouble guessing the correct source of the feedback. Such lack of differentiation might have occurred because our participants were too early in their training to have sufficient knowledge about other health care professional students or to have adopted a professional identity of their own. To students who are still forming their own identity, the skills on which they gave each other feedback (interviewing and teamwork) may not vary much by professional role.
Another possible explanation for the lack of difference in feedback ratings across groups is that students may have seen themselves and their team members as part of the same group, the ISPE team. This explanation is consistent with the social identity framework, which states that self-categorization to a particular group may vary by context. In different contexts, different group identities may be more or less salient, dependent on accessibility. Accessibility is defined as “the availability of the group identity to an individual as determined by his or her current goals, task orientation and available contextual cues.”7 The students in our study provided each other with feedback after they had worked in their teams through a complex exercise, which may have made the group identity of the team more accessible than the individuals’ professional identity (especially given the students’ stage of training). This resonates with observations by others that health care professional students have positive attitudes toward IPE despite the presence of stereotypes.18,19 If formation of a group identity at the team level indeed explains our findings, this would reinforce calls for IPE. The teams in our study, however, were formed for the purpose of a short educational session only. How this plays out in other health care teams consisting of more seasoned clinicians is unclear.
Lastly, it is possible that the overall high ratings of the feedback eliminated the ability to discriminate between groups. Students may have rated the feedback as useful and positive because the feedback they received was indeed useful and positive or because they have insufficient experience with receiving and/or evaluating feedback. We did not analyze the actual content of feedback provided, but we found a modest correlation between usefulness and positivity ratings. This may indicate that students mostly liked the feedback received because it was positively reinforcing. The generally positive comments may have occurred because of social desirability, a confounder identified in studies measuring attitudes toward IPE.20,21 Several aspects of our study design were aimed at mitigating these potentially confounding effects. First, students provided their ratings of feedback through an anonymous survey instrument and were informed that the data would not be shared with feedback providers. Second, the study investigator who distributed the feedback and associated surveys was not associated with the ISPE or any other curricular activity for any of the students in the study. Third, we collected feedback ratings after the ISPE had been completed such that students would not expect to reconvene with the same groups and, hence, might feel more comfortable providing honest opinions.
Our study has important limitations. First, approximately one-third of the students did not access the feedback and did not rate the feedback they received, which could indicate that they had a less favorable attitude toward interprofessional feedback. In particular, only half of the dentistry students accessed the feedback, and this group rated the positivity of feedback lower than other groups of students. In concordance with these findings, a study looking at readiness for interprofessional learning found that dentistry students had the lowest scores.22 This could indicate that dentistry students are less interested in IPE in general or, alternatively, were merely less engaged with the particular IPE activity offered. However, the positivity ratings provided by the dentistry students who did participate were quite high, and their usefulness ratings were similar to those of other students, indicating that they did see value in the interprofessional feedback they received. Second, the study focused on learners early in their clinical years. As a result, most students had limited prior experience with teamwork and limited previous exposure to other professions. Therefore, extrapolation of our findings to learners and practitioners with more experience may not be possible. Studies on multisource feedback suggest that physicians are generally open to feedback from others,23–31 but these reports do not differentiate between physicians’ perceptions of feedback from other physicians versus from other health care professionals. Preliminary findings of studies among resident physicians and staff nurses suggest more prominent in-group versus out-group differentiation and less receptivity to interprofessional feedback relative to intraprofessional feedback.32,33 Lastly, we only looked at perceptions of feedback and did not assess the actual quality of feedback, nor did we investigate whether perceptions had any bearing on actual performance improvement as a result of feedback received. The literature suggests that several factors modify the complex relationship between feedback and performance improvement. These modifying factors include the perceived positivity (or negativity) of feedback, the task on which feedback is provided, the self-efficacy of the feedback recipient, and the perceived trustworthiness of the source.34,35 Perceived trustworthiness of the feedback source likely plays a role in interprofessional feedback, in particular in combination with the task on which feedback is given, but literature on how this plays out in the health professions is sparse.
In summary, our findings suggest that early clinical learners have a positive attitude toward interprofessional feedback. Whether students retain this attitude over the course of training and whether positive attitudes translate into actual performance improvement remains to be studied. In addition, our findings indicate that there is room to improve students’ self-efficacy in giving interprofessional feedback. As Bainbridge and Wood36 have articulated, “learning from” others may be one of the more complex aspects of IPE because it is thought to require trust in the other person’s knowledge base and skill set and needs to be facilitated in a context of equal value. If we believe that “learning to give and receive timely, sensitive, and instructive feedback with confidence helps health professionals improve their teamwork and team-based care,”2 we should integrate these principles into IPE curricula and perform the necessary research to understand how to successfully accomplish this.
Acknowledgments: The authors wish to thank Dr. Maria Wamsley and Ms. Laura Bekes for their assistance with data collection for the study.
1. Kohn LT, Corrigan J, Donaldson MS. To Err Is Human: Building a Safer Health System. 2000.Vol 6. Washington, DC: National Academy Press.
2. Core Competencies for Interprofessional Collaborative Practice: Report of an Expert Panel. 2011.Washington, DC: Interprofessional Education Collaborative Expert Panel.
3. Learning together to work together for health. Report of a WHO study group on multiprofessional education of health personnel: The team approach. WHO Technical Report Series 1988. World Health Organ Tech Rep Ser. 1988;769:172.
4. Rasker PC, Post WM, Schraagen JM. Effects of two types of intra-team feedback on developing a shared mental model in command & control teams. Ergonomics. 2000;43:11671189.
5. Willey K, Freeman M. Completing the learning cycle: The role of formative feedback when using self and peer assessment to improve teamwork and engagement.Proceedings of the 17th Annual Conference of the Australasian Association for Engineering Education: Creativity, Challenge, Change. 2006.Auckland, New Zealand: Australasian Association for Engineering Education.
6. Blickensderfer E, Cannon-Bowers JA, Salas E. Beyerlein MM, Johnson DA. Theoretical bases for team self-correction: Fostering shared mental models.Advances in Interdisciplinary Studies of Work Teams. 1997:Vol 4. Greenwich, Conn: JAI Press; 249279.
7. Burford B. Group processes in medical education: Learning from social identity theory. Med Educ. 2012;46:143152.
8. Hudson B. Interprofessionality in health and social care: The Achilles’ heel of partnership? J Interprof Care. 2002;16:717.
9. Reeves S. Using the sociological imagination in the interprofessional field. J Interprof Care. 2011;25:317318.
10. Salas E, Cooke NJ, Rosen MA. On teams, teamwork, and team performance: Discoveries and developments. Hum Factors. 2008;50:540547.
11. Burke CS, Salas E, Wilson-Donnelly K, Priest H. How to turn a team of experts into an expert medical team: Guidance from the aviation and military communities. Qual Saf Health Care. 2004;13(suppl 1):i96i104.
12. Zwarenstein M, Reeves S. Knowledge translation and interprofessional collaboration: Where the rubber of evidence-based care hits the road of teamwork. J Contin Educ Health Prof. 2006;26:4654.
13. Wood L, Hassell A, Whitehouse A, Bullock A, Wall D. A literature review of multi-source feedback systems within and without health services, leading to 10 tips for their successful design. Med Teach. 2006;28:e185e191.
14. Lockyer J. Multisource feedback in the assessment of physician competencies. J Contin Educ Health Prof. 2003;23:412.
15. Wenrich MD, Carline JD, Giles LM, Ramsey PG. Ratings of the performances of practicing internists by hospital-based registered nurses. Acad Med. 1993;68:680687.
16. Lockyer J, Violato C, Fidler H. Likelihood of change: A study assessing surgeon use of multisource feedback data. Teach Learn Med. 2003;15:168174.
17. Wamsley M, Staves J, Kroon L, et al. The impact of an interprofessional standardized patient exercise on attitudes toward working in interprofessional teams. J Interprof Care. 2012;26:2835.
18. Hean S, Clark JM, Adams K, Humphris D, Lathlean J. Being seen by others as we see ourselves: The congruence between the ingroup and outgroup perceptions of health and social care students. Learn Health Soc Care. 2006;5:1022.
19. Tunstall-Pedoe S, Rink E, Hilton S. Student attitudes to undergraduate interprofessional education. J Interprof Care. 2003;17:161172.
20. Stone N. Evaluating interprofessional education: The tautological need for interdisciplinary approaches. J Interprof Care. 2006;20:260275.
21. Hoffman SJ, Harnish D. The merit of mandatory interprofessional education for pre-health professional students. Med Teach. 2007;29:e235e242.
22. Coster S, Norman I, Murrells T, et al. Interprofessional attitudes amongst undergraduate students in the health professions: A longitudinal questionnaire survey. Int J Nurs Stud. 2008;45:16671681.
23. Joshi R, Ling FW, Jaeger J. Assessment of a 360-degree instrument to evaluate residents’ competency in interpersonal and communication skills. Acad Med. 2004;79:458463.
24. Hesketh E, Anderson F, Bagnall G, et al. Using a 360 degrees diagnostic screening tool to provide an evidence trail of junior doctor performance throughout their first postgraduate year. Med Teach. 2005;27:219233.
25. Brinkman WB, Geraghty SR, Lanphear BP, et al. Effect of multisource feedback on resident communication skills and professionalism: A randomized controlled trial. Arch Pediatr Adolesc Med. 2007;161:44.
26. Massagli TL, Carline JD. Reliability of a 360-degree evaluation to assess resident competence. Am J Phys Med Rehabil. 2007;86:845852.
27. Ogunyemi D, Gonzalez G, Fong A, et al. From the eye of the nurses: 360-degree evaluation of residents. J Contin Educ Health Prof. 2009;29:105110.
28. Murphy DJ, Bruce D, Eva KW. Workplace-based assessment for general practitioners: Using stakeholder perception to aid blueprinting of an assessment battery. Med Educ. 2008;42:96103.
29. Meng L, Metro DG, Patel RM. Evaluating professionalism and interpersonal and communication skills: Implementing a 360-degree evaluation instrument in an anesthesiology residency program. J Grad Med Educ. 2009;1:216220.
30. Chandler N, Henderson G, Park B, Byerley J, Brown WD, Steiner MJ. Use of a 360-degree evaluation in the outpatient setting: The usefulness of nurse, faculty, patient/family, and resident self-evaluation. J Grad Med Educ. 2010;2:430434.
31. Garra G, Wackett A, Thode H. Feasibility and reliability of a multisource feedback tool for emergency medicine residents. J Grad Med Educ. 2011;3:356360.
32. Vesel T. Residents’ perceptions of interprofessional feedback. Poster presented at: Association of American Medical Colleges Medical Education Meeting; November 6, 2014; Chicago, Ill.
33. van Schaik S. Interprofessional feedback between nurses and physicians: In-group versus out-group bias. Talk presented at: Association of American Medical Colleges Medical Education Meeting; November 7, 2014; Chicago, Ill.
34. Kluger AN, DeNisi A. The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254.
35. Hattie J, Timperley H. The power of feedback. Rev Educ Res. 2007;77:81112.
36. Bainbridge L, Wood VI. The power of prepositions: Learning with, from and about others in the context of interprofessional education. J Interprof Care. 2012;26:452458.