Secondary Logo

Journal Logo

Research Reports

Facilitated Reflective Performance Feedback

Developing an Evidence- and Theory-Based Model That Builds Relationship, Explores Reactions and Content, and Coaches for Performance Change (R2C2)

Sargeant, Joan PhD; Lockyer, Jocelyn PhD; Mann, Karen PhD; Holmboe, Eric MD; Silver, Ivan MD, MEd, FRCPC; Armson, Heather MD; Driessen, Erik PhD; MacLeod, Tanya MSc; Yen, Wendy MA; Ross, Kathryn MSc; Power, Mary MBA

Author Information
doi: 10.1097/ACM.0000000000000809

Abstract

Giving and receiving feedback is now understood to be a complex process. Central elements include the interaction between the feedback provider and recipient, the recipient’s active role, and the recipient’s focused planning for improvement.1–3 Current perspectives of feedback use a sociocultural lens to understand the relationships among the learner (or recipient), the teacher (or provider), and the environment. Specifically, such approaches explore the feedback dyad, their interactions with each other and with the feedback data, and the influence of context and culture.4–7

Using a sociocultural lens illuminates numerous factors that influence feedback acceptance and use. These factors include, among others, feedback timeliness and specificity, the nature or valence (positive, negative) of the feedback, perceptions of feedback credibility, provider–recipient relationship, context and culture, the recipient’s personal goals, and the recipient’s self-efficacy.8–14 Recipients of performance feedback do not always accept and use the feedback, and, importantly, the feedback can evoke emotional reactions if it disconfirms the recipient’s self-perceptions.8–16 Strong emotional reactions often occur because one’s performance is integrally linked to one’s sense of self; disconfirming feedback can be difficult to accept. Earlier research has demonstrated that physicians receiving performance feedback first compared the feedback with how they saw themselves—that is, with their own self-assessment. If the feedback confirms their self-perceptions, they tend to react more favorably and accept it more readily than if it is disconfirming.13–17 Further study, exploring how physicians and learners self-assess their performance and apply feedback to inform these self-assessments and guide subsequent actions, has confirmed not only the complexity of accepting and using feedback but also the diverse influences on these processes.18–20

One influence on the feedback process is facilitation. Specifically, facilitation of the feedback conversation can have a positive effect on the acceptance and use of feedback.21,22 Facilitation of feedback means actively engaging recipients in a reflective, intentional conversation about their performance and the feedback they receive with the goal of increasing their understanding of both. Physicians receiving multisource feedback found reflection beneficial and empowering, even many months after the feedback had been received. Similarly, residents reported that feedback facilitation can clarify the feedback and enable their acceptance and use of it.2123 Although gaps exist between the provision and use of feedback, such evidence suggests that facilitation can enhance feedback use. The purpose of this study, therefore, was to develop and conduct feasibility testing of an evidence- and theory-informed model for facilitating performance feedback for physicians so as to enhance their acceptance and use of the feedback.

Developing a Feedback Model: Theoretical Perspectives and Evidence

To develop the feedback model, we drew on earlier research that highlights not only the factors that influence giving, receiving, accepting, and using feedback but also perspectives that enable an increased understanding of these influences. Three theoretical and evidence-based approaches appear to have been effective in enhancing feedback acceptance and use in varying contexts: (1) humanist or person-centered approaches which engage the individual in feedback ownership, goal setting, and action planning24; (2) informed self-assessment approaches that facilitate integration of external feedback with self-perceptions of performance18; and (3) the science of behavior change, a framework of psychological domains for assessing and addressing barriers to feedback use and change.25

Humanist or person-centered approaches

Humanism and person-centered approaches draw on the cognitive and behavioral sciences; the goal is to enhance individual self-awareness and engagement. The feedback recipient is a partner in the discussion, not just a receiver of information.24,26 Facilitators guide the recipients’ reflection on the feedback they have received about their performance, their personal goals, their integration of the feedback, and their plans for improvement and development.21,22,26 Facilitators use coaching to increase the recipient’s self-awareness and self-direction; they support the recipient’s personal development and goal achievement. Coaching, a humanist technique, involves both facilitating the feedback recipient’s identification of goals for change and developing a plan to meet them.15,27–30

Informed self-assessment approaches

The second conceptual framework, “informed self-assessment,” recognizes that individuals can use external and internal data (i.e., feedback) to generate an appraisal of their own performance.14,18–20,31 External data take many forms ranging from informal verbal feedback to formal workplace-based assessments. Internal data include self-perceptions of one’s performance and one’s emotional state. It is now evident that external feedback is essential to informed self-assessment, and reflection is frequently integral to feedback acceptance. Context, culture, and relationships, along with individual perceptions and emotional state, all influence the recipient’s acceptance and use of external feedback.

The science of behavior change

Finally, understanding the science of behavior change is critical to facilitating and coaching individual change and improvement in response to external feedback. Multiple individual and social factors influence behavior and change including knowledge, skills, motivation, self-efficacy, environmental elements, and social influences.25,32 Exploring these systematically in the feedback conversation can contribute to performance change.

Using these three perspectives as a lens, we reviewed existing feedback models arising from others’ and our own work.2,14,18–20,28,31,33 We drew on and incorporated features that reflect these perspectives.

Method

Design

We undertook an iterative, multistage, qualitative study guided by two recognized research frameworks. The first is the UK Medical Research Council (UK-MRC) guideline for studying complex interventions. Its four stages are (1) development, including modeling process and outcomes; (2) feasibility testing; (3) evaluation of effectiveness; and (4) sustainability.34 The current report covers UK-MRC Stage 1 (development and modeling) and UK-MRC Stage 2 (feasibility testing). Feasibility testing addresses multiple challenges in studying complex interventions (e.g., acceptability, compliance, stability of the intervention, consistency of delivery, recruitment and retention, influence of varying contexts) that require resolution prior to moving to effectiveness or outcome studies (Stage 3 and 4).34

The second perspective guiding the research is “realist evaluation,” which studies the social system or context in which an intervention occurs as well as the intervention itself.35 Realist evaluation seeks to understand how the individual components of an intervention work and how they interact with the people and within the context involved. It asks the question, “What works for whom and under what conditions?” It proceeds from the premise that educational interventions—such as giving and receiving feedback—are complex social phenomena occurring within dynamic social systems. Realist evaluation uses mixed data collection approaches consistent with those recommended in the UK-MRC Stage 1 and 2.

Using these frameworks, we conducted the research in four stages: (1) modeling, (2) facilitator preparation, (3) model feasibility testing, and (4) model refinement. Using content and thematic analysis,36 we analyzed data from each stage to inform the subsequent stage. Ethical approval for this study was provided by the Dalhousie University health sciences research ethics board, Halifax, Nova Scotia. The study period was July 2011 to September 2013.

Participating populations and organizations

Participants in Stage 1a (part of modeling, see below) were family physicians who had participated in an earlier study on informed self-assessment.18 They were enrolled in a formal Canadian continuing education program, the Foundation for Medical Practice Education (FMPE), which develops and supports accredited practice-based, small-group learning for family physicians.37

In Stages 1b (modeling), 2 (facilitator preparation), and 3 (feasibility testing), three physician regulatory bodies interested in enhancing their physician assessment and feedback processes participated: the American Board of Internal Medicine (ABIM), the College of Physicians and Surgeons of Ontario (CPSO), and the College of Physicians and Surgeons of Nova Scotia (CPSNS). Each has a formal physician assessment and feedback program that provides a lengthy standardized report to physicians. The goal of each program is practice improvement. (See Supplemental Digital Appendix 1 for brief descriptions of these organizations and the assessment programs and reports used in this study, http://links.lww.com/ACADMED/A291.)

Research stages, data collection, and analysis

Table 1 outlines the four research stages and the following aspects of each: the task, participating organizations, data collection methods, and sample size.

Table 1
Table 1:
Research Stages, Tasks, Participants, Data Collection, and Sample Sizes in a Study to Develop and Test an Evidence- and Theory-Based Model of Facilitated Reflective Performance Feedback

Stage 1: Modeling.

Stage 1 involved two substeps: first, a verbal review and critique of the feedback model and, second, facilitated sessions using the model, followed by debriefing interviews.

Step 1a: Verbal review and critique of the feedback model. The FMPE physician participants had identified in earlier research the features of their formal small-group learning sessions that facilitated feedback acceptance and use and informed self-assessment. These features included a safe environment, mutual trust and respect, credible data, and collaborative planning for change.18 We invited one of the two original study groups to critique our model for facilitating feedback, and we analyzed the resulting focus group transcript using content analysis.

Step 1b: Facilitated feedback sessions using the model and debriefing interviews. After participants from the verbal review substep confirmed that the model made sense and included important components of effective feedback, we tested the model. We recruited one volunteer physician experienced as an assessor from each of the ABIM and CPSO programs (total n = 2), and we trained them to facilitate a feedback session using the model. We recruited another two physician volunteers from each program (total n = 4) who were receiving their “live” performance review within the next few months to participate in a facilitated feedback session. The trained facilitators used the model to discuss each physician’s report with him or her in a session lasting about 45 minutes. Feedback sessions were videotaped.

Following the feedback session, two members of the research team (E.H., I.S.) conducted a structured debriefing interview with each facilitator and recipient pair using the session video to identify specific verbal and nonverbal communication techniques and phrases helpful in each phase of the feedback model. Another goal of the debriefing interviews was to determine the participants’ perceived value of each phase and the overall process (see Supplemental Digital Appendix 2 at http://links.lww.com/ACADMED/A291). The debriefing interviews were audiotaped.

The videotapes (facilitated feedback) and audiotapes (debriefing) were transcribed. Three members of the research team (J.S., J.L., and K.M.) conducted content and interpretive analysis of the transcripts to understand participants’ responses to the overall model and its specific phases and to identify especially effective strategies and phrases used by the facilitators. They also analyzed the videotapes to identify nonverbal communication techniques.

Stage 2: Facilitator preparation (workshop).

After considering and incorporating the refinements from the facilitators and participants in Stage 1, we recruited an additional eight volunteer physician facilitators (four from ABIM, two each from CPSO and CPSNS; see Table 1) who had all been in practice for over 20 years and had supervised students and residents. They participated in the workshop at their own site; each workshop was conducted by members of the research team (E.H. at ABIM; I.S. and J.S. at CPSO; J.S. and K.M. at CPSNS). The facilitators received a working list of helpful phrases (developed in Stage 1) to use with feedback-receiving physicians at each feedback phase. Workshop activities, some of which were based on suggestions from Stage 1 participants, included reviewing the theory and evidence informing the model, interactive discussion of the model phases and strategies, and hands-on practice using the model with debriefing to discuss how the model worked. Workshops were similar across sites; however, because of time constraints, the length of practice time varied from about 45 minutes to about 120 minutes.

Stage 3: Model feasibility testing (facilitated feedback sessions using the model and debriefing interviews).

We recruited eight physician volunteers (again, four from ABIM and two each from CPSO and CPSNS) to participate in a facilitated feedback discussion about their recent formal performance assessment report. We told the volunteers that the facilitator was trained and worked within their program; we explained that in addition to a feedback discussion, there would be a structured debriefing session at which multiple facilitators and feedback recipients for each program would be present. The eight feedback-receiving physicians varied in experience: one had been in practice 5 years, four for 11 to 20 years, and three for over 20 years. All four physicians for ABIM were general internists; for CPSO and CPSNS, three were subspecialists and one was a family physician.

As in Stage 1, the purpose of the debriefing sessions was to explore the elements of the model which were most helpful, to identify specific communication strategies and phrases, and to propose revisions (see Supplemental Digital Appendix 3 at http://links.lww.com/ACADMED/A291). The feedback discussions lasted approximately 45 minutes, and the debrief sessions, led by a member of the research team, lasted approximately 1.5 hours. Both were audiotaped.

Analysis.

We analyzed the transcripts of the feedback sessions and debriefing interviews using content and thematic analysis36 to better understand the model, its phases, and their effectiveness, and to identify useful phrases for each phase. The analysis was guided by the goal of realist evaluation—that is, to facilitate an understanding of what works for whom (i.e., feedback facilitators and recipients) under what circumstances.35 Eight members of the research team (J.S., J.L., K.M., E.H., I.S., H.A., E.D., and T.M.) participated in the analysis. During analysis, we addressed each phase of the feedback model sequentially, so as to understand facilitator communication approaches and the recipient’s responses (i.e., what worked) and to identify meaningful phrases within each phase (see Supplemental Digital Appendix 4, Coding framework, at http://links.lww.com/ACADMED/A291). We also identified general facilitative approaches.

At least three team members individually analyzed, first, each feedback session and, then, the respective debriefing interview, to explore findings from the feedback session and to better understand the factors influencing the recipient’s responses. Following these individual analyses, we shared our written responses and met by teleconference to compare our findings, resolve differences, and seek further understanding as needed. Specifically, we compared our findings of effective strategies, helpful phrases, and feedback-influencing factors for each of the four feedback phases within the individual sessions so that we could, in turn, determine those unique to each phase and those generalizable across all four. Next, we compared findings across the sessions within each program and, finally, across the three programs to identify program-specific and more generalizable findings and contextual influences. We added commonly identified helpful facilitator phrases, by phase, to an inventory of facilitative strategies.

Stage 4: Model refinement.

To refine the model, the full research team met regularly through e-mail and teleconference to discuss the results of the analyses, to query findings in light of the theoretical perspectives guiding the study, and to confirm the inventory of helpful phrases used by facilitators in each phase.

Results

Findings support the facilitated feedback model, its four phases, and the theoretical perspectives informing them. Further, they contribute to understanding the elements that enhance recipient engagement with, acceptance of, and use of feedback.

The four phases are (1) build rapport and relationship, (2) explore reactions, (3) explore content, and (4) coach for performance change. Hence, we refer to it as the R2C2 Facilitated Feedback Model. Table 2 provides the goal of each phase, representative facilitation phrases, and guiding notes. The following summarizes responses to the model and each phase.

Table 2
Table 2:
Final Evidence- and Theory-Based Facilitated Feedback Model: The R2C2 Facilitated Feedback Model

Overall, facilitators reported that the model made sense and the four phases generally flowed in a logical manner. Having a structure for the feedback discussion was helpful. To illustrate, one facilitator remarked, “It actually puts your mind at ease because you know what you’re supposed to do … it made it more structured, less scattered” (Site 2, Facilitator 1). Whereas some facilitators reported using the phases more linearly, others used them in an iterative and open manner throughout the discussion.

All eight physician recipients reported that the feedback process was helpful. They specifically appreciated the reflection stimulated by the model and its phases. One recipient commented:

It causes you to self-reflect. So this process of looking at your anxiety, looking at what your concerns are, addressing over and over again “Do you agree with the content?” brings out that feeling of self-reflection. It’s very appropriate: self-reflection and self-improvement. And it puts that bug in your head … “just because you graduated [from] medical school 20 years ago and you have all these patients who think you’re the best, doesn’t mean you can’t improve.” (Site 2, Recipient 4)

Phase 1: Build rapport and relationship

Feedback recipients consistently shared that both being formally assessed and receiving performance reports are sensitive and often intimidating activities. Hence, the feedback conversation requires thoughtfulness. They stressed that the facilitator should take the time to build a relationship with the recipient, learn about his or her practice context and challenges, and explore any concerns he or she might have regarding the assessment process. Facilitators and recipients emphasized that building and maintaining mutual respect and trust was the foundation for meaningful conversation about performance assessment.

Assessor X wanted to know about me. You know, “tell me your story.” We all have different stories about what our practices are like. And so it’s good to start with that so that [the assessor] know[s] what kind of practice you have, show[s] respect for that.… (Site 1, Recipient 6)

If you have that connection, if you have that rapport, it makes the whole process go so much better. (Site 2, Assessor 1)

Phase 2: Explore reactions to the performance data

Exploring recipients’ general reactions to their feedback report and to specific items revealed whether the report confirmed their own views of their performance (i.e., their own self-assessments). Facilitators found that asking open, nonjudgmental questions, such as “What were your initial reactions to the report?” “Did anything surprise you in your report?” or “How do these data compare with how you thought you were doing?” enabled recipients to feel comfortable sharing personal, even emotional, reactions to the feedback data. Recipients confirmed this finding and reported that the approach was effective—that is, the facilitators’ open and respectful questions and reflective listening created a safe, respectful environment for sharing honest reactions.

And then you get into the report and [you ask] … “were you surprised?” And then that’s when you’re going to see the anger if there is any. And then you acknowledge what you’re hearing and you listen attentively, and then you reflect back what you hear them saying. And all of a sudden during the interview, probably somewhere between halfway and three-quarters through, you find out as you’ve talked together and listened and reflected on what they’ve said, the anger has dissipated and they’re starting to think, “maybe there’s something I can do about this. Maybe there is something I can change.” (Site 3, Assessor 1)

Phase 3: Explore understanding of the content

At this phase, the feedback conversation transitions from soliciting the physician’s reactions to the report to ensuring a clear understanding of the report’s content and the opportunities it affords for change and learning. The facilitator seeks to clarify any data that might be unclear and to guide the physician in recognizing strengths, as well as performance gaps and opportunities for change. The goal is to enable physicians to identify one or two specific opportunities arising from the feedback data that are especially important to them and which they wish to address.

What was most important was the opportunity to explore the person, Recipient 1, in particular what he felt was the most striking about the data. Which allowed us to focus on that aspect, out of all the data there, and then to understand how he’s going to address that and move forward. (Site 1, Assessor 2)

Phase 4: Coach for performance change

In Phase 4, the focus is the physician’s development of realistic goals and an accompanying action plan to work toward the goals, enabled through the facilitator’s coaching. Facilitator–recipient dialogue at this phase concerns the nature of the feedback recipient’s goals, actions to be taken, factors that will enable and hinder implementation, and specific strategies to address these, especially the barriers.

The facilitator said, “Tell me your three goals for next year, and then we’ll talk about how you can achieve them.” And it gets people reflecting that everyone has goals. You don’t always put them right out but if you focus on them, you can do that. (Site 3, Recipient 1)

Whereas facilitators reported feeling least prepared for the coaching role, feedback recipients reported that this activity was central to enabling them to be able to change and required more attention.

In summary, an overall result of the facilitated feedback session appeared to be to enable physicians to recognize their performance assessment data positively as a personal opportunity and not negatively as an imposed threat or risk. The facilitator role seemed to enhance the feedback recipients’ ability to take ownership of their performance data; facilitated feedback seemed to empower physicians to plan to make the identified practice change(s).

I think it’s important for the assessed doctor to feel somewhat in control and not on the defensive, and to feel that they can control the conversation about what they want to discuss, as long as everything gets addressed through the meeting. So there should be an atmosphere created such that it’s a peer giving advice, as if two people in the same office just coaching or giving advice to someone who maybe needs a little help with something. Like a colleague-to-colleague consultation and not an atmosphere of intimidation. (Site 2, Recipient 2)

Discussion

The R2C2 Facilitated Feedback Model and its four phases

Our careful four-stage model development plan and feasibility testing results seem to indicate that the R2C2 Facilitated Feedback Model and its four phases are acceptable and stable across three different physician contexts. Our findings indicate that, as we hoped, each of the four phases (Relationship, Reactions, Content, Coaching) evidenced one or more of the theoretical and evidence-based perspectives guiding the model creation (see Table 2).

Phase 1, building rapport and relationship, drew on humanistic and person-centered approaches that focus on demonstrating a genuine interest in the physician recipient and creating respect and trust.24,26 The participants indicated that asking opening questions and getting to know one another made giving and receiving feedback easier.

In Phase 2, exploring reactions, person-centered approaches continued to guide the conversation. Specifically, the concepts of informed self-assessment led to the exploration of the feedback recipients’ reactions to the report.18 It was important for recipients to know that their reactions and views were taken seriously.

Phase 3, exploring content, was also guided by person-centered approaches, in which facilitators asked feedback recipients to consider the performance report, whether they understood it or had any questions, and what data were of most importance to them.24,26 This phase, in keeping with informed self-assessment, encouraged feedback recipients to consider performance gaps that they may be reluctant to address, plus any factors influencing the gaps.

Finally, in Phase 4, coaching for performance change, facilitators successfully applied person-centered approaches24,26 and principles of behavior change25 to coaching the feedback recipient in identifying goals and developing an action plan.

The central intent of the R2C2 feedback process is to empower physicians to take ownership of their performance data and responsibility for their use.

Reflection and coaching

The feedback model builds on earlier research exploring factors influencing the effectiveness of feedback. These include relationships and culture,1,5,6 emotional reactions to feedback,22,38 the role of facilitation and discussion in informing self-assessment and feedback assimilation,22,39 the roles of reflection21,22,24 and coaching,29,30 and the format of the feedback discussion.21,28,40 The contributions of two elements in particular, reflection and coaching, deserve further attention.

Although not explicitly stated, each phase of the R2C2 model is moderated through facilitated reflection on the feedback data and the data’s meaning to the feedback-receiving physician. Previous researchers have found this technique (facilitated reflection) to be effective in person-centered counseling approaches.24,26 Reflection is a strategy for learning, “an intellectual and affective activity in which individuals engage to explore their experiences in order to lead to new understandings and appreciations.”41(p19) The open-ended questions used in the R2C2 model are designed to encourage just such understandings and appreciation at each phase: explaining one’s practice and context, exploring reactions to the feedback,22,26 interpreting the feedback,31,41,42 and considering options for and factors influencing change.15,43

Facilitator participants noted that coaching was a helpful and relatively unfamiliar skill to enable physicians to make changes. Executive coaching is used in organizations to support leadership improvement and skill development.27,29 Coaching to foster physicians’ professional development, based on their own personal performance data, involves facilitating their acknowledgment of a need for change, identification of related goals, and development of a plan to achieve the goals.15,44 Importantly, in the R2C2 Facilitated Feedback Model, coaching follows relationship building and exploring the recipient’s reaction to and understanding of his or her performance feedback data. Feedback recipients reported finding the coaching helpful, whereas facilitators generally reported that it was a skill that did not come naturally to them. Coaching differs from more traditional directive teaching, and facilitators requested more attention to coaching skills in their preparation workshop.

Reviewing the perspectives of facilitators and feedback recipients from three institutions at multiple stages of development enhanced our understanding of the relationships among the feedback facilitator and the recipient; the feedback data, context, and culture; and the feedback model itself and the facilitation phases.

Limitations and areas for further research

Limitations of this study include the small number of Stage 3 participants (n = 8) and the limited number of specialties represented (n = 5). Other limitations include the voluntary nature of the facilitators’ and feedback recipients’ participation and the generally high levels of the feedback recipients’ performance, as evidenced by their formal performance reports. The latter is an important consideration; although a few participants expressed minor surprise at one or two specific aspects of their performance data, most did not. Developing a recruitment strategy that would encourage the participation of physicians who receive disconfirming assessments would enable more rigorous evaluation of the R2C2 Facilitated Feedback Model.

Additionally, study results raise questions regarding practical use and sustainability of the R2C2 Facilitated Feedback Model. The model requires a time commitment of facilitators and recipients alike (about 45 minutes) for the feedback session. Facilitators also participate in a faculty development workshop (about two hours). The workshop should include experiential practice time, and it should emphasize coaching techniques. To inform approaches to sustainability (Stage 4 of the UK-MRC guidelines34), we propose rigorously exploring varied models of providing the sessions and faculty development—for example, through professional and regulatory associations, during professional and continuing education conferences, through initiating peer feedback, and through coaching programs in which physicians identify a trusted colleague for the coaching.

Further research steps in this program of study include assessing the effectiveness (Stage 3 of the UK-MRC guidelines34) of the R2C2 Facilitated Feedback Model (e.g., Have the sessions been effective in promoting physician practice change?). Research directions also include exploration of the R2C2 Facilitated Feedback Model in residency and undergraduate medical education. For example, might the model be useful to facilitate feedback discussions with learners who receive workplace assessments, especially those who receive multiple reports and need to make sense of them collectively? Could such facilitated feedback discussions with trainees encourage reflection and the development of learning goals and plans? We also propose that the model may be useful for enabling workplace and competency-based education and assessment where the focus is on using feedback to coach competency development.45

In summary, the theory- and evidence-based R2C2 Facilitated Feedback Model appears stable and helpful for physicians in facilitating their reflection on and use of formal performance assessment feedback. Further research will determine its usefulness with physicians and physicians-in-training across a broader spectrum of performance.

Acknowledgments: The authors recognize the major funder of this research, the Society for Academic Continuing Medical Education. They also thank the research partners who provided funding and/or in-kind support: the American Board of Internal Medicine, the College of Physicians and Surgeons of Ontario, and the College of Physicians and Surgeons of Nova Scotia.

References

1. Archer JC. State of the science in health professional education: Effective feedback. Med Educ. 2010;44:101–108
2. Sargeant J, Mann KCantillon P, Wood D. Feedback in medical education: Skills for improving learner performance. ABC of Learning and Teaching in Medicine. 20102nd ed. Chinchester, UK BMJ Books:29–32
3. van de Ridder JMM, Stokking KM, McGaghie WC, ten Cate OT. What is feedback in clinical education? Med Educ. 2008;42:189–197
    4. Lantolf JP, Thorne SLvan Patten B, Williams J. Sociocultural theory and second language learning. Theories in Second Language Acquisition: An Introduction. 2007 Mahwah, NJ Lawrence Erlbaum:201–224
    5. Archer J, Sargeant JWalsh K. Successful feedback: Embedded in the culture. Oxford Textbook of Medical Education. 2013 Oxford, UK Oxford University Press:564–573
    6. Watling C, Driessen E, van der Vleuten CP, Vanstone M, Lingard L. Beyond individualism: Professional culture and its influence on feedback. Med Educ. 2013;47:585–594
    7. Evans C. Making sense of assessment feedback in higher education. Rev Educ Res. 2013;83:70–120
    8. Hattie J, Timperley H. The power of feedback. Rev Educ Res. 2007;77:81–112
    9. Shute VJ. Focus on formative feedback. Rev Educ Res. 2008;78:153–189
    10. Kluger AN, DeNisi A. The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254–284
    11. Brett JF, Atwater LE. 360 degree feedback: Accuracy, reactions, and perceptions of usefulness. J Appl Psychol. 2001;86:930–942
    12. DeNisi AS, Kluger AN. Feedback effectiveness: Can 360-degree appraisals be improved? Acad Manage Exec. 2000;14:129–139
      13. Sargeant J, Mann K, Sinclair D, van der Vleuten C, Metsemakers J. Challenges in multisource feedback: Intended and unintended outcomes. Med Educ. 2007;41:583–591
      14. Sargeant J, Eva KW, Armson H, et al. Features of assessment learners use to make informed self-assessments of clinical performance. Med Educ. 2011;45:636–647
      15. Overeem K, Driessen EW, Arah OA, Lombarts KM, Wollersheim HC, Grol RP. Peer mentoring in doctor performance assessment: Strategies, obstacles and benefits. Med Educ. 2010;44:140–147
      16. Overeem K, Wollersheim HC, Arah OA, Cruijsberg JK, Grol RP, Lombarts KM. Evaluation of physicians’ professional performance: An iterative development and validation study of multisource feedback instruments. BMC Health Serv Res. 2012;12:80
      17. Eva KW, Regehr G. “I’ll never play professional football” and other fallacies of self-assessment. J Contin Educ Health Prof. 2008;28:14–19
      18. Sargeant J, Armson H, Chesluk B, et al. The processes and dimensions of informed self-assessment: A conceptual model. Acad Med. 2010;85:1212–1220
      19. Lockyer J, Armson H, Chesluk B, et al. Feedback data sources that inform physician self-assessment. Med Teach. 2011;33:e113–e120
      20. Mann K, van der Vleuten C, Eva K, et al. Tensions in informed self-assessment: How the desire for feedback and reticence to collect and use it can conflict. Acad Med. 2011;86:1120–1127
      21. Sargeant J, McNaughton E, Mercer S, Murphy D, Sullivan P, Bruce DA. Providing feedback: Exploring a model (emotion, content, outcomes) for facilitating multisource feedback. Med Teach. 2011;33:744–749
      22. Sargeant JM, Mann KV, van der Vleuten CP, Metsemakers JF. Reflection: A link between receiving and using assessment feedback. Adv Health Sci Educ Theory Pract. 2009;14:399–410
      23. Delva D, Sargeant J, Miller S, et al. Encouraging residents to seek feedback. Med Teach. 2013;35:e1625–e1631
      24. Rogers CR Freedom to Learn: A View of What Education Might Become. 1969 Columbus, Ohio C. E. Merrill Publishing Company
      25. Cane J, O’Connor D, Michie S. Validation of the theoretical domains framework for use in behaviour change and implementation research. Implement Sci. 2012;7:37
      26. Goodstone MS, Diamante T. Organizational use of therapeutic change: Strengthening multisource feedback systems through interdisciplinary coaching. Consult Psychol J Pract Res. 1998;50:152–163
      27. Thorn PM, Raj JM. A culture of coaching: Achieving peak performance of individuals and teams in academic health centers. Acad Med. 2012;87:1482–1483
      28. Nicol DJ, Macfarlane-Dick D. Formative assessment and self-regulated learning: A model and seven principles of good feedback practice. Stud Higher Educ. 2006;31:199–218
      29. Ives Y. What is “coaching”? An exploration of conflicting paradigms. Int J Evid Based Coach Mentoring. 2008;6:100–113
      30. Eva KW, Regehr G. Effective feedback for maintenance of competence: From data delivery to trusting dialogues. CMAJ. 2013;185:463–464
      31. Eva KW, Armson H, Holmboe E, et al. Factors influencing responsiveness to feedback: On the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ Theory Pract. 2012;17:15–26
      32. Michie S, Johnston M, Abraham C, Lawton R, Parker D, Walker A“Psychological Theory” Group. . Making psychological theory useful for implementing evidence based practice: A consensus approach. Qual Saf Health Care. 2005;14:26–33
      33. Teunissen PW, Stapel DA, van der Vleuten C, Scherpbier A, Boor K, Scheele F. Who wants feedback? An investigation of the variables influencing residents’ feedback-seeking behavior in relation to night shifts. Acad Med. 2009;84:910–917
      34. Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M. Developing and evaluating complex interventions: The new Medical Research Council guidance. Int J Nurs Stud. 2013;50:587–592
      35. Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—A new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005;10(suppl 1):21–34
      36. Liamputtong P Qualitative Research Methods. 20093rd ed. Victoria, Australia Oxford University Press
      37. Foundation for Medical Practice Education Web site. http://www.fmpe.org/index.asp. Accessed April 14, 2015
      38. McConnell MM, Eva KW. The role of emotion in the learning and transfer of clinical skills and knowledge. Acad Med. 2012;87:1316–1322
      39. Kruger J, Dunning D. Unskilled and unaware of it: How difficulties in recognizing one’s own incompetence lead to inflated self-assessments. J Pers Soc Psychol. 1999;77:1121–1134
      40. Kluger AN, Van Dijk D. Feedback, the various tasks of the doctor, and the feedforward alternative. Med Educ. 2010;44:1166–1174
      41. Boud D, Keogh R, Walker DBoud D, Keogh R, Walker D. Promoting reflection in learning: A model. Reflection: Turning Experience Into Learning. 1985 London, UK Routledge Falmer:19–40
      42. Driessen EW, Overeem K, van Tartwijk JDornan T, Mann K, Scherpbier A, Spencer J. Learning from practice: Mentoring feedback and portfolios. Medical Education: Theory and Practice. 2011 New York, NY Churchill Livingston Elsevier:211–227
      43. van der Leeuw RM, Slootweg IA, Heineman MJ, Lombarts KMJMHvan der Leeuw RM. Explaining how faculty members act upon residents’ feedback to improve their teaching performance. From Feedback to Action: Physicians’ Teaching Performance in Residency Training. 2013 Zutphen, the Netherlands Wohrmann Print Service:143–158
      44. Gagliardi AR, Wright FC, Victor JC, Brouwers MC, Silver IL. Self-directed learning needs, patterns, and outcomes among general surgeons. J Contin Educ Health Prof. 2009;29:269–275
      45. Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32:676–682

      Supplemental Digital Content

      © 2015 by the Association of American Medical Colleges