The Educational Alliance as a Framework for Reconceptualizing Feedback in Medical Education

Telio, Summer MD; Ajjawi, Rola PhD; Regehr, Glenn PhD

doi: 10.1097/ACM.0000000000000560
Articles

Feedback has long been considered a vital component of training in the health professions. Nonetheless, it remains difficult to enact the feedback process effectively. In part, this may be because, historically, feedback has been framed in the medical education literature as a unidirectional content-delivery process with a focus on ensuring the learner’s acceptance of the content. Thus, proposed solutions have been organized around mechanistic, educator-driven, and behavior-based best practices. Recently, some authors have begun to highlight the role of context and relationship in the feedback process, but no theoretical frameworks have yet been suggested for understanding or exploring this relational construction of feedback in medical education. The psychotherapeutic concept of the “therapeutic alliance” may be valuable in this regard.

In this article, the authors propose that by reorganizing constructions of feedback around an “educational alliance” framework, medical educators may be able to develop a more meaningful understanding of the context—and, in particular, the relationship—in which feedback functions. Use of this framework may also help to reorient discussions of the feedback process from effective delivery and acceptance to negotiation in the environment of a supportive educational relationship.

To explore and elaborate these issues and ideas, the authors review the medical education literature to excavate historical and evolving constructions of feedback in the field, review the origins of the therapeutic alliance and its demonstrated utility for psychotherapy practice, and consider implications regarding learners’ perceptions of the supervisory relationship as a significant influence on feedback acceptance in medical education settings.

S. Telio is clinical instructor, Department of Psychiatry, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada.

R. Ajjawi is senior lecturer in medical education, Centre for Medical Education, Medical Education Institute, University of Dundee, Dundee, Scotland.

G. Regehr is professor, Department of Surgery, and associate director (research), Centre for Health Education Scholarship, Faculty of Medicine, University of British Columbia, Vancouver, British Columbia, Canada.

Editor’s Note: A Commentary by D.F. Weinstein appears on pages 559–561.

Funding/Support: During the period of this project, Dr. Telio was provided salary support through the University of British Columbia, Faculty of Medicine, Clinical Educator Fellowship Program.

Other disclosures: None reported.

Ethical approval: Reported as not applicable.

Correspondence should be addressed to Summer Telio, Centre for Health Education Scholarship, Faculty of Medicine, University of British Columbia, Suite 3300, 910 W. 10th Ave., Vancouver, BC, Canada V5Z 4E3; telephone: (604) 875-4111, ext. 67644; e-mail: summertomaszek@gmail.com.

Article Outline

Feedback has long been recognized as one of the key factors in enabling effective learning in the health professions. It has been the subject of much scholarship and many recommendations to ensure that it is enacted well. Unfortunately, despite this acknowledgment, attention, and effort, current feedback practices in medical education are broadly understood to be suboptimal. There is increasing recognition that a likely source of the ongoing difficulties with feedback rests in the medical education community’s dominant conception of the feedback process as a mechanism for delivering content about performance to the learner with a focus on getting the learner to “accept” the feedback.

In this article, we offer a thematic review of the medical education literature on feedback, informed by relevant readings from the broader education literature, to highlight this recurring conceptualization that underlies the current discourse. To this end, we briefly describe historical and modern articulations of the mandate to offer feedback, we summarize descriptions of the problems with current feedback practices, and we highlight the dominant approach to redressing these problems using prominent examples from the literature. We then describe a growing trend toward reconceptualizing feedback as a more dialogic process in which context and relationship are the dominant factors in enabling and enacting behavior change. Finally, we provide a targeted review of the psychotherapy literature on the origins of and research related to the “therapeutic alliance,” and we offer the “educational alliance” as a framework for supporting and extending the reconceptualization of feedback in medical education. We conclude by discussing some implications of this framework for educational practices and scholarship.

Back to Top | Article Outline

The Current State of Feedback

Feedback has always been recognized as an essential component of medical training. Well before modern times, Plato and other ancient Greek scholars were noted to describe the importance of supervision and feedback.1 Galen of Pergamon (AD 129–c. 216), for example, was documented as providing the following feedback to his medical pupils:

He then gives them kindly-meant advice in regard to their clothing, their behaviour, and the language they should use with patients: he recommends them cleanliness and a proper attention to their hair and forbids them to eat onions or garlic before visiting a patient, or to drink too much wine, lest they annoy the sufferer by the offensive odor from their mouths and stink like goats.1

Today, the Accreditation Council for Graduate Medical Education in the United States explicitly requires that U.S. residency programs incorporate feedback into routine practice and provide each trainee with semiannual performance evaluation through feedback.2 The Royal College of Physicians and Surgeons of Canada, which defines the accreditation standards for all Canadian postgraduate training programs, sets similar requirements for feedback:

There must be honest, helpful and timely feedback provided to each resident. Documented feedback sessions must occur regularly, at least at the end of every rotation.… There should also be regular feedback to residents on an informal basis.3

Consistent with these national standards, the scholarly literature repeatedly emphasizes the importance of feedback in medical training.4–6

Yet despite the clear importance of feedback in medical education, the concept remains phenomenologically fraught, complex, and not fully delineated. Authors in medical education have recognized several discrepancies and tensions related to feedback. For example, a myriad of studies have highlighted discrepancies between supervisors’ and learners’ perceptions regarding the nature of feedback. Some survey-based studies have suggested that faculty report regular provision of feedback, yet learners deny its receipt.7–9 These findings have been elaborated by more in-depth studies of students and preceptors. In a study on the practices of expert clinical teachers, for example, Irby10 noted that students did not identify receipt of feedback that he had observed being provided, which suggests that students may underestimate the amount of feedback they receive. On the other hand, studies of preceptors suggest that many factors and competing goals influence the decision to offer feedback in any given situation, such as preserving collegiality within the teaching relationship and encouraging student motivation.11 Kogan et al12 suggested that these multiple factors and goals generate tensions. It is possible, therefore, that the moment-by-moment resolution of these tensions might, in fact, lead to fewer actual feedback interactions than faculty believe they are providing.13

Further, although learners voice a strong desire for more elaborate and more frequent feedback,14 research has highlighted tensions between this desire and satisfaction when actually receiving feedback. Mann et al,15 for example, described how the desire for feedback is in tension with the fear of critical or negative performance appraisal. Moreover, there is mounting evidence that satisfaction with feedback is linked to the feedback’s emotional impact on the recipient rather than solely its quantity or validity. This finding was illustrated in a randomized trial in which feedback was given to medical students in the context of a surgical knot-tying intervention.16 The results suggested that although constructive feedback was more helpful to learning, students expressed more satisfaction with praise, leading the study authors to conclude that what students actually desire is reassurance.

Perhaps more alarming than these discrepancies and tensions in the perceptions of feedback is the range of concerns identified in research regarding the actual impact of feedback: Feedback is not always accepted by the recipient, it does not consistently lead to changes in behavior or self-concept, and it can actually have a detrimental impact on later performance. In a comprehensive meta-analysis of over 130 studies from a variety of domains, for example, Kluger and DeNisi17 highlighted a high rate of ineffective, and often detrimental, feedback interventions. Overall, they found that feedback was associated with a mean effect size of 0.4; however, 38% of feedback interventions resulted in performance deterioration not attributable to sampling or other errors. High variability in the effectiveness of feedback has been identified in other meta-analyses as well.18,19

Both the broader education18–20 and the medical education21–23 literatures have tended to respond to these concerns by identifying the common features of effective and ineffective feedback. The identification of such features has often resulted in the construction of context-free “best practice” recommendations—that is, prescriptive guidelines geared to maximize the impact of feedback. For example, Pendleton et al24 described a set of prescribed steps (commonly known as “Pendleton’s rules”) by which feedback should be given in order to maximize its effectiveness. The now-proverbial “feedback sandwich” has also been explicitly described as a procedure for effective delivery.25 The medical education literature has generally endorsed these types of best practice recommendations, which have been published in articles directed toward a range of medical specialists, including radiologists, obstetricians, internists, emergency physicians, and palliative care doctors.4–6,22,26–30

Back to Top | Article Outline

Expanding Our Thinking About Feedback

Despite efforts to mitigate the problem of variable effectiveness through use of standardized best practice techniques for delivery, feedback continues to be a suboptimal and fraught phenomenon.31 The lack of progress in this area may, in part, be related to a lack of emphasis on the context in which feedback occurs. That is, in medical education, feedback is traditionally conceptualized as a unidirectional delivery process: Feedback is provided by the supervisor and directed toward the trainee. Thus, the best practice recommendations focus on feedback content and delivery and give little attention to the recipient or to the context of the supervisory relationship in which the feedback is being experienced. This perspective is perhaps most clearly expressed in Shute’s comprehensive review of the feedback literature:

[A]lthough the teacher may also receive student-related information and use it as the basis for altering instruction, I focus on the student (or more generally, the “learner”) as the primary recipient of formative feedback herein.19

Hattie and Timperley18 similarly conceptualize feedback as being provided by an agent and seem to relegate contextual elements, such as the supervisory relationship, to a position of lesser significance.

Recently, authors have begun to explicitly question the conceptualization of feedback as “a one-way stream—from teacher to learner”32 and to voice criticisms of the resulting formulaic feedback practices.32–37 As Bing-You and Trowbridge suggest,

Medical educators may have been too focused on a narrow view of feedback. Building an approach or system around a few teacher-specific behavioral principles of feedback (eg, timely, specific) is inadequate. An approach to improving feedback incorporates teacher-based behaviors, learner-based cognitive principles, and a focus on the teacher-learner relationship.34

In response to such challenges, researchers have begun exploring factors that influence receptivity and incorporation of feedback from the learner’s perspective. Perhaps the most prominent finding of this work is the recognition that recipients’ beliefs about and attitudes toward the source clearly influence the impact of feedback. Bing-You et al,38 for example, examined perceived source credibility. In-depth, semistructured interviews revealed that residents discounted feedback from supervisors whom they felt lacked clinical knowledge or experience or from supervisors whom they judged as demonstrating poor interpersonal skills toward them, as both clinical expertise and interpersonal behavior were linked to the degree of respect residents described holding toward their supervisors.

Sargeant and colleagues39,40 completed a series of studies on multisource feedback given to general practitioners from patients, medical colleagues, and coworkers. In one study, they found that a variety of factors influenced the incorporation of feedback, including its source. Feedback from medical colleagues was viewed as lacking credibility, in large part because of the infrequency with which physicians observe each other in practice.39 In a subsequent publication, this group found that the quality of the relationship with the feedback source was also critical in recipients’ interpretation and use of external information.40

Consistent with these findings, Eva and colleagues41 found that, in addition to the internal cognitive and emotional processes of the recipient, the feedback recipient’s perception of source credibility was an important factor in feedback acceptance. In this study, which used focus groups with learners at various levels of practice from eight health professional training programs, source credibility was influenced by whether the feedback provider had observed the recipient’s performance, whether the provider was perceived to understand the recipient’s role in that context, and the nature of the relationship between the provider and the recipient. In particular, this study identified a need for recipients to believe that feedback was delivered from a position of beneficence in order for feedback to have an impact.

Finally, Watling et al42 also emphasized the centrality of source in the determination of feedback credibility. In a study using a grounded theory approach, 22 junior faculty were asked to reflect on experiences that were influential during their clinical training. The analysis yielded a model of clinical learning that included feedback and highlighted the significance of credibility judgments. Feedback from teachers was rarely integrated without scrutiny. Perceived alignment of the teacher with a learner’s personal and professional values was influential in determining the credibility of feedback.

These studies represent a vanguard movement in the consideration and exploration of the contextual and, in particular, the relational aspects of feedback. Factors contributing to source credibility have been noted, including the provider’s capacity for observation as well as the recipient’s estimations of the provider’s clinical skill, of the provider’s understanding of the recipient’s role, and of the provider’s beneficence and alignment with the recipient’s values.40–42 Yet the field of medical education still lacks a coherent conceptual framework for understanding how these factors might interact and work together to bring about decisions to incorporate feedback in a way that leads to behavior change. In this regard, we suggest that the psychotherapy literature holds great potential in furthering our analysis and exploration of educational feedback. In the field of psychotherapy, comprehensive theoretical models have long been employed to explain the process by which the therapeutic relationship affects how external information is incorporated, leading to changes in the self-concept, behavior, and skills of the feedback recipient. In the next section of this article, we therefore review some key concepts from psychotherapy that might translate well to a model of feedback effectiveness in medical education.

Back to Top | Article Outline

The Therapeutic Alliance in Psychotherapy

The evolution of theory currently occurring in the feedback literature in medical education parallels the historical progression of theoretical orientations in psychotherapy. The earliest psychotherapy theorists generally assumed that the process of clinical improvement in psychotherapy was a direct result of interpretations made to patients by therapists.43 Psychotherapy patients were believed to have transformations in their knowledge, self-concept, and behavior simply as a result of gaining information about their conduct from the expert observer. Over time, theoretical orientations evolved, and, with this evolution, the psychoanalytic community came to understand that the provision of expert insight alone is insufficient to evoke change in patients. Rather, expert insight is provided in the context of a relationship, and the features of that relationship have much bearing on the extent to which the expert’s proffered interpretations induce change.44 This revolution in psychotherapeutic thought has much to import with respect to the theoretical conceptualization of feedback in medical education. The shift from the dominant importance of therapists’ interpretations to the significance of the therapeutic relationship is mirrored in the burgeoning recognition of the significance of the supervisor–trainee relationship in the medical education literature. Thus, psychotherapy theory may provide a theoretical framework with which to expand the current understanding and exploration of feedback incorporation in medical education. Detailed inquiries regarding the nature of the therapeutic relationship have revealed a host of embedded factors that influence change in feedback recipients.

Greenson45 coined the term “working alliance” to emphasize the patient’s capacity to work jointly with the therapist toward change. Bordin46 later outlined three components of the working alliance between therapist and patient: (1) a mutual understanding of the purpose or goal of therapy; (2) an agreement about how to work toward that goal or the tasks of therapy; and (3) the patient’s liking, trusting, and valuing of the therapist. As this concept gained prominence, the term “working alliance” was gradually replaced by the term “therapeutic alliance.” The modern connotation is free from any specific therapeutic orientation, and today the concept of the therapeutic alliance is thought to be central to psychotherapy theory across modalities.47

The therapeutic alliance (which encompasses the three previously described components of the working alliance) is defined by the quality of the patient–therapist relationship.48 Importantly, however, therapists are known to overestimate the quality of their working relationships with patients and may not be attuned to ruptures in the therapeutic alliance.48–51 Thus, as the psychotherapy literature has repeatedly shown, it is the patient’s perception, rather than the therapist’s estimation, of the therapeutic alliance that is key. Horvath summarizes this as follows:

[T]he client’s subjective evaluation of the relationship, rather than the therapist’s actual behavior, has the most impact on therapy outcome. Specifically, it has been found that it was not the objectively measured level of the therapist’s empathy, congruence, or unconditional regard per se that had the most powerful impact on the therapy outcome, rather, it was the client’s perception of these qualities that foretold the success of the helping process.48

As a result of this repeated finding, the therapeutic alliance has come to be defined in the psychotherapy field by the alliance as experienced by the patient.

The impact of this patient-experienced therapeutic alliance is significant in terms of its effect size, with studies repeatedly demonstrating the quality of the therapeutic alliance to be the most robust predictor of therapy outcome, surpassing the impact of specific therapeutic techniques.48,51–56 Patients who perceive alliances with their therapists as healthy are more likely to exhibit improvements in symptoms and beneficial changes in self-concept and behavior47,53,57,58 across therapeutic modalities.47,59 Poor therapeutic alliances have been associated with unilateral termination, where patients elect to stop treatment without revealing this decision to the therapist.57,58,60,61 Moreover, a bevy of research has indicated that informing the therapist of the patient’s perception of the therapeutic alliance can avert negative outcomes such as therapy dropout or failure to attain clinical improvement.50,53 In this research, therapeutic alliance quality is measured through validated questionnaires completed by the patient throughout the course of therapy. These tools assess the patient’s perceptions regarding the mutual understanding of goals in treatment, the patient’s sense of mutual agreement about the way in which to work toward these goals, and the patient’s sentiments toward the therapist. When the questionnaire results are provided to the therapist, thereby informing him or her of a concerning alliance and potential treatment failure, the therapist can address problems in the therapeutic relationship with the patient in subsequent sessions and avert negative outcomes.

Back to Top | Article Outline

Exploring the Construct of an “Educational Alliance”

As previously described, there are striking parallels in the therapist–patient and supervisor–trainee relationships in terms of the provision of feedback directed at effecting changes in knowledge, self-concept, and behavior. Just as a patient forms a therapeutic alliance with a therapist, a trainee could be conceptualized to form an “educational alliance” with a supervisor. If so, appreciation of the therapeutic alliance and its importance in effecting patient change may provide a valuable basis for understanding and exploring the influences of the educational alliance in effecting trainee change in the feedback process. For example, given the recognized importance of the patient’s perception of the therapeutic alliance, the quality of the educational alliance should accordingly be judged from the trainee’s perspective. Further, key aspects of the therapeutic alliance—namely, the unity of goals, agreement on how to reach those goals, and the bond between therapist and patient—may have conceptual translations to the educational alliance. Indeed, using the educational alliance as a lens reframes the feedback process from one of information transmission (from supervisor to trainee) to one of negotiation and dialogue occurring within an authentic and committed educational relationship that involves seeking shared understanding of performance and standards, negotiating agreement on action plans, working together toward reaching the goals, and co-creating opportunities to use feedback in practice. Thus, we propose that the construct of the educational alliance holds significant potential as a framework for the theoretical conceptualization of feedback and has implications for future research and practice.

For example, using the construct of the educational alliance would imply that previously articulated “best practices” in feedback delivery, although not necessarily wrong, are conceptually (and therefore functionally) insufficient. Such best practices should not be treated as catechisms, “rules,” or steps in a procedure that, if done well, will deterministically lead to feedback incorporation. Rather, these practices should be considered as a set of tools that can be selected amongst and used strategically to achieve the goal of establishing and maintaining an effective educational alliance with the learner. Thus, faculty development within this framework would focus on the effective use of these tools in the establishment and maintenance of the educational alliance rather than exclusively on the enactment of the practices. Further, emphasis would be placed on seeking evidence from the learner that he or she feels a strong educational alliance with the supervisor, not on the faculty member’s own sense of a positive educational alliance.

Using the construct of the educational alliance would also imply that setting the stage for effective feedback interactions does not occur only at the time of feedback itself. Rather, an educational alliance framework suggests that the learner is likely to be actively exploring and testing the supervisor’s commitment to the learning process from the first moment of their first meeting. Almost immediately, the learner will be asking himself or herself questions such as the following: Does this supervisor care about me as a person? Am I present in this person’s mind? Does this supervisor care about my goals in this context? Is he/she trying (and able) to understand where I am starting from and where I want to get to? Does this supervisor have my best interests at heart? Is this relationship about my becoming the best clinician I can be, or are there other agendas here? Recognizing that the learner may be closely examining the supervisor’s commitment to the educational alliance very early in the relationship reinforces the importance of being authentically interested in the learner upon introduction and suggests why failing to demonstrate authentic interest early in the relationship may result in the later reluctance of the learner to “listen” to the supervisor’s valuable feedback.

More broadly, the construct of the educational alliance raises questions about whether current tools completed by trainees to rate supervisor effectiveness are as meaningful as they could be, because it is doubtful that these tools effectively assess the educational alliance as perceived by the learner. Well-established, reliable, and validated tools exist for measuring the patient’s sense of the therapeutic alliance.53 This raises questions regarding how easily and effectively these tools might be adapted for use in the educational context, as well as about when such tools should be invoked to ensure that the supervisor understands the learner’s perception of the educational alliance and can actively correct a poor alliance if one is identified. Means of addressing ruptures in the educational alliance could then be considered, including supervisor-initiated discussions regarding the teaching relationship or third-party mediation.

Of course, there are important differences between psychotherapy and supervision in medical education. Therefore, questions regarding the limits of the therapeutic alliance as a model for the educational alliance must also be raised. For example, given that the therapeutic alliance is generally measured in the context of a longer-term relationship, what is the equivalent relational process for the brief teaching and feedback encounters that exist in many outpatient contexts or for the very short-term relationships that exist in contexts such as simulation debriefings? How quickly do the learner’s initial assessments of the educational alliance occur, and what sorts of interactions enable a good alliance (or proxy for it) in these brief-encounter situations? As another example, given well-established phenomena in the therapeutic domain such as transference and countertransference, where are the boundaries on educational relationships? Do these boundaries mirror those normally drawn in the therapeutic context, or do we have to draw the lines differently in the context of supervision in medical education?

Back to Top | Article Outline

In Sum

Although feedback has been recognized as important throughout the history of medical education,23,62 the literature reveals a mismatch between supervisors’ and learners’ perspectives on feedback16–20 and limited feedback “acceptance”17,18 despite clear and explicit recommendations regarding best practices for its delivery.22 Traditional theories tend to construct feedback as unilateral and educator driven, with little exploration of the role of the learner or of the supervisory relationship in this process.18 Recently, efforts have been made to reconceptualize feedback as a bilateral, context-based dialogue.34,37 The importance of perceived source credibility in mediating the assimilation of feedback has been suggested, but it has not been fully delineated.41,42 A substantial gap in the medical education literature remains with respect to understanding the effect of context, in particular the supervisory relationship, on feedback incorporation. The psychotherapy literature suggests that relationship-based factors contribute significantly to changes in knowledge, identity, and behavior.44 Attributes of the supervisory relationship in medical education may be of parallel significance with respect to feedback incorporation. The “educational alliance,” derived from the psychotherapy literature’s “therapeutic alliance,”48 may offer an innovative framework from which to explore this potentially significant influence on feedback incorporation.

We have offered several potential implications for considering feedback as occurring in the context of an educational alliance; however, these are largely speculations and raise more questions than answers. With this idea in mind, we propose a new set of research questions that will likely be beneficial to the community of educators and researchers exploring feedback as an educational practice: What characterizes positive educational alliances between supervisors and their trainees? How do trainees experience negative educational alliances? What attributes of the educational alliance influence feedback incorporation and trainees’ future engagement in learning? How might systems-based issues, such as brief rotations and the strong culture of summative assessment, influence the ability to develop an effective educational alliance? By pursuing such questions, informed by the theoretical frameworks of the psychotherapy literature, we may be able to develop a more meaningful understanding of the context in which feedback functions, and to reorient the focus of the discussion in medical education from addressing feedback as a set of mechanisms for effective delivery to addressing feedback as a process of negotiation in the environment of a supportive educational relationship.

Back to Top | Article Outline

References

1. Puschmann T A History of Medical Education. 1966 New York, NY Harner Publishing
2. Accreditation Council for Graduate Medical Education. Common Program Requirements. 2011 [no longer available]. http://www.acgme.org/acgmeweb/Portals/0/dh_dutyhoursCommonPR07012007.pdf. Accessed September 14, 2012
3. Royal College of Physicians and Surgeons of Canada. General Standards Applicable to All Residency Programs B Standards. 2011 revised 2013. http://www.royalcollege.ca/portal/page/portal/rc/common/documents/accreditation/accreditation_blue_book_b_standards_e.pdf. Accessed September 12, 2014
4. Wood BP. Feedback: A key feature of medical training. Radiology. 2000;215:17–19
5. Richardson BK. Feedback. Acad Emerg Med. 2004;11:e1–e5
6. Branch WT Jr, Paranjape A. Feedback and reflection: Teaching methods for clinical settings. Acad Med. 2002;77(12 pt 1):1185–1188
7. Sender Liberman A, Liberman M, Steinert Y, McLeod P, Meterissian S. Surgery residents and attending surgeons have different perceptions of feedback. Med Teach. 2005;27:470–472
8. Jensen AR, Wright AS, Kim S, Horvath KD, Calhoun KE. Educational feedback in the operating room: A gap between resident and faculty perceptions. Am J Surg. 2012;204:248–255
9. Yarris LM, Linden JA, Gene Hern H, et al.Emergency Medicine Education Research Group (EMERGe). Attending and resident satisfaction with feedback in the emergency department. Acad Emerg Med. 2009;16(suppl 2):S76–S81
10. Irby DM. What clinical teachers in medicine need to know. Acad Med. 1994;69:333–342
11. Ende J, Pomerantz A, Erickson F. Preceptors’ strategies for correcting residents in an ambulatory care medicine setting: A qualitative analysis. Acad Med. 1995;70:224–229
12. Kogan JR, Conforti LN, Bernabeo EC, Durning SJ, Hauer KE, Holmboe ES. Faculty staff perceptions of feedback to residents after direct observation of clinical skills. Med Educ. 2012;46:201–215
13. Burack JH, Irby DM, Carline JD, Root RK, Larson EB. Teaching compassion and respect. Attending physicians’ responses to problematic behaviors. J Gen Intern Med. 1999;14:49–55
14. Duffield KE, Spencer JA. A survey of medical students’ views about the purposes and fairness of assessment. Med Educ. 2002;36:879–886
15. Mann K, van der Vleuten C, Eva K, et al. Tensions in informed self-assessment: How the desire for feedback and reticence to collect and use it can conflict. Acad Med. 2011;86:1120–1127
16. Boehler ML, Rogers DA, Schwind CJ, et al. An investigation of medical student reactions to feedback: A randomised controlled trial. Med Educ. 2006;40:746–749
17. Kluger A, DeNisi A. The effects of feedback interventions on performance: Historical review, a meta-analysis and a preliminary feedback intervention theory. Psychol Bull. 1996;119:254–284
18. Hattie J, Timperley H. The power of feedback. Rev Educ Res. 2007;77:81–112
19. Shute VJ. Focus on formative feedback. Rev Educ Res. 2008;78:153–189
20. Davies D, Jacobs A. Sandwiching complex interpersonal feedback. Small Group Behav. 1985;16:387–396
21. Hewson MG, Little ML. Giving feedback in medical education: Verification of recommended techniques. J Gen Intern Med. 1998;13:111–116
22. Bienstock JL, Katz NT, Cox SM, Hueppchen N, Erickson S, Puscheck EE. To the point: Medical education reviews—providing feedback. Am J Obstet Gynecol. 2007;196:508–513
23. van de Ridder JMM, Stokking KM, McGaghie WC, Ten Cate OTJ. What is feedback in clinical education? Med Educ. 2008;42:189–197
24. Pendleton D, Schofield T, Havelock P The Consultation: An Approach to Learning and Teaching. 1985 Oxford, UK Oxford University Press
25. LeBaron SW, Jernick J. Evaluation as a dynamic process. Fam Med. 2000;32:13–14
26. Brown N, Cooke L. Giving effective feedback to psychiatric trainees. Adv Psychiatr Treat. 2009;15:123–128
27. Chowdhury RR, Kalu G. Learning to give feedback in medical education. Obstet Gynaecol. 2004;6:243–247
28. Gigante J, Dell M, Sharkey A. Getting beyond “good job”: How to give effective feedback. Pediatrics. 2011;127:205–207
29. Thomas JD, Arnold RM. Giving feedback. J Palliat Med. 2011;14:233–239
30. Wong A. Review article: Teaching, learning, and the pursuit of excellence in anesthesia education. Can J Anaesth. 2012;59:171–181
31. Klaber B. Effective feedback: An essential skill. Postgrad Med J. 2012;88:187–188
32. Krackov SK. Expanding the horizon for feedback. Med Teach. 2011;33:873–874
33. Archer JC. State of the science in health professional education: Effective feedback. Med Educ. 2010;44:101–108
34. Bing-You RG, Trowbridge RL. Why medical educators may be failing at feedback. JAMA. 2009;302:1330–1331
35. Walsh K. The rules. BMJ. 2005;331:574
36. Watling CJ, Kenyon CF, Schulz V, Goldszmidt MA, Zibrowski E, Lingard L. An exploration of faculty perspectives on the in-training evaluation of residents. Acad Med. 2010;85:1157–1162
37. Watling CJ, Lingard L. Toward meaningful evaluation of medical trainees: The influence of participants’ perceptions of the process. Adv Health Sci Educ Theory Pract. 2012;17:183–194
38. Bing-You RG, Paterson J, Levine MA. Feedback falling on deaf ears: Residents’ receptivity to feedback tempered by sender credibility. Med Teach. 1997;19:40–44
39. Sargeant J, Mann K, van der Vleuten C, Metsemakers J. “Directed” self-assessment: Practice and feedback within a social context. J Contin Educ Health Prof. 2008;28:47–54
40. Sargeant J, Armson H, Chesluk B, et al. The processes and dimensions of informed self-assessment: A conceptual model. Acad Med. 2010;85:1212–1220
41. Eva KW, Armson H, Holmboe E, et al. Factors influencing responsiveness to feedback: On the interplay between fear, confidence, and reasoning processes. Adv Health Sci Educ Theory Pract. 2012;17:15–26
42. Watling C, Driessen E, van der Vleuten CP, Lingard L. Learning from clinical work: The roles of learning cues and credibility judgements. Med Educ. 2012;46:192–200
43. Mitchell AM, Black MJ Freud and Beyond: A History of Modern Psychoanalytic Thought. 1996 New York, NY Basic Books
44. Gabbard GO, Westen D. Rethinking therapeutic action. Int J Psychoanal. 2003;84(pt 4):823–841
45. Greenson RR. The working alliance and the transference neurosis. Psychoanal Q. 1965;34:155–181
46. Bordin ES. The generalizability of the psychoanalytic concept of the working alliance. Psychother Theory Res Pract. 1979;16:252–260
47. Hill CE, Knox S. Processing the therapeutic relationship. Psychother Res. 2009;19:13–29
48. Horvath AO. The therapeutic relationship: From transference to alliance. J Clin Psychol. 2000;56:163–173
49. Hannan C, Lambert MJ, Harmon C, et al. A lab test and algorithms for identifying clients at risk for treatment failure. J Clin Psychol. 2005;61:155–163
50. Lambert M. Presidential address: What we have learned from a decade of research aimed at improving psychotherapy outcome in routine care. Psychother Res. 2007;17:1–14
51. Norcross JC Psychotherapy Relationships That Work: Therapist Contributions and Responsiveness to Patients. 2002 New York, NY Oxford University Press
52. Ahn HN, Wampold BE. Where oh where are the specific ingredients? A meta-analysis of component studies in counseling and psychotherapy. J Couns Psychol. 2001;48:251–257
53. Burlingame G, Strauss B CORE Battery–Revised: An Assessment Toolkit for Promoting Optimal Group Selection, Process and Outcome. 2006 New York, NY American Group Psychotherapy Association
54. Del Re AC, Flückiger C, Horvath AO, Symonds D, Wampold BE. Therapist effects in the therapeutic alliance–outcome relationship: A restricted-maximum likelihood meta-analysis. Clin Psychol Rev. 2012;32:642–649
55. Martin DJ, Garske JP, Davis MK. Relation of the therapeutic alliance with outcome and other variables: A meta-analytic review. J Consult Clin Psychol. 2000;68:438–450
56. Safran JD, Muran CJ Negotiating the Therapeutic Alliance: A Relational Treatment Guide. 2000 New York, NY Guilford Press
57. Samstag LW, Batchelder ST, Muran JC, Safran JD, Winston A. Early identification of treatment failures in short-term psychotherapy. An assessment of therapeutic alliance and interpersonal behavior. J Psychother Pract Res. 1998;7:126–143
58. Tryon GS, Kane AS. Relationship of working alliance to mutual and unilateral termination. J Couns Psychol. 1993;40:33–36
59. Høglend P, Hersoug AG, Bøgwald KP, et al. Effects of transference work in the context of therapeutic alliance and quality of object relations. J Consult Clin Psychol. 2011;79:697–706
60. Tryon GS, Kane AS. The helping alliance and unilateral termination. Couns Psychol Q. 1990;3:233–238
61. Tryon GS, Kane AS. Client involvement, working alliance, and type of therapy termination. Psychother Res. 1995;5:189–198
62. Ende J. Feedback in clinical medical education. JAMA. 1983;250:777–781
© 2015 by the Association of American Medical Colleges