To the Editor: Multisource feedback (MSF) is a mechanism where various individuals who interact with a physician comment on their interactions. These comments are used to assess the physician’s performance in graduate and postgraduate medical education. Below I offer my review of the implementation of MSF, which reveals a number of shortcomings. Collectively, they show that MSF should not be used to assess a physician’s competence.
In the United States, MSF is a component of the American Board of Medical Specialties (ABMS) maintenance of certification (MOC) programs and is a requirement in assessing residents, mandated by the Accreditation Council of Graduate Medical Education.
Weiss1 advocates that MSF be used in MOC programs. He assumes that a patient’s self-report of health care interactions informs us about the quality of the care he or she received and can be generalized to a practitioner’s competence.
I disagree2 and maintain that there are questions regarding MSF as implemented by the ABMS. The rationale for the selection and number of patients and peers in the ABMS MOC protocols are arbitrarily determined and do not correlate to any clear standard.
Wright and colleagues3 report on the reliability of using MSF to aid in the assessment of physicians in the United Kingdom. Because of the risk that MSF may misinform the assessment of a practitioner’s functioning, they propose that MSF at this time should be used only to provide formative assessments of physician performance. Holmboe and Ross4 propose that every patient participating in MSF should provide a descriptive statement about his or her clinical experience, not simply complete checklists.
Ensuring physicians’ competence is a critical component of residency and postresidency education. With changes, MSF could serve as a key component of these processes. To accomplish this, clearer knowledge of patient raters must be obtained. This includes (1) the reason they sought care, (2) how they selected their physician, (3) the treatment they expected, (4) the treatment they received, and (5) whether they will see the physician again. The patient would also complete a number of Likert scales, including ones that focus on quality of care received, the physician’s ability to meet expectations, and the physician’s ability to answer questions.
All this information would be used to assess the physician’s care from the patient’s perspective. That assessment could be coupled with other physician performance data to evaluate the physician’s competence. In this context, MSF could become a meaningful component of a protocol to assess physician competence.
Sidney Weissman, MD
Past president, American Association of Directors
of Psychiatric Residency Training, and professor of
clinical psychiatry, Feinberg School of Medicine,
Northwestern University, Chicago, Illinois;
1. Weiss KB. Commentary: Shifting the focus of physician assessment in the United States. Acad Med. 2012;87:1660–1661
2. Weissman SH. Does maintenance of certification ensure maintenance of competency? Psychiatric Times. 2010;27:17
3. Wright C, Richards SH, Hill JJ, et al. Multisource feedback in evaluating the performance of doctors: The example of the UK General Medical Council patient and colleague questionnaires. Acad Med. 2012;87:1668–1678
4. Holmboe ES, Ross K. Commentary: Realizing the formative potential of multisource feedback in regulatory-based assessment programs. Acad Med. 2012;87:1657–1659