There is a substantial international body of research that supports the value of the use of multisource feedback (MSF) for physician performance assessment. In this issue, Wright and colleagues add to this body of literature by examining two instruments designed to collect MSF from large cohorts of patients and colleagues in the context of physician performance evaluation in the United Kingdom. In the United States there is a reticence by leading physician organizations to adopt the use of patient and peer feedback in key evaluations such as maintenance of specialty board certification or licensure. This commentary explores this incongruity in light of the 50th anniversary of Thomas Kuhn’s seminal work, The Structure of Scientific Revolutions. It is concluded that, although the paradigm of using patient and peer evaluation in the United States appears to be shifting, the profession, by way of not providing MSF to the public, is losing its role in being the unique provider of public information on physician performance.
Dr. Weiss is senior vice president for institutional accreditation, Accreditation Council for Graduate Medical Education, and clinical professor of medicine, Northwestern University Feinberg School of Medicine, Chicago, Illinois.
Correspondence should be addressed to Dr. Weiss, Accreditation Council for Graduate Medical Education, 515 North State St., Chicago, IL 60654; e-mail: firstname.lastname@example.org.
Editor’s Note: This is a commentary on Wright C, Richards SH, Hill JJ, et al. Multisource feedback in evaluating the performance of doctors: The example of the UK General Medical Council patient and colleague questionnaires. Acad Med. 2012;87:1668–1678.
Change happens and science advances, most often in small measurable steps rather than single great improvements. 2012 is the 50th anniversary of the publication of Thomas Kuhn’s1 The Structure of Scientific Revolutions. This book laid the foundation for a new way of thinking about scientific discovery. The term “paradigm shift” is the popularized shorthand we have used to talk about his concepts.
Kuhn believed that progression in science was not always driven by an orderly accumulation of scientific facts, but rather, by a blending of advancement in discovery within a culture that was, in a sense, ready for change. Kuhn noted that such changes in paradigms can start within smaller groups of scientific communities as part of the advancement of science. Then, at some unanticipated moment, these changes can rapidly advance new thinking into the scientific mainstream and, thereby, significantly move scientific practice forward.
It is in the light of this anniversary of Kuhn’s work that the article in this issue by Wright and colleagues2 provides us with an important opportunity to reflect on how we use multisource feedback (MSF) in evaluating physicians in practice. The concept of MSF is not new. For practicing physicians it has generally been applied using small-scale efforts and within the context of performance review in single settings rather than as part of a larger evaluation scheme for professional self-regulation. The article by Wright and colleagues adds to the science in this area by exploring the feasibility of large-scale efforts to gain feedback on physicians through peers and patients. The authors’ study provides additional evidence on how this type of feedback can contribute to our understanding of physician performance.
Although direct input by peers, patients, and health professionals as part of multisource physician assessment is being tested or used on a large scale in the United Kingdom and other countries, until recently there has been little effort to adopt such assessment in the United States. The reticence to advance these methods in the United States was demonstrated when the American Board of Medical Specialties (ABMS), in adopting its 2009 standards for maintenance of certification (MOC) program policies, decided against requiring patient and peer surveys for assessment of individual physician practice as part of the MOC program.3 Instead, the ABMS included patient and peer surveys as a “developing standard” to be viewed as optional for any of its 24 member boards. As of 2012, only a few of the 24 ABMS member boards have chosen either to test or implement a patient or peer survey; the majority still have not committed to using such feedback. Similarly, the Federation of State Medical Boards (FSMB), in its framework for policy on maintenance of licensure, has no mention of a requirement for patient or peer surveys for individual physician practice.4
In the United States, well-validated surveys measuring patients’ experience of care have been used for decades as a component for assessing quality of care by hospitals and health plans.5 Some of these regulatory organizations have chosen to sample patient experience at the level of the individual physician. However, because individual physician sampling for patient experience and peer surveys adds to the cost of the assessment process, most hospitals and health plans are reluctant to consider this additional component for their surveys without having a specific requirement to do so.
The six core competencies (Patient Care, Medical Knowledge, Interpersonal and Communication Skills, Practice-Based Learning and Improvement, Professionalism, and Systems-Based Practice) jointly sponsored by the Accreditation Council for Graduate Medical Education and the ABMS have provided an impetus for assessing patient experience and interprofessional assessment, mostly focused on inter-personal and communication skills. Currently, during their proportionately few formative years of training, residents and fellows in the United States are being assessed for these competencies; however, in their lifetime of clinical practice, physicians in the United States are still not required to seek this type of formal feedback from their patients and peers.
Given the current of the ABMS and FSMB policies, it is worth asking whether our national paradigm for the evaluation of physician performance has shifted. The pessimist might argue that there has been little progress. For the optimist, however, the ABMS policy of “developing standards” provides a public declaration of the growing importance of these approaches to physician evaluation and a tactical way to test the environment and professional culture for readiness for the change. Yet, while the profession seeks to engage this type of professional evaluation cautiously, the public is not as patient. Successful commercial efforts such as Angie’s List (www.angieslist.com) and Consumers’ Checkbook (www.checkbook.org), which allow consumers to engage in public feedback and discussion about service providers, are signaling that change is happening, with or without the profession in the lead.
The seeming discomfort of the U.S. physician workforce with assessing individual physician practice through patient and peer surveys is, at least according to Thomas Kuhn, the expected outcome of a paradigm shift. So, perhaps the paradigm is shifting. If that is the case, the next signal of change will likely need to come from the profession by way of its lead organizations adopting policies which emphasize the importance of engaging in MSF during the practicing physician’s lifelong professional journey.
Acknowledgments: The author thanks Patricia Surdyk, PhD for reviewing this commentary.
Other disclosures: None.
Ethical approval: Not applicable.
Disclaimer: The views expressed represent personal opinion and are not necessarily the views or policies of the Accreditation Council for Graduate Medical Education.
1. Kuhn TS The Structure of Scientific Revolutions. 1962 Chicago, Ill University of Chicago Press
2. Wright C, Richards SH, Hill JJ, et al. Multisource feedback in evaluating the performance of doctors: The example of the UK General Medical Council patient and colleague questionnaires. Acad Med. 2012;87:1668–1678
5. . Agency for Healthcare Research and Quality. CAHPS (Consumer Assessment of Healthcare Providers and Systems). www.ahrq.gov/cahps
. Accessed August 23, 2012.