Commentary: Urgently Needed: A Safe Place for Self-Assessment on the Path to Maintaining Competence and Improving Performance

Abstract

Traditional continuing medical education (CME), necessary for keeping physicians current and competent, is insufficient in translating physician practice into better patient outcomes. CME, then, must be transformed from a system of episodic interventions to a more personalized, contextual, flexible, and targeted process within a continuing professional development framework. The core of this transformation must be a formal process of physician self-assessment. Unfortunately, health care providers tend toward inaccurate self-assessment, regardless of training, specialty, or manner of self-assessment. Therefore, the development of an external validation system conducted by credible, informal peer review in a safe environment is essential. Clinicians must be able to access practice and patient data without concerns about accuracy, timeliness, confidentiality, attribution, or unintended consequences. New analytical tools are also needed to illuminate the data compilations and present them in compelling, individualized, and comparative formats. However, such developmental work will not be possible without strong community leadership coordinating a collaboration of resources and a sharing of data. Ensuring physician competence has long been an issue for medical societies, state licensing boards, and others invested in improving patient care. Now it's time to get serious. Current efforts at massive health care reform provide the perfect opportunity to fully integrate physician self-assessment and performance improvement into the larger health care system through a continuing professional development model. Practitioners in CME have been far too complacent with the current practices. A shift to a focused and dedicated sense of urgency must occur to ensure physicians' continuous learning and change.

Physicians hold the key to health care reform, but they cannot unlock its promise without paying considerable attention to their own performance, both as individual health care providers and as drivers of the larger health care system. Within the debate about evidence-based medicine, pay for performance, and other measures designed to funnel medical care into predetermined, cost-effective, health-improving silos, we believe that the transformation of the continuing education system needed to keep physicians current and competent amidst rapidly changing standards of care has been neglected and must be addressed.

We have worked in continuing medical education (CME) for more years than we'd like to admit and have seen CME activities develop erratically in response to advances in medical research that have led to a better understanding of disease and improvements in diagnostic tools and therapeutic options. Thankfully, the random lectures that were common in the 1960s on subjects of interest to presenters, often delivered and received without much instructional forethought, have given way to educationally sound presentations built around learners' needs and preferred learning styles. CME has been embraced by licensing and credentialing boards as well as by the broader health care community, making it a condition of continued practice. The Accreditation Council for Continuing Medical Education¹ reported more than 100,000 certified CME activities in 2008, providing nearly 800,000 hours of instruction to more than 10 million physicians. But quantity does not trump quality. We need better CME programs in the future, not just more programs.

You only need to look at the results of recent research into disparities in patient care, techniques and treatments that are over- or underused or flat-out ineffective, as well as a host of other practice issues to realize that the current system has not prevented physicians from often falling short in providing optimal patient care. The result has too often been dissatisfying patient outcomes and increasing costs.² In response, a host of often conflicting disease management guidelines, performance measures, and treatment algorithms are rapidly being issued in the name of evidence-based medicine. But physicians are often unaware of these new guidelines, confused by them, in disagreement with them, or simply too attached to familiar ways to follow them. Physicians do what their experience tells them works—what they know and trust.

Not a Panacea but a Pathway to Evidence-Based CME

Evidence-based medicine, though rapidly growing in use, is still in its infancy. The educational framework needed to ensure that all health care providers are practicing effective, patient-centered, cost-conscious medicine is in its infancy as well. The traditional knowledge-based, faculty-driven, large-group didactic training that has dominated CME for decades has failed the modern litmus test. Research shows one of the most common forms of continuing education, the didactic lecture, to be the least effective means of improving physicians' attitudes and changing practice behaviors, despite being physicians' most preferred CME format.³ We as CME providers have often liked this format as well; it is easier and less costly to produce than other more dynamic formats, and it reaches large audiences. But when researchers started measuring the impact of various CME activities on improved physician performance, they found little proven benefit to many of the familiar and popular formats.⁴

Today, evidence-based CME needs to join evidence-based medicine as a dynamic duo on a path to improved patient outcomes within an efficient and effective health care system. We must transform CME to conform to the broader designation of continuing professional development (CPD), extending beyond mere knowledge and its application to encompass other key attributes such as skills, behaviors, processes of care, and improved patient health rather than the episodic, standardized, cognitive testing used as a surrogate to measure physician competence. A new educational framework integrating CME into the overall landscape of CPD must be developed in partnership with the American Medical Association (AMA), licensing and credentialing boards, specialty societies, health care providers, medical schools, hospitals and clinics, patient advocacy groups, payers, and anyone with a stake in high-quality health care. Like modern medicine itself, new CME programs must be personalized, contextual, and flexible courses, targeted toward both individualized and systemic needs. Most of all, they must be self-directed, lifelong, and constantly evolving. These programs also must objectively support the attainment and maintenance of levels of competency and improved patient health for each individual physician.

A Move Toward Self-Assessment

It is time for change. And the core of that change must be a formal, externally validated process of physician self-assessment. Only then can we tailor CME activities to meet individual physician needs in the context of his or her day-to-day practice and interactions with the overall health care system. Because physicians affect and are also affected by the systems in which they practice, their self-assessment activities must also incorporate these interactions so that both the individual physician and the overall health care system will improve.

Unfortunately, research shows that, regardless of training, specialty, disease state involved, domain of self-assessment, and manner of comparison between self- and external assessment, health care providers often fail to accurately self-assess.⁵ Health care providers overestimate their own competence, particularly those providers whose performance on external assessments (i.e., peer observation and evaluation, chart audits, cognitive tests, use of case vignettes, and other methods beyond personal assessment) is low. There are a variety of reasons for this overestimating, including the bias of our learned optimism and self-confidence, which we all need to get us through the day.

Another reason for inaccurate self-assessment involves the lack of truly reflective practice. Reflection enables the clinician to not only analyze his or her behavior but also to evaluate strengths and weaknesses, consider new options, and integrate new knowledge into routine practice. Reflective practice can help reconcile feedback that may be inconsistent with a physician's self-perception, facilitating the eventual assimilation and positive acceptance of the constructive feedback.⁶ There is little doubt that physicians think a lot about what they do, despite long hours and hectic working conditions. But this informal reflection must now become a purposeful, structured endeavor that results in a written record of its processes and outcomes. Some physicians take naturally to such structured activities; others find them difficult to manage and may need support. However, a key requirement in developing this process is that physicians perceive the environment in which any formal written reflection is generated and stored as safe, confidential, and protected from discovery. Modern media, especially social media platforms, which are now being used for CME, do not guarantee the level of protection that physicians will require.

As self-assessment reveals relative strengths and weaknesses, it also provides a pathway for educational interventions to address domains in which gaps have been found. Thus, self-assessment provides a more rational approach to decision making with regard to desirable educational interventions than does the hit-or-miss approach currently in use or the varying methods of external entities, such as the process by which individual states regulate relicensure. Physicians may undergo cycles of self-assessment after each intervention in order to measure gains and provide sorely needed data on how well the intervention worked. Appropriately linking self-assessment with educational interventions will also permit physicians to track their CME/CPD credit histories and, if necessary, report them for relicensure, recertification, or credentialing. The compilation of such data provides the foundation for individualized learning portfolios.

An Assurance of a Safe Environment for External Validation

Given the generally poor ability of individuals to draw appropriate conclusions from self-assessment, external validation with credible, informal peer review by colleagues is essential. Yet, physicians are naturally concerned about who sees the information and what will be done with it. How long will it be kept? How secure is it? These questions and others are still unanswered. One reliable mechanism for external validation, then, may be the use of professional and specialty societies, which could provide anonymous reviews, perhaps from physicians with comparable practice profiles. If more serious weaknesses are discovered that compromise competence, new mechanisms must be created that offer more concerted feedback, ranging from mentoring and training through counseling and mandated additional learning and testing.⁷

Externally validated self-assessment is already being used to some degree in Performance Improvement (PI) CME, promulgated by the AMA. PI CME draws on practice-based data and guidelines to assist physicians in understanding their actual performance patterns and the gaps in their competence. It provides physicians with the information necessary to guide their self-assessment. As part of the learning process, physicians set goals for change and engage in structured learning activities to improve performance. Maintenance of certification provides another avenue for external input and review in the self-assessment process. The American Board of Internal Medicine has developed Web-based, self-directed Practice Improvement Modules (PIMs) that require the candidate to review patient charts or perform patient surveys to evaluate and improve practice performance. These PIMs differ from traditional self-evaluation modules in that they focus on individual quality improvement and practice performance rather than medical knowledge alone. On the basis on the reported outcomes, the PIMs allow the clinician to routinely set up and track improvement goals.

A Need for Robust Analytical Tools

PI initiatives have been slow to expand, in part because they are expensive, time consuming, and, many fear, potentially career ending. Nevertheless, they are particularly important because they provide real-time, relevant data on the individual physician's performance. Average physicians, unless they work in the few highly functioning integrated health care systems, see little or no aggregate data that reflect their day-to-day performance. These data “black holes” span not only the practice profile (e.g., number of patients seen, demographics, diagnoses, severity) but also relevant processes of care and patient outcomes. Such data gathering requires the development of engines and infrastructures that can compile, format, and post a broad variety of relevant data sets in a manner that satisfies serious physician concerns about accuracy, timeliness, confidentiality, attribution, and unintended consequences.

Moreover, we will need new analytical tools to illuminate the data compilations and present them in compelling form, such as credible comparators to provide validation. These may include evidence-based performance measures compiled and validated by national organizations or informal benchmarks drawn from group data from other physicians with similar practice profiles. Individuals must be able to review their performance data as trends over time, enhanced by data analysis and formatting that take into account learner styles and preferences. Methods must also be found to analyze derivative measurement approaches such as the formulation of a composite score across several different categories of data and attributes or a score derived by aggregating data over a defined period of time. Such analyses will be useful not only to the individual physician but throughout the health care system, from medical schools to medical centers.

A Call for Leadership

The changes necessary to move from traditional, episodic CME to continuous, comprehensive, lifelong CME/CPD seem massive and daunting. Nevertheless, the movement toward real physician performance assessment and improvement is inexorably under way. It may take some time to achieve robust agreement to move forward. However, formulating a clear research agenda could help in both building agreement and implementing the underlying vision. Furthermore, the research agenda should focus primarily on feasibility testing of new methods and approaches prior to large-scale development and resource commitment. Above all, the research agenda must be a living document, flexible, transparent, and credible.

With few exceptions, such developmental work will not be possible without massive collaboration and sharing of data and resources across many different organizations, particularly national data repositories. Nothing practical can be developed without continuous, strong leadership from a community of like-minded leaders with high purpose, who are drawn from across the traditional fault lines of “the House of Medicine,” a metaphor connoting the micro- and macrosystems that espouse the ethics, professionalism, self-regulation, politics, and philosophy defining and transcending the realm of organized medicine, and encompassing the myriad of associations, societies, and accrediting, certifying, and licensing bodies—each with one or more missions on behalf of the medical profession. This developmental initiative could mean a single newly formed body as proposed at the 2007 Macy Conference on CME⁸ or, more practically, several existing bodies functioning together. These bodies would form a cohesive leadership community to articulate a shared vision in practical and compelling terms, secure ongoing political and financial support from the appropriate organizations, and champion implementation in the face of controversy or frank pushback. Such a concerted approach would send an unambiguous signal that the House of Medicine accepts the responsibilities of self-regulation and is acting to ensure physician competence, optimize patient outcomes, and improve health system performance.

Acknowledgments:

The authors wish to thank Robert M. Galbraith, MD, Richard E. Hawkins, MD, and Eric S. Holmboe, MD, for their contributions to this manuscript.

Funding/Support:

None.

Other disclosures:

None.

Ethical approval:

Not applicable.