Ziegelstein, Roy C. MD; Fiebach, Nicholas H. MD
In 1999, the Accreditation Council for Graduate Medical Education (ACGME) introduced a program intended to better evaluate whether residents are developing into competent physicians and professionals.1 The ACGME’s general competency and outcome assessment initiative, called the ACGME Outcome Project, emphasizes the importance of assessing the outcomes of residency education in six general competencies that were identified through a research and review process between January 1998 and February 1999. These six competencies include two new ones: practice-based learning and improvement (PBLI) and systems-based practice (SBP). For PBLI, the ACGME concluded that “residents must be able to investigate and evaluate their patient care practices, appraise and assimilate scientific evidence, and improve their patient care practices.”2 For SBP, the conclusion was that residents must “demonstrate an awareness of and responsiveness to the larger context and system of health care and the ability to effectively call on system resources to provide care that is of optimal value.”2
Whereas the meaning and intent of the four other general competencies (e.g., patient care, medical knowledge, interpersonal and communication skills, and professionalism) are generally self-evident, PBLI and SBP may be conceptually difficult for both residents and faculty. Methods for teaching these concepts are needed if PBLI and SBP are to be introduced into residency training and incorporated into lifelong education and practice. Metaphors and parables, often used as educational tools to allow learners to grasp new concepts, allow individuals to compare subjects that are complex and unknown to subjects that are more easily understood and that are known to the learner. At Johns Hopkins Bayview Medical Center in Baltimore, Maryland, we used the metaphors “the mirror” and “the village” to introduce the concepts of PBLI and SBP, respectively.
The word metaphor is derived from the Greek metapherein, or “to transfer.”3 Introducing complex new concepts in medical education may be facilitated by transferring their meaning through the use of metaphors.4,5 At Johns Hopkins Bayview Medical Center, we presented conferences to internal medicine residents and faculty, incorporating the mirror and the village metaphors to explain the objectives of PBLI and SBP. We likened PBLI to holding a mirror up to ourselves to document, assess, and improve our practice. Specific tools (e.g., resident learning portfolios, chart self-audits, and Morbidity and Mortality Morning Reports) were likened to mirrors that reflect residents’ practice and allow them to identify variances from practice guidelines and from material learned in evidence-based medicine training.
We introduced SBP using the metaphor of the village made famous by former First Lady Hillary Clinton when she said, “It takes a village to raise a child.”6 In our conferences, we used the village metaphor to emphasize that the physician must work together with a community of health care providers to deliver optimal patient care, much as parents must work within a family and society raise a healthy child. The importance of nonphysician members of the health care team and of changes in health care delivery (e.g., shorter lengths of stay, increasing use of subacute facilities) were highlighted to stress the importance of the metaphorical village in modern practice. Specific training activities (e.g., multidisciplinary patient care rounds, nursing evaluations of residents, and quality assessment-systems improvement exercises) were used to emphasize that the resident–physician is one part of the health care system.
We introduced PBLI and SBP at the end of the 2001–02 academic year and again at the beginning of the 2002–03 academic year in a series of three conferences, two with the residents as the target audience and one with the faculty. During an approximately five-month period, specific PBLI and SBP activities (see below) were either newly implemented or ones already in existence were discussed in the context of the mirror and the village. At the end of this period, residents completed a questionnaire to assess the impact of some of these activities. Responses to items on the questionnaire were based on five-point Likert scales. In the first part of the questionnaire, we asked residents whether specific training activities had enhanced their abilities in PBLI and SBP. In the second part of the questionnaire, residents retrospectively reported their competence in PBLI before and after three specific PBLI activities. Retrospective pre– and post–self-assessments have been shown to be sensitive and valid measures of educational training interventions.7 Residents’ completion of the questionnaire was voluntary and anonymous. The Institutional Review Board of the Johns Hopkins Bayview Medical Center approved our study.
We performed statistical analyses using Stata Statistical Software Release 7.0 (Stata Corporation, College Station, Texas) to compare the residents’ retrospective preactivity and postactivity responses. The distributions of responses to some of these questionnaire items were not normally distributed, so Wilcoxon signed-rank tests were performed in addition to paired t tests. The results of the nonparametric tests were similar to those of the t tests for each item, so we report only the latter results.
The Mirror: Practice-Based Learning and Improvement Activities
To teach PBLI, we introduced three separate activities into the residency program: a weekly inpatient morbidity and mortality morning report (MMMR), continuity clinic chart self-audits, and resident learning portfolios. These activities are discussed using the metaphor of the mirror. The MMMR serves as a metaphorical mirror that reflects residents’ and faculty’s medical practice in the inpatient setting. Once each week, residents and medical students meet with the program director at morning report to review all mortality and selected morbidity from the inpatient medical services during the previous week. In a forum that encourages free and open discussion of personal and systems-practice issues, residents identify possible errors in practice that could improve outcomes. This weekly discussion at morning report sends an important message that PBLI is an expected regular activity for all residents and physicians.
Chart self-audits are used to illustrate the importance of holding a mirror up to one’s own ambulatory practice and examining the reflection critically to identify areas for improvement. Each resident performs a yearly self-audit on three charts in their continuity clinic using a protocol which is then discussed with the clinic attending. For 2001–02, self-audits were completed by 38 residents (95%). They listed 89 separate “areas for [self] improvement” (range 0–5 self-improvement items identified by each resident, mean 2.3). The three most common ambulatory practice improvements the residents identified were discussion and documentation of preventive interventions, patient profiles, and advance health care directives.
Resident learning portfolios enable residents to document and assess their abilities as clinicians, teachers, and scholars. Suggested components of the portfolio include a list of patients and their diagnoses, residents’ individual learning objectives, goals for improvement, evaluations, attendance at program conferences, scholarly activity, procedure log, presentations and teaching activities, awards, and meaningful patient experiences. The portfolios are discussed at individual feedback sessions with the program director. In one ambulatory PBLI exercise for the portfolio, residents were advised to determine how many of their patients with hypertension in the resident continuity clinic were being prescribed medications and had achieved adequate blood pressure control according to the guidelines of the Sixth Report of the Joint National Committee on Prevention, Detection, Evaluation, and Treatment of High Blood Pressure.8 In this exercise, we also provided residents with a copy of a study by one of their predecessors in which he did a similar exercise in the continuity clinic using a chart review of patients treated for hypertension.9 In an inpatient PBLI exercise for the portfolio, residents were advised to determine how many of their medical inpatients with congestive heart failure and without a specific contraindication they had discharged from the hospital on an angiotensin converting enzyme inhibitor or on a beta-blocker.10 Residents are asked to come to their regular feedback sessions with the program director prepared not only to have the mirror held up to them, but also to discuss what they think of their own progress using the portfolio as a mirror.
The Village: Systems-Based Practice Activities
To teach SBP, we introduced three separate activities into the residency program: daily multidisciplinary inpatient rounds, monthly nursing evaluations of residents, and quality assessment-systems improvement exercises. These activities are discussed using the metaphor of the village. At daily multidisciplinary inpatient rounds the various contributors to the care of medical inpatients meet to discuss patient care and anticipated postdischarge needs. These rounds emphasize that it takes providers with different expertise and focus to provide optimal patient care, in other words the village. In these rounds, residents on inpatient ward rotations discuss their patients’ progress, functional status, and postdischarge care plans with a team of health care providers including nurse case managers, physical and occupational therapists, and social workers.
Using information from these rounds and from the daily interactions with residents at the bedside and inpatient care activities on the inpatient ward and intensive care unit, nurse managers compile a monthly consensus evaluation for each resident using an evaluation form that addresses the six ACGME competencies. These evaluations often provide important information about residents’ patient care, interpersonal and communication skills, and professionalism not otherwise obtained and allow the residents to see that the opinions of many members of the village are important in assessing their professional development.
Quality assessment-system improvement exercises allow residents to appreciate the importance of the many contributors to the care of ambulatory patients in the continuity clinic. As part of a seminar in the ambulatory block curriculum for postgraduate year (PGY) 2 and 3 residents, small groups of residents and faculty audit clinic charts to assess systems elements that might improve practice. For example, in 2001–02, quality assessment-system improvement exercises examined mammography for women older than age 50 and identified the following system barriers: scheduling process, patient access, physician awareness, patient attitudes, and follow-up and tracking. Changes in referral, scheduling and tracking procedures have since been implemented. The quality assessment-system improvement exercise for 2002–03 was an examination of blood pressure control for clinic patients with diagnosed hypertension.
Thirty-eight of 44 (86.4%) residents returned the questionnaires assessing the impact of PBLI and SBP activities. Some residents did not provide responses for every item on the questionnaire. Because all residents had not participated in a quality assessment-system improvement exercise by the time they were asked to complete the questionnaire, the statement about this exercise had substantially fewer respondents than did statements for the other activities.
Residents agreed that five activities (weekly MMMR, multidisciplinary rounds, chart self-audit, nursing evaluations, and quality assessment-systems improvement exercises) were effective in implementing specific aspects of PBLI and SBP (see Table 1). The effectiveness of resident learning portfolios was not specifically assessed in the first part of the questionnaire because this activity had only been recently introduced. Of the 38 respondents, 33 (86.8%) either agreed strongly (23 respondents, 60.5%) or somewhat (ten respondents, 26.3%) that adding the MMMR to the conventional morbidity and mortality monthly departmental conference helped them to develop and maintain a willingness to learn from and use errors to improve the systems or processes of care, a concept that is central to PBLI. Similarly, 35 of the 38 respondents (92.1%) either agreed strongly (19 of 38 respondents, 50%) or somewhat (16 of 38 respondents, 42.1%) that multidisciplinary rounds improved their ability to understand, access and utilize the resources, providers, and systems necessary to provide optimal patient care, a concept that is central to SBP. The majority of residents also rated chart self-audits, nursing evaluations, and quality assessment-systems improvement exercises positively.
Approximately five months after the introduction of three PBLI activities, we asked residents to rate retrospectively their abilities to identify areas for improvement and to implement strategies to enhance their knowledge of, skills in, and attitudes about the processes of medical care (1 = very able to 5 = very unable). Residents responded that before the introduction of each of the three PBLI activities (MMMR, clinic chart self-audit, and resident learning portfolio), their PBLI abilities were modest (mean ratings of 2.5 for each of the three PBLI activities) (see Table 2). Self-reported PBLI abilities after the introduction of these activities improved significantly (mean ratings for MMMR, self-audits, and portfolios were 1.6 [p < .001], 1.6 [p < .001], and 2.0 [p = .016], respectively).
In 1999, the ACGME introduced the six general competencies, including two new competencies, PBLI, and SBP. The ACGME specified elements of these two new competencies and suggested resident evaluation methods. Residency program directors were expected to educate faculty and trainees about the Outcome Project and the general competencies. The ACGME attempted to assist program directors in this effort by developing and maintaining an Outcome Project Web site that includes a section on implementation of the competencies entitled Recognize Success Via imPlementation (RSVP).11 This RSVP section describes activities that may be used to teach and assess the general competencies. Although many of these activities can help programs implement the competencies, a method is needed for introducing the competencies that allows residents and faculty to understand their purpose and intent.
As a recent study showed, residency program directors need assistance introducing, implementing, and evaluating the general competencies. Although program directors felt that they were fairly informed about and understood the general competencies, they were not well-prepared to meet them.12 PBLI and SBP were two of the three competencies for which program directors felt they needed the most help developing curricular materials. Although educating program directors is a key step in ensuring the successful introduction of the general competencies in residency education, methods must also be developed to assist program directors in educating residents and faculty about the competencies’ meaning and purpose. The meaning and intent of the other general competencies (e.g., patient care, medical knowledge, interpersonal and communication skills, and professionalism) are generally self-evident, but PBLI and SBP may be conceptually difficult for both residents and faculty.
As we noted earlier, metaphors can help learners to understand new concepts by comparing them to more familiar ones. In an article entitled “Teaching Abstract Concepts by Metaphor,” Sutherland noted that “metaphor is an excellent technique for introducing unfamiliar or complex material because a connection between the unknown and . . . invisible constructs can be understood through concrete and extant phenomena.”4 Metaphors allow difficult-to-understand subjects to be related in readily understood terms, particularly when more conventional, time-consuming means may not be as effective.5
We used the metaphors of the mirror and the village to introduce PBLI and SBP to our institution. We used the mirror to illustrate the importance of looking carefully at one’s practice to assess and improve it. The metaphor of the mirror emphasizes the importance not only of self-reflection and self-assessment, but also of faculty’s evaluation and assessment of residents that occur as part of training. We used the village to emphasize the importance of the larger community of health care providers, comparing the intent of SBP to the concept espoused by then First Lady Hillary Rodham Clinton in her speech to the Democratic National Convention on August 27, 1996.13 Just as “it takes a village” emphasizes the importance of a healthy society to ensure the care and well-being of a child, we noted that SBP emphasizes the importance of the medical community and the larger context and system of health care in providing the best care to our patients.
The results of our study show that residents felt that activities that were introduced using the metaphors of the mirror and the village were successful in teaching the key concepts of PBLI and SBP. Most respondents reported that all five activities they were asked about (nursing evaluations, multidisciplinary rounds, quality assessment-systems improvement exercises, MMMR, and chart self-audit) were helpful to them. Particularly noteworthy is the fact that 86.8% of our respondents agreed that the introduction of weekly MMMR helped them to develop and maintain a willingness to learn from and use errors to improve the systems or processes of care; 23 of the 38 residents (60.5%) agreed strongly with this statement. Residents also retrospectively reported significant improvement in their ability to engage in PBLI after their participation in the three specific activities used to highlight this competency.
Our study had several limitations. We did not specifically measure the residents’ and faculty’s understanding and acceptance of the metaphors of the mirror and the village. The training activities used to highlight these metaphors in the contexts of PBLI and SBP were not explicitly designed to implement these competencies, and we did not conduct a formal needs assessment. Importantly, with regard to evaluating the impact of these activities, we report only the retrospective self-assessments of residents from a single, medium-sized training program in internal medicine. Nevertheless, the metaphorical concepts and highlighted activities appear to have face validity, and our results provide preliminary evidence of their effectiveness.
The effectiveness of this approach in other residency programs will depend on the resources dedicated to the implementation of PBLI and SBP in the curriculum. At a minimum, program directors must discuss the metaphors and specific PBLI and SBP activities in conferences and small-group sessions. Introduction of the basic concepts to residents and faculty can be accomplished in three hour-long conferences each academic year. One or two separate hour-long conferences with residents will likely be necessary to discuss the importance, purpose, and content of specific tools (e.g., resident learning portfolios). To fully introduce these competencies, program directors must work not only with faculty and residents, but also with other health care providers to change the culture of the program, if not the hospital and ambulatory clinic. Multidisciplinary patient care rounds, monthly evaluation of residents by nursing staff, and weekly MMMR require considerable time and effort and therefore must be seen as having a positive effect not only on the education of residents, but also on the care of patients.
Other innovative activities may also be used to illustrate the importance of the mirror and the village metaphors. For example, our residency program piloted a posthospitalization follow-up project to determine whether providing residents with follow-up information about their former inpatients could add to their learning.14 This project emphasized that the care of the patient does not stop at the boundaries of the “inpatient village,” but must involve the patient’s primary care provider and provide information about the accuracy of the discharge diagnosis, the response to therapy, and the patient’s quality of life at home.
The focus of residency education has changed in recent years. Whereas content and process were stressed previously, there is a greater emphasis now on outcomes. The essence of the ACGME Outcome Project, as noted on the ACGME Web site, is to have residency programs provide “evidence showing the degree to which program purposes and objectives are or are not being attained, including achievement of appropriate skills and competencies by students.”15 In addition to ensuring that the content of a curriculum and the venue in which residents are educated meets appropriate standards, programs must now continuously evaluate the competencies of residents, with greater emphasis on residents sharing in this process. Indeed, the ACGME notes that in the competency-based model, programs must not only demonstrate that they have the potential to educate residents, but “programs will be asked to show how residents have achieved competency-based educational objectives and in turn, how programs use information drawn from evaluation of those objectives to improve the educational experience of the residents.”15
In conclusion, our study has shown that the metaphors of the mirror and the village may be used to introduce the conceptually difficult new general competencies, practice-based learning and improvement and systems-based practice, into residency programs. These metaphors allow learners to understand the meaning and intent of these competencies by comparing them to two familiar and easily understood ideas. Our study indicates that these new competencies have been successfully introduced into our program by incorporating specific activities that address the competencies and by using the metaphors the mirror and the village to illustrate their importance. Other programs may find these metaphors and specific activities helpful in implementing training in the new ACGME competencies.
The authors gratefully acknowledge the assistance of Maryann Z. Fiebach with the statistical analyses. The authors are also especially grateful to the Johns Hopkins Bayview internal medicine residents for their enthusiastic participation in this project.