Batalden, Maren K. MD, MPH; Warm, Eric J. MD; Logio, Lia S. MD
The Accreditation Council for Graduate Medical Education (ACGME) mandates that internal medicine residencies include a didactic program based on the core knowledge content of internal medicine.1 Many residency programs fulfill this requirement with traditional noon conferences even though the evidence for the efficacy of the classic, lecture-based conference series on long-term knowledge retention is conflicting,2–5 and the evidence for the impact of that kind of learning model on patient care is, at best, minimal.6 Over the past several years, work hours restrictions,7 decreasing pharmaceutical support for food at educational conferences,8–10 and emerging teaching philosophies5,11–13 have pushed residency program directors to consider alternatives to the traditional daily noon conference. Leaders of several residency programs have sought to improve the delivery of core curricula by instituting a weekly academic half day (AHD) in which learning is concentrated over a longer block of time. Despite the many narrative descriptions of individual AHD curricula available on program Web sites, little published literature on the AHD exists.14 This article details the experiences of three distinctive internal medicine residency programs whose leaders replaced the traditional noon conference curriculum with an AHD. Although each program’s AHD developed independently of the other two, retrospective comparative review reveals instructive similarities and differences, which may be useful for other residency directors seeking to redesign their educational programs. In keeping with suggested guidelines for describing innovations in medical education,15 we describe the generalizable problem that inspired the innovation, delineate alternative solutions at three sites, describe some initial outcomes, and reflect on the potential import for graduate medical education.
The Problem: Limitations of Noon Conference
Though the three residency programs we describe are diverse in setting and size (see Table 1), similar deficits in the traditional noon conference structure for delivering core curriculum occurred at all three. Sustaining resident attendance at noon conferences was difficult. Clinical responsibilities often kept or pulled learners away from teaching sessions. Duty hours changes packed new pressures into the schedule and curtailed time for shared thinking and discussion. Without pharmaceutical industry support for food, the noon conference was losing its ability to attract learners.
The typical noon conferences at all three programs entailed passive PowerPoint (Microsoft, Redmond, Washington) lectures with little audience interaction. Faculty articulated no expectations for residents to prepare for the noon conference in advance and made no attempt, when the conference was over, to assess learner retention of the material that had been presented. Although the educational programs at each of the three residencies aligned with some larger core curricular plan, the timing of particular conferences was often determined more by faculty availability than by the natural progression of topics for resident learners. Faculty taught some topics repeatedly (sometimes with the same lectures minimally updated and delivered year after year) and failed to attend to other topics. Faculty focused on transmitting information; they rarely asked learners in noon conferences to practice skills or solve problems, and they only infrequently addressed learning in any of the core ACGME competency domains other than medical knowledge.
The noon conference was largely a curricular design of convenience intended to minimize interruptions in patient care work flow. Independently, leaders at all three programs desired a better alternative. Although they shared similar frustrations with the traditional noon conference, the immediate catalyst for change at each of their programs was distinct. At Cambridge, the creation of a nonteaching hospitalist service opened a window of opportunity; at Cincinnati, loss of funding for lunch tipped the balance; at Cornell, a new program director (L.L.) brought a new curricular vision. Encouraged by the experience of other residency programs (most prominently Tulane’s innovative “Friday School”16), the leaders of the residency programs launched their own AHDs.
Three Approaches to Innovation: AHD Design and Implementation
At all three institutions, program directors and associate program directors convened AHD design teams that included faculty, residents, and chief residents. Table 1 describes the four- to six-month planning process at each site. Design teams drew on a variety of available internal and external resources. Cornell’s program leaders were unique in reaching out across their academic institution to Columbia University’s Center for Education Research and Evaluation for support.
At all three sites, the AHD curriculum described herein focuses on inpatient education because separate curricular structures deliver ambulatory curriculum.17 In addition to the AHD curriculum, residents at each site participate in a variable 8 to 16 hours weekly of additional learning sessions, including grand rounds, morbidity and mortality conferences, journal club sessions, safety and quality conferences, resident-led lectures and presentations, radiology and pathology conferences, and reflective practice seminars.
Leaders at all three programs aimed to improve resident and faculty satisfaction and resident learning outcomes. Design teams in all three sites articulated guiding principles. Retrospective review identified six shared core principles across all three AHDs:
1. Protect time and space to facilitate learning
2. Nurture active learning in residents
3. Choose and sequence curricular content deliberately
4. Develop faculty
5. Encourage resident preparation and accountability for learning
6. Employ a continuous improvement approach to curriculum development and evaluation
As Table 1 conveys, all three design teams operationalized—sometimes by intent and sometimes secondary to pragmatic considerations—the shared principles differently from one another.
Protect time and space to facilitate learning
The program directors and/or associate program directors at all three sites recognized that extending the teaching sessions beyond the previous 50-minute noon conference lecture would facilitate increased learner engagement with the material. The longer block of time not only permits greater choice in pedagogy and offers learners more flexibility to internalize concepts at their own pace but also creates a weekly shared retreat from clinical duties that nurtures learner investment in education and cultivates a learning community.
All sites experienced initial resistance from the hospital’s clinical leaders who expressed concerns about the AHD’s impact on patient flow and coverage. Though net weekly didactic time with the AHD schedule has actually decreased relative to the previous noon conference schedule, the time in which residents are unavailable is more concentrated. All three programs (see Table 1 and Table 2) have succeeded in meeting hospitalized patients’ needs during the AHD without significant difficulty. At Cambridge, the Department of Medicine invested in a single midlevel provider who supports the hospitalist service during the weekly AHD. AHD clinical coverage at Cincinnati and Cornell is provided by already-employed hospitalists, fellows, and other residents at no additional cost.
Though the three programs share a commitment to protected time and space for learning, the AHD sessions at each site are not of equal length, as Table 1 shows. Cincinnati is unique in sponsoring a second, shorter weekly AHD reprise (known as AHD Echo) for learners unable to attend the principal weekly AHD session. Residents who miss AHD programming at Cambridge and Cornell can find teaching materials posted on the intranet; Cornell leaders are also exploring the possibility of making conference video recording available. At Cambridge and Cincinnati, residents from all three postgraduate years learn together in the traditional “one-room” schoolhouse, whereas room-size constraints in Manhattan have necessitated separate academic time for each postgraduate year at Cornell.
Nurture active learning in residents
All three AHD programs prioritize active learning, but each employs different strategies, as Table 1 describes. We include a sample lesson plan from the University of Cincinnati’s AHD as Supplementary Digital Appendix A, http://links.lww.com/ACADMED/A122.
AHD leaders have encouraged an active pedagogy in an evolutionary fashion across three years by encouraging and supporting faculty innovation. Specialist faculty serve as “course directors” for four-week curricular blocks and work in partnership with the associate program director (M.B.) to plan their blocks. The associate program director coaches AHD faculty members to shift their focus from delivering information to creating opportunities for learners to apply information through practice. The associate program director also helps the specialist faculty members to experiment with a variety of educational methodologies: small-group and case-based problem solving, real-time exploration of the evidence, panel discussion and debate, role-play and simulation, skills practice, games, and the engagement of patients as teachers.
During three years, the associate program director at Cambridge has identified best practices for active learning and shared these practices through biannual faculty conferences and monthly e-mails highlighting faculty creativity in curricular design and pedagogy. Each year, the associate program director asks course directors to incorporate an increasing number of established exercises into their respective blocks. In academic year 2011–2012, for example, each discipline-specific block included the following: (1) Reader’s Digest, a weekly review article prepared for small-group discussion that includes both a clinical case and questions, (2) Mystery Case, a complex clinical case presented in an unfolding manner across two hours during which residents work in small groups to answer evaluation and management questions before engaging in a large-group discussion with a multidisciplinary faculty panel, and (3) Lightning Lit Review, in which small groups of residents work outside of class to prepare five-minute summary presentations of selected articles and to frame a single question for discussion with the large group.
one of the program’s five chief residents designs each AHD conference. The chief resident typically opens a session with the learning objectives. Next, a faculty member provides a quick “theory burst” or five-minute introductory lecture. Residents then break into facilitated small groups to work on cases and problems prepared by the chief resident in consultation with the faculty advisor. Depending on the complexity of the case or exercises, groups may be admixed with regard to learner stage (e.g., third-year students with second-year residents), or segregated (e.g., fourth-year students with fourth-year students, interns with interns). Faculty members or chief residents serve as facilitators for each group. Facilitators receive training on how to focus the conversation at the residents’ learning edge—that is, to reinforce and expand each resident’s understanding of familiar concepts and introduce new ideas. Many activities are skills based (e.g., writing orders, demonstrating communication skills, solving acid–base problems, performing a physical exam maneuver, adjusting a ventilator), and facilitators can immediately evaluate learners’ ability to perform such tasks. After the small groups work through the activities, the larger group debriefs and reemphasizes the learning objectives.
leaders structure all sessions using a team-based learning format.18 Two designated associate program directors (of five total associate program directors) or the chief resident leader works with faculty to identify learning objectives, define cases and questions, identify reference material, and develop workbooks for each AHD session. Before each session, residents receive one or two key articles to read. In the first 5 to 10 minutes, residents use interactive audience response system software to answer independently three to five readiness questions. After individuals answer the questions independently, teams of five to six residents (assigned by the associate program director) work to answer the same questions together and then submit a second round of responses. Because the software displays on the screen all responses to the questions, the exercise allows individuals to identify their own knowledge gaps and compare themselves with their peers.
After considering the questions and before receiving correct answers, the resident teams proceed to cases presented in a workbook and work as a team on the associated learning activities. Teams use Ipad computers (Apple Corporation, Cupertino, California) to conduct real-time searches of the literature. These searches promote self-directed learning and build skills in using electronic clinical decision support. After teams complete the case-related exercises in the workbook, faculty facilitators and content experts work with the large group to identify correct answers and articulate explicit clinical reasoning.
Choose and sequence curricular content deliberately
Program directors and leaders at all three sites designed their AHD curriculum with the intent of delivering content sequenced according to a more explicit educational logic than the previous daily noon conference schedule permitted. The Cambridge team used syllabi from prepared American Board of Internal Medicine study materials; Cincinnati’s team used an internal medicine textbook and other extant materials; Cornell’s AHD leaders built curricular content around the In Training Exam (ITE) objectives. At all three sites, leaders sought to schedule content with developing complexity over the course of each year and over the course of a three-year residency program. Table 1 describes the different approaches.
Learners at all levels benefit from reviewing core principles. Engaging clinical cases in small groups usually permits learners enough flexibility to interact with the material in a developmentally appropriate manner. Still, the one-room schoolhouse format used at Cambridge and Cincinnati has posed challenges (see also Table 2). Upper-level residents are more likely than more junior learners to express frustration about sessions which combine learners across postgraduate years. Leaders at both Cambridge and Cincinnati have satisfied upper-level learners by, at times, dividing the large group into small groups by postgraduate year and by designing different higher-level exercises or guiding questions on the same topic for more advanced learners. Further, upper-level residents at Cambridge have increasingly, on their own initiative, assumed responsibility for AHD teaching, which encourages higher-level engagement with material.
The AHD curricula at both Cambridge and Cincinnati include cross-cutting longitudinal themes that weave across the AHD sessions. Leaders at Cambridge have identified evidence-based medicine, patient-centered care, and systems improvement as themes; leaders at Cincinnati have named EKG reading, geriatrics, and evidence-based medicine. The AHD curricula at all three programs differ in the degree to which topics outside of clinical biomedicine (e.g., health care systems delivery, quality improvement, communication skills, professionalism, leadership, advocacy, bioethics, research methods) are integrated. As mentioned, the AHD at all three programs fits into an array of other curricular didactics and experiences which facilitate learning in other domains.
The leaders of all three programs have prioritized faculty development. The mandate for more active resident learning requires clinical faculty to develop new teaching skills. In all of our institutions, program directors and associate program directors have organized faculty workshops, volunteered as AHD educational coaches, provided residents’ explicit feedback to faculty, and made choices about which faculty to include as AHD teachers. The leaders have consistently encouraged faculty to let go of some content; lecture-based teaching places a premium on covering large quantities of information, whereas all three AHD models value, instead, more meaningful engagement with fewer topics. As Table 1 describes, the faculty at Cambridge have retained a central role in the design and delivery of the core curriculum. At Cincinnati, the five chief residents, and at Cornell, the five associate program directors and the chief resident, have assumed primary leadership for AHD educational methodology and session planning while each AHD faculty member plays the role of content area expert.
Encourage resident preparation and accountability for learning
At all three programs, the AHD has raised expectations regarding learner preparation for conferences and accountability for learning outcomes. As Table 1 describes, residents at all three institutions receive reading material in advance of each weekly session. At Cambridge, residents complete a 30-item board-style exam at the end of every four-week discipline-specific block. At Cincinnati, residents take a brief multiple-choice test at the end of each AHD session. At Cornell, sessions begin with a quiz based on the reading as prescribed by the team-based learning format.18
Employ a continuous improvement approach to curriculum development and evaluation
Leaders at all three institutions developed the AHD programs with a strong commitment to formative evaluation, a willingness to learn from mistakes, and an intent to encourage evolution and continuous improvement. Program evaluation at all three institutions includes both qualitative and quantitative feedback from faculty and residents. Table 2 highlights challenges that program directors have encountered in the first three years of their AHD experiences and describes their responses.
Outcomes: The Innovation’s Impact
All three programs endeavored to improve learning outcomes for residents and to enhance resident and faculty satisfaction. We acquired the majority of the evaluative information we present about the AHD programs through routinely administered resident surveys and examination. The Cambridge Health Alliance institutional review board found that AHD-specific surveys of residents and faculty at Cambridge met criteria for exemption; program leaders at Cincinnati and Cornell collected no data from human participants outside of data collected for routine residency program self-evaluation. With the AHD in its first year of implementation, the Cornell program has limited available outcome data. Data from Cambridge and Cincinnati, summarized in Table 3, suggest significant improvement in resident conference attendance and resident satisfaction.
Faculty enthusiasm for the AHD is also high, as supported by anecdotal evidence at Cincinnati and Cornell. On a survey in academic year 2010–2011, 100% of Cambridge faculty surveyed (n = 20) endorsed that they agreed or strongly agreed that they were excited to continue to develop the AHD model; over 90% of faculty reported that they agreed or strongly agreed that they had tried new teaching techniques because of the AHD and that the AHD had increased their collaboration with faculty colleagues.
Though we cannot claim causality, ITE scores at both Cambridge and Cincinnati have improved since the implementation of the AHD, suggesting that the new learning program may have had a positive effect on learning outcomes. At both programs, residents’ performance as measured by their ITE percentile scores had for several years fallen slightly between postgraduate years 1 and 2. After AHD initiation, both programs observed small improvements in ITE scores for residents between postgraduate years 1 and 2 (Table 4). Neither program observed a consistent effect on the change in ITE scores between postgraduate years 2 and 3 after AHD initiation.
Though difficult to measure quantitatively, we have also observed that the AHD has had a positive impact on the overall intellectual climate within the residency program. In surveys during academic years 2009–2010 and 2010–2011, 95% of surveyed Cambridge residents (n across both years = 41) agreed or strongly agreed that the AHD contributed to the creation of a “stimulating learning environment within the residency program,” and 90% of the surveyed faculty (n across both years = 35) agreed or strongly agreed that the AHD had “improved the learning environment” within the residency program. In the four years before the advent of the AHD in Cincinnati, only 4 of the 20 chief residents (20%) went on to pursue academic careers; in the three years since the introduction of the AHD, 12 of the 15 chief residents (80%) have chosen academic positions. Program leaders in all three sites have also noted improved resident recruitment after AHD initiation as demonstrated by increased numbers of qualified applicants and increasing numbers of positions filled by highly ranked applicants. Other changes in the three programs over the three-year time period, however, make attributing improvements in resident recruitment directly or solely to the AHD difficult.
Implications: Lessons for Medical Educators
Graduate medical educators are familiar with the limitations of the traditional lecture-based noon conference series for resident learning. Here we have detailed three successful approaches to the design and implementation of a curricular alternative, the AHD. The AHD designs differ considerably across sites in session length and relative degree of responsibility of specialist faculty for session planning. Despite differences among sites, each program’s AHD demonstrates fidelity to six core principles not previously in evidence in their respective noon conference programs: (1) protecting time and space to facilitate learning, (2) nurturing active learning in residents, (3) choosing and sequencing curricular content deliberately, (4) developing faculty, (5) encouraging resident preparation and accountability for learning, and (6) employing a continuous improvement approach to curriculum development and evaluation. Residency program directors might, in a manner less disruptive than creating a weekly AHD, substantially improve their noon conference series by adopting even just one of these principles. However, the six principles have face validity, and—importantly—are common across three independent, successful AHD models. Future study might endeavor to link any one of these principles to resident satisfaction and learning outcomes during residency. Other researchers might also examine whether the AHD model leads to durable outcomes in professional development or patient care.
The AHD provides an educational environment in which learners and faculty have sufficient time to engage in integrated learning as called for in the recent Carnegie Foundation report on the future of medical education.19 As active participants in the curriculum, residents develop habits of inquiry and improvement, another mandate from the Carnegie Foundation report. Challenges faced by medical educators seeking to prepare the next generation of leaders for a health care system undergoing rapid transformation are real, and the AHD represents one innovative means of meeting that challenge during graduate medical education.
Other disclosures: None.
Ethical approval: Program leaders at Cincinnati and Cornell collected no data from human participants outside of data collected for routine residency program self-evaluation. Surveys administered to faculty and residents at Cambridge were exempted by the Cambridge Health Alliance institutional review board.
Previous presentations: Maren Batalden presented the Cambridge Health Alliance AHD in a poster at the national meeting of the Society of General Internal Medicine, April 21, 2010. Lia Logio and colleagues presented the NYP/Weill Cornell model as an APDIM workshop in April 2012. Eric Warm and colleagues presented the University of Cincinnati AHD as an APDIM workshop in April 2010.
2. Cacamese SM, Eubank KJ, Hebert RS, Wright SM. Conference attendance and performance on the In-Training Examination in internal medicine. Med Teach. 2004;26:640–644
3. McDonald FS, Zeger SL, Kolars JC. Factors associated with medical knowledge acquisition during internal medicine residency. J Gen Intern Med. 2007;22:962–968
4. McDonald FS, Zeger SL, Kolars JC. Associations of conference attendance with internal medicine In-Training Examination scores. Mayo Clin Proc. 2008;83:449–453
5. Van Hoof TJ, Monson RJ, Majdalany GT, Giannotti TE, Meehan TP. A case study of medical grand rounds: Are we using effective methods? Acad Med. 2009;84:1144–1151
6. Davis D, O’Brien MA, Freemantle N, Wolf FM, Mazmanian P, Taylor-Vaisey A. Impact of formal continuing medical education: Do conferences, workshops, rounds, and other traditional continuing education activities change physician behavior or health care outcomes? JAMA. 1999;282:867–874
7. Jagsi R, Shapiro J, Weissman JS, Dorer DJ, Weinstein DF. The educational impact of ACGME limits on resident and fellow duty hours: A pre-post survey study. Acad Med. 2006;81:1059–1068
8. Brennan TA, Rothman DJ, Blank L, et al. Health industry practices that create conflicts of interest: A policy proposal for academic medical centers. JAMA. 2006;295:429–433
9. Campbell EG, Rao SR, DesRoches CM, et al. Physician professionalism and changes in physician–industry relationships from 2004 to 2009. Arch Intern Med. 2010;170:1820–1826
10. Fugh-Berman A, Brown SR, Trippett R, et al. Closing the door on pharma? A national survey of family medicine residencies regarding industry interactions. Acad Med. 2011;86:649–654
11. Holmboe ES, Bowen JL, Green M, et al. Reforming internal medicine residency training. A report from the Society of General Internal Medicine’s task force for residency reform. J Gen Intern Med. 2005;20:1165–1172
12. Thomas KG, Thomas MR, York EB, Dupras DM, Schultz HJ, Kolars JC. Teaching evidence-based medicine to internal medicine residents: The efficacy of conferences versus small-group discussion. Teach Learn Med. 2005;17:130–135
13. Weinberger SE, Pereira AG, Iobst WF, Mechaber AJ, Bronze MSAlliance for Academic Internal Medicine Education Redesign Task Force II. . Competency-based education and training in internal medicine. Ann Intern Med. 2010;153:751–756
14. Chalk C. The academic half-day in Canadian neurology residency programs. Can J Neurol Sci. 2004;31:511–513
15. Kanter SL. Toward better descriptions of innovations. Acad Med. 2008;83:703–704
16. Wiese JG, Guidry M, Deming S, Donald C. No more noon conference: The Friday school alternative. Poster presented at: Association of Program Directors in Internal Medicine Spring Meeting; April 2–5, 2006; Philadelphia, PA
17. Warm EJ, Schauer DP, Diers T, et al. The ambulatory long-block: An Accreditation Council for Graduate Medical Education (ACGME) educational innovations project (EIP). J Gen Intern Med. 2008;23:921–926
18. Michaelson L, Parmelee D, McMahon KK, Levine RE Team-Based Learning for Health Professions Education: A Guide to Using Small Groups for Improving Learning. 2008 Sterling, Va Stylus Publishing
19. Cooke M, Irby DM, Sullivan W, Ludmerer KM. American medical education 100 years after the Flexner report. N Engl J Med. 2006;355:1339–1344