As residents identify a patient population of interest, they submit proposals for CLARs on the basis of their interests and area of focus that will allow them to meet LPM year one objectives and prepare for the year two practicum. Each CLAR has templates for describing the work proposed for the rotation and for reporting and reflecting on what was actually accomplished. These templates are designed to help the resident focus his or her CLAR work on (1) defining and understanding the population of interest, (2) exploring the current care and gaps in care, (3) describing the microsystem(s) of relevance to that care, and (4) understanding the resident's selected microsystem's processes, cultures, and prior experiences with change. Faculty coaches approve residents' rotation proposals and assess the resident work produced. CLAR learning objectives and templates, practicum proposal templates, and evaluations are available on the DHLPMR Web site.20
In some cases, residents begin DHLPMR with a clear area of population-health-related interest; in others, the CLARs help residents define a suitable focus, which may differ significantly from where they started. As an example, one current resident came to our program with an interest in childhood obesity. Her first year focused on learning about the pediatric population served by DHMC and the extent to which obesity was diagnosed, documented, and addressed; assessing the current outcomes and processes of care for obese children and children at risk for obesity; assembling a team interested in the issue; understanding the barriers to optimal treatment; and working to develop a pediatric obesity program which is now the focus of her practicum. A different resident came to the program knowing she was interested in the outpatient management of medically complex patients, but without a defined starting place. Her work in the CLARs helped her to better define the population she wished to work with and to develop a specific intervention using a patient-centered care plan in the electronic health record. Another resident began the program with an interest in adult diabetic patients. After initial work on inpatient diabetes care, however, she determined that a more suitable focus for her work would be improvement of outpatient diabetes care at one of DHMC's satellite clinics where she regularly saw patients. Thus, although her first CLAR explored a population of patients hospitalized with diabetes, her later rotations involved learning about the diabetics cared for at the Lyme clinic, understanding their processes of outpatient care to propose changes in that care, and working with a team of providers, nurses, and staff from the clinic to collect data and understand how best to implement the changes which are now the substance of her practicum. A surgery resident who entered the program with experience and interest in surgical oncology developed an interest in trauma care as she completed her first CLAR. She spent the remainder of her first year learning more about trauma care at DHMC and about the particular challenges of rural trauma care. She is now completing a practicum that involves the population of trauma patients who arrive here directly as well as those who are transferred from outside hospitals.
During both LPM years, residents continue to see patients in their home clinical departments and remain an active part of the care team. We feel it is imperative that residents maintain the clinical skills they have already acquired and maintain relationships and credibility as they seek to lead change in those same settings.
Longitudinal public health experience
All residents spend at least one month working with a governmental public health agency. We have elected to spread this time across one or two years of the residency rather than have it concentrated in a one-month rotation. This allows residents to develop a lasting relationship with a public health mentor, and to participate in work that is longitudinal and meaningful for both the resident and the agency. As with CLARs, residents and their mentors agree on a project and work product for this longitudinal experience. In some cases, this governmental public health experience is a direct outgrowth of the work of the resident's practicum; examples of such projects include participation in state efforts for smoking cessation and developing state preventive health service recommendations. In other cases, residents may choose to learn about an aspect of public health on which they have not previously focused; examples of such projects include analysis of statewide emergency preparedness, review of prescription narcotic overdoses, and assessing maternal-child health services provided by a regional public health office.
MPH degree and other didactic sessions
The MPH program at The Dartmouth Institute for Health Policy and Clinical Practice is well suited to meet the needs of our residents, who must meet the MPH requirements to complete our program. The courses offer multiple opportunities for DHLPMR residents to use real data and to work in real clinical settings to fulfill coursework requirements, which are completed during the course of two years, with almost all the coursework occurring in LPM year one. In the fall term course on the Continuous Improvement of Health Care, residents “rehearse” the efforts that might be needed to improve care in a given setting; our resident working with pediatric obesity used this opportunity to review the literature on pediatric obesity and existing guidelines and to consider how these might be incorporated into her practice setting. The winter term course on Statistics for Improvement allows residents to analyze and effectively display data collected during their CLARs. The spring term course on Clinical Microsystems allows residents to work with clinicians and staff who will be affected by their practicum work; in the case of our resident working on outpatient diabetes care, a team of nurses, staff, and providers from the Lyme clinic participated in the class with her to facilitate understanding and productivity during her practicum. Table 1 displays the close alignment of MPH coursework and the work done by residents in their four sequential first-year rotations leading to the practicum.
During both years of the program, one half-day each week is reserved for program “didactics.” Examples of these didactic sessions include monthly case-based seminars on public health and critical issues in health care, as well as monthly sessions on leadership development and seminars on writing for publication. There is also a monthly journal club led by residents. Evening seminars are held every month except August and are open to the medical center community at large; they cover a wide range of topics in preventive medicine, public health, and health care improvement.
A portion of this protected didactic time is devoted to “work rounds” at least two times each month. This provides residents an opportunity to update faculty and other residents on their progress in preparing for or executing the practicum, the chance to rehearse important presentations, and an opportunity to receive feedback on their work. These sessions are attended by program faculty and coaches, as well as our improvement specialist.
In all of these didactic activities, the opportunity for both residents and faculty to interact with peers from other disciplines has been extraordinarily important. Learning about how other systems and settings work, and coming to understand how the culture of surgery and the culture of psychiatry, for example, both differ and resemble each other, is powerful.
Practicum review and approval
Each resident develops a practicum proposal21 with his or her coach during the second half of LPM year one as the first year of residency comes to an end. All practicum proposals must be approved by the PRB before completion of the first LPM year. This 20-person review board includes hospital physician and nurse leadership, faculty practice leadership, clinical chairs, the director of GME, LPM residents, the New Hampshire director of public health, and members of the DHLPMR Program Management Group (PMG). This board is responsible for assuring that the proposed practicum meets the requirements of preventive medicine training, fulfills residency learning objectives, and is consistent with institutional priorities. The board meets once to twice monthly for two hours. Its members are selected by and invited to participate by the DHLPMR PMG. There is no set membership term. Residents are encouraged to attend review board meetings as observers to learn about the practicum proposal and review process.
Residents do both a formative and summative presentation to the PRB. In addition to giving a 10-minute talk summarizing their proposed practicum, residents submit a written proposal to be reviewed in depth by two review board members. The extensive and thoughtful formative review provided for each proposal almost always results in substantive changes in the proposal before its summative review. Feedback from the formative review often results in changes not just to the proposal itself, but to the resident's understanding of the work of the practicum, and can lead to refinement and narrowing of focus for the target population or the proposed changes, more careful assessment of the potential sustainability of the proposed changes, or greater precision in the description of measures.
During the practicum year, residents are directly engaged in their work of improving health care for their defined population of patients. This involves measurement of patient and system need, work with frontline providers, development of leadership skills during implementation of changes, and continuous evaluation of both processes and outcomes. Residents are identified as leaders of these change initiatives, and, as such, their day-to-day work involves a wide range of activities. At the conclusion of the practicum year, residents complete a practicum report, which delivers information about the target population; the context, processes, and outcomes of care; improvements made and plans for sustaining them; enabling and inhibiting factors; and lessons learned. The practicum report template follows Davidoff and Batalden's22 proposed guidelines for publications on quality-improvement work. Residents present the results of their practicum to the PRB and the DHMC community at large and participate in facilitated reflection about what they learned about leadership, what they learned about making change, and how the changes will be sustained.
When the residency's learning objectives and curriculum were being conceptualized, we anticipated that residents would need regular, ongoing comprehensive coaching by a faculty member from the resident's home clinical department. It was clear that coaches would need dedicated time and personal commitment to assist residents in completing their practicum. Thus, the coach selection and development process started well before the first resident was accepted. Coaches are expected to meet the following criteria:
- Academic. Must be a local leader in scholarly productivity (publications, grants, presentations, teaching) and a recognized mentor to junior faculty and younger learners.
- Clinical. Must be an excellent clinician and respected among colleagues in his or her field.
- Change. Must have a track record for successfully leading change in technical, service, or cost-related improvement efforts within his or her department.
- Education. Preferably is a graduate of Dartmouth's Center for Evaluative Clinical Sciences (now The Dartmouth Institute for Health Policy and Clinical Practice); must understand and support the specialty of preventive medicine and appreciate the essential services necessary to promote public health.
The PMG, whose role and membership is described in detail below, worked with the department chairs to identify individuals as potential coaches and invited them to become part of a DHLPMR coaching faculty.
Coach development has been led by DHMC's director of quality education, management, and research. The coaches meet monthly to learn coaching skills and discuss their experiences. These meetings focus on the ongoing development of the program, the successes and challenges of coaching, improving resident and program evaluation, and resident recruitment. The opportunity to serve as a DHLPMR coach has been helpful in recruiting several new faculty to DHMC. Ongoing assessment of coaches offers them feedback on their contributions to both the program and residents; conversely, the program greatly benefits from coaches' input and evaluation. The program supports coaches for their time, offering up to 20% time for coaches with more than one resident.
A full-time PhD improvement specialist has also been hired to work closely with the residents and coaches. Her expertise in both quantitative and qualitative research, statistical analysis, and microsystems has led to better practicum experiences, has improved resident, coach, residency, and institutional learning about quality and improvement, and has enabled the program to assure each resident sufficient statistical and quantitative research support regardless of his or her coach's background or training.
Residents bring to their experience curiosity, commitment to do the best for patients, enthusiasm, and high motivation to improve the systems in which they work. They also have a deep, and often overlooked, knowledge of the systems they are trying to improve. Only residents already in a DHMC residency or fellowship can apply to DHLPMR. It is important that the home program director be highly supportive of the resident's application; we also ask for letters of support from faculty and the department chair. With permission, we review applicants' initial ERAS application and transcripts. All applicants must submit a separate application for the MPH program. Applicants are interviewed by the program director and at least two additional faculty, coaches, or members of the DHLPMR PMG. It is vital that applicants also meet with current DHLPMR residents. The PMG makes admission decisions.
To date, 34 residents have applied, 31 residents have been accepted, 12 residents have completed the program, and 3 residents have withdrawn because of career plans to concentrate solely in a clinical discipline. We had one resident our first year in 2003, and, since that time, have admitted four to six new residents each year. We have approval for 20 positions per year.
In addition to formal review of practicum proposals, residents are regularly evaluated by faculty, coaches, and others with whom they work. Evaluations are facilitated by the use of an online portfolio. Residents are evaluated (and complete self-assessments) on their CLAR work and the longitudinal public health work. They are also evaluated on their MPH coursework and participation in the regular didactic sessions. All residents take the preventive medicine in-training exam. Meetings with the program director or associate program director every six months are structured around an assessment which is tied to the expected level of achievement of the capabilities described earlier.
The PMG includes the program director, two associate program directors (one based at DHMC and one at Concord), two assistant program directors, two program managers, an evaluation specialist, the leader of our coach development program, an improvement specialist, and a resident representative. The PMG meets for two hours every other week. Agenda items typically include resident recruitment, program and resident evaluation, online learning portfolio development, and resident academic progress through the program. A daylong annual program retreat is attended by the members of the PMG and selected coaches and residents. This provides an opportunity to comprehensively review the program and plan improvements. Substantive changes, such as new assessment tools and the implementation of regular didactics for one half-day per week, have resulted from PMG retreat discussions and planning.
Our residency advisory committee (RAC) exists to assist the program director in reviewing overall program content and effectiveness, resident progression, and evaluations of the program. To date, no terms of membership have been set. The RAC meets twice a year; meetings regularly include resident presentations, a discussion of their work, and a review of each resident's performance. RAC meetings allow us to deeply inquire into one or more program component(s) and provide an additional opportunity to discuss program evaluations. They contribute to the development of the coaching faculty and help update program staff on national developments in the field of preventive medicine. These two-day meetings offer the program faculty, residents, and staff direct access to leaders in the field of preventive medicine, and they have contributed substantially to the ongoing improvement and development of the program. We recognize that we have attracted many to the work and leadership of this single GME program, and we work very actively to function as careful stewards of their time and their contributions.
Table 2 presents examples of the practicum work completed by our residents. Our graduates have continued their work in a variety of settings. Several have stayed on at DHMC and Concord as junior faculty, usually in positions that combine clinical practice with continued work on patient-care improvement. Others have successfully secured employment that includes time for system improvement as well as patient care. One graduate has followed a more “traditional” preventive medicine path and is now at the Centers for Disease Control and Prevention. Several have sought further training in fellowship programs.
Implications for others in GME
Locally, microsystem leaders, department chairs, and senior leadership have all seen that residents can successfully lead system change in an academic health care setting. They have also seen the challenges met (and not met) in making such change, and they have a new appreciation for the educational implications of such experiences. Much has been learned about how care is actually provided at DHMC and about what facilitates and frustrates change efforts. The interdisciplinary nature of the residents, coaching faculty, and PRB has brought together individuals from many settings and specialties to work together in a new way. Given their in-depth knowledge of and experience with the competencies of practice-based learning and improvement and systems-based practice, our residents are seen as resources, not only at the clinical microsystem level, but also for GME at DHMC.
On a national level, there has been interest in DHLPMR from both the GME community and the preventive medicine world. Many are curious about how to meld the experience of leading the improvement and change of patient care with the day-to-day clinical commitments of their residents. Others are attracted to the innovation of integrating the LPM experience with the resident's home department to effect change in an environment that the resident is both accountable to and familiar with. Still others in the preventive medicine community are excited about the new paradigm that the program brings to training in the field—improving the health of a population of patients within traditional institutions of care.
Moving this program from an idea to a reality involved several steps and the contributions of many people. Essential steps of the journey we followed were
- Creation of a shared aim.
- Leaders of GME, graduate education (MPH), and health care operations must be united around developing leadership learning and a commitment to focus local resources toward that aim.
- Identification of relevant local resources and strengths within them.
- The Dartmouth Institute for Health Policy and Clinical Practice offers specialized learning about the small systems of health care—clinical microsystems—and the leadership of change for improvement, in addition to offering all of the required MPH coursework. Many of our clinical faculty have taken advantage of master-degree study at The Dartmouth Institute. Our health care organization and its leaders are deeply committed to making changes for the improvement of the health of the population served. Those serving the public health needs of the setting were genuinely interested in working with younger learners.
- Formulation of a developmental path forward.
- We learned that this plan should integrate and build on local strengths and resources with a realistic understanding of the budgets and regulatory requirements involved. We plan to regularly revisit this.
- A proactive approach to communication with residency review committees and specialty board representatives.
- Because “combined” programs require approval of both educational accreditors and specialty certifying boards, concrete plans for the “learning journey” of each type of combined resident need to be reviewed in advance with the appropriate representatives.
- Construction of real learning opportunities that are continually improved with internal and external feedback.
- Designing preparatory sessions for the conduct of the practicum year, for example, has substantially improved the content of the practicum.
- Use of multidisciplinary management.
- The PMG draws on the skills of educators, evaluators, physicians, and administrators.
- Regular improvement program operations.
- Early on, we developed a learning portfolio to help us manage and evaluate the resident's learning experiences; now, we are working on the second-generation learning portfolio. We have continued to experiment with different ways of enabling the work of our coaching faculty—creating clearer definitions of resident work products and clearer expectations of coaching faculty regarding their roles in mentoring residents. Our annual program retreat offers an opportunity to spend longer, creative time on agreed-upon developmental needs for the program.
- Regular communication with the context in which the program lives.
- Through resident and faculty presentations, direct involvement of operations leaders, rotating terms on the PRB, coach and departmental program reviews, structured recruiting sessions planned with other departmental representatives, and other responses to ad hoc invitations, we seek to “keep the program current” in the eyes of the leaders of the setting in which the program lives.
In summary, these formative steps form a skeletal outline for the development of this program. We believe that each setting will have different strengths and different opportunities, but the general work of creating leaders able to improve the health of the population can proceed in many settings with the will and commitment to “making it happen.”
The authors have invited and received the help and participation of many as they have created this program, and they are deeply grateful for that. Specific thanks for their foundational and ongoing work in the creation of this program are due to Mariah Capurso, Thomas A. Colacchio, MD, Marvin Dunn, MD, Dan Eubank, MD, Paul Gardent, MBA, Dominick Geffken, MD, MPH, James Heimarck, MHA, David C. Leach, MD, Stephen Liu, MD, MPH, Joy McAvoy, and Eugene C. Nelson, DSc, MPH.
Additionally, the authors have benefited greatly from the contributions of past and current residency advisory committee members: George Anderson, MD, MPH, Jared Barton, MD, Brian Boulay, MD, Erica Boulay, MD, John Collins, JD, Jeffery R. Davis, MD, MPH, Daniel Eubank, MD, Gail Fayre, MD, Paul B. Gardent, MBA, CPA, Travis Harker, MD, MPH, Robert Harmon, MD, MPH, Sue Hassmiller, PhD, RN, William Kassler, MD, MPH, Adam Keller, MPH, Linda Kinsinger, MD, MPH, Jose T. Montero, MD, Patricia A. Nolan, MD, MPH, Gerald O'Connor, PhD, DSc, H. Worth Parker, MD, James Squires, MD, and Michael Zubkoff, PhD.
1 The Institute of Medicine. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press; 2001.
2 Wennberg JE, Cooper MM. The Dartmouth Atlas of Health Care. Hanover, NH: The Trustees of Dartmouth College; 1996.
3 McGlynn EA, Asch S, Adams J, et al. The quality of health care delivered to adults in the United States. N Engl J Med. 2003;348:2635–2645.
4 Nelson EC, Homa K, Mastanduno M, et al. A healthcare system's experience with publicly reporting comprehensive quality and cost data: A transparency initiative. Jt Comm J Qual Patient Saf. 2005;31:573–584.
5 Institute of Medicine. Training Physicians for Public Health Careers. Washington, DC: National Academy Press; 2007.
6 Maeshiro R. Public health practice and academic medicine: Promising partnerships regional medicine public health education centers—Two cycles. J Public Health Manag Pract. 2006;12:493–495.
8 Ducatman AM, Vanderploeg JM, Johnson M, et al. Residency training in preventive medicine: Challenges and opportunities. Am J Prev Med. 2005;28:403–412.
9 Medicare Payment Advisory Commission. Report to the Congress: Medicare Payment Policy. Hospital Inpatient and Outpatient Services: Assessing Payment Adequacy and Updating Payments. Washington, DC: MedPAC; 2007.
10 Nelson EC, Batalden PB, Huber TP, et al. Microsystems in health care: Part 1. Learning from high performing front-line clinical units. Jt Comm J Qual Saf. 2002;28:472–493.
11 Nelson EC, Batalden PB, Homa K, et al. Microsystems in health care: Part 2. Creating a rich information environment. Jt Comm J Qual Saf. 2003;29:5–15.
12 Godfrey MM, Nelson EC, Wasson JH, Mohr JJ, Batalden PB. Microsystems in health care: Part 3. Planning patient-centered services. Jt Comm J Qual Saf. 2003;29:159–170.
13 Wasson JH, Godfrey MM, Nelson EC, Mohr JJ, Batalden PB. Microsystems in health care: Part 4. Planning patient-centered care. Jt Comm J Qual Saf. 2003;29:227–237.
14 Batalden PB, Nelson EC, Mohr JJ, et al. Microsystems in health care: Part 5. How leaders are leading. Jt Comm J Qual Saf. 2003;29:297–308.
15 Mohr JJ, Barach P, Cravero JP, et al. Microsystems in health care: Part 6. Designing patient safety into the microsystem. Jt Comm J Qual Saf. 2003;29:401–408.
16 Kosnik LK, Espinosa JA. Microsystems in health care: Part 7. The microsystem as a platform for merging strategic planning and operations. Jt Comm J Qual Saf. 2003;29:452–459.
17 Huber TP, Godfrey MM, Nelson EC, Mohr JJ, Campbell C, Batalden PB. Microsystems in health care: Part 8. Developing people and improving work life: What front-line staff told us. Jt Comm J Qual Saf. 2003;29:512–522.
18 Batalden PB, Nelson EC, Edwards WH, Godfrey MM, Mohr JJ. Microsystems in health care: Part 9. Developing small clinical units to attain peak performance. Jt Comm J Qual Saf. 2003;29:575–585.
19 Nelson EC, Batalden PB, Godfrey MM. Quality by Design: A Clinical Microsystems Approach. San Francisco, Calif: Jossey Bass; 2007.
© 2008 Association of American Medical Colleges
22 Davidoff F, Batalden P. Toward stronger evidence on quality improvement. Draft publication guidelines: The beginning of a consensus project. Qual Saf Health Care. 2005;14:319.