Andolsek, Kathryn M. MD, MPH; Murphy, Gwendolyn PhD; Nagler, Alisa JD, EdD; Moore, Peggy R.; Schlueter, Joanne; Weinerth, John L. MD; Cuffe, Michael S. MD, MBA; Dzau, Victor J. MD
Graduate medical education (GME) in the United States necessitates tremendous resources to offset its costs, which are only partially compensated by Medicare’s investment of over $9.5 billion.1 Duke University Hospital, as is true of many sponsoring institutions, supports a large number of GME positions well over the number of positions for which it receives Medicare funding.
Implementation of the new Common Program Requirements of the Accreditation Council on Graduate Medical Education (ACGME) and the next accreditation system add additional “unfunded mandates” to program directors. Pressures to optimize financial productivity while simultaneously revising GME educational processes have increased the role strain on academic faculty.2 As with clinical practice, there are increasing expectations for evidence-based medical education,3–6 an additional challenge for educators. Most program directors are highly creative teachers, often selected for their recognized passion for education, involvement with learners, creativity, and inventiveness. Resourcing program directors with strategic funding can empower them educationally, kindle (or rekindle) their excitement and creativity, and improve the retention of these individuals.7 Programs supported in this way can help medical educators to develop, evaluate, and disseminate best practices, resulting in cost-effective education. In this article, we describe the role of the Duke Graduate Medical Education’s Innovation Fund, supported by the GME Quasi-Endowment,* as a grassroots effort in nurturing program director innovations and creativity to enhance residents’ and fellows’ education and, ultimately, patient care.
As of this writing (2012), Duke University Hospital is the sponsoring institution for over 980 residents in 77 ACGME-accredited programs, 1 nonaccredited ACGME program, and 60 institutionally sponsored, non-ACGME programs.
In 2006, the chancellor for health affairs at Duke University and president and CEO of Duke University Health System (V.J.D.) and the board of directors created a $50 million quasi-endowment with health system reserves—the Duke Graduate Medical Education Quasi-Endowment—and deployed it for the support of Duke GME. With anticipated investment income generated by the fund, it was designed to infuse new financial support into GME for a decade or more. Over 70% of the funds available each year was allocated to direct program infrastructure support of both program directors and administrative staff. The remainder, approximately 29%, was designated to the Duke Innovation Fund for annual applications and special chancellor initiatives.
The mission of the Duke Innovation Fund is to enhance the learning environment and contribute to quality care. Its goal is to support faculty to create innovative approaches to teaching and learning. It is expected that these innovations will result in scholarly work and, through improved teaching and learning, improve patient outcomes. It is also anticipated that they will enhance the programs’ competiveness in leveraging additional resources. The process and results described below represent the first five years’ experience, 2007–2011.
The Awards Process
Oversight for the quasi-endowment was delegated to a GME oversight committee chaired by the vice president for medical affairs of the health system (M.S.C.) and the designated institutional official of Duke University Hospital (J.L.W.). Their first decision was to not allocate any funds during the first year of the quasi-endowment while formal program goals, accounting, and requirements were developed.
The oversight committee developed broad project guidelines and charged an Innovation Fund committee (IFC) with maintaining fiscal responsibility and promoting project success. The IFC, chaired by the associate director of GME (K.A.—a physician and former program director)—consists of representatives from the Office of GME (OGME). These include two doctoral-level educators, a representative from hospital finance, and the OGME chief of staff. A dedicated Innovation Fund project manager was added after two years. The IFC delineated the following criteria for successful projects. They should
* be innovative and likely to enhance teaching and learning in GME,
* be collaborative in creating common curriculum and assessment tools among two or more programs or departments,
* develop interventions that can be adapted by other programs,
* be focused on measuring educational (and ideally patient) outcomes,
* be short-term with high impact,
* be sustainable,
* be matched with in-kind funds, and
* promote scholarship.
The IFC designed an application process that was endorsed by the GME oversight committee. Application materials were developed and made available to program directors on a password-protected area of the OGME Web site. These include application templates and frequently asked questions. Electronic posters from prior successful projects were posted on the site. The process and materials have been refined yearly on the basis of participants’ annual feedback.
Program directors, along with other faculty and trainees, are encouraged to develop sustainable short-term (1–3 years) initiatives facilitating curricular development and assessment of one or more of the ACGME competencies. Funds must facilitate active collaboration among programs and departments and (ideally) measurably improve patient care. Suggested funding is in the range of $5,000 to 100,000 per year for up to three years. The maximum available for all programs combined is $2 million yearly. Funds cannot support existing trainees or program director time. When possible, they should be used to leverage additional resources, enhancing the sustainability of the projects. Grantees agree to present outcomes internally to Duke colleagues and in peer-reviewed venues.
Initially, a call for proposals was planned twice yearly. After the first cycle, this was decreased to an annual basis. Each IFC member reviews and scores each application individually using a standardized checklist. Points are awarded based on how well the proposal meets defined criteria. Each proposal is discussed, and consensus is reached on its relative merit without consideration of available funds. Budgetary items are assessed as to whether they meet criteria for funding. The summary, scorecard, notes, and IFC analysis are sent to the GME oversight committee that then makes its recommendations to the chancellor for the final selection. Projects are generally funded in order of their “score” until available funds are allocated.
Funded projects can receive full or partial funding for up to three years. Principal investigators (PIs) may apply for one subsequent cycle of funding for an additional three years. An annual electronic survey sent to PIs gathers feedback to determine whether the project is on task and within the scope of the original stated objectives. These progress reports assist programs to stay on task and provide the IFC and project manager with feedback on how to better support the projects. They also identify successes that can be shared.
The PIs submit final reports on completed projects detailing successes, challenges, plans for sustainability, and potential future projects. The IFC has increasingly created templates to facilitate the process for the initial application and budget, the progress report, and the final report. The budget template details expenses by quarter and is audited to ensure fiscal responsibility and progress toward project goals. Budget amounts for many commonly requested items, such as travel to a national meeting, computers, and administrative support, have been standardized.
An annual celebration highlights projects and successes. All PIs are encouraged to present a poster, regardless of where they are in project implementation. In addition, two to four projects are selected for a podium presentation. Electronic versions—using Slide Rocket (San Francisco, California)—of all of the posters are displayed on Duke’s GME Web site.
Beginning in FY2011, a 10% overhead rate has been applied to each funded project to support necessary management and oversight. This funds the project manager, project design management, and statistical support through the research management team (RMT), and the annual celebration. The RMT is available at no charge to PIs both before submission and after they are awarded funding. Partnering with the RMT has allowed the creation of tools that can be shared across projects.
Overall Experience With Proposals and Awards
The first funding was deployed in January 2008. Through December 2011, 105 proposals have been submitted, and 78 have been funded.
Funding requests have ranged from $3,651 to $348,750, with an average of $118,711. Total awards have ranged between $2,363 and $348,750. The average award was $66,391, which represents 42% of requested funds. As mentioned earlier, there is an annual “ceiling” of $2 million dollars allocated each year to all programs. So far, most projects have required fairly modest support. In the five years reported here, 12 projects (15%) each received less than $10,000; 20 (26%) received less than $20,000; and 43 (55%) received less than $50,000 (see Figure 1). On the other end of the cost spectrum, 4 projects (5%) each received more than $200,000. As of June 30, 2011, $5,540,000 had been committed and $3,461,000 had been spent. Because of effective stewardship of the quasi-endowment, funds are currently projected to continue 2½ years beyond the initial 10-year goal.
Table 1 lists the numbers of proposals submitted and approved and related information, by funding cycle, from January 2008 to July 2011. (No funding was made during the first year of the Innovation Fund.) All of Duke’s 13 clinical departments are represented. Many projects involve program, department, and medical center collaboration, as well as collaboration with entities of Duke University such as Duke University Fuqua’s School of Business, the Center for Documentary Film Studies, the School of Nursing, and the Duke University Health System Patient Safety Office.
Most PIs have accepted the grant award even when it provided less than the original request. Two PIs declined the funding that was awarded. One submitted a revised (and much enhanced) proposal for the subsequent funding cycle. This was approved, the funding accepted by the PI, and the project subsequently implemented.
Figure 2 describes funding by budget category. The majority of funds have been used for personnel, such as administrative support and project coordinators or managers. Program director, or PI, time is not funded and is expected to be contributed “in kind.”
Figure 3 classifies the Duke Innovation projects by the following educational categories:
* Teaching skills. Training the learner how to perform one or more specific skills, such as neonatal intubation
* Simulation. Training the learner in skills using mannequins, and “real” or virtual simulation
* Assessment and evaluation. Developing innovative ways to evaluate the learner
* Technology. Use of software and hardware to enhance teaching or evaluation (e.g., Webcasts)
* Educational resources. Materials made available for ongoing use (e.g., libraries of information, representative teaching scenarios, evaluation templates)
* Institution-wide educational projects. Projects implemented and made available across the entire institution
* Patient outcomes. For example, learning about and evaluating return visits to the emergency room, procedures completed in less time, or chart audits reflecting adherence to quality measures
* Professional development. Development for program directors, core faculty, and program coordinators
* Feedback. Strategies to enhance formative feedback to learners, and the documentation of such feedback
* Curriculum expertise. Basic knowledge and skills for an array of programs, typically not discipline-specific (e.g., health policy; communication)
* Duke partnerships. Collaboration outside of the Duke University School of Medicine (e.g., the School of Business, School of Nursing, Undergraduate Center for Instructional Technology)
* Interdisciplinary teamwork. Interaction among other health care team members
Many projects include elements of two or more categories. Projects most commonly involve innovative methods of assessment and evaluation.
Thirty-seven projects have been completed as of 2011, all of which were within budget. However, not all projects required all of the funds initially requested. Remaining funds from completed projects are reclaimed and applied toward future quasi-endowment growth. A total of $165,000 has been recovered from 11 completed projects.
Two projects were closed by the Innovation Committee because of lack of progress and/or failure to adhere to reporting requirements. One project requested additional funding after the first project year, and this funding was denied. Three submitted separate new proposals to further build on their projects. These were approved.
Twenty-eight of 35 completed projects (80%) have been completed on time. Fifteen projects in progress requested additional time, ranging from 3 to 12 months, which has been granted. The average duration of projects was 27 months, with a range from 6 months to 36 months.
Progress reports are submitted annually. Table 2 presents PIs’ views of the impacts of their projects from the annual progress reports. When asked to identify their projects’ major impact, most PIs described increased collaboration, enhanced interest in medical education, and greater use of technology. Ninety-two percent of PIs reported that these projects would not have been able to have been accomplished without the quasi-endowment funds. One PI expressed the view of many: that the funding allowed “the opportunity to develop something that … will benefit not only trainees here at Duke, but those at other institutions, as well as their patients.”
Examples of Specific Projects
The neurological exam for the non-neurologist
This one-year project resulted in a professionally produced DVD, illustrating how a “non-neurologist” should perform a basic neurological examination. The project grew out of the neurology program director’s observation that neurology was no longer a required medical student clerkship nor necessarily a rotation for all internal medicine residents. Neurological consults were called too frequently because of lack of skill or confidence and not because of a true “medical need.” The video has been posted online and as a podcast. This proved to be an extremely useful tool for a wide variety of learners. This instructional strategy is currently being replicated by two other innovation projects: the essential eye exam for the nonophthalmologist and the essential rheumatologic exam for the nonrheumatologist.
Improving evaluations using core competencies
The emergency medicine program director devised an electronic tool to evaluate core competencies. Faculty select from a drop-down menu of descriptors of a resident’s competency-based performance. Faculty can select common, behaviorally specific phrases to provide concrete suggestions for improvement. The result is greater uniformity in how faculty evaluations are stated, more useful information, and greater satisfaction from residents and faculty because of the specificity, clarity, and ease of feedback. Another program director plans to adapt this strategy, substituting narratives from the specialty’s emerging milestones.
Teaching health policy using live and Webcast/podcast strategies
The Office of Graduate Medical Education collaborated with Duke University’s Fuqua School of Business to offer a health policy lecture series to all residents and fellows addressing the ACGME systems-based practice competency. The content was offered live and by Web- and podcast. Seventy-seven residents and fellows from 8 (of 13) departments participated. There were no statistically significant differences in posttest scores between the live and Web/podcast groups. All participants reported a need for knowledge in health policy and valued the hybrid format for content delivery. The live group enjoyed the networking opportunities with a multidisciplinary group of colleagues. Because of its success, a second Innovation Fund project has developed a formal GME concentration in health policy.
Documenting medicine through film
This project paired 14 Duke residents and fellows from 9 departments with faculty from the Center for Documentary Studies to produce documentary films exploring medical issues.8 PIs believe that the experience of creating documentary film fosters insights into the societal context that informs the health and treatment of patients. This, in turn, can potentially improve treatment strategies and reduce costs. Participants report:
It’s made me a much more sensitive physician and a better listener—and that helps create much more individualized care for my patients.
After producing this project, I’m much more patient and compassionate about why people make the treatment decisions they do; in medicine we often look at problems in terms of numbers, say “X [number] of patients do this … with documentary you really give those numbers a face, a life, and help people work on a new level.”
Work culture and local community economics can significantly influence a worker’s decision to report work-related injuries. However, these factors are not readily visible in the exam room, and too often medical providers assume an overreporting of injuries under worker’s compensation. As a senior resident in occupational and environmental medicine, I frequently provide care for injured workers. This project explores one coal-miner’s experience following a work-related injury. It highlights the importance of understanding the worker’s experience—both in terms of workplace safety and socioeconomic position in the community. This is also my father’s story. His open and personal account of the medical care he receives following his injury is one of the most important lessons I’ll carry with me.
In the first five years, all 13 clinical departments benefited from at least one innovation project. Yearly progress reports provide insight into their impact. PIs are encouraged to work with the RMT to use project-specific, validated, and reliable evaluation methods and tools, incorporate informal feedback in project development, and analyze qualitative and quantitative evidence.
* On yearly progress reports, PIs have identified positive outcomes such as improved in-service training exams, faculty evaluations, and advising.
* PIs perceived that residents have assumed greater responsibility for their own learning.
* Attitudes toward patient safety have improved.
* There have been high-quality performance improvement projects institutionally and within departments and divisions.
* Clinical examination skills, and interest in medical education, have increased.
* Collaboration among faculty and trainees has improved.
* Curricular resources have been identified or developed.
* Groups that have not traditionally collaborated have discovered value working together. Some programs have applied the models/strategies/tools developed by another project to their own.
* A seminar on empathetic communication developed for internal medicine and pediatric residents was adapted for surgical residents with the intent to share the seminar with programs nationally.
* One department was awarded the 2012 teaching award of its academic association, an achievement that the vice chair attributes to national recognition of several innovations either initially piloted or evaluated through Innovation Fund grants.
* Projects are being shared and published in peer-reviewed venues and academic journals,9–20 and several others are in preparation for submission or awaiting journal decisions.
* There have been at least 68 posters, abstracts, and peer-reviewed presentations at venues such as the Accreditation Council for Graduate Medical Education, the Association of American Medical Colleges, the Institute for Healthcare Improvement, the University Healthcare Consortium, and specialty-specific conferences. Funding has allowed PIs to leverage additional funds from their own departments, specialty associations, and agencies such as the National Institutes of Health. PIs have submitted funding requests to the Picker Institute, Health Resources and Services Administration, and the Agency for Healthcare Research and Quality.
Environmental forces are dramatically changing the practice of medicine, which makes new ways of teaching and learning essential. The pressures noted by Irby and Wilkerson21 in 2003 are even more relevant today. The Institute of Medicine has called for increased rigor of research and funding to support educational innovation.22 There is a need for educators to share what works and what does not.23 However, it is challenging to obtain the funds necessary to allow for rigorous medical education research. Because most educational innovation is not formally funded and there are few traditional grant sources, one-third of successfully published educators have considered pursuing other fields.24
A few institutions have responded by providing seed money to allow teaching faculty time for educational innovation. Generally, these have had an effect on faculty for medical students.25,26 The Mayo Clinic provided both protected time and project funds but focused on a single course or program.27 In contrast, Duke’s Innovation Fund has more than five years of experience with its approach, with positive impacts on GME, scholarship, and sustainable projects.
What the projects did well
A Duke project cost an average of $66,391 during the five years reported here. The least expensive project cost $2,363, and over half of the projects cost less than $50,000. Costs were low partly because program directors were expected to contribute their own effort “in kind,” and the institution already funds some protected program director time. Choosing to defer any spending during the first year of the quasi-endowment, in retrospect, was an excellent decision. Markets were favorable, and the investment income generated increased the corpus by over 20%. Despite the economic downturn, effective stewardship contributed to the sustainability of projects and the overall program. Recovering unused funds and closing suboptimally performing projects saved money. The IFC was frequently able to trim budgets by identifying more cost-effective options for PIs. Many innovations have been incorporated into routine training. “Enduring” instructional strategies such as DVDs, Webcasts, or podcasts were effective modalities by which curricula “lived on.” “Train the trainer” models increased the capacity of existing faculty. Process enhancements were frequently embedded into the institutional electronic evaluation system, allowing broad accessibility across programs.
A centralized IFC promoted consistency across project years. The group was sufficiently small and flexible to respond quickly to feedback. In response, many improvements were created such as templates that streamlined the application and progress reports. The IFC identified consistent resource needs. Because most PIs lacked access to or were unfamiliar with statistical support, it was difficult for them to include a robust evaluation component. The RMT arranged to consult with PIs before and during the application process. The RMT also helped identify and refine project aims, design an appropriate evaluation prospectively, create and manage databases, and help manage projects. Costs for these types of activities, which were frequently overlooked, could then be built into the budget. Similarly, a dedicated medical center librarian was identified to assist PIs in researching the medical literature and classifying project deliverables. PIs frequently aren’t aware of, or haven’t sufficiently leveraged, low- or no-cost resources.
An annual professional development workshop offers guidance in project design and submissions. An expanded Web page facilitates access to ongoing and completed projects, networks PIs with similar interests, and helps to avoid duplication and redundancy. An electronic application, progress report, and budget tools have been developed. The IFC is matching institutional offices with PIs to attempt to monetize some projects. To further promote scholarly work, the IFC requires the final report to include an abstract.
There have also been challenges. The most significant has been the “dissemination” of the completed projects across the entire GME enterprise, despite holding annual celebrations, creating a dedicated Web site, and hosting periodic forums for the sharing of projects. Budgets were at times unrealistic. This was particularly true of simulation projects in “simulation-naïve” programs. Although the majority of PIs were extremely effective in their roles, at times, a minority of those who were not program directors struggled to obtain sufficient “buy-in” from necessary members of the team. Attempts to commercialize projects and returning any “profit” back to the fund are in a nascent phase.
Certainly, other choices regarding the management of the quasi-endowment and innovation support may have been made. The projects could have been funded from the quasi-endowment’s interest only, and spared the principle. This would have sustained the funds far “longer” but at the potential cost of having an indeterminate amount of funding year to year, especially if committed multiyear projects led to years when new projects could not be entertained. In addition, this approach would have been contradictory to the mandate of the quasi-endowment to support infrastructure as well as innovation. To meet both goals, a ceiling (and minimum) was set for annual expenditures, taking into account the regular infrastructure support and previously committed funds.
An IFC composed of program directors might have analyzed applications differently. As constituted, a more “neutral” group seemed effective because they could also serve as consultants to PIs from the preapplication to the implementation phases. Given their broad knowledge base and familiarity with institutional programs, they could frequently “match” one or more program directors to coordinate efforts on shared projects and identify cost-effective resources of which PIs were unaware.
The Increasing Importance of Grassroots Innovation
As academic health centers continue to come under greater financial pressure, their core and historic ability to innovate in GME and in all missions is threatened. Creating opportunities to foster grassroots innovation is vital to their educational success; small investments can foster sustaining projects and improve GME’s vitality. New models of scholarship and its inquiry will be required.28 Over a decade ago, Wilkerson and Irby29 identified the four foundations of a comprehensive faculty development program. One was “organizational development (which recognized that empowering faculty members to excel in their roles as educators requires organization policies and procedures that encourage and reward teaching and continual learning).”
Duke’s GME Innovation Fund is a potent tool for such organizational development. It has demonstrated that infusing relatively modest amounts fosters innovation and empowers program directors. The results are sustainable creative strategies to teach and evaluate residents, promote interdisciplinary collaboration, and enhance educational satisfaction.
Acknowledgments: The authors wish to thank Ms. Tammy Tuck, Office of GME chief of staff.
Funding/Support: The Duke GME Innovations Fund.
Other disclosures: None.
Ethical approval: The work was exempted by the Duke University School of Medicine institutional review board.
* A quasi-endowment comprises funds that the health system (not a donor) has set aside to resource a specific purpose, but that it could choose to take away. (See http://fbs.admin.utah.edu/endowment/definition/quasi/.) Cited Here...