Reuben, David B. MD; Bachrach, Peter S. PhD; McCreath, Heather PhD; Simpson, Deborah PhD; Bragg, Elizabeth J. PhD, RN; Warshaw, Gregg A. MD; Snyder, Rani MPA; Frank, Janet C. DrPH
By 2030, the U.S. population aged 65 years and older will exceed 70 million—approximately twice the number in 2000.1 In the future, as now, physicians and other health care professionals who are not geriatricians will provide the majority of health care for older people.2 Although some professionals will seek continuing education in geriatrics, many will receive geriatrics training only within the context of core training in their medical school, residency, or subspecialty training. This nongeriatrician workforce must have the attitudes, knowledge, and skills needed to provide high-quality care for older persons. Recognizing this need, in 2005 the American Geriatrics Society (AGS) identified five major goals for the future of geriatric medicine, including increasing the number of health care professionals who employ the principles of geriatric medicine in caring for older persons.3
Progress toward meeting this goal has been substantial but inadequate. From 1998 to 2000, the percentage of graduating medical students who reported that appropriate time had been devoted to geriatrics training increased by more than 5%.4 Nevertheless, in 2000, 39% of graduating medical students reported that the time devoted to instruction in geriatrics was inadequate.4 That year, the John A. Hartford Foundation funded a $5 million Geriatrics Curriculum Grants Initiative through the Association of American Colleges (AAMC) to support the design and implementation of geriatrics curricula in 40 U.S. medical schools.
In 2001, the Donald W. Reynolds Foundation awarded approximately $19.8 million to 10 medical schools (Cornell University, the Medical College of Wisconsin [MCW], the University of Hawaii, the University of Iowa, the University of Michigan, the University of Nebraska, the University of Rochester, the University of South Carolina [USC], Virginia Commonwealth University [VCU], and Yale University) to strengthen geriatrics training for medical students, residents, and practicing physicians. Each institution could receive up to $500,000 in total costs (including no more than 20% overhead) per year for four years (concluding in 2005) and was required to contribute a match of $1 for every $2 of foundation support. In addition, each institution had to make a formal commitment to sustain its program after grant funding ended. This initiative represented the largest single private foundation effort in geriatric education at these 10 schools and was followed in 2003 and 2006 by two additional cohorts of 10 medical schools each. Most of the schools requested and were granted a no-cost extension year. By 2006, the initial cohort completed their grants and Reynolds Foundation support ended at these 10 schools.
Before funding the third cohort of schools, the Reynolds Foundation commissioned the UCLA Multicampus Program in Geriatric Medicine and Gerontology to conduct an evaluation of the program's first cohort. We developed a multimethod evaluation plan to address the foundation's primary question: “Were these 10 funded programs successful at strengthening the geriatric training of medical students, residents, faculty, and practicing physicians?”
The evaluation, conducted in 2005–2006 at the conclusion of the grants (which ran from 2001 to 2005), focused on the structure, process, and outcome changes as a result of these programs, including dissemination activities to enhance geriatric education at other institutions. The study was approved by the UCLA institutional review board.
We undertook three main evaluation activities: (1) structured interviews of the program directors at each institution, (2) content analysis of key administrative data provided by the Reynolds Foundation, and (3) analysis of relevant secondary survey data. Each of these activities is described below. We used these activities to gather a broad perspective from those who were responsible for the program as well as the learners. This approach also allowed us to examine changes in structure, process, and outcomes with a framework that has been used extensively in evaluating quality.5 Although individual programs examined a wide variety of satisfaction, knowledge, attitudes, and performance of clinical skills to evaluate their specific training efforts, these were not standardized across programs or available in detail for this evaluation.
Program director interviews
Two experienced UCLA researchers (P.S.B. and another staff member) conducted hourlong structured telephone interviews with the 11 program directors of the 10 funded programs (one program had cochiefs) in January 2006. The interview protocol was designed to probe program leaders' perceptions and insights regarding accomplishments, obstacles and barriers, consolidation of geriatric infrastructure, key educational programming, successful dissemination methods, impacts on patient care, increases in departmental/divisional visibility, and unmet needs. Before the interviews, program leaders received, via e-mail, a set of talking points that listed these topic areas as well as limitations of funding and advice for the Reynolds Foundation to allow them to gather appropriate materials and to focus their thoughts.
With permission of the interviewees, we audiorecorded all interviews. The interviewers content-analyzed transcriptions of each of the interviews they conducted to identify themes from the interviews. Each interviewer then reviewed the content analysis of the other interviewers' transcriptions. Discrepancies between the two interviewers about content or themes were reviewed with another author (H.M.) to resolve these discrepancies and develop a final list of common themes across sites.
Evaluation of administrative data
The Reynolds Foundation provided two forms of administrative data to the evaluation team for this project: reports written by Reynolds Foundation Advisory Committee members who conducted site visits, and final reports submitted by project directors to the Reynolds Foundation. We abstracted administrative data from these archival data sets across four domains: (1) physician training, (2) environment, (3) program success, and (4) products. We then classified usable data using an iterative process of two coders who read each document, discussed the presence or absence of a particular datum, and obtained consensus on its appropriate domain. Factors examined in these site visit reports included major accomplishments, unanticipated accomplishments, unmet goals, barriers to meeting goals, and an overall evaluative section highlighting strengths, weaknesses, and recommendations.
Data from the final reports included numbers of physicians trained, environmental changes, perceived program successes, and products including new or revised courses, curricula/modules, training methods, rotations (developed or modified), tools (developed or modified), and internal and external dissemination. We did not have direct information about how programs spent their funds. So, to determine the reach of the funding, we determined the relative number of learners at each level of training (percent of total learners for each level of trainee, based on the median for 10 institutions).
Secondary survey data
An online survey was conducted by the Association of Geriatric Academic Program Directors-Status of Geriatrics Workforce Study in the spring of 2005.6 All directors of geriatric academic programs at the 125 allopathic medical schools accredited by the Liaison Committee on Medical Education were included. The response rate for allopathic schools was 71%, with 89 of 125 schools reporting. We used this data source to obtain information on numbers of faculty and professional staff working in divisions/departments of geriatrics and perceived obstacles to increased geriatrics training.
Finally, we used two additional secondary data sources in the evaluation. A subset of data from the AAMC Medical School Graduation Questionnaire administered to graduating senior medical students in 2001 (before grant receipt), in 2005 (at the conclusion of grant awards), and in 2007 (the most recent available data) provided cohort-specific and national norms. The AAMC data focused on the appropriateness and amount of exposure to geriatrics/gerontological training during medical school. Chi-square analyses were used to compare responses between Reynolds programs with non-Reynolds programs at each year and to compare 2001, 2005, and 2007 responses at Reynolds schools. The second source was drawn from the annual report on medical schools published in the Journal of the American Medical Association (JAMA) on medical school enrollment to obtain calculation of the percentage of U.S. medical students who received training under the first cohort of Reynolds funding.7–11
The required match of $1 for every $2 of Reynolds funding was exceeded by all institutions. None of the program directors described unmet needs in their interviews. Reynolds schools developed programs that largely incorporated geriatric content into the medical student curriculum (63% of learners), but considerable resources were also devoted to resident (18%), postresidency fellowship (2%), and community-based nonfaculty practitioner (17%) training.
At the medical student level, geriatric content was integrated within existing topics such as physician–patient communication, end-of-life care, chronic disease management, and mental health. These changes were well received, and several program directors mentioned that the response from both students and faculty exceeded expectations. At the basic science level, programs reported seamless infusion of aging content into anatomy and biology coursework. Seven schools developed new courses to teach geriatrics, and courses were substantially revised to incorporate more geriatrics content at seven schools (Table 1). Only two schools did neither. Eight schools revised residency curricula, and eight developed new programs for teaching community-based physicians.
All 10 institutions reported structural changes in clinical education as well, including newly developed or revised geriatric rotations for their trainees or placement of students in new clinical situations (e.g., rotating through the geriatrics inpatient unit at Cornell, mandatory home visits at Rochester). All schools either added and/or revised their clinical rotations: Seven added new clinical rotations, nine revised existing clinical rotations, and six schools did both. For example, Michigan developed new geriatric rehabilitation and gynecological rotations, Rochester's emergency medicine rotation now includes functional assessment on geriatric patients, and Yale added geriatric rounds to its VA psychiatry residency requirements.
To provide faculty resources for increased geriatrics education, eight institutions engaged in formal faculty development programs with the intent that once faculty were more focused on geriatrics in the curriculum, they would continue to teach geriatrics long after the Reynolds funding ended. For example, 28 faculty at USC, including the department chairs for emergency medicine and surgery, participated in the school's Faculty Scholars in Aging program. The University of Michigan implemented a nine-month geriatrics faculty development program for medical and surgical specialties and subspecialties of internal medicine, and VCU provided 50 faculty scholarships to the Virginia Geriatric Conference.
Nine schools reported changes in infrastructure. Typically, these changes included more systematic interactions and collaborative efforts between program faculty and offices of medical education and evaluation researchers, as well as closer cooperation with medical school deans. For example, the office of educational services at MCW facilitated resource use and evaluation between the medical student and residency/fellow-related geriatric education initiatives from internal medicine, family and community medicine, and psychiatry and behavioral medicine. The University of Michigan's geriatrics program developed a partnership with the school's education evaluation unit, and VCU added staff to coordinate programs and provide administrative support, which enabled the program to work as a whole. Overall, these types of changes resulted in the integration of geriatrics into the basic structure of medical education at these schools.
Five institutions added faculty and six added staff to meet the demands of increased training. For example, Cornell created a new position of educational coordinator for the division of geriatrics, Hawaii added a full-time residency coordinator, and Iowa added a director of e-learning. Four schools reported changes in clinical care as a result of the grant, including the University of Iowa and VCU, which created or enhanced inpatient geriatric consult services, and Cornell, which created an acute care of the elderly unit. Finally, six medical schools added AGS student chapters, which provided additional publicity for the programs (e.g., sponsored sessions at MCW drew more than 100 students to selected events) and may promote entry into careers in geriatrics.
The 10 funded institutions incorporated a variety of educational approaches to teaching geriatrics (Table 2). All schools added new lectures and new interspecialty clinical experiences. Most used case discussions, standardized patients, online educational materials, and compact discs/videos and sent students to new sites (eight programs). Examples of innovative approaches included a Senior Mentor Program for students at USC, Cornell's “Fast-Forward Rounds” that link residents with third-year medical students, a geriatric gynecology curriculum developed at Hawaii, a Web-based Geriatrics Education Consult system curriculum at Michigan, a geriatric anesthesia module at Rochester, and a geriatric cardiology module at MCW. Reynolds support also allowed schools to pilot and refine innovative approaches to teaching and learning, such as Michigan's Web-based portfolios. In addition, a variety of products (e.g., pocket cards, personal digital assistant tools, modules, assessment instruments) were developed that could be exported to other institutions. More detailed information about many of these is available through the Portal of Geriatric Online Education (POGOe) (http://knowledgemap2.mc.vanderbilt.edu/pogoe/pogoe2) or from the first author, who can provide contact information for the individual program.
The grantee institutions provided geriatric training at all levels, from medical students and residents to faculty and nonfaculty community practitioners (Table 3). On average, each institution trained more than 1,000 medical students, 500 residents, 100 faculty, and 700 nonfaculty community physicians during the grant period. Based on 2001–2005 JAMA medical school matriculation data, Reynolds funding of these 10 schools reached 7% of all U.S. medical students during the grant period. In addition to providing training for primary care residents, the Reynolds institutions provided geriatrics training across 22 non-primary-care disciplines (Table 4). Each Reynolds site trained at least two non-primary-care disciplines, with MCW providing training to 13 non-primary-care disciplines and Michigan and VCU providing geriatric training to 12. Besides primary care disciplines, psychiatry (at 8 programs) and surgery (at 7 programs) were the next most common specialties to receive geriatric training under the grants.
Using data from the AAMC Medical School Graduation Questionnaire, we compared schools that received Reynolds funding with those that did not for the years 2001, 2005 (when the evaluation was conducted), and 2007, which are the most recently available data (Figure 1a–f). In 2001, responses to questions related to geriatrics education from medical students at Reynolds-funded schools were similar to those at non-Reynolds-funded schools. However, during the course of the award period, students' reported geriatrics experiences dramatically increased at the Reynolds-funded schools. By 2005, students at first-cohort Reynolds-supported schools reported higher levels of geriatrics/gerontology education, more exposure to expert geriatric care from their attendings, and more use of small-group exercises and interdisciplinary approaches when compared with students at non-Reynolds-award schools. Graduating Reynolds school seniors also reported more use of small-group exercises and interdisciplinary approaches to increase geriatric knowledge. For the four geriatrics/gerontology items asked again in 2007, most of the gains at the Reynolds schools were maintained; only graduates' perceptions of four years of geriatrics education dropped significantly. Interestingly, graduates at non-Reynolds schools reported similar experiences in 2007 to those of 2005 graduates from Reynolds schools.
As of April 2006, when Reynolds Foundation support ended and at the end of this evaluation period, 78 papers and 334 conference presentations had been produced as a result of the Reynolds initiative. Papers were published in top journals such as the Journal of the American Geriatrics Society, JAMA, and the Gerontologist on such diverse topics as geriatric standardized patients, geriatric clinical decision making, and the evaluation of geriatric training programs.
Dissemination extended beyond the traditional methods of journal publications and conference presentations. POGOe, an online repository of geriatric educational materials supported by the Reynolds Foundation, the Family Medicine Digital Resources Library (FMDRL), and the AAMC MedEdPORTAL (Providing online Resources to Advance Learning) emerged as efficient and productive methods to disseminate toolkits, Web-based educational modules, and other types of products, such as newly developed screening and assessment tools. “Graduation and exportation” of program trainees also seemed to be a successful dissemination method; for example, two Hawaii trainees established geriatric programs at other institutions: Kaiser in California and Morehouse College in Atlanta.
Additionally, all first-cohort schools disseminated their geriatric expertise and products via external requests for assistance and consultation. These included consultation requests from other institutions that were preparing their own Reynolds grant applications, and multiple requests for newly developed pocket cards from the University of Nebraska, as well as various slide sets and curricula. Iowa's GeriSims modules have been particularly broadly disseminated.
The Reynolds programs also reported dissemination of geriatric content internally through cross-pollination by faculty from other departments and divisions who had been recruited to teach geriatric content. As a result, the Reynolds programs successfully integrated geriatric content into traditionally nongeriatric disciplines, such as surgery, aesthesia, dermatology, neurology, ophthalmology, rheumatology, emergency medicine, and obstetrics–gynecology (see Table 4).
In recognition of the rising numbers of older Americans, medical education in the United States has begun to respond to the need to train medical students, residents, and practicing physicians to adequately care for this population. Private foundations and federal and state governments have provided support for medical education in geriatrics. In particular, the John A. Hartford Foundation has had a long-standing commitment to geriatrics education at the medical student, primary care residency, and practicing physician levels and, more recently (with Atlantic Philanthropies), in surgical and related specialties. During the past decade, the Donald W. Reynolds Foundation has provided support to two departments of geriatrics, 30 medical schools to improve medical education at their institutions, and 4 “consortium” schools to develop faculty to teach geriatrics. Based on the examination of the first 10 schools funded through the current Reynolds initiative, this funding strategy was highly successful in achieving its goals. As a result of the strategy's success, a previously unplanned fourth cohort of 10 medical schools has been funded.
First-cohort Reynolds programs were able to capture the attention of the pivotal educators at their institutions and became integrated into the fabric of medical education, both during the period of grant support and after funding ended. For example, geriatrics was frequently used as the content area for new curricular innovations (e.g., standardized patients, portfolios of patient encounters) that will likely be expanded to other content areas. Some of these innovations, such as video triggers, foreshadowed impending use in competency-based assessment (e.g., by the Accreditation Council for Graduate Medical Education) and/or in licensure examinations.12 Geriatrics faculty in these 10 schools became close partners with faculty in the offices of medical education, which may also facilitate the insertion of additional geriatrics content in the future. Numerous current and future physicians received geriatric training, many curricular changes were implemented within institutions, and geriatric content infiltrated well beyond expected audiences, including surgical specialties. Moreover, the students recognized these changes, reporting substantially more geriatrics education and being better prepared to care for older persons compared with medical students at schools that did not receive Reynolds funding. Most first-cohort programs invested heavily in faculty development both within and outside geriatrics, which will facilitate sustainability of program initiatives with associated long-lasting benefits to future trainees. Multiple products with broad dissemination potential were also developed and have been requested and adopted by non-Reynolds institutions.
The ability of the Reynolds program to affect geriatrics education beyond primary care specialties is particularly noteworthy. These physicians provide care for large numbers of older patients but typically receive little, if any, training in geriatrics. As of 2003, only 27 of 120 residency and fellowship programs had Accreditation Council for Graduate Medical Education requirements for geriatrics training.13
As a condition of the awards, first-cohort institutions have committed to sustaining programmatic changes beyond Reynolds funding, including support for faculty and staff additions. Less clear are the institutional commitments to update newly developed educational tools and technologies as the knowledge base grows.
Although the results of this evaluation are positive, some caveats should be outlined. First, with the exception of the structured interviews, none of the data we examined were specifically collected for external program evaluation use. Second, the use of qualitative interviews and the small sample size precluded analysis beyond descriptive statistics. However, the quantitative context provided by AAMC and JAMA data support the qualitative results. Third, because there was no uniform specification of goals across programs (i.e., each program specified its own goals and could only be judged against itself), cross-site comparisons were not possible. In addition, data from internal evaluations conducted by the individual sites (e.g., on trainee knowledge and attitudes) used a variety of instruments and sampling frames precluding their use for the overall program evaluation. Fourth, for most analyses, there was no control group. To compete successfully for funding, these 10 schools had to demonstrate their capacity for geriatric education through evidence that successful efforts were already underway in geriatrics and had to document substantial institutional commitment. Thus, it is impossible to know what these institutions would have accomplished even without funding. Finally, it must be recognized that although participating students, residents, and faculty received excellent training in geriatrics, it is too soon to judge the long-term effect of this initiative on the quality of care that older patients receive and on patient-level clinical outcomes.
This evaluation may also have value for schools that have not received Reynolds Foundation funding. In addition to learning about successful strategies that other schools have used to infiltrate geriatrics into undergraduate and graduate medical education, many of the educational products developed by these institutions are available through POGOe, FMDRL, and MedEdPORTAL. Moreover, the individual programs have been generous in sharing lessons learned and approaches with institutions that have not yet received Reynolds Foundation funding.
In summary, the investment of the Reynolds Foundation in the development of geriatrics training at 10 U.S. medical schools has substantially influenced undergraduate, graduate, and practicing physician education. New curricula and approaches have been integrated into the general medical education at these institutions, and faculty have been trained to provide ongoing education. The process of medical education has been changed, and some short-term outcomes have been demonstrated. Nevertheless, the full impact of these programs remains to be determined as first-cohort trainees enter practice or other health-related careers.
The authors appreciate the help of the principal investigators of the Reynolds Geriatrics Education grants for interview data and for reviewing sites-specific administrative data for accuracy.
This study was supported by a grant from the Donald W. Reynolds Foundation and the UCLA Older Americans Independence Center (Grant 5P30AG028748).