Secondary Logo

Journal Logo

Aging Q3: An Initiative to Improve Internal Medicine Residents’ Geriatrics Knowledge, Skills, and Clinical Performance

Moran, William P. MD, MS; Zapka, Jane ScD; Iverson, Patty J. MA; Zhao, Yumin PhD, MS; Wiley, M. Kathleen MD; Pride, Pamela MD; Davis, Kimberly S. MD

doi: 10.1097/ACM.0b013e31824d4a10
Geriatrics Education and Training
Free
SDC

A growing number of older adults coupled with a limited number of physicians trained in geriatrics presents a major challenge to ensuring quality medical care for this population. Innovations to incorporate geriatrics education into internal medicine residency programs are needed. To meet this need, in 2009, faculty at the Medical University of South Carolina developed Aging Q3Quality Education, Quality Care, and Quality of Life. This multicomponent initiative recognizes the need for improved geriatrics educational tools and faculty development as well as systems changes to improve the knowledge and clinical performance of residents. To achieve these goals, faculty employ multiple intervention strategies, including lectures, rounds, academic detailing, visual cues, and electronic medical record prompts and decision support. The authors present examples from specific projects, based on care areas including vision screening, fall prevention, and caring for patients with dementia, all of which are based on the Assessing Care of Vulnerable Elders quality indicators. The authors describe the principles driving the design, implementation, and evaluation of the Aging Q3 program. They present data from multiple sources that illustrate the effectiveness of the interventions to meet the knowledge, skill level, and behavior goals. The authors also address major challenges, including the maintenance of the teaching and modeling interventions over time within the context of demanding primary care and inpatient settings. This organized, evidence-based approach to quality improvement in resident education, as well as faculty leadership development, holds promise for successfully incorporating geriatrics education into internal medicine residencies.

Dr. Moran is professor and division director, Division of General Internal Medicine and Geriatrics, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina.

Dr. Zapka is professor, Division of Biostatistics and Epidemiology, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina.

Ms. Iverson is program director, Aging Q3, Division of General Internal Medicine and Geriatrics, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina.

Dr. Zhao is research associate, Division of General Internal Medicine and Geriatrics, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina.

Dr. Wiley is associate professor, Division of General Internal Medicine and Geriatrics, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina.

Dr. Pride is assistant professor, Division of General Internal Medicine and Geriatrics, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina.

Dr. Davis is assistant professor, Division of General Internal Medicine and Geriatrics, Department of Medicine, Medical University of South Carolina, Charleston, South Carolina.

Correspondence should be addressed to Dr. Moran, MUSC 1255 Rutledge Tower, 135 Rutledge Ave., Charleston, SC 29425; telephone: (843) 792-5386; fax: (843) 792-7283; e-mail: moranw@musc.edu.

Adults aged 65 years and older currently constitute almost 13% of our nation’s population, and for many of these individuals, living longer means living with chronic conditions that limit one’s ability to perform many of the activities of daily life and increase one’s need for health care services. In 2008, the Institute of Medicine noted that unless academic health centers (AHCs) take action immediately, the health care workforce will be unable to effectively meet the needs of this growing population.1 Today, there are fewer than 7,200 certified geriatricians practicing in the United States, a number that continues to decline each year.1 The low reimbursement rate in medicine in general and the high complexity of patient care in geriatrics, coupled with the negative stereotypes of older adults and the perceived professional dissatisfaction compared with other specialties, have diverted students and residents from pursuing careers in the field.2 As a result, faculty at AHCs across the country have begun to recognize the need for curriculum innovation in geriatrics education for all physician trainees, regardless of their planned career paths.3 To meet this need, we sought to integrate geriatrics education into our existing general internal medicine services to improve residents’ skills in assessing and managing the medical challenges of caring for geriatrics patients.

Our program, Aging Q3Quality Education, Quality Care, and Quality of Life—is a four-year initiative at the Medical University of South Carolina (MUSC). In the quality education component of the program, general internal medicine faculty develop content to improve the quality of geriatrics education, encourage and observe residents in demonstrating the appropriate clinical skills required for caring for older adults, and reinforce the application of that knowledge and skill during patient care activities. In the quality care component, residents apply the knowledge and skills that they have learned in a supportive clinical environment. The quality-of-life component represents the final goal of the program—to improve patient outcomes as a result of improved education and care processes.

The Aging Q3 program is geared toward 96 internal medical residents at MUSC who are responsible for over 20,000 visits annually in the internal medicine continuity clinic and for staffing four inpatient medicine services. Twenty-seven general internal medicine clinical faculty (including 2 geriatricians) and 15 outpatient and 12 inpatient faculty physicians, with extensive educational and supervisory responsibility, oversee the program. Because residency training emphasizes experiential learning, we use innovative strategies to build residents’ geriatrics knowledge, skill level, and competencies while they practice medicine.4,5

In this article, we reflect on the development and current state of the Aging Q3 program, share examples and data from the implementation process and the measured impact of the program to date, and discuss lessons that we have learned throughout the process.

Back to Top | Article Outline

Developing the Aging Q3 Program

The MUSC environment resembles many other AHC environments in which faculty expertise in geriatrics is limited, yet the population under care is dominated by older adults. Recognizing this challenge, in 2008, the MUSC leadership formulated a strategic plan to commit internal resources and garner external resources to build an infrastructure to enable and sustain quality geriatrics education and care. The resulting initiative, Aging Q3, came to fruition in 2009 with the commitment of internal resources and an award from the Donald W. Reynolds Foundation.6

Back to Top | Article Outline

Guiding principles

Following this award, we and other MUSC faculty began to develop educational and clinical care improvement projects, each spanning a three-month period. We used as the basis for the development of these projects the Assessing Care of Vulnerable Elders (ACOVE) framework, which includes 16 quality-of-care indicators that have been shown to have a positive association with better care outcomes for older adults.7-9 We expect to develop, implement, and evaluate projects for each of the 16 ACOVE indicators within the four years of our initiative. The entire project will then will be repeated in two-year cycles.

In addition, we incorporated three important principles10 into the development of the Aging Q3 projects: (1) physician behavior change is more likely to occur if interventions are focused on multiple roles in the educational and clinical process (e.g., faculty, residents, nurses, and staff), (2) interventions are more effective if a combination of strategies is put into place, each of which has been individually proven to improve care (such combinations include lectures, brief point-of-care educational interventions such as academic detailing, which is based on the pharmaceutical marketing strategy in which faculty preceptors deliver brief, key messages during face-to-face encounters at the point of patient care, point-of-care cues, and electronic medical record [EMR] prompts and decision support), and (3) multiple objectives and data sources are used for monitoring and evaluating the interventions’ effectiveness. In designing Aging Q3, we strived to create projects that would foster an educational environment in which residents can learn the importance and health consequences of an aging issue, learn how to assess and treat that condition, and are prompted to do so at the bedside and in the clinic.

In the following sections, we discuss further the development, implementation, and evaluation of the projects that address three specific ACOVE indicators, which have been incorporated into the first year of internal medicine residency training at MUSC—vision screening, fall prevention, and caring for patients with dementia.

Back to Top | Article Outline

Convening faculty workgroups

In 2009, the MUSC leadership convened one interdisciplinary workgroup of five to eight faculty with a grant-funded faculty chair for each of the 16 ACOVE indicators. Each workgroup is responsible for three major tasks: (1) drafting the learning objectives and developing the educational content for their project, including writing lectures and academic detailing materials, (2) designing and implementing education strategies for their project, such as patient screening cues to action, reminders and data collection mechanisms in the EMR, ongoing monitoring strategies, and course correction adjustments, and (3) evaluating the effectiveness of their project in terms of the knowledge, skill level, and participation rate of the residents, faculty, and staff.

All workgroup chairs first serve on a workgroup to learn the Aging Q3 project development process. Workgroup chairs agree to a three-month commitment to do each of the following—serve on a workgroup; chair a workgroup; and implement, evaluate, and write up a report about the outcomes of the project. The total time commitment for each chair is approximately two to three hours biweekly for one year. During each two-year cycle, 16 faculty members will lead projects and participate in the development of reports based on their project’s outcomes.

The MUSC leadership focused on developing interdisciplinary workgroups and on recruiting a resident to champion each project and ensure that the residents’ perspectives were included in the development of the project.

Back to Top | Article Outline

Developing educational content

The first challenge for each workgroup is to identify learning objectives, based on the ACOVE quality indicators and other published evidence. For example, the vision screening workgroup focused their learning objectives on an understanding of the vision-threatening eye diseases of aging and the symptoms of eye disease, and a demonstration of a funduscopic examination (see Table 1). Next, each workgroup develops tailored lectures for three distinct audiencesresidents, faculty, and clinic staff.11

Table 1

Table 1

Workgroups also develop a brief, one- to two-page summary of the key issues related to their ACOVE indicator based on their learning objectives. At the bedside, faculty review this academic detailing sheet with the residents to briefly remind them of the factual information about the ACOVE indicator and the skills necessary for its evaluation and treatment.12,13 Workgroups then develop a focused patient question asked by the medical assistant or cue to action for faculty to use at the bedside to remind the residents to address the issue. For example, to address fall prevention, the medical assistant should ask the patient two screening questions when interviewing a patient: “Have you fallen in the past year?” and, if the patient says yes, “Have you fallen two or more times in the past year?” The cue for residents to remember to address these questions is included on a blue sheet of paper that is attached to the patient’s billing sheet and is mirrored in the corresponding EMR prompt (see below). Visual cues on each exam room door also remind the residents of the ACOVE indicator, and posters around the practice mirror the academic detailing sheet. Finally, workgroups develop an online survey to assess whether residents meet the specified project objectives.14

Back to Top | Article Outline

Implementing Aging Q3 Projects

As the workgroups develop the educational content for the individual projects, they also determine which resident and patient impact and outcome measures are essential and feasible for the evaluation of each project. Although aimed primarily at residents, the Aging Q3 program also targets faculty, clinic staff, and clinical systems in both inpatient and outpatient settings to achieve education, skill level, and performance goals.15,16 Because residents already have little time for added curriculum, we cannot expect a single educational intervention to reach all MUSC internal medicine residents in a three-month interval. To compensate, the workgroups employ multiple intervention strategies for each project, as shown in Figure 1.

Figure 1

Figure 1

Workgroups are also challenged to identify feasible monitoring methods for each project. Evaluation data sources that workgroups have chosen in the past include lecture attendance logs, project participation logs including which faculty were detailing which residents, and surveys of resident participants. They also have tracked staff participation in in-service sessions and the feasibility of implementing initial assessments at patient registration to supplement the other components in the project.

Workgroups found that the use of the EMR was key to prompting residents to both complete their assessments and collect the appropriate patient data. Using cue questions, the EMR progress note template for patients 65 years and older reminds residents to assess and/or address a specific ACOVE indicator. Using decision support embedded in the progress note template, residents respond to the EMR prompts. For example, if a patient answers yes to the question “Have you fallen two or more times in the past year?” the EMR will prompt the resident to complete a fall history, perform a Timed Up and Go test, and, if the patients fails the test, refer him or her to a physical therapist.17 Workgroups use the residents’ responses to the EMR-prompted questions as one source of monitoring and outcomes data.

Back to Top | Article Outline

Evaluating Aging Q3 Projects

At the end of the three-month project, the workgroup chair, with the support of the Aging Q3 evaluation team, is responsible for interpreting the data collected and developing a report on the project’s outcomes. Given that faculty are very busy themselves, the first several workgroups developed and refined a guide with examples to streamline the process for future workgroups at the evaluation stage of the project. This standardized but flexible guide is designed to facilitate a discussion of the important steps including setting priorities and designing content and methods for evaluating a project. The guide also includes how to consider evidence, why each ACOVE indicator was chosen, and an explanation of the process measures, as well as feasible knowledge, attitude, skill level, behavior, and outcome objectives. A sample guide is available from the authors on request. Because each project has different requirements, faculty created a generic logic model for each workgroup to use as a guide for tailoring their ACOVE-indicator-specific inputs, processes, and outcomes (see Chart 1).

Chart 1

Chart 1

Workgroups also use the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework to track specific, essential project activities to monitor the overall project performance.18,19 The RE-AIM framework urges workgroup chairs to consider (1) the project’s reach or the participation rate of each target population, (2) the appropriate and feasible effectiveness measures, (3) the adoption of Aging Q3 projects at the systems level and by other institutions, (4) the project’s implementation or the integrity, quality, and consistency of implementing the project, and (5) the maintenance of the project at the individual and systems levels.18,19 In the next section, we evaluate three projects—those focused on the ACOVE indicators of vision screening, fall prevention, and caring for patients with dementia—using the RE-AIM framework.

Back to Top | Article Outline

Reach

Monitoring the reach of each project is key. If the target participants—faculty, clinic staff, and residents—did not take part in the project, then its goals would not be met. We considered faculty participation to be crucial because their participation and support are essential to reaching our primary audience—residents. Involving clinic staff is also vital because they are the ones who provide the residents’ cues to action during patient care. Workgroups use a performance monitoring table to define the reach of each project and to make comparisons across projects (see Table 2). Workgroup chairs also receive weekly progress reports, which allow them to identify those faculty who are not participating and those residents who need detailing.

Table 2

Table 2

Back to Top | Article Outline

Effectiveness

Workgroups use effectiveness measures to assess whether the project has met its specified objectives. Each workgroup develops five to six pre- and posttest questions for an online survey of residents. Questions assess the important cognitive, attitudinal, skill level, and clinical behavior objectives of the project. For example, the vision screening workgroup survey includes case studies for the residents to address, and the fall prevention workgroup survey includes measures of residents’ self-efficacy.20

One key measure of a project’s effectiveness is the skill acquisition of the participating residents. For example, the vision screening skill objective is to learn how to perform a funduscopic exam. Faculty observe the residents and evaluate their performance. Recently, 73% of residents performed an adequate funduscopic exam (see Table 2). Another key measure of a project’s effectiveness is the residents’ adherence to clinical standards. For example, the vision screening workgroup promotes the standard of referring an older adult to an ophthalmologist if he or she has not had an eye exam in the past year. EMR reports indicate that residents referred 43% of eligible patients to the MUSC ophthalmology department, a number that excludes referrals to community eye specialists because these referrals are recorded only in an EMR text note and, therefore, cannot be extracted for measurement (see Table 2).

Back to Top | Article Outline

Adoption

A long-term goal of the Aging Q3 program is widespread adoption or that others will employ projects at a systems level at their institutions. Workgroups collect adoption data including the cost of implementation and level of resources and expertise required for implementation. To track the adoption of these projects, workgroups keep records of the continuing medical education credits provided to regional geriatrics groups; abstracts, papers, and panels at national conferences; and Web resources, such as the Donald W. Reynolds Foundation Portal of Geriatric Online Education.21

Back to Top | Article Outline

Implementation

Implementation refers to the integrity of the project or the quality and consistency of the delivery of the different components. The standardized guide, introduced above, encourages decision making based on clinical evidence and the delineation of explicit learning objectives and the design of systems changes. For example, the blue sheet that is attached to the patient’s billing sheet by reception staff begins the process; it includes questions such as “Does vision impair your daily activities?” The number of participants in each project cues leadership to whether the key players are being reached and are implementing the project as designed (see Table 2). The design and testing of the EMR cues and decision rules are particularly important and require workgroups to brainstorm and pretest to ensure accuracy and ease of use.

Back to Top | Article Outline

Maintenance

Maintaining a project at the individual staff member level refers to how well that individual consistently practices a behavior over the long term. Individual maintenance is particularly challenging to monitor given the nature of residency, in which participation in training groups constantly turns over and a plethora of priorities compete for residents’ time and attention. The primary goal of the Aging Q3 program is to enable residents to identify, evaluate, and treat or refer older adults with ACOVE conditions in an effective manner within the context of a brief yet busy patient encounter.

Maintaining a project at the institutional level, on the other hand, refers to the extent to which a behavior becomes the standard across an institution.22 Sustaining an educational intervention in an already-crowded curriculum is a challenge faced by faculty in geriatrics education.23 Each workgroup considers avenues for organizational-level adoption on a project-by-project basis. For example, after meeting the vision screening skill objective, the MUSC leadership purchased a digital retinal camera to encourage future attention to eye disease. In addition, the MUSC leadership added a query regarding falls to the preventive care and screening table in the outpatient EMR. This systems-wide change enabled and reinforced the value of individual physicians performing a regular proactive falls risk assessment whenever seeing older adult patients. Finally, the MUSC quality improvement group added the complete workgroup detailing sheet and assessment regarding caring for patients with dementia—specifically, how to distinguish between dementia, delirium, and depression—to their online library of inpatient assessments.

Back to Top | Article Outline

Challenges Faced, and Going Forward

The primary, long-term goal of the Aging Q3 program is to systematically integrate geriatrics knowledge and skills into the curriculum and clinical processes of residency training. We have set this goal in a time of economic cutbacks when faculty are challenged to do more clinical and educational work in less time. The workgroups have generally been successful in introducing new evidence-based content into the residency curriculum without disrupting the existing clinical workflow.24 They have accomplished this with the support of enthusiastic leaders and champions at our institution as well as those at the Donald W. Reynolds Foundation.

Differences between inpatient and outpatient venues have presented challenges for the workgroups, as they have for medical educators in other disciplines.25 Different clinical venues have different clinical areas of emphasis, which affects how each project is employed at each site. For example, faculty in the outpatient internal medicine practice focused on teaching residents to identify, evaluate, and treat patients with dementia; in the general internal medicine and geriatrics inpatient service, however, faculty focused on teaching residents to distinguish dementia from depression and delirium, to avoid the use of restraints in practice, and to formally work more closely with caregivers. Between these two settings, there were differences not only in the content objectives but also in the challenges that accompany different staffing patterns (clinic medical assistants versus floor nurses and discharge planners), teaching interactions (outpatient precepting versus inpatient rounds), and documentation systems (EMRs versus paper records). With each project, workgroup chairs became more aware of and attentive to the implications and feasibility of the objectives for projects carried out in two disparate venues.

Our experiences in the first year of the Aging Q3 program underscore the need in medicine for physician leadership development, a challenge increasingly discussed in the literature.26,27 Workgroup chairs must master the clinical content, develop the appropriate materials, and design and implement systematic prompts and cues for residents, all in a three-month period of time. Given the coemergence within medicine of lifelong learning goals and practice-based improvement, we see the need to build physicians’ skills to promote high-performance leadership26,27 by teaching them how to build consensus around a shared vision.28 The logic model that each workgroup develops provides faculty with a guide to instruct their work so that it is highly valued by each new workgroup chair, who then is not required to develop new guidelines for each project.

The EMR has proven critical to the success of the Aging Q3 program because it prompts residents to take action and provides decision support and a data collection mechanism for each project. Our EMR is limited, however, to the outpatient practice. For the inpatient service, only paper screens of patients, prompts to action, and manual data collection are available. Still, the EMR not only makes the Aging Q3 program more effective in the short term but also will be critical to sustaining curriculum and quality improvements.

Given the differences in residents’ schedules and learning styles, a multipronged educational approach is essential because one intervention cannot reach all residents. As such, the Aging Q3 program emphasizes learning during clinical rotations rather than through lectures. In fact, we observed a fairly low resident attendance rate at the Aging Q3 lectures, which we attributed to the fact that many residents come from different training facilities, such as emergency and clinical care units, from which attendance at such events is often low. However, the high rate of participation in academic detailing sessions assured us that educational content was delivered to the residents. In addition, residents who are unable to attend the lectures can instead view a recorded version online through the Aging Q3 Web site.29 Unfortunately, we are unable to track the number of residents who view the lectures online, and therefore we cannot assess the full reach of these resources. In the end, the complete education sequence for each project—faculty development resources, resident lectures, academic detailing sessions, cues to action and prompts at the point of care, and EMR decision support—has achieved a high rate of faculty and resident participation and measurable improvements in the quality of geriatrics care in a three- to six-month period of time.

Yet, as others in the literature have acknowledged,30 the difficulty lies not only in designing and implementing such projects but also in closely evaluating their impact. We acknowledge that our surveys, self-reported data, and short-term follow-ups are limitations of the Aging Q3 program. The use of the RE-AIM framework has forced workgroups to consider the tradeoffs in resources that are necessary when implementing such projects and tracking outcomes data.

Systematically integrating geriatrics education into the curriculum for all internal medicine residents at MUSC has been a difficult task, especially when faced with a shortage of geriatrics faculty. In this article, we have presented an educational model that may overcome some of these barriers. Furthermore, with the combined focuses on quality-of-care standards and standards for geriatrics competencies, it may increase the quality of institutional geriatrics training.22 We must continue to develop effective and efficient strategies to improve resident education in geriatrics to address the shortage of practitioners who are trained to treat the increasing numbers of older adults, to teach future generations of physicians, and to conduct research on the quality of geriatrics care.31

Acknowledgments: The authors acknowledge the efforts of the faculty at the University of South Carolina Columbia and residents and faculty from the Medical University of South Carolina who take part in the Aging Q3 program.

Funding/Support: The Donald W. Reynolds Foundation funded in part the Aging Q3 program with a 2009 grant, which the Medical University of South Carolina matched.

Other disclosures: None.

Ethical approval: The institutional review board at the Medical University of South Carolina deemed the Aging Q3 program exempt.

Back to Top | Article Outline

References

1. Institute of Medicine.. Committee on the Future Health Care Workforce for Older Americans. Retooling for an Aging America: Building the Health Care Workforce. 2008. Washington, DC National Academies Press
2. Fineberg HV. Retooling for an aging America. Medscape J Med. 2008;10:188
3. Laditka SB, Fischer M, Mathews KB, Sadlik JM, Warfel ME, Serv Q. There’s no place like home: Evaluating family medicine residents’ training in home care. Home Health Care. 2002;21:1–17
4. Lee M, Wilkerson L, Reuben DB, Ferrell BA. Development and validation of a geriatric knowledge test for medical students. J Am Geriatr Soc. 2004;52:983–988
5. Ackerly DC, Sangvai DG, Udayakumar K, et al. Training the next generation of physician–executives: An innovative residency pathway in management and leadership. Acad Med. 2011;86:575–579
6. D.W. Reynolds Foundation.. Aging and quality of life. http://www.dwreynolds.org/Programs/National/Aging/Aging.htm Accessed January 11, 2012
7. Wenger NS, Solomon DH, Amin A, et al. Application of Assessing Care of Vulnerable Elders—3 quality indicators to patients with advanced dementia and poor prognosis. J Am Geriatr Soc. 2007;55:S457–S463
8. Higashi T, Shekelle PG, Adams JL, et al. Quality of care is associated with survival in vulnerable older patients. Ann Intern Med. 2005;143:274–281
9. Rand Corporation.. Assessing Care of Vulnerable Elders: A Rand Health Project. http://www.rand.org/health/projects/acove.html Accessed January 11, 2012
    10. Thomas DC, Leipzig RM, Smith LG, Dunn K, Sullivan G, Callahan E. Improving geriatrics training in internal medicine residency programs: Best practices and sustainable solutions. Ann Intern Med. 2003;139:628–634
    11. Laditka SB, Jenkins CL, Eleazer GP, Kelsey SG. Geriatric expertise among medical school faculty: Preparing for the challenges of an aging population. Educ Gerontol. 2007;33:469–482
    12. Soumerai SB, Avorn J. Principles of educational outreach (‘academic detailing’) to improve clinical decision making. JAMA. 1990;263:549–556
    13. Schattner P, Markey P, Mathews M. Changing GPs’ clinical behaviour. What can divisions do? Aust Fam Physician. 2001;30:300–304
    14. Harris ED. Morning report. Ann Intern Med. 1993;119:430–431
    15. Mazotti L, Moylan A, Murphy E, Harper GM, Johnston CB, Hauer KE. Advancing geriatrics education: An efficient faculty development program for academic hospitalists increases geriatric teaching. J Hosp Med. 2010;5:541–546
    16. Johnson PA, Bookman A, Bailyn L, Harrington M, Orton P. Innovation in ambulatory care: A collaborative approach to redesigning the health care workplace. Acad Med. 2011;86:211–216
    17. Caton C, Wiley MK, Zhao Y, Moran WP, Zapka J. Improving internal medicine residents’ falls assessment and evaluation: An interdisciplinary, multistrategy program. J Am Geriatr Soc. 2011;59:1941–1946
    18. Glasgow RE, Nelson CC, Strycker LA, King DK. Using RE-AIM metrics to evaluate diabetes self-management support interventions. Am J Prev Med. 2006;30:67–73
    19. Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health. 1999;89:1322–1327
    20. Janz NK, Champion VL, Strecher VJGlanz K, Rimer BK, Lewis FM eds.. The health belief model—Theory, research and practice. Health Behavior and Health Education: Theory, Research and Practice. 2002.3rd ed San Francisco, Calif Jossey-Bass
    21. Donald W. Reynolds Foundation. Portal of Geriatric Online Education. http://www.pogoe.org/ Accessed January 11, 2012
    22. Leipzig RM, Granville L, Simpson D, Anderson MB, Sauvigne K, Soriano RP. Keeping granny safe on July 1: A consensus on minimum geriatrics competencies for graduating medical students. Acad Med. 2009;84:604–610
    23. van Zuilen MH, Mintzer MJ, Milanez MN, et al. A competency-based medical student curriculum targeting key geriatric syndromes. Gerontol Geriatr Educ. 2008;28:29–45
    24. Walker L, Sullivan G, Lindberg M, Cushman R, Gruman C. Geriatrics in generalist resident training—Attitudes and educational needs reported by recent graduates. Gerontol Geriatr Educ. 1998;19:79–88
    25. Thomas KG, West CP, Popkave C, et al. Alternative approaches to ambulatory training: Internal medicine residents’ and program directors’ perspectives. J Gen Intern Med. 2009;24:904–910
    26. Levine SA, Chao SH, Brett B, et al. Chief resident immersion training in the care of older adults: An innovative interspecialty education and leadership intervention. J Am Geriatr Soc. 2008;56:1140–1145
    27. Prather SE, Jones DN. Physician leadership: Influence on practice-based learning and improvement. J Contin Educ Health Prof. 2003;23:S63–S72
    28. Boland GW. From herding cats toward best practices: Standardizing the radiologic work process. AJR Am J Roentgenol. 2009;193:1593–1595
    29. Iverson PJ. Aging Q3 Web site. http://mcintranet.musc.edu/agingq3 Accessed January 11, 2012
    30. Simon SR, Fabiny AR, Kotch J. Geriatrics training in general internal medicine fellowship programs: Current practice, barriers, and strategies for improvement. Ann Intern Med. 2003;139:621–627
    31. Landefeld CS, Callahan CM, Woolard N. General internal medicine and geriatrics: Building a foundation to improve the training of general internists in the care of older adults. Ann Intern Med. 2003;139:609–614
    © 2012 Association of American Medical Colleges