Falls are the leading cause of unintentional injury nationally among people aged 65 years and older.1 Systematic reviews of randomized trials have shown that exercise programs can reduce falls in this population.2–4 To identify patients at risk for falls, consensus guidelines from the American and British Geriatrics Societies recommend that health care professionals ask about falls, with gait and balance assessment as a key component to further evaluate patients at high risk (those who have fallen, or who have a gait or balance problem).5 Findings from the gait and balance assessment should in turn lead to tailored exercise recommendations for the patient. Implementing a fall prevention program that adheres to clinical guidelines, however, requires addressing the significant logistical barriers that impede the integration of fall prevention activities into primary care.6,7
Electronic health record (EHR) tools may help primary care providers (PCPs) deliver recommended care through enhanced decision support and quick access to patient education materials and relevant resources in the community, but little published work exists on EHR tools for fall prevention in primary care. Two studies explored the use of EHR tools to support fall prevention activities, focusing on initial development and testing of tools but not on regular, continued use in clinical practice.8,9 A third study improved the quality of care for patients with falls and urinary incontinence in 5 community practices, but included only 1 site that used a fully functional EHR.10 Here we report on the redesign and local tailoring of a nationally developed electronic clinical reminder for fall prevention to integrate it with PCP workflow in 3 community-based outpatient clinics (CBOCs). This undertaking is built upon a multiyear effort by the US Veterans Health Administration (VHA) Office of Geriatrics and Extended Care to support PCPs in providing better geriatric care.
The VA Greater Los Angeles Healthcare System (VAGLAHS) Institutional Review Board determined that the study’s quality improvement component (PCC 2009-091302) was not human subjects research. The study’s formative evaluation (PCC 2011-020177) was approved as minimal risk human subjects research.
VAGLAHS is 1 of 5 integrated delivery systems within the VA Desert Pacific Healthcare Network, which serves Veterans in Southern California and Southern Nevada. VAGLAHS has 3 ambulatory care centers, a tertiary care facility, and 10 CBOCs. The VA Desert Pacific Healthcare Network is 1 of 21 clinical networks in the VHA.
In late 2006, the nationwide VA External Peer Review Program began conducting medical record reviews to determine how often outpatients aged 75 years and older were asked about previous falls, and whether appropriate care to prevent future falls was provided to those Veterans with 2 falls or 1 fall with injury in the previous year. These data showed wide variation in VHA health care facilities’ quality of care and significant room for improvement in national performance.11
In 2007, under the auspices of the VA Office of Geriatrics and Extended Care, a national VA workgroup developed an electronic clinical reminder for falls to be used in primary care. Within the VHA, clinical reminders prompt health care providers to implement recommended practices as part of the Computerized Patient Record System, VHA’s EHR. Clinicians complete reminders by filling out structured templates,12 which can lead automatically to options for placing orders or making referrals. Completing the reminder generates structured documentation that is added to the clinician’s progress note, and may be edited as needed. Although PCPs and nurses may feel overtaxed with the current load of reminders,12 they may serve as useful tools for patient care if properly adapted to clinicians’ workflow, and may also help VHA facilities to meet performance targets.
During 2008, a confluence of events led to local pilot testing of the national workgroup falls reminder in 3 VAGLAHS CBOCs. First, by October 2008, the national workgroup had completed development of a clinical reminder for fall prevention. Second, during 2008, VAGLAHS had formed a local workgroup to improve the quality of care for falls.11 Third, in early 2008, the VAGLAHS workgroup leader provided an educational presentation on falls at a joint staff meeting of the 3 CBOCs, thereby establishing a relationship with the CBOC site directors. Fourth, a PCP member of the local workgroup who also worked at 2 of the 3 CBOCs suggested testing the national workgroup’s reminder.
As a result of these events, in 2009, the VAGLAHS workgroup collaborated with 3 CBOCs to pilot the national workgroup reminder as 2 linked parts: (1) a nurse reminder to identify older Veterans at high risk for future falls and (2) a PCP follow-up reminder for Veterans identified as being at high risk based on the nurse reminder. An audit of this pilot test revealed that among 483 PCP reminders completed between September 1, 2009 and February 28, 2010, 84% had a box checked indicating that “no further fall assessment was needed,” thereby bypassing the fall assessment (D. A. Ganz, unpublished data). Anecdotal reports suggested the reminder was too time-consuming and complex for regular use. In light of this information, we set out to redesign the clinical reminder. This article describes the methods and results of the redesign project.
Quality Improvement Project
The 3 VAGLAHS CBOCs were selected as demonstration sites for the redesign project because nurse managers and physician site directors of the CBOCs had already built a successful working relationship with the improvement team in piloting the national workgroup’s clinical reminder. In addition, the proportion of individuals aged 75 years and older at these sites was particularly high: in 2010, the 3 clinics served 3296 Veterans aged 75 years and older, representing 32% of the Veterans seen by these clinics for primary care services (D. A. Ganz, unpublished data). The clinics employed 16 PCPs (10 physicians and 6 nurse practitioners) and 21 staff nurses [11 registered nurses (RNs) and 10 licensed vocational nurses (LVNs)].
The reminder redesign project took place between July 2010 and June 2011. The project was theory-driven,11 and built upon the Assessing Care of Vulnerable Elders (ACOVE) quality improvement efforts for falls in non-VHA primary care settings. ACOVE interventions consist of 5 key elements10,13,14:
* Case finding (ie, identification of patients at risk)
* Efficient collection of condition-specific clinical data
* Medical record prompts
* Patient and family education materials, including linkages to community resources
* Physician decision support and physician and staff education
The first 4 elements and part of the fifth (physician decision support) were directly incorporated into the electronic clinical reminder. Physician and staff education were provided through an in-service described later.
Success for this project was defined as achieving regular reminder use without a sustained flow of resources from the research team. To achieve this goal, no additional time was given to the CBOC staff for implementation beyond these employees’ standard administrative time. Nurse managers were primarily responsible for coordinating the project with the study team, with the physician site directors approving actions at key points. Research resources supported the investigators’ time to provide technical assistance to the CBOCs. This assistance included provider education, development of local community resource lists, and provision of detailed specifications to the VAGLAHS programmer who developed the redesigned reminder.
To implement and evaluate the intervention, we made 3 site visits to the CBOCs. The first (in August 2010) involved visiting each clinic to assess the physical space, determine clinic staffing, workflow and responsibilities, and meet the staff who would be implementing the project. We used this first site visit to determine the existing clinic care model to tailor the reminder to existing clinic setups and workflows.12
At the first site visit, we received extensive feedback from PCPs and nurses that they were experiencing “reminder overload,” given the number of other required clinical reminders. PCPs reported that they did not use the existing falls reminder because it took too long to complete. Meanwhile, nurses did not believe that delegation of some of the PCP’s work to the nurse would be feasible given competing demands. We also learned that the proper fitting and use of assistive devices was an area where clinic personnel sought additional guidance.
On the basis of this information, we decided to focus the redesigned PCP clinical reminder on gait, balance, and strength assessment and limit the amount of delegation to nurses to screening for falls, history-taking on assistive device use and circumstances of the most recent fall (if applicable), and provision of educational materials. These decisions were driven by a desire to increase the rate of completed fall assessments over the previous version of the reminder, where the fall evaluation was more complete but was rarely used. We consciously excluded important components of a fall assessment (eye examination, medication review, cognitive evaluation, and functional status assessment), as these components represent good geriatric care for all adults aged 75 years and older regardless of fall risk and providers could complete these components separately from the fall reminder.
To create the redesigned reminder (see Appendix Supplementary Digital Content 1, http://links.lww.com/MLR/A401, for overview of reminder operation, screenshots, and monthly clinical reminder activity), we worked closely with a VAGLAHS programmer, first creating a blueprint of the reminder template using a word processor, and then iteratively testing the reminder created by the programmer using a dummy patient record. We also worked with the programmer to ensure that data could be collected from the reminder to generate reports. The report specifications were created so as to match the numerator and denominator of relevant ACOVE-3 quality indicators (see Appendix Supplementary Digital Content 1, http://links.lww.com/MLR/A401).15 Report findings are preliminary because they have not yet been validated against a medical record review.
In February 2011 we made a second site visit to present the redesigned reminder at a joint staff meeting of the 3 clinics, including a demonstration of how the gait, balance, and strength examination could be integrated into existing clinic workflow. Physical medicine and rehabilitation colleagues presented an in-service education session on fitting assistive devices to a combined audience of physicians and nurses. Each participant received a syllabus with the class information and patient education materials for future use. Shortly after the in-service, we activated the redesigned reminder.
We conducted a third site visit at each CBOC from July through September of 2011. This set of site visits was a formative evaluation to identify barriers and facilitators to implementation. The evaluator was a separate member of the research team who was not directly involved in the planning and implementation of the program; her involvement with the CBOCs began during this stage. Each site visit lasted one half day, during which time the evaluator conducted semistructured interviews and direct observation of clinic operations.
The semistructured interviews (see Appendix Supplementary Digital Content 1, http://links.lww.com/MLR/A401, for interview guides) were scheduled in advance of the site visit where possible; however, because of the busy schedules of interviewees, many interviews were arranged during the site visit and occurred flexibly around patient care and other commitments. Interview guides were used flexibly depending on how much time the interviewee had available. With interviewees’ verbal consent, the evaluator took notes using a smart pen (Livescribe Echo), which doubled as an audio recorder and allowed playback of specific conversations based on their location in the notes.
During the interview, clinical reminder walk-throughs took place using a dummy patient record in the Computerized Patient Record System. During the walk-through, the nurse or PCP completed an actual falls reminder using data provided by the interviewer about an imaginary patient. The interviewer watched, took notes, and asked questions as needed (see interview guide in Appendix Supplementary Digital Content 1, http://links.lww.com/MLR/A401). The purpose of the walk-through was to enhance the concreteness and specificity of interviewees’ feedback about the strengths and weaknesses of the clinical reminder with respect to usability.
The evaluator conducted 8 nurse interviews and 5 PCP interviews (4 or 5 interviews, including at least 1 PCP interview, occurred at each site). Nurse interviews ranged from about 7–23 minutes; PCP interviews ranged from about 10–33 minutes. The direct observation component of the site visit supplemented and confirmed interview findings regarding patient flow through the clinic. The focus was on each clinic’s physical layout, the fullness of the waiting room, and any related change in patient flow.
The evaluator iteratively reviewed handwritten notes from direct observations and interviews, and recordings from the smart pen for repetitions and key words or phrases. The coding process did not involve transcripts or specialized software; instead, the evaluator grouped similar ideas into common themes in a spreadsheet.16 For example, similar words such as “time,” “long,” “cumbersome,” and “slow” would be categorized together. As new themes emerged, the evaluator made additional entries to the spreadsheet. The evaluator then shared the completed spreadsheet with the principal investigator, and recategorized data as needed to clarify ambiguities. When questions arose, the principal investigator and evaluator together reviewed notes or played back a particular section of an interview from the smart pen to resolve uncertainties in categorization.
The redesigned falls clinical reminder retained the same basic structure from the national workgroup’s version: a nurse screening reminder and a PCP follow-up reminder for Veterans who screened positive from the nurse reminder. A care model (Fig. 1) demonstrates the activities covered by the reminder components.
Preliminary data from reminder reports indicate that from reminder activation in February 2011 through the end of January 2012, 2943 unique Veterans aged 75 years and older visited the 3 clinics, of whom 2264 (77%) had the nurse clinical reminder completed. Among those screened by the nurse clinical reminder, 472 Veterans (21%) screened positive: 262 Veterans had 2 falls in the last year or 1 fall with injury since their last visit to a physician, and another 210 Veterans were afraid of falling. Among these 472 Veterans, 318 were given patient educational materials (which could include home exercises, locations for community exercise programs, and/or footwear advice) and 402 were given a home safety checklist.
Among 472 Veterans screening positive on the nurse reminder, PCPs indicated that further evaluation was not needed for 202 of these individuals; PCPs chose to perform a gait, balance, and strength evaluation on 231 Veterans. Of these 231 people, 162 (70%) had a gait, balance, or strength problem on evaluation and were free of advanced dementia or poor prognosis, meeting inclusion criteria for exercise. Thirty-nine of these 162 individuals (24%) were offered physical therapy or exercise (or the patient had recent physical therapy, recent supervised exercise program, or declined physical therapy).
Our evaluation of clinic operations at the third site visit showed that all 3 clinics shared a similar workflow and team structure, which generally conformed to the information obtained directly from clinic staff at the first site visit. For each clinic, Veterans arrived, checked in at a front desk, and then were met by a nurse or seated in a waiting room. After doing a basic check in with the nurse, including the nurse portion of the reminder, Veterans either returned to waiting area or were brought to the PCP. The PCP visit, including the PCP portion of the falls reminder, was then completed. The final check out with the nurse routed Veterans to any follow-up appointments or various other tests that needed to be completed. At each site providers were split into teams, with each team consisting of at least 1 of each of the following: (1) an LVN, who was usually responsible for check in and check out procedures; (2) a PCP; and (3) an RN, who played more of a coordination role and could act as relief to keep the flow moving if Veterans required extra attention or the LVN needed support.
Some variations of workflow were noted. For example, some nurses had individual rooms to check Veterans, whereas other clinics used a common nurses’ office and a shared examination room. In addition, reported waiting times varied both within and between clinics, congruent with observations that waits varied, as did fullness of the clinic waiting room. If the patient was late for the visit, nurses said that they might complete their reminders after the patient’s time with the PCP, to try to make up time. The nurse reminder was reported to take 2–5 minutes, with the total nurse check-in usually around 10–20 minutes long. Although 2 sites planned to use stickers on patient files to flag cases with falls, they reported actual use of these stickers to be variable. When fall risk was a significant concern, nurses reported that they might write notes on the patient file or speak directly to the PCP, but at no site was communication about fall risk systematically reported or communicated outside of the reminder.
Table 1 shows findings from interviews with clinic PCPs and nurses at the third site visit. Three groups of themes included: (1) the problems nurses and PCPs experienced with the clinical reminder; (2) aspects of the clinical reminder and implementation that worked well; and (3) suggestions for improvement. Within the theme discussing problems, 3 subthemes emerged. The first was the amount of effort required to complete the PCP reminder. Fourteen PCP comments (11 directly from the PCP and 3 from nurses, attributed to the PCP) mentioned the reminder as long both in size and in terms of time to complete, that PCPs would like to be able to skip through their reminder, and that the PCP reminder was duplicating efforts, either between the nurse and PCP or within the PCP visit’s other tasks. In contrast, only 2 nurses suggested that the nurse component of the reminder was lengthy in size or in terms of time to completion. For the second problem subtheme, 2 nurse and 3 PCP comments cited long wait times for referrals and a lack of resources, especially for physical therapy. Finally, 4 nurse comments noted patient factors that could impede the success of the reminder, mainly patient lack of motivation to agree to changes and patient transportation being seen as a challenge due to the patient’s physical limitations. Interviewees did not mention any unintended negative clinical consequences of reminder use.
Positive feedback about the reminder and implementation process included 6 comments about the assistive device training, 8 comments about the implementation team and the organization of the implementation, including having the opportunity to provide feedback about the reminder during its redesign. Nurse comments also suggested that their reminder was a good length, that the handouts were useful, and that the reminder had positive impacts on care delivery, namely routing Veterans to appropriate care and empowering Veterans through behavioral modification.
A number of constructive suggestions were also identified, mostly relating to the format of the reminder. These ranged from having more space to add free text comments to shortening or rearranging different sections. Ideas for additional program options were also presented, including a fall prevention telehealth clinic.
We redesigned an electronic clinical reminder for fall prevention to improve its usefulness in primary care, building on the existing VHA administrative, clinical, and informatics infrastructure. The nurse component of the reminder was generally well-accepted by users in the 3 CBOCs, as indicated by nurses’ more frequent positive comments about the reminder and implementation process. This acceptance may stem from the content of the nurse reminder, which covers history taking and patient education, activities that are congruent with nurses’ role in the clinic. In addition, the iterative changes we incorporated into the nurse reminder helped create a product that was more tailored to the nurses’ needs and preferences. Finally, increased acceptance of the nurse reminder may reflect the greater simplicity of the nurse reminder component, as compared with the PCP component.
In contrast to the nurse experience, the PCP experience with the project and the reminder signaled less buy-in and satisfaction. Interviewed PCPs noted that the burden of existing clinical reminders is already high, the formal process of the reminders is redundant with their own clinical judgment about what patient evaluation and management are needed, and there are limited downstream referral options for Veterans found to need additional services, such as physical therapy. However, interviewed PCPs were not always aware of the downstream options available to them, especially the community resource list. In total, there were roughly twice as many comments given by or attributed to PCPs, compared with nurses, relating to problems with the reminder.
PCPs’ comments suggest that the PCP reminder component warrants further redesign, and in the interim, the reminder may need to be set up so that PCPs can opt out of it more easily. A complementary strategy would involve reviewing all reminders currently in use for Veterans aged 75 years and older, and determining whether some could be turned off for this age group, to allow more time for geriatric syndromes such as falls to be addressed. Our interview findings are limited by deriving from a convenience sample of PCPs and nurses based on their availability at the time of our site visit. These findings may therefore be neither representative nor comprehensive. However, we gleaned enough information from the interviews to inform the next cycle of reminder redesign, which was our primary goal.
Importantly, the effect of reminder use on clinical outcomes was not tested in this study, but there is reason to expect that falls outcomes will be improved by increasing the rate at which care processes driven by the reminder are performed. The individual processes are supported by evidence that they improve outcomes,15,17 and better performance on these falls process indicators was associated with better scores on the Falls Efficacy Scale outcome measure in a cohort of patients aged 75 years and older who screened positive for falls or fear of falling.18 In this project, we focused on achieving sustained use of the reminder, and preliminary data suggest the reminder is being used without continued intervention by the study team.
However, we are concerned about the potentially limited effect of the reminder on population outcomes. First, 23% of eligible Veterans were not screened for falls or fear of falling. We allowed screening to be postponed to a subsequent visit if there were competing demands on the nurse’s time, which may have led to the observed results. Second, there was a significant drop-off between screening activities and the provision of exercise as captured by the reminder report. The reminder report may underestimate the true rate of referral to exercise programs, as the report does not capture care before or subsequent to reminder use. Even so, the rate of referral to exercise programs is low. On the basis of our interview data, this low referral rate may result from a combination of low patient motivation, lack of uniform PCP awareness of the community exercise handout, and the deterrent of long waits for physical therapy. Future cycles of quality improvement will need to address these issues.
Partnership within VAGLAHS was critical for the reminder redesign. For example, collaboration between the implementation team and the Physical Medicine and Rehabilitation department resulted in a continuing professional education training session on proper use and fitting of assistive devices, a need that was identified by all 3 clinics. Although this activity was not part of our original implementation plan, it facilitated buy-in from the participating clinics and was germane to our quality improvement effort. In addition, close collaboration between VAGLAHS programmers and the implementation team was needed to create both the redesigned reminder and the automated reports indicating how the reminder was being used.
What path subsequent reminder development, implementation, and dissemination should take remains unclear. Both “bottom-up” and “top-down” approaches to implementation exist,19 the former implying a more organic, participatory approach from frontline staff and the latter relying more on centralized requests for implementation from high levels of management. Currently, we have espoused a more organic diffusion pattern, allowing other sites at the VHA to test and provide feedback on the clinical reminders without any directive or external incentives.
For this project, we provided extensive technical assistance by working with programmers to develop the reminder within the constraints imposed by the Computerized Patient Record System, updating a draft community resource list provided by a site social worker, and providing continuing professional education credits. More importantly, the project team was highly motivated to see the project through. Although the reminder itself does not need to be rebuilt from scratch for use at new sites, other activities, such as creating community resource lists and delivering educational sessions on falls, are inherently local efforts and would need to be repeated by local champions. Successful spread will therefore require local involvement.
In addition to continuing to refine the clinical reminder, we intend to study the similarities and differences of the 3 clinics to other sites with respect to site characteristics, such as organizational culture, that might affect the adoption and use of the clinical reminder. The reminder technology is only a tool, and we hope that continued close relationships with the sites will, over time, help us identify better approaches to improving care for older Veterans.
The authors thank Ahnnya Slaughter, RN, Lee Ochotorena, RN, and Caroline Goldzweig, MD for their support in clinical reminder development and implementation; Hilary Siebens, MD and Kaye Harmston, PT for assisting in provider training; Kenneth Shay, DDS, MS and Barbara Hyduke, MSA for supporting the reminder development process at a national level; Gery Ryan, PhD for guidance regarding the formative evaluation, and all the employees of the 3 participating clinics and VAGLAHS falls workgroup for their involvement.