Identifying and overcoming implementation challenges: Experience of 59 noninstitutional long-term services and support pilot programs in the Veterans Health Administration

Abstract

Background: 

From 2010 to 2013, the Department of Veterans Affairs (VA) funded a large pilot initiative to implement noninstitutional long-term services and supports (LTSS) programs to support aging Veterans. Our team evaluated implementation of 59 VA noninstitutional LTSS programs.

Purpose: 

The specific objectives of this study are to (a) examine the challenges influencing program implementation comparing active sites that remained open and inactive sites that closed during the funding period and (b) identify ways that active sites overcame the challenges they experienced.

Methodology: 

Key informant semistructured interviews occurred between 2011 and 2013. We conducted 217 telephone interviews over four time points. Content analysis was used to identify emergent themes. The study team met regularly to define each challenge, review all codes, and discuss discrepancies. For each follow-up interview with the sites, the list of established challenges was used as a priori themes. Emergent data were also coded.

Results: 

The challenges affecting implementation included human resources and staffing issues, infrastructure, resources allocation and geography, referrals and marketing, leadership support, and team dynamics and processes. Programs were able to overcome challenges by communicating with team members and other areas in the organization, utilizing information technology solutions, creative use of staff and flexible schedules, and obtaining additional resources.

Discussion: 

This study highlights several common challenges programs can address during the program implementation. The most often mentioned strategy was effective communication. Strategies also targeted several components of the organization including organizational functions and processes (e.g., importance of coordination within a team and across disciplines to provide good care), infrastructure (e.g., information technology and human resources), and program fit with priorities in the organization (e.g., leadership support).

Implications: 

Anticipating potential pitfalls of program implementation for future noninstitutional LTSS programs can improve implementation efficiency and program sustainability. Staff at multiple levels in the organization must fully support noninstitutional LTSS programs to address these challenges.

Noninstitutional long-term services and supports (LTSS) are an alternative to institutional care in a nursing home by providing a variety of home and community-based services in settings ranging from recipients’ own residences to various congregate living arrangements (^{Wysocki et al., 2012}). As demand for noninstitutional LTSS grows, the U.S. health care system is rebalancing the long-term care sector with an array of noninstitutional programs to supplement already-existing institutional services (^{Hudson, 2010}). The Affordable Care Act, in particular, provided an impetus for rebalancing by authorizing preexisting noninstitutional LTSS program expansion and the creation of new noninstitutional LTSS program options (^{Kapp, 2014}). Nonetheless, policymakers, payment programs, and LTSS providers face challenges in rebalancing, such as getting funding to initiate or expand programs and providing enough capacity to handle the influx of older adults who prefer to receive care in the community. The demand for noninstitutional LTSS is likely to continue to grow as the population of older adults 65 and older begins to utilize formal long-term care services.

Similar to the private sector, the Veterans Health Administration (VA) is facing a growing population of older adult enrollees. VA officials project that the total number of enrollees aged 65 and older will increase from 4.1 to 4.7 million and the number of high-priority Veterans will increase from 500,000 to 1.0 million between 2013 and 2023. Prior to the Affordable Care Act, the passage of the 1999 Millennium Act (Public Law 106-117), gave all VA enrollees access to noninstitutional LTSS programs, allowing an unprecedented number of older Veterans to remain in their homes and communities in spite of significant health care needs (^{Shay, Hyduke, & Burris, 2013}). Given the shifting demographics, it is critical to the VA to find additional ways to accommodate the needs and preferences of its growing patient population eligible for nursing home level care.

The VA has begun to address this need by implementing noninstitutional LTSS programs to support aging Veterans. Along with VA leadership commitment to transform the VA to a 21st century organization (T21 initiative), part of the VHA’s 2009 Office of Geriatrics and Extended Care (GEC) strategic plan (^{VA Office of Strategic Integration 2013}) was to offer a uniform set of Veteran-centric care and services and to continuously improve geriatric care. As a result, many efforts are ongoing in the VA. Noninstitutional alternatives to institutional care have been expanded with more than 167 pilot programs initiated through local efforts and “transformational” support based on models validated as successful in other settings (^{Shay et al., 2013}). These services provide long-term care to chronically ill Veterans in their own homes under the coordinated care of an expert team of providers; services are provided either directly or by contract with community-based agencies. For GEC T21 initiative, the VA adopted several specific types of noninstitutional LTSS programs including Care Management, Dementia Support, Home-Based Primary Care, Geriatric Primary Care, Hospital at Home, and the Program of All-Inclusive Care for the Elderly (^{Kuwahara, Saito, Arima, & Ohmi, 2011}). The programs target Veterans (a) who need skilled services, case management, and assistance with activities of daily living or instrumental activities of daily living; (b) who live alone or who are isolated; and/or (c) who have caregivers who are experiencing stress (^{Smith & Wolfe, 2010}).

The VA’s Office of GEC, the sponsor of the noninstitutional LTSS initiative, commissioned our team to conduct a mixed methods program evaluation focused on the implementation process of these noninstitutional LTSS pilot programs. Although the selected programs were based on established private-sector noninstitutional LTSS models, implementation success is not necessarily guaranteed. Effective and efficient program implementation and subsequent sustainability require substantial effort especially when adapting to a new setting like the VA. Implementation science literature suggests that there are many factors that can expedite/impede implementation of a new program (e.g., resources, leadership support, communication; ^{Chaudoir, Dugan, & Barr, 2013}; ^{Damschroder & Lowery, 2013}; ^{Kadu & Stolee, 2015}; ^{Lukas et al., 2007}; ^{Tomoaia-Cotisel et al., 2013}).

Despite the previous research, much of the literature on program implementation has been in the acute care setting and has not targeted programs for older adults. However, two articles were written about the Hospital Elder Life Program, which aims to reduce delirium and prevent decline in both cognition and physical functioning in the hospital setting. Bradley et al. found that the Hospital Elder Life Program sites were faced with challenges in getting strong leadership support, finding advocates for the programs, and getting needed resources for the program (^{Bradley, Schlesinger, Webster, Baker, & Inouye, 2004}; ^{Bradley, Webster, Baker, Schlesinger, & Inouye, 2005}). Although these findings are reflective of implementation challenges in the literature, it is unclear if the findings are applicable outside the hospital setting especially in cases where noninstitutional LTSS programs provide services that span multiple care settings. Often, LTSS programs provide care via interdisciplinary teams, which could result in better communication and coordination, thereby offsetting the difficulty of care provision spanning multiple settings. In addition, these articles are over a decade old and may not reflect changes that have occurred in health care delivery system over that time (i.e., the move to provide more care through noninstitutional LTSS).

Thus, given the gaps in the literature, the objectives of this study are to (a) examine the challenges influencing program implementation comparing active sites that remained open and inactive sites that closed during the funding period and (b) identify strategies active sites used to overcome the challenges they experienced. Conducting the study in the VA gave us the opportunity to examine implementation of a large number of noninstitutional LTSS programs adapted from the private sector applied in a national integrated health care setting with both national oversight and local variation in how services are organized.

Methods

Study Design and Sample

This study is a 3-year process evaluation of LTSS program implementation at 59 sites spanning seven model types from fiscal year 2010 to 2013 (FY11–FY13). Table 1 displays a description of each program model type and key program operations and infrastructure requirements. We conducted qualitative interviews with key informants from each site at four time points over the course of 3 years—early implementation (Time 1), mid-implementation (Time 2 and Time 3), and sustainability (Time 4). Oversight and approval of this evaluation was provided by the Research and Development Committee at the VA Boston Healthcare System, and study participants gave informed consent.

Interview Guide Development

We developed a semistructured qualitative interview guide featuring questions about the structure of the program, implementation progress, and organizational practices (see Figure 1). The program questions concerned how the program was being run and progress on program implementation, which would allow us to understand how well their program plans and activities were working. For example, we asked, “How is your program structured?” and “Could you please describe the progress you have made with program implementation?”

Figure 1: Sample interview questions

Table 1: Program type and description of program operations and infrastructure

To explore organizational practices, we developed questions derived from implementation literature on organizational structures and processes involved in program implementation guided by the Organizational Transformation Model (OTM; ^{Lukas et al., 2007}; ^{VanDeusen Lukas et al., 2010}). The OTM was developed in seven private sector hospitals and is included within the Consolidated Framework for Implementation Research (^{Damschroder & Lowery, 2013}). The OTM has been used in the VA setting to examine program implementation (^{Kertesz et al., 2014}; ^{Pogoda, Cramer, Rosenheck, & Resnick, 2011}). It includes five drivers of implementation, including (a) an impetus to change, (b) leadership commitment, (c) meaningful staff engagement, (d) alignment of improvement efforts with organizational priorities and resources, and (e) integration across traditional organizational boundaries. These drivers of change are hypothesized to influence components of the health care system including the mission, vision, and strategies that set its direction, value and norms, the operational functions and processes that represent providing services, and infrastructure (e.g., information technology [IT] and human resources [HR]; ^{Lukas et al., 2007}). We would expect that evidence of these OTM drivers and basic components of the health care system would result in a more supportive environment for program implementation.

Data Collection

Prior to data collection, GEC identified points of contact (POCs) at each site. In addition to the POCs, we also identified additional key informants at each site based on the POC’s assessment that the additional staff could provide information about program implementation. We contacted key informants by e-mailing each an invitation to participate and then scheduled informants to do individual or group interviews depending on participants’ preference. We began interviewing key informants during the winter of FY2011.

Two team members participated in each interview; one conducted the interview, and the other took detailed interview notes. With permission from the key informants, we audio-recorded the interviews to supplement our interview notes. All coauthors (except KS) participated as both interviewers and notetakers across the 59 sites. We trained the team in interviewing methods to reduce interviewer bias and in standardized note-taking practices. Prior to each data collection period, the lead interviewer reviewed the prior recordings from past interviews to help guide probes. For example, if key informants at a site had mentioned that they had a previous challenge with securing equipment, the interview team would follow up on that challenge and ask whether it was still a problem and how the site overcame or resolved the issue.

Table 2 displays the number and types of staff interviewed by model type. We conducted 217 telephone interviews over the four time points (i.e., 59 sites at Time 1, 55 sites at Time 2, 55 sites at Time 3, and 48 sites at Time 4) and conducted group interviews 44% of the time. We interviewed 140 unique key informants over the course of the study, many multiple times at different time points. On average, there were 95 key informants interviewed per time period, with a minimum of one and a maximum of seven key informants interviewed per site. For both active and inactive sites, the POCs were mostly consistent over time with only 12 changes (11 for active sites and only 1 for inactive sites) over the four time periods (217 interviews). When a change occurred, the POC was already part of the team or had other team members participate on the call that had been previously interviewed. Thus, key informants participating in the evaluation were familiar with program implementation and factors affecting implementation.

Table 2: Number of unique staff interviewed by model type and staff role

Data Analysis

We used Rapid Assessment Procedures (^{Beebe, 2014}; ^{Meyer & Avery, 2009}) to conduct analysis and report back to the sponsor of the overall evaluation. It was not feasible to have the recordings transcribed given the rapid turnaround of feedback to the sponsor. However, to assure that the notes were accurate, the second team member reviewed each recording after the interview and supplemented the notes by adding in any missing statements. The resulting documents were similar to a transcript except that we did not include every statement the interviewer said and did not record things like pauses and extraneous phrases like “hmm.”

The lead interviewer at each site created a program summary for each site by transferring interview quotes from the notes into a structured matrix organized by each interview question. The coder also transferred additional evidence regarding that topic from other places in the notes as necessary. The site summaries allowed us to organize the data by topic/question and also transfer the data into Excel where the remaining analysis would take place. Once the 59 individual site summary matrices were complete, the primary analyst (OA) created a cross-site summary matrix with the data from all 59 site summaries (^{Miles & Huberman, 1994}). The first column of the spreadsheet listed each site, and the remaining columns contained data grouped by interview question. The primary analyst (OA) then conducted content analysis, grouping relevant quotes into initial emergent categories reflecting challenges affecting implementation across all interview questions. The identified challenges were then entered into a new matrix, and the cells within the matrix contained exemplary quotes. Once coding across all sites was complete, the primary analyst (OA) and secondary analyst (JS) reviewed the evidence for each identified factor, defined the parameters of each factor, and recoded when coding discrepancies were found. Analysts agreed 90% of the time on the initial coding and through discussion and review of primary interview data reached consensus on the remaining 10% of data. Other team members who participated in data collection then gave feedback on results.

For each follow-up period (Times 2–4), we used the challenges identified at Time 1 as our a priori themes. We applied the same process for analysis (i.e., individual site summaries and cross-site summary analysis) at each time point. We coded one new emergent challenge called “referrals and marketing,” which arose at Time 2, and was used in the analysis of data from Times 3 and 4. We then compared the challenges for active and inactive sites to see if there were any similarities or differences. We also followed the same analysis process to identify strategies to overcome challenges.

Results

On average, we found that each site faced two to three challenges as they rolled out and sustained their programs. Sites reported the most challenges during the first time point. Table 3 presents the six types of challenges, their definitions created from summarizing the data, and representative quotes that illustrate challenges and how sites overcame those challenges. Across the seven program model types, most of the sites reported a similar pattern of challenges. Ordered from most to least mentioned, the common challenges included (a) HR and Staffing, (b) Infrastructure, (c) Team Dynamics and Processes, (d) Referrals and Marketing, (e) Resources and Geography, and (f) Leadership Perspectives. The Program for All-Inclusive Care of the Elderly model type was the only outlier and most often reported evidence of within the team dynamics and processes challenge.

Table 3: Examples illustrating the six challenges influencing long-term services and supports program implementation and strategies to overcome each challenge

We also compared the challenges for sites active over the entire 3-year period and those that became inactive or closed prior to FY2013 (e.g., the end of grant funding). We found that the 48 active sites and 11 inactive sites both faced challenges mentioned above. However, inactive sites were not able to overcome one or more of the challenges they faced, and the challenges led to them closing prior to or by the end of the funding period. For example, the challenges by the inactive sites were issues on hiring staff to run the program, team dynamics especially around collaboration within VA networks or with outside VA entities, and leadership support in terms of the program being a low priority for funding given other foci.

The second objective of this study is to identify ways active programs overcame the challenges they experienced over the course of the evaluation. We present this information for each type of challenge below.

HR and Staffing Strategies

The reported HR and staffing issues included staff turnover, constraints with existing staff, and delays in hiring. Sites used various strategies to address these staffing issues.

• Staff turnover. Sites overcame staff turnover challenges in several ways from cross-training remaining staff for coverage until replacements were hired, fine tuning or realigning staff to workload and program needs, providing financial incentives or salary increase to attract and retain valued staff, and spending time with staff to address their needs and provide training when necessary.
• Constraints with existing staff. Sites used video conferencing technology to enable providers to engage remotely with staff teams at different locations and shared staff across departments.
• Delays in hiring. Sites faced with delays in hiring overcame this challenge by way of patience and persistence in the face of slow administrative processes and continued communication with HR.

Infrastructure Strategies

Infrastructure challenges span from management and operations issues, systems and technical needs, to community resources. Sites used a variety of strategies to address these challenges.

• Management and operation. Many sites dealt with supply issues by obtaining special funding, getting a credit card for the program service line, and borrowing supplies and equipment from other departments. In addition, one site mentioned working on a policy to enable nurses to have the ability of entering orders, which had in the past limited referrals.
• Systems and technical needs. Here, strategies used include finding a resource person to provide knowledge of the VA ordering process, using video technology to manage the increasing demand, and better coordination and communication between sites and their community partner providers for their programs.
• Community resources. One site addressed challenges faced with community resources through improved communication with the community partner, specifically through the social worker efforts to establish more collegial cooperation.

Teams Dynamics and Processes Strategies

Communication was the main strategy used to improve teams’ dynamics and processes. In particular, some sites kept open lines of communications with program stakeholders such as participants and community partners to obtain feedback and improve processes. Communication for some sites required educating new providers and/or the community partners on aspects of the way the VA provides services in order to improve interactions within program teams and across stakeholders.

Referrals and Marketing Strategies

Referral strategies include bypassing referral providers for direct access to potential participants in a program (e.g., case finding), program promotion, and creating a consult service or a new function in the electronic medical record to remind and ease the referral process on providers. Sites improved marketing over time by making connections with other departments through active attendance at meetings, as well as educating staff and patient about their program—including pamphlets and equipment demonstrations.

Resources Allocation and Geography Strategies

The issue of space as a resource was dealt with through remote work, advocating for flex time where staff work longer shifts to reduce days of work, and providing technology (e.g., laptops) so staff could share space more effectively.

Leadership Strategies

Sites mentioned few strategies for overcoming lack of leadership support given that the leadership challenge was the least mentioned over the grant period. One site mentioned using the following strategy: having a program representative participate in rounds and other meetings where leaders were present to increase visibility and attract leadership to their program.

Discussion

This study sought to identify the challenges LTSS programs experienced over the course of a large-scale evaluation and describe strategies sites used to overcome these challenges. We found six challenges influenced implementation of the noninstitutional LTSS programs. We saw little variation in these factors across model types or by active or inactive status suggesting that the challenges were applicable regardless of program type or ability to sustain a program over the grant period.

To date, very little research on program implementation has been conducted outside of the acute care setting, particularly within the LTSS realm. Interestingly, our findings regarding implementation challenges were consistent with the research on program implementation, which has shown that there are certain issues that need to be addressed during program implementation (^{Bradley et al., 2004}, ²⁰⁰⁵; ^{Damschroder & Lowery, 2013}; ^{Feldstein & Glasgow, 2008}; ^{Kadu & Stolee, 2015}; ^{Tomoaia-Cotisel et al., 2013}). For example, research suggests that it is important to have leaders to support implementation and sustainability by making decisions about resources such as staff time and commitment to the program (^{Bradley et al., 2004}; ^{Ijkema, Langelaan, van de Steeg, & Wagner, 2014}). In addition, team dynamics and processes are crucial for gaining internal support for the program, linking the new program with other existing geriatric programs, and otherwise supporting program implementation, which may run into periods where project operations are inefficient (^{Bradley et al., 2004}; ^{Reuben, 2002}). Thus, to improve implementation and sustainability of noninstitutional LTSS programs, it is important to work with staff and leadership to understand the value of the program. In addition, the program should advocate for support to deal with challenges within the organization like infrastructure (e.g., HR, resources, and geography). Finally, positive interdisciplinary team dynamics and integration are important for both positive teamwork and coordination through referral and marketing processes that enhance program implementation.

We also presented strategies to overcome each type of challenge, which can aid programs experiencing similar issues. We feel that this is an important contribution to the literature especially because there is very little written about LTSS program implementation. Interestingly, staff reported very few strategies for overcoming leadership challenges. We believe that this may be the case because leadership was the least reported challenge at all time points in the evaluation, and leadership support was initially required for participation in the grant initiative. In addition, throughout the grant period, the sponsor gave sites assistance on how to overcome local leadership resistance by providing training sessions on preparing business plans (including return-on-investment analysis when applicable) and ways to present findings to leadership to garner support for the program. Sites also participated in calls for each model type where problems such as leadership support could be discussed and sites could share best practices and learn from each other.

Upon reviewing the strategies, we think there are three cross-cutting areas in which these sites targeted their strategies including organizational functions and processes (e.g., importance of effective coordination across disciplines via referrals and marketing, staffing), infrastructure (e.g., IT and HR), and program fit with priorities in the organization (e.g., assuring leadership buy-in given multiple priorities). These areas all fall into the health care organization itself and are consistent with ways to provide high-quality health care when the appropriate organizational supports (e.g., OTM drivers) are in place. We hypothesize that this is the case because the GEC T21 initiative already had the OTM drivers in place including an impetus to change health care with the rebalancing of institutional and noninstitutional care in the VA, national leadership support through funding for these programs and strong administrator support for participating sites, program alignment within GEC service in the VA, integration across boundaries because of the interdisciplinary nature of the LTSS programs, and staff engagement with the programs since the grants were submitted by them. Thus, with the drivers in place, the strategies targeted changing components of the health care system. However, we did see that the most often mentioned cross-cutting strategy was communication. Although the OTM model alludes to the importance of communication for effective change, it was not one of the top drivers in the model. A recent review of implementation literature focused on communication (^{Manojlovich, Squires, Davies, & Graham, 2015}) advocates for the inclusion of communication as a driver of implementation. Our findings suggest that communication was often used to overcome implementation challenges and is a facilitator to implementation.

Several considerations are important when interpreting these findings. Although we assessed a large number of sites and model types, we had a smaller number of key informants at some sites. However, the respondents we spoke to were the most knowledgeable about the programs at their site. Future efforts should incorporate as many staff perspectives as feasible given budget constraints. In addition, this multisite evaluation focused on a process evaluation, not clinical outcomes and costs, which were part of another GEC-sponsored effort (^{Intrator et al., 2013}). Therefore, findings do not address programs’ long-term outcomes; rather they provide insight on program implementation and strategies to overcome these challenges. Additional studies are thus needed to assess the impact of GEC programs that has had and continues to have many initiatives rolled out to address the various needs of a diverse Veteran population. For example, in addition to the 59 noninstitutional LTSS programs we assessed, the VA funded another 108 programs through the Transforming VA Care to the 21st Century Strategic Plan. Furthermore, the VA has funded the Veterans Directed Home and Community-Based Program to provide Veterans (and their caregivers) more access, choice, and control over noninstitutional care services working in collaboration with the private sector. This evaluation was conducted in the VA, a large integrated network of hospitals. We believe that the challenges as a whole are applicable to private sector, in particular, results regarding leadership support, team dynamics and processes, and referrals and marketing. Although the VA has specific infrastructure of its own (e.g., IT, HR, contracting), other organizations are faced with their own sets of challenges dealing with various infrastructure or cross-departmental issues that can enable or inhibit program implementation.

Practice Implications

This article’s largest practice implication is sharing strategies used to overcome program implementation challenges that can be used by other programs hoping to adopt these LTSS models inside or outside the VA. We saw that many of the strategies played to the strengths of LTSS programs themselves, that utilize interdisciplinary teams. The strategies we observed are reflective of ways an organization can successfully implement and support LTSS programs including having a strategy that sets its direction and aligns with priorities, effective operational clinical care functions and processes, and infrastructure (IT, HR, facility management) that support patient care (^{Lukas et al., 2007}). Future research is needed to understand the role communication plays as a facilitator of implementation.