To explore organizational practices, we developed questions derived from implementation literature on organizational structures and processes involved in program implementation guided by the Organizational Transformation Model (OTM; Lukas et al., 2007 ; VanDeusen Lukas et al., 2010). The OTM was developed in seven private sector hospitals and is included within the Consolidated Framework for Implementation Research (Damschroder & Lowery, 2013). The OTM has been used in the VA setting to examine program implementation (Kertesz et al., 2014 ; Pogoda, Cramer, Rosenheck, & Resnick, 2011). It includes five drivers of implementation, including (a) an impetus to change, (b) leadership commitment, (c) meaningful staff engagement, (d) alignment of improvement efforts with organizational priorities and resources, and (e) integration across traditional organizational boundaries. These drivers of change are hypothesized to influence components of the health care system including the mission, vision, and strategies that set its direction, value and norms, the operational functions and processes that represent providing services, and infrastructure (e.g., information technology [IT] and human resources [HR]; Lukas et al., 2007). We would expect that evidence of these OTM drivers and basic components of the health care system would result in a more supportive environment for program implementation.
Prior to data collection, GEC identified points of contact (POCs) at each site. In addition to the POCs, we also identified additional key informants at each site based on the POC’s assessment that the additional staff could provide information about program implementation. We contacted key informants by e-mailing each an invitation to participate and then scheduled informants to do individual or group interviews depending on participants’ preference. We began interviewing key informants during the winter of FY2011.
Two team members participated in each interview; one conducted the interview, and the other took detailed interview notes. With permission from the key informants, we audio-recorded the interviews to supplement our interview notes. All coauthors (except KS) participated as both interviewers and notetakers across the 59 sites. We trained the team in interviewing methods to reduce interviewer bias and in standardized note-taking practices. Prior to each data collection period, the lead interviewer reviewed the prior recordings from past interviews to help guide probes. For example, if key informants at a site had mentioned that they had a previous challenge with securing equipment, the interview team would follow up on that challenge and ask whether it was still a problem and how the site overcame or resolved the issue.
Table 2 displays the number and types of staff interviewed by model type. We conducted 217 telephone interviews over the four time points (i.e., 59 sites at Time 1, 55 sites at Time 2, 55 sites at Time 3, and 48 sites at Time 4) and conducted group interviews 44% of the time. We interviewed 140 unique key informants over the course of the study, many multiple times at different time points. On average, there were 95 key informants interviewed per time period, with a minimum of one and a maximum of seven key informants interviewed per site. For both active and inactive sites, the POCs were mostly consistent over time with only 12 changes (11 for active sites and only 1 for inactive sites) over the four time periods (217 interviews). When a change occurred, the POC was already part of the team or had other team members participate on the call that had been previously interviewed. Thus, key informants participating in the evaluation were familiar with program implementation and factors affecting implementation.
We used Rapid Assessment Procedures (Beebe, 2014 ; Meyer & Avery, 2009) to conduct analysis and report back to the sponsor of the overall evaluation. It was not feasible to have the recordings transcribed given the rapid turnaround of feedback to the sponsor. However, to assure that the notes were accurate, the second team member reviewed each recording after the interview and supplemented the notes by adding in any missing statements. The resulting documents were similar to a transcript except that we did not include every statement the interviewer said and did not record things like pauses and extraneous phrases like “hmm.”
The lead interviewer at each site created a program summary for each site by transferring interview quotes from the notes into a structured matrix organized by each interview question. The coder also transferred additional evidence regarding that topic from other places in the notes as necessary. The site summaries allowed us to organize the data by topic/question and also transfer the data into Excel where the remaining analysis would take place. Once the 59 individual site summary matrices were complete, the primary analyst (OA) created a cross-site summary matrix with the data from all 59 site summaries (Miles & Huberman, 1994). The first column of the spreadsheet listed each site, and the remaining columns contained data grouped by interview question. The primary analyst (OA) then conducted content analysis, grouping relevant quotes into initial emergent categories reflecting challenges affecting implementation across all interview questions. The identified challenges were then entered into a new matrix, and the cells within the matrix contained exemplary quotes. Once coding across all sites was complete, the primary analyst (OA) and secondary analyst (JS) reviewed the evidence for each identified factor, defined the parameters of each factor, and recoded when coding discrepancies were found. Analysts agreed 90% of the time on the initial coding and through discussion and review of primary interview data reached consensus on the remaining 10% of data. Other team members who participated in data collection then gave feedback on results.
For each follow-up period (Times 2–4), we used the challenges identified at Time 1 as our a priori themes. We applied the same process for analysis (i.e., individual site summaries and cross-site summary analysis) at each time point. We coded one new emergent challenge called “referrals and marketing,” which arose at Time 2, and was used in the analysis of data from Times 3 and 4. We then compared the challenges for active and inactive sites to see if there were any similarities or differences. We also followed the same analysis process to identify strategies to overcome challenges.
On average, we found that each site faced two to three challenges as they rolled out and sustained their programs. Sites reported the most challenges during the first time point. Table 3 presents the six types of challenges, their definitions created from summarizing the data, and representative quotes that illustrate challenges and how sites overcame those challenges. Across the seven program model types, most of the sites reported a similar pattern of challenges. Ordered from most to least mentioned, the common challenges included (a) HR and Staffing, (b) Infrastructure, (c) Team Dynamics and Processes, (d) Referrals and Marketing, (e) Resources and Geography, and (f) Leadership Perspectives. The Program for All-Inclusive Care of the Elderly model type was the only outlier and most often reported evidence of within the team dynamics and processes challenge.
We also compared the challenges for sites active over the entire 3-year period and those that became inactive or closed prior to FY2013 (e.g., the end of grant funding). We found that the 48 active sites and 11 inactive sites both faced challenges mentioned above. However, inactive sites were not able to overcome one or more of the challenges they faced, and the challenges led to them closing prior to or by the end of the funding period. For example, the challenges by the inactive sites were issues on hiring staff to run the program, team dynamics especially around collaboration within VA networks or with outside VA entities, and leadership support in terms of the program being a low priority for funding given other foci.
The second objective of this study is to identify ways active programs overcame the challenges they experienced over the course of the evaluation. We present this information for each type of challenge below.
HR and Staffing Strategies
The reported HR and staffing issues included staff turnover, constraints with existing staff, and delays in hiring. Sites used various strategies to address these staffing issues.
- Staff turnover. Sites overcame staff turnover challenges in several ways from cross-training remaining staff for coverage until replacements were hired, fine tuning or realigning staff to workload and program needs, providing financial incentives or salary increase to attract and retain valued staff, and spending time with staff to address their needs and provide training when necessary.
- Constraints with existing staff. Sites used video conferencing technology to enable providers to engage remotely with staff teams at different locations and shared staff across departments.
- Delays in hiring. Sites faced with delays in hiring overcame this challenge by way of patience and persistence in the face of slow administrative processes and continued communication with HR.
Infrastructure challenges span from management and operations issues, systems and technical needs, to community resources. Sites used a variety of strategies to address these challenges.
- Management and operation. Many sites dealt with supply issues by obtaining special funding, getting a credit card for the program service line, and borrowing supplies and equipment from other departments. In addition, one site mentioned working on a policy to enable nurses to have the ability of entering orders, which had in the past limited referrals.
- Systems and technical needs. Here, strategies used include finding a resource person to provide knowledge of the VA ordering process, using video technology to manage the increasing demand, and better coordination and communication between sites and their community partner providers for their programs.
- Community resources. One site addressed challenges faced with community resources through improved communication with the community partner, specifically through the social worker efforts to establish more collegial cooperation.
Teams Dynamics and Processes Strategies
Communication was the main strategy used to improve teams’ dynamics and processes. In particular, some sites kept open lines of communications with program stakeholders such as participants and community partners to obtain feedback and improve processes. Communication for some sites required educating new providers and/or the community partners on aspects of the way the VA provides services in order to improve interactions within program teams and across stakeholders.
Referrals and Marketing Strategies
Referral strategies include bypassing referral providers for direct access to potential participants in a program (e.g., case finding), program promotion, and creating a consult service or a new function in the electronic medical record to remind and ease the referral process on providers. Sites improved marketing over time by making connections with other departments through active attendance at meetings, as well as educating staff and patient about their program—including pamphlets and equipment demonstrations.
Resources Allocation and Geography Strategies
The issue of space as a resource was dealt with through remote work, advocating for flex time where staff work longer shifts to reduce days of work, and providing technology (e.g., laptops) so staff could share space more effectively.
Sites mentioned few strategies for overcoming lack of leadership support given that the leadership challenge was the least mentioned over the grant period. One site mentioned using the following strategy: having a program representative participate in rounds and other meetings where leaders were present to increase visibility and attract leadership to their program.
This study sought to identify the challenges LTSS programs experienced over the course of a large-scale evaluation and describe strategies sites used to overcome these challenges. We found six challenges influenced implementation of the noninstitutional LTSS programs. We saw little variation in these factors across model types or by active or inactive status suggesting that the challenges were applicable regardless of program type or ability to sustain a program over the grant period.
To date, very little research on program implementation has been conducted outside of the acute care setting, particularly within the LTSS realm. Interestingly, our findings regarding implementation challenges were consistent with the research on program implementation, which has shown that there are certain issues that need to be addressed during program implementation (Bradley et al., 2004 , 2005 ; Damschroder & Lowery, 2013 ; Feldstein & Glasgow, 2008 ; Kadu & Stolee, 2015 ; Tomoaia-Cotisel et al., 2013). For example, research suggests that it is important to have leaders to support implementation and sustainability by making decisions about resources such as staff time and commitment to the program (Bradley et al., 2004 ; Ijkema, Langelaan, van de Steeg, & Wagner, 2014). In addition, team dynamics and processes are crucial for gaining internal support for the program, linking the new program with other existing geriatric programs, and otherwise supporting program implementation, which may run into periods where project operations are inefficient (Bradley et al., 2004 ; Reuben, 2002). Thus, to improve implementation and sustainability of noninstitutional LTSS programs, it is important to work with staff and leadership to understand the value of the program. In addition, the program should advocate for support to deal with challenges within the organization like infrastructure (e.g., HR, resources, and geography). Finally, positive interdisciplinary team dynamics and integration are important for both positive teamwork and coordination through referral and marketing processes that enhance program implementation.
We also presented strategies to overcome each type of challenge, which can aid programs experiencing similar issues. We feel that this is an important contribution to the literature especially because there is very little written about LTSS program implementation. Interestingly, staff reported very few strategies for overcoming leadership challenges. We believe that this may be the case because leadership was the least reported challenge at all time points in the evaluation, and leadership support was initially required for participation in the grant initiative. In addition, throughout the grant period, the sponsor gave sites assistance on how to overcome local leadership resistance by providing training sessions on preparing business plans (including return-on-investment analysis when applicable) and ways to present findings to leadership to garner support for the program. Sites also participated in calls for each model type where problems such as leadership support could be discussed and sites could share best practices and learn from each other.
Upon reviewing the strategies, we think there are three cross-cutting areas in which these sites targeted their strategies including organizational functions and processes (e.g., importance of effective coordination across disciplines via referrals and marketing, staffing), infrastructure (e.g., IT and HR), and program fit with priorities in the organization (e.g., assuring leadership buy-in given multiple priorities). These areas all fall into the health care organization itself and are consistent with ways to provide high-quality health care when the appropriate organizational supports (e.g., OTM drivers) are in place. We hypothesize that this is the case because the GEC T21 initiative already had the OTM drivers in place including an impetus to change health care with the rebalancing of institutional and noninstitutional care in the VA, national leadership support through funding for these programs and strong administrator support for participating sites, program alignment within GEC service in the VA, integration across boundaries because of the interdisciplinary nature of the LTSS programs, and staff engagement with the programs since the grants were submitted by them. Thus, with the drivers in place, the strategies targeted changing components of the health care system. However, we did see that the most often mentioned cross-cutting strategy was communication. Although the OTM model alludes to the importance of communication for effective change, it was not one of the top drivers in the model. A recent review of implementation literature focused on communication (Manojlovich, Squires, Davies, & Graham, 2015) advocates for the inclusion of communication as a driver of implementation. Our findings suggest that communication was often used to overcome implementation challenges and is a facilitator to implementation.
Several considerations are important when interpreting these findings. Although we assessed a large number of sites and model types, we had a smaller number of key informants at some sites. However, the respondents we spoke to were the most knowledgeable about the programs at their site. Future efforts should incorporate as many staff perspectives as feasible given budget constraints. In addition, this multisite evaluation focused on a process evaluation, not clinical outcomes and costs, which were part of another GEC-sponsored effort (Intrator et al., 2013). Therefore, findings do not address programs’ long-term outcomes; rather they provide insight on program implementation and strategies to overcome these challenges. Additional studies are thus needed to assess the impact of GEC programs that has had and continues to have many initiatives rolled out to address the various needs of a diverse Veteran population. For example, in addition to the 59 noninstitutional LTSS programs we assessed, the VA funded another 108 programs through the Transforming VA Care to the 21st Century Strategic Plan. Furthermore, the VA has funded the Veterans Directed Home and Community-Based Program to provide Veterans (and their caregivers) more access, choice, and control over noninstitutional care services working in collaboration with the private sector. This evaluation was conducted in the VA, a large integrated network of hospitals. We believe that the challenges as a whole are applicable to private sector, in particular, results regarding leadership support, team dynamics and processes, and referrals and marketing. Although the VA has specific infrastructure of its own (e.g., IT, HR, contracting), other organizations are faced with their own sets of challenges dealing with various infrastructure or cross-departmental issues that can enable or inhibit program implementation.
This article’s largest practice implication is sharing strategies used to overcome program implementation challenges that can be used by other programs hoping to adopt these LTSS models inside or outside the VA. We saw that many of the strategies played to the strengths of LTSS programs themselves, that utilize interdisciplinary teams. The strategies we observed are reflective of ways an organization can successfully implement and support LTSS programs including having a strategy that sets its direction and aligns with priorities, effective operational clinical care functions and processes, and infrastructure (IT, HR, facility management) that support patient care (Lukas et al., 2007). Future research is needed to understand the role communication plays as a facilitator of implementation.
Beebe J. (2014). Rapid qualitative inquiry: A field guide to team-based assessment
. (2nd ed.). Lanham, MD: Rowman & Littlefield.
Bradley E. H., Schlesinger M., Webster T. R., Baker D., & Inouye S. K. (2004). Translating research into clinical practice: Making change happen. Journal of the American Geriatrics Society
, 52(11), 1875–1882. doi:10.1111/j.1532-5415.2004.52510.x
Bradley E. H., Webster T. R., Baker D., Schlesinger M., & Inouye S. K. (2005). After adoption: Sustaining the innovation. A case study of disseminating the hospital elder life program. Journal of the American Geriatrics Society
, 53(9), 1455–1461. doi:10.1111/j.1532-5415.2005.53451.x
Chaudoir S. R., Dugan A. G., & Barr C. H. (2013). Measuring factors affecting implementation
of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implementation Science
, 8, 22. doi:10.1186/1748-5908-8-22
Damschroder L. J., & Lowery J. C. (2013). Evaluation of a large-scale weight management program using the consolidated framework for implementation
research (CFIR). Implementation Science
, 8, 51. doi:10.1186/1748-5908-8-51
Feldstein A. C., & Glasgow R. E. (2008). A practical, robust implementation
and sustainability model (PRISM) for integrating research findings into practice. Joint Commission Journal on Quality and Patient Safety
, 34(4), 228–243.
Hudson R. B. (2010). More change than continuity? The evolution of long-term care policy. Public Policy & Aging Report
, 20(1), 2.
Ijkema R., Langelaan M., van de Steeg L., & Wagner C. (2014). What impedes and what facilitates a quality improvement project for older hospitalized patients? International Journal for Quality in Health Care
, 26(1), 41–48. doi:10.1093/intqhc/mzt079
Intrator O., Cai S., Gidmark S., Meucci M., Kinosian B., Phibbs C., Shay K.(2013). The Geriatrics and Extended Care Data and Analyses Center: evaluation of the VHA Office of Geriatrics and Extended Careã(tm)s transformational pilot projects
, Paper presented at: AcademyHealth Annual Research Meeting; 2013 Jun 24, Baltimore, MD.
Kadu M. K., & Stolee P. (2015). Facilitators and barriers of implementing the chronic care model in primary care: a systematic review. BMC Family Practice
, 16, 12. doi:10.1186/s12875-014-0219-0
Kapp M. B. (2014). Home and community-based long-term services and supports: Health reform's most enduring legacy. Saint Louis University Journal of Health Law & Policy
, 8, 9.
Kertesz S. G., Austin E. L., Holmes S. K., Pollio D. E., Schumacher J. E., White B., & Lukas C. V. (2014). Making housing first happen: Organizational leadership in VA's expansion of permanent supportive housing. Journal of General Internal Medicine
, 29(Suppl. 4), 835–844. doi:10.1007/s11606-014-3025-4
Kuwahara Y., Saito A., Arima K., & Ohmi H. (2011). Center of excellence for atomically controlled fabrication technology. Journal of Nanoscience and Nanotechnology
, 11(4), 2763–2776.
Lukas C. V., Holmes S. K., Cohen A. B., Restuccia J., Cramer I. E., Shwartz M., & Charns M. P. (2007). Transformational change in health care systems: an organizational model. Health Care Management Review
, 32(4), 309–320. doi:10.1097/01.HMR.0000296785.29718.5d
Manojlovich M., Squires J. E., Davies B., & Graham I. D. (2015). Hiding in plain sight: Communication theory in implementation
science. Implementation Science
, 10, 58. doi:10.1186/s13012-015-0244-y
Meyer D. Z., & Avery L. M. (2009). Excel as a qualitative data analysis tool. Field Methods
, 21(1), 91–112. doi:10.1177/1525822X08323985
Miles M. B., & Huberman A. M. (1994). Qualitative data analysis: An expanded sourcebook
. (2nd ed.). Thousand Oaks, CA: Sage Publications.
Pogoda T. K., Cramer I. E., Rosenheck R. A., & Resnick S. G. (2011). Qualitative analysis of barriers to implementation
of supported employment in the Department of Veterans Affairs. Psychiatric Services
, 62(11), 1289–1295. doi:10.1176/appi.ps.62.11.1289
Reuben D. B. (2002). Organizational interventions to improve health outcomes of older persons. Medical Care
, 40(5), 416–428.
Shay K., Hyduke B., & Burris J. F. (2013). Strategic plan for geriatrics and extended care in the veterans health administration: background, plan, and progress to date. Journal of the American Geriatrics Society
, 61(4), 632–638. doi:10.1111/jgs.12165
Smith M., & Wolfe B. M. (2010). Center of excellence designation: Pros and cons: Comment on “Reduced access to care resulting from centers of excellence initiatives in bariatric surgery. Archives of Surgery
, 145(10), 997–998.
Tomoaia-Cotisel A., Scammon D. L., Waitzman N. J., Cronholm P. F., Halladay J. R., Driscoll D. L., … Stange K. C. (2013). Context matters: the experience of 14 research teams in systematically reporting contextual factors important for practice change. Annals of Family Medicine
, 11(Suppl. 1), S115–S123. doi:10.1370/afm.1549
VA Office of Strategic Integration. (2013). VHA T21 implementation & sustainment guidance
. Washington, DC: Department of Veterans Affairs.
VanDeusen Lukas C., Engle R. L., Holmes S. K., Parker V. A., Petzel R. A., Nealon Seibert M., … Sullivan J. L. (2010). Strengthening organizations to implement evidence-based clinical practices. Health Care Management Review
, 35(3), 235–245. doi:10.1097/HMR.0b013e3181dde6a5
Wysocki A., Butler M., Kane R. L., Kane R. A., Shippee T., & Sainfort F. (2012). AHRQ comparative effectiveness reviews. Long-term care for older adults: A review of home and community-based services versus institutional care
. Rockville, MD: Agency for Healthcare Research and Quality (US).
Keywords:Copyright © 2018 Wolters Kluwer Health, Inc. All rights reserved
implementation; long-term services and support programs; qualitative methods