Secondary Logo

Journal Logo

Original Articles

Tailored to Fit

How an Implementation Framework Can Support Pragmatic Pain Care Trial Adaptation for Diverse Veterans Affairs Clinical Settings

Purcell, Natalie PhD, MPA*,†; Becker, William C. MD‡,§; Zamora, Kara A. MA*,†; McGrath, Sarah L. MA*; Hagedorn, Hildi J. PhD∥,¶; Fabian, Eva R. MPH*; McCamish, Nicole MA*; Seal, Karen H. MD, MPH*,†

Author Information
doi: 10.1097/MLR.0000000000001376
  • Free


Over 100 million Americans—nearly one third of the US population—suffer from chronic pain or pain that persists for at least 6 months.1,2 Military veterans suffer disproportionately, with over 50% reporting chronic pain.3 War-related trauma can amplify chronic pain symptoms, increasing risk for disability and overuse of pain medication, including opioids.4,5 Currently, the rate of opioid-related deaths among veterans is nearly double that of US adults overall.6 Chronic pain care and opioid prescribing occur predominantly in primary care settings, yet most primary care providers (PCPs) are ill-equipped to manage chronic pain and opioid misuse due to a lack of training and time, financial disincentives, and other systemic constraints.7–9 In response, the Department of Veterans Affairs (VA) and other health care organizations have recognized a critical need to develop new, nonpharmacological approaches to chronic pain care.10,11

In parallel, the VA is implementing a “Whole Health” model at its health care facilities nationwide.12 In this model, a Whole Health “partner” or health coach assists patients in completing a personalized health inventory13 evaluating their experiences across multiple dimensions of health and well-being (eg, energy, rest, nutrition, relationships, surroundings). Using the completed inventory, the veteran collaborates with his/her care team to develop a personal health plan14 consisting of “SMART” goals (specific, measurable, action-oriented, realistic and time-bound goals) grounded in the veteran’s personal values. Veterans are offered education, tools, and support (eg, coaching) to help them meet their goals.14 Also key to the Whole Health model is connecting veterans to wellness programs and complementary and integrative health (CIH) modalities, such as yoga, Tai Chi, and meditation.

With the Comprehensive Addiction Recovery Act of 2016,15 and in response to the national opioid crisis,16 VA has promoted Whole Health and CIH care for veterans with chronic pain. To date, trials have demonstrated the effectiveness of multiple CIH modalities,17–19 interdisciplinary pain programs,18,20 and nonpharmacological approaches such as cognitive-behavioral therapy and exercise19,21,22 in treating chronic pain. However, no trial has provided evidence for the effectiveness of the Whole Health model in treating chronic pain. Similarly, no trial has identified best practices for implementing Whole Health for chronic pain care.

Under the auspices of the Pain Management Collaboratory (PMC)—a cooperative agreement funded by the National Institutes of Health, the Department of Defense, and the VA23,24—we are conducting the first large-scale pragmatic trial that evaluates the Whole Health model’s effectiveness in treating veterans with chronic pain: Whole Health Options and Pain Education (wHOPE). The wHOPE study is a Hybrid Type-125 pragmatic implementation-effectiveness trial to be conducted at 5 geographically distinct VA locations across the United States. Nearly 800 veterans with moderate to severe chronic pain will be randomized to receive a Whole Health Team (WHT) intervention, a less intensive Primary Care Group Education (PC-GE) intervention, or usual VA primary care. The WHT intervention consists of an interdisciplinary collocated team of CIH/Whole Health medical and ancillary providers, including a Whole Health coach; the team uses the Whole Health model to engage and empower patients, mobilizing nonpharmacological and CIH approaches for chronic pain management. The PC-GE intervention is a modified, group-based Cognitive-Behavioral Therapy for Chronic Pain (CBT-CP). These interventions will be compared with each other, and both to usual care.

The study’s primary aim is to determine whether veterans randomized to WHT are more likely than those randomized to PC-GE and usual care to: (1) experience reduced pain interference, (2) decrease use of pain medications, including opioids, (3) engage in more nonpharmacological or CIH pain management activities, and (4) experience improvements functioning, quality of life, and mental health symptoms. The secondary aim is to conduct a process evaluation and budget impact analysis to support scaling and dissemination of effective interventions. Results of this study will contribute to the PMC’s overall mission to build a national-level infrastructure that supports nonpharmacological pain management in veterans and military personnel.

Consistent with this mission, we conducted a developmental formative evaluation to tailor implementation of the study interventions to the needs, preferences, and resources of each VA enrollment site. We engaged local stakeholders and collected site-specific data to enhance trial pragmatism across several domains, including trial setting, participant eligibility, recruitment methods, intervention delivery, and measurement strategy.26,27 In this manuscript, we report the results of the formative evaluation and describe our iterative process of qualitative data collection, stakeholder-engagement, and evidence-based quality improvement (EBQI).28 We explain how this process prepared diverse VA settings for the implementation of new pain care interventions and for the launch of our trial.


Informed by the PARiHS (Promoting Action on Research Implementation in Health Systems) framework29,30 we gathered site-specific information relevant to wHOPE study implementation. The PARiHS framework requires attention not only to existing evidence (the scientific rationale for intervention implementation), but also to context (various local contextual factors likely to affect implementation) and to facilitation (strategic approaches to guiding/supporting local implementation). Our formative evaluation incorporated each of these elements.

We began with an initial evidence-based trial design, evaluated and informed by the PMC. Then, to better understand the context for study implementation, we conducted semistructured qualitative interviews with a diverse sample of frontline clinicians, administrators, veterans, and other stakeholders at all 5 wHOPE enrollment sites (the VA Health Care Systems in Portland, San Francisco, St Louis, Tampa, and Connecticut) and 1 additional site selected to serve as a backup should enrollment be insufficient at the other sites (Little Rock). Participating stakeholders and veterans at each site were purposively selected by the local lead investigator based on their work in pain care, primary care, mental health, and/or Whole Health. Each received a personal invitation to participate (for VA staff, by email; for veterans, by mail and follow-up phone call). Across all sites, 74 stakeholders, including 49 VA staff and 25 veterans, completed 30–60-minute telephone interviews (Table 1). Staff participants included physicians, nurse practitioners, registered nurses, psychologists, and other clinicians working across primary care, pain specialist programs, and Whole Health programs. The purpose of the interviews was to: (1) assess the feasibility and acceptability of the study interventions; and (2) identify potential facilitators of and barriers to: (a) implementing and staffing each clinical study arm and (b) recruiting and enrolling veterans into the study.

TABLE 1 - Formative Evaluation Participants
Site Physicians (MD) NP RN Psychologists Other Providers* Total VA Staff Veteran Patients Total Participants
 Portland VA 2 0 3 3 2 10 6 16
 San Francisco VA 2 2 2 3 1 10 3 13
 St Louis VA 2 1 0 2 5 10 3 13
 Tampa VA 5 1 0 4 0 10 4 14
 West Haven VA 4 1 0 1 0 6 4 10
 Little Rock VA (backup site) 1 0 0 1 1 3 5 8
 Interview subtotals 16 5 5 14 9 49 25 74
EBQI meetings
 Portland VA 3 0 1 4 4 12 0 12
 San Francisco VA 4 2 1 2 2 11 1 12
 St Louis VA 3 0 2 7 9 21 1 22
 Tampa VA 3 0 0 6 0 9 1 10
 West Haven VA 4 1 1 6 3 15 0 15
 EBQI subtotals 17 3 5 25 18 68 3 71
*Other providers includes: pharmacists, recreational therapists, occupational therapists, social workers, chiropractors, dieticians, Whole Health coaches, peer support specialists, and unspecified provider types.
EBQI indicates evidence-based quality improvement; NP, nurse practitioners; RN, registered nurses; VA, Veterans Affairs.

We audio-recorded all interviews and analyzed them using a rapid analysis method developed for health services research.31,32 At least 2 trained analysts independently listened to each interview and prepared a written summary using a templated matrix organized by topical areas. The analysts then collaborated with the principal investigators (PIs) to review all matrices, identify recurring themes, and refine the description of each theme.

After completing interviews, we conducted an on-location EBQI meeting28 at each enrolling site. EBQI meetings were attended by the key study stakeholders at the site, including the local site PI, VA leaders, veteran representatives, and frontline clinicians involved in pain care, primary care, mental health, and Whole Health. Participants were purposively selected and personally invited to participate by the lead investigator for the local site and the study PI. Across all sites, 71 stakeholders attended (Table 1). At each EBQI meeting, the PIs described the trial interventions, reviewed evidence for them, highlighted flexible components, and summarized themes from completed interviews. The PIs then facilitated a discussion to troubleshoot potential barriers and identify site-specific strategies and adaptations to facilitate implementation. Qualitative analysts attended each EBQI meeting to take rapid ethnographic notes32,33 on identified barriers, facilitators, and implementation strategies.

Finally, the study PIs collaborated with local site investigators through regular meetings and ad hoc communication to tailor implementation plans, further adjust the study protocol, and launch the trial.


Acceptability and Feasibility of the Study Interventions

Stakeholders were asked to assess the overall acceptability and feasibility of each study intervention and its core components (see Table 2, “initial design” rows). Most veterans, clinicians, and other stakeholders reacted positively to the description of a holistic, team-based approach to pain care, and most thought that the WHT intervention would be desirable to both providers and patients. In the words of a psychologist from Tampa VA: “Trying innovative things to help patients wrap their heads around a mind-body approach to chronic pain is a great thing.” Many veterans and clinicians voiced excitement at the prospect of increasing access to integrative health modalities for pain care, often referencing personal, positive experiences with CIH and Whole Health. “It’s empowering—making decisions on your healthcare based on what’s important to you,” volunteered a veteran peer specialist from Portland, “it provides… a sense of being heard and provides ownership.”

TABLE 2 - Description of Active Study Interventions: Initial Design and Modifications Based on Formative Evaluation
Active Study Interventions Team Intervention Components Overall Structure
Whole Health Team
 Initial Design Medical provider (eg, primary care doctor or nurse practitioner) Mental health provider (eg, psychologist, social worker) CIH provider specializing in an integrative treatment modality Health coach (ancillary) Providers see patients as a team—all 3 clinical providers are present all visits with the same patient (in the same room, at the same time). The team helps the patient develop a personalized health plan—a care plan that is tailored to the patient’s needs and based on the patient’s goals and preferences. The personal health plan includes referrals to wellness programs and CIH treatments—for instance, yoga, Tai Chi, or acupuncture. Patients receive weekly coaching to help them reach their care goals. (1) Personal health inventory visit (completed with Whole Health coach) (2) Initial Whole Health Team visit (creation of personal health plan) (3) At least 3-follow-up Whole Health Team visits (4) Final Whole Health Team visit (5) Concurrent Health Coaching (8 sessions)
 Modifications Two required clinicians:  (1) Medical provider  (2) CIH provider of any kind (eg, yoga or Tai Chi instructor, acupuncturist) Inclusion of mental health provider is optional WHT clinicians are no longer required to be colocated for every visit, which will likely lead to variation in the dose and intensity of the WHT visits. The medical provider must be present at the minimum number of required study visits to deimplement and prescribe nonopioid medications. Prescribing by the WHT arm can be negotiated with patient’s regular PCP on a case-by-case basis.
 Initial design Mental health provider (eg, psychologist) trained in delivery of CBT-CP Enrolled patients participate in group CBT-CP, facilitated by a trained mental health provider. Patients attend 5 sequential weekly group sessions with other veterans to learn practical, behavior-based strategies for coping with chronic pain. (1) CBT-CP orientation (2) Weekly CBT-CP modules (5 sessions) (3) Discharge planning (4) Optional “Booster” sessions
 Modifications Multiple therapists may cross-cover groups Trainees may serve as therapists Optional “open-group” format (more consistent with drop-in classes); study participants will start with whichever of the 5 manualized PC-GE sessions is scheduled after their required orientation session.
Eligibility (for both arms)
 Initial design Eligibility: any VA patients with: (1) Pain every day or nearly every day for ≥6 mo; (2) PEG score ≥5 Patients are excluded if suicidal, cognitively impaired, in hospice, or otherwise unable to participate. PCPs must give written permission to recruit their patients.
 Modifications PCPs will not be required to provide written permission for study staff to contact their patients for recruitment.
CBT-CP indicates Cognitive-Behavioral Therapy for Chronic Pain; CIH, complementary and integrative health; PC-GE, Primary Care Group Education; PCP, primary care provider; VA, Veterans Affairs; WHT, Whole Health Team.

At some facilities, clinicians expressed concern about potential duplication of or competition with existing pain care resources and lack of sufficient CIH availability to support referrals from the study intervention team. Other clinicians and veterans voiced skepticism about the WHT intervention, doubting that CIH and Whole Health approaches could be effective pain care. “Meditation, mindfulness and all that googly-glop just doesn’t work…” asserted one Connecticut veteran. “All these [CIH] resources really don’t matter,” agreed a Tampa pain clinic provider, “[Veterans] laugh them off.”

Participants varied in their assessment of the acceptability and feasibility of the PC-GE study arm as well. Many clinicians believed that the study would help increase access to an important evidence-based intervention (CBT-CP). Many veterans also found the PC-GE arm appealing, citing the opportunity to connect with other veterans in a group setting. “I’ve seen miracles happen in those groups,” shared a San Francisco veteran. Veterans felt that their peers might have more credibility than clinicians when talking about pain and making pain-related recommendations: “If you get a guy sitting there who’s an amputee… and he’s talking to five or six guys [about chronic pain], they’re going to listen to him.”

Clinicians, however, cautioned that mental health groups can be poorly attended and have high attrition. A Tampa PCP acknowledged her reluctance to refer veterans to pain education groups like PC-GE, “A lot of people come out angry, [saying] ‘they’re trying to convince me that I don’t have pain.’” Indeed, several veterans reacted negatively to the PC-GE group description, questioning mental health care for pain. “The conditions you are going to be dealing with are real conditions; they’re not hypothetical, not mental,” shared a Tampa veteran, “You can talk till the cows come home; it’s not going to resolve the chronic condition or relieve pain.”

Implementation Barriers and Facilitators

At most sites, staff stakeholders identified the limited availability of clinical staff time as the primary barrier to successful trial implementation. They felt that staffing the study interventions using their health care system’s current local resources (ie, without designated funds to hire new clinicians) could be challenging. Clinicians described significant constraints on their availability due to rigid productivity and access metrics. As an RN from Portland explained, it can be “a lot of hard work… just keeping your head above water.” Other implementation barriers commonly cited by staff/clinicians included limited clinical space and scheduling support.

To overcome these barriers, staff stressed the need to clarify resource requirements and to secure executive leadership’s support from the start. They felt facility leadership should explicitly endorse “protected time” for the clinicians delivering the study interventions to avoid their reassignment to other work. “If the higher ups are not on board, nothing will come of it,” cautioned a Tampa physician.

Participants also stressed the need for broad, inclusive, consistent communication about the study and its aims to all potentially impacted VA staff. They recommended careful internal messaging targeted at frontline staff and middle management to overcome the perception that the study is competing with current programs for limited resources. Several participants recommended emphasizing that the patients served by wHOPE are patients who would otherwise be served by existing clinics and clinicians, and who would be harder to successfully manage within primary care than within coordinated interventions like WHT and PC-GE. “If Whole Health can lighten [PCP’s] burden in a way by offering modalities and resources that will be meaningful for the patients… there will be enthusiasm from primary care,” noted a Tampa PCP. “Advertise the interventions [as]… providing for the veteran and for the provider as well…show mutual benefit,” suggested a Little Rock pharmacist.

EBQI meetings generated ideas about other strategies to overcome resource and staffing barriers. Clinicians and staff generally agreed that maximizing flexibility in staffing and visit structure would be beneficial—for example, allowing different types of providers and trainees to serve as study interventionists where their scopes of practice would permit. Proposed structural modifications included allowing rolling admission into CBT-CP groups in the PC-GE arm (Connecticut) and permitting WHT providers to see patients sequentially or in varied configurations as indicated and practical (Portland).

Participants at all sites discussed how to utilize telehealth to overcome resource and staffing constraints—for example, allowing clinicians at a central VA site to see patients across multiple community clinic sites. Other commonly cited implementation facilitators to mobilize included enthusiastic Whole Health champions, local expertise in chronic pain care, experience with and interest in research, and consistency between VA priorities and the study aims—namely, shared commitments to reducing opioids, improving chronic pain treatment, and expanding Whole Health/CIH offerings.

Study Recruitment Barriers and Facilitators

Among stakeholders and veterans, the most commonly cited barrier to veteran recruitment was potential skepticism about CIH and mental health interventions for pain. “I have pain… And, it’s not a mental issue,” said a St Louis veteran, “Acupuncture is not going to put collagen back in my neck.” Some veterans also expressed concern that participating in the study would mean giving up their pain medications or jeopardizing their disability benefits should their pain improve. “There’s fear involved,” acknowledged a Little Rock veteran, “What if they take my drugs away from me?” “The first thing that came to my mind” upon hearing about the study, said a St Louis veteran, was, “[this is] an effort to reduce our benefits.”

Suggested strategies for overcoming these barriers included using plain and direct language in recruitment materials (eg, “treating your pain with less medication” instead of “non-pharmacological treatments”) and emphasizing improvement in quality of life over medication reduction. In describing the PC-GE intervention, veterans and clinicians recommended explaining how CBT can be helpful in managing pain without implying that pain is just “in your head.” Veterans and staff also recommended emphasizing that the study interventions add care choices without taking any away: “When veterans have some choices and they have more than one thing to go towards, people feel freedom and ownership,” explained a rehabilitation therapist in St Louis.

Another commonly cited recruitment barrier was the required time commitment for study participation and travel—a barrier that was more pronounced at sites serving wide geographic areas. “I genuinely believe in giving back and helping other veterans,” observed a Portland-area veteran, “but deal-breakers for me are inflexibility… having to take time off from my work day and supporting my family to do this kind of stuff.” Suggested strategies to mitigate this barrier included: clarifying time commitments and participation requirements up front; maximizing appointment options; and reimbursing travel expenses. Participants also recommended making the pool of eligible participants as broad as possible by minimizing exclusion criteria. Finally, they suggested making services accessible through telehealth technologies (eg, VA Video Connect to home).

Stakeholders also addressed potential barriers to enlisting PCP support with recruitment. Among clinicians and administrators, there was a general consensus that PCPs do not have time to add anything else to their visits and are unlikely to remember to refer patients to the study. In the words of a St Louis PCP: “Anything to add, even box-clicking, is more than primary care can handle.” Suggestions for overcoming this barrier included ensuring that providers understand how the study will help with their workload and benefit their patients. Alternately, multiple participants recommended making study recruitment relatively independent of PCPs—for example, by focusing on direct patient recruitment and accepting referrals from a wider range of clinicians.

Modifications to Study Design and Strategy

Researchers made several modifications to the study design based on the formative evaluation process. These modifications, detailed in Table 2 (see “modifications” rows), were designed to maximize study pragmatism. Modifications focused on introducing additional elements of flexibility into intervention-team composition and mode of care delivery, allowing individual sites to modify the interventions to match local needs and resources while retaining essential features. The researchers also adopted several of the strategies recommended by stakeholders to address anticipated barriers to arm implementation, patient recruitment, and patient retention in the study. These identified barriers and adopted strategies are detailed in Table 3.

TABLE 3 - Implementation Issues and Stakeholder-identified Solutions, Strategies, and Work-Arounds
Area of Concern Identified Issue Stakeholder-identified Solutions, Strategies, and Work-Arounds
Intervention implementation/assembling clinical teams Limited clinician availability (qualified clinicians already have full schedules) Perception that clinicians are already stretched thin (due to demanding productivity and access metrics) Limited/inadequate clinical space Limited administrative/scheduling support Maximize flexibility in arm staffing and visit structure. Itemize resource requirements (including clinician/staff time); secure executive leaders’ explicit commitment to provide these resources. Seek “protected time” for clinicians delivering study interventions. Allow delivery of services via telehealth. Frame study as a service to help the medical center manage complex patients who would otherwise require primary care resources.
Securing VA-wide buy-in for study Potential skepticism about value of research Perceived competition with existing programs for patients and other resources Broad, inclusive, consistent communication about the study, its aims, and potential benefits to patients and the health care system. Emphasize how study can help medical center meet its metrics and mandates (eg, reducing opioid prescriptions, improving chronic pain treatment, expanding Whole Health/CIH offerings). Leverage enthusiastic Whole Health champions as study champions.
Provider collaboration in patient recruitment PCPs do not have time to help with recruitment, describe study to patients PCPs may be leery of referring patients to a study that includes randomization; not sure what options patient will receive Communicate study aims & anticipated patient benefits to PCPs. Explain how study will help PCPs with their workload, especially challenging/complex chronic pain patients. Embed study staff within primary care to make hand-off’s easy. Do not rely on PCPs as primary recruitment tool—for example, focus on direct patient recruitment; accept other referral sources.
Direct patient recruitment Skepticism about CIH effectiveness Doubts that CBT-CP is real pain care Concern that participating in study means pain medications will be taken away Concern that participating in the study would mean losing disability benefits Use plain and direct language in recruitment materials. Emphasize help for pain and improvement in quality of life (not medication reduction/elimination). Emphasize that the study adds care choices; does not take any away. Make the pool of eligible participants as broad as possible (minimizing exclusion criteria).
Patient retention Track record of poor group attendance/high drop-out rates for group-based interventions Patient transportation challenges/difficulty coming to appointments Patient scheduling challenges/difficulty committing to regular appointments Clarify time commitments/expectations up front. Make services accessible through telehealth technologies (eg, video tele-conferencing or VA Video Connect to home). Maximize appointment/scheduling options. Provide transportation reimbursement where possible.
CBT-CP indicates Cognitive-Behavioral Therapy for Chronic Pain; CIH, complementary and integrative health; PCP, primary care provider; VA, Veterans Affairs.


After decades of biomedically oriented pain care where passive, low-impact treatments have dominated, the VA and health care systems nationwide are seeking more holistic treatment paradigms that promote active self-management, incorporate CIH modalities, and focus on well-being, functioning, and quality of life.19,20 With this paradigm shift, researchers are challenged to develop innovative strategies for conducting large, pragmatic effectiveness trials and identifying intervention-implementation strategies for diverse clinical settings.21 To aid in this effort, we have described a systematic formative evaluation that enabled multiple VA sites to prepare for the first major trial of the Whole Health model for chronic pain.

Hybrid implementation-effectiveness trials like the wHOPE study must maintain a careful balance between their pragmatic and explanatory aims: they must be able to validly and reliably examine the effectiveness of the interventions under study; yet, they must be sufficiently practical to test those interventions in real clinical contexts.25–27 Given the critical need to implement new pain care models in diverse, real-word clinical settings, we aimed to make the wHOPE study as pragmatic as possible while retaining the essential features of a randomized controlled trial and ensuring adequate intervention fidelity. Our systematic formative evaluation enabled us to maximize study pragmatism across several domains, including trial setting, participant eligibility, recruitment methods, intervention delivery, and measurement strategy.

The enhanced pragmatism of our modified trial design can be visualized with the Pragmatic Explanatory Continuum Indicator Summary (PRECIS-2) tool (Fig. 1). Published in 200926 and refined in 2015,27 the tool was designed to aid investigators in assessing trial design pragmatism and making informed design decisions across multiple domains. The PRECIS-2 tool draws attention to the inherent tension between pragmatic and explanatory research designs; decisions made to increase pragmatism can decrease a study’s explanatory potential. The modifications made to the wHOPE study illustrate this tension: increasing flexibility in intervention delivery methods and team composition will make implementation across diverse sites more feasible but could also result in differences in intervention delivery across sites. Variability in implementation, if too great, could result in a lack of intervention fidelity. This, in turn, could threaten the validity of comparisons across arms and make it difficult to identify intervention effects if, for example, the interventions become too similar. For this reason, our protocol includes fidelity monitoring to ensure that core elements of both active interventions are implemented and maintained. Careful fidelity monitoring will also allow us to describe variability where it exists, especially in cases where observed outcomes differ from those expected.

wHOPE PRECIS-2. VA indicates Veterans Affairs.

The most significant pragmatic modifications to the intervention structure—in particular, allowing rolling admission to PC-GE groups and sequential as well as colocated WHT visits—may affect intervention efficacy in ways that are not yet clear. Arguably, they may also reduce the distinction between the study interventions and usual primary care, especially in light of VA’s nationwide efforts to incorporate Whole Health approaches into primary care, to ensure widespread access to CIH, and to routinely provide CBT-CP for veterans with chronic pain. In short, the trial modifications that facilitate implementation of the interventions at individual sites may decrease the likelihood of observing meaningful differences between study arms.

Nonetheless, it is crucial to achieve sufficient pragmatism to conduct a successful trial in diverse clinical contexts and to inform even broader dissemination. Our systematic formative evaluation, informed by the PARIHS framework,29,30 enabled us to calibrate a reasonable balance. By methodically engaging a broad swath of study stakeholders, we secured stakeholder buy-in and commitments from all sites to implement the study interventions. We also developed evidence-informed, site-specific implementation plans to maximize study feasibility and accessibility, tailoring our research design to accommodate identified changes.

Our formative evaluation strategy included several replicable components that enhanced pragmatism and laid the groundwork for successful trial implementation. Those components, outlined in Table 4, include the following:

  • Semistructured, qualitative interviews with representatives from all major stakeholder groups (clinicians, administrators, and veterans) at all potential implementation sites: These enabled the study team to gather in-depth feedback on intervention acceptability and feasibility, as well as suggestions for improving both.
  • Performing a rapid-turnaround analysis31,32of interview data: This facilitated the swift identification—and report back—of central themes relevant to implementation, including potential local facilitators and champions, likely barriers and strategies to overcome them, and possible design modifications to improve local fit.
  • Structured, facilitated, on-location EBQI meetings28with major stakeholders: These meetings helped sites prepare for local implementation by: familiarizing participants with the study and preliminary implementation plans; mobilizing themes from the interview data to focus the discussion on potential barriers/issues; engaging stakeholders in a brainstorming/problem-solving dialogue; and securing stakeholder buy-in.
  • Preparing and analyzing rapid ethnographies32,33after each EBQI meeting: This enabled capture of identified implementation themes, potential challenges, and proposed solutions to support decision-making.
  • Engaging local site investigators in routine communication with central study PIs and local site investigators: This enabled the study team to finalize and codify decisions regarding intervention structure and implementation and allowed iterative, site-specific refinement with attention to intervention fidelity.

TABLE 4 - Formative Evaluation Components
Action Description Purpose Product
(1) Stakeholder interviews Conduct qualitative, semistructured interviews. Target representatives from all major stakeholder groups and sites. Gather feedback on intervention acceptability/feasibility, potential issues/barriers. Solicit suggestions for improving both. Written summary matrix for each interview audio-recording
(2) Rapid analysis to identify themes Analyze qualitative interview data to rapidly identify themes for report back to stakeholders. Identify likely barriers and strategies to overcome them. Identify possible design modifications to improve local fit. Identify potential local facilitators and champions. List of key themes organized by topic area List of barriers and facilitators to implementation, provider collaboration, patient recruitment & retention
(3) EBQI meetings Facilitate structured, on-location EBQI meetings including all major stakeholders. Present study aims, active interventions, preliminary implementation plans. Present interview data/summary themes. Discuss barriers and issues requiring clarification and refinement. Engage stakeholders in an active brainstorming/problem-solving. Plan for local implementation (tailored changes to plans/protocols). Identify potential solutions, strategies, and work-arounds for any identified issues/barriers. Assess the feasibility/desirability of proposed strategies/solutions. EBQI presentation slides Meeting minutes summarizing identified issues as well as potential solutions, strategies, and work-arounds
(4) Quick ethnography Prepare rapid ethnographies of each EBQI meeting. Capture central themes, issues, and proposed solutions for investigator review/decision-making. Ethnographic summary of discussion and key decision points
(5) Iterative refinement Maintain regular communication between central study PIs and local site investigators. Codify decisions about modifications to study design, intervention structure, or implementation plans. Revisit and refine plans as needed. Make key decisions collaboratively while allowing for iterative refinement. Maintain intervention fidelity. Revised study protocol
EBQI indicates evidence-based quality improvement; PIs, principal investigators.

Conducting this step-wise, multicomponent formative evaluation was an important precursor to the successful launch of the first large trial to test VA’s Whole Health model—a major clinical undertaking with the potential to markedly shift chronic pain treatment paradigms.

Because the wHOPE study is being conducted as a pragmatic clinical trial at diverse VA sites across the nation, we envision that the study interventions will be ready for broad dissemination at the conclusion of the trial. Future researchers can follow the formative evaluation steps delineated here to make meaningful decisions about study design in pragmatic trials and to find an appropriate balance between their pragmatic and explanatory aims. This structured approach can facilitate the conduct of other large-scale pragmatic clinical trials, the implementation of novel pain care models in real-world clinical settings, and the dissemination of interventions found to be effective.


1. Institute of Medicine (US) Committee on Advancing Pain Research, Care, and Education. Relieving Pain in America: A Blueprint for Transforming Prevention, Care, Education, and Research. Washington (DC): National Academies Press; 2011.
2. Johannes CB, Le TK, Zhou X, et al. The prevalence of chronic pain in United States adults: results of an Internet-based survey. J Pain. 2010;11:1230–1239.
3. Kerns RD, Otis J, Rosenberg R, et al. Veterans’ reports of pain and associations with ratings of health, health-risk behaviors, affective distress, and use of the healthcare system. J Rehabil Res Dev. 2003;40:371–379.
4. Outcalt SD, Ang DC, Wu J, et al. Pain experience of Iraq and Afghanistan Veterans with comorbid chronic pain and posttraumatic stress. J Rehabil Res Dev. 2014;51:559–570.
5. Beckham JC, Crawford AL, Feldman ME, et al. Chronic posttraumatic stress disorder and chronic pain in Vietnam combat veterans. J Psychosom Res. 1997;43:379–389.
6. Bohnert ASB, Ilgen MA, Galea S, et al. Accidental poisoning mortality among patients in the Department of Veterans Affairs health system. Med Care. 2011;49:393–396.
7. Reuben DB, Alvanzo AAH, Ashikaga T, et al. National Institutes of Health Pathways to Prevention Workshop: the role of opioids in the treatment of chronic pain. Ann Intern Med. 2015;162:295–300.
8. Becker WC, Fraenkel L, Kerns RD, et al. A research agenda for enhancing appropriate opioid prescribing in primary care. J Gen Intern Med. 2013;28:1364–1367.
9. Krebs EE, Bergman AA, Coffing JM, et al. Barriers to guideline-concordant opioid management in primary care—a qualitative study. J Pain Off J Am Pain Soc. 2014;15:1148–1155.
10. Institute of Medicine (US) Committee on Advancing Pain Research, Care, and Education. National Pain Strategy: A Comprehensive Population Health-Level Strategy for Pain. Washington, DC: National Academies Press; 2016.
11. Dowell D, Haegerich TM, Chou R. CDC guideline for prescribing opioids for chronic pain—United States, 2016. JAMA. 2016;315:1624–1645.
12. Krejci LP, Carter K, Gaudet T. Whole Health: the vision and implementation of personalized, proactive, patient-driven health care for veterans. Med Care. 2014;52(suppl 5):S5–S8.
13. US Department of Veterans Affairs, Veterans Health Administration. MyStory: personal health inventory—whole health for life [internet]. 2019. Available at: Accessed October 11, 2019.
14. Simmons LA, Drake CD, Gaudet TW, et al. Personalized health planning in primary care settings. Fed Pract. 2016;33:27–34.
15. Comprehensive Addiction and Recovery Act. Public Law 114-198. 2016. Available at: Accessed October 11, 2019.
16. United States Department of Health and Human Services. HHS Acting Secretary declares public health emergency to address national opioid crisis. 2017. Available at: Accessed April 20, 2018.
17. Peterson K, Anderson J, Ferguson L, et al. Evidence Brief: the Comparative Effectiveness of Selected Complementary and Integrative Health (CIH) Interventions for Preventing or Reducing Opioid Use in Adults With Chronic Neck, Low Back, and Large Joint Pain. April, 2016. In: VA Evidence Synthesis Program Evidence Briefs [Internet]. Washington (DC): Department of Veterans Affairs; 2011.
18. Cheatle MD. Biopsychosocial approach to assessing and managing patients with chronic pain. Med Clin North Am. 2016;100:43–53.
19. Kligler B, Bair MJ, Banerjea R, et al. Clinical policy recommendations from the VHA State-of-the-Art Conference on non-pharmacological approaches to chronic musculoskeletal pain. J Gen Intern Med. 2018;33 (suppl 1):16–23.
20. Kerns RD, Krebs EE, Atkins D. Making integrated multimodal pain care a reality: a path forward. J Gen Intern Med. 2018;33(suppl 1):1–3.
21. Becker WC, DeBar LL, Heapy AA, et al. A research agenda for advancing non-pharmacological management of chronic musculoskeletal pain: Findings from a VHA state-of-the-art conference. J Gen Intern Med. 2018;33(suppl 1):11–15.
22. Hylands-White N, Duarte RV, Raphael JH. An overview of treatment approaches for chronic pain management. Rheumatol Int. 2017;37:29–42.
23. NIH-DoD-VA Pain Management Collaboratory. 2017. Available at: Accessed October 11, 2019.
24. National Institutes of Health (NIH). Federal agencies partner for military and veteran pain management research. 2017. Available at: Accessed October 11, 2019.
25. Curran GM, Bauer M, Mittman B, et al. Effectiveness-implementation hybrid designs: combining elements of clinical effectiveness and implementation research to enhance public health impact. Med Care. 2012;50:217–226.
26. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62:464–475.
27. Loudon K, Treweek S, Sullivan F, et al. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350:h2147.
28. Rubenstein LV, Meredith LS, Parker LE, et al. Impacts of evidence-based quality improvement on depression in primary care: a randomized experiment. J Gen Intern Med. 2006;21:1027–1035.
29. Rycroft-Malone J. The PARIHS framework—a framework for guiding the implementation of evidence-based practice. J Nurs Care Qual. 2004;19:297–304.
30. Kitson A, Harvey G, McCormack B. Enabling the implementation of evidence based practice: a conceptual framework. BMJ Qual Saf. 1998;7:149–158.
31. Hamilton AB. Qualitative methods in rapid turn-around health services research. Health Services Research & Development Cyberseminar; December 11, 2013. Available at: Accessed June 3, 2016.
32. Palinkas LA, Mendon SJ, Hamilton AB. Innovations in mixed methods evaluations. Annu Rev Public Health. 2019;40:423–442.
33. Ackerman S, Gleason N, Gonzales R. Using rapid ethnography to support the design and implementation of health information technologies. Stud Health Technol Inform. 2015;215:14–27.

pragmatic trials; formative evaluation; implementation science; qualitative methods; chronic pain; integrative health; veterans

Copyright © 2020 Wolters Kluwer Health, Inc. All rights reserved.