Secondary Logo

Journal Logo

Supplement Article

The Setting-Intervention Fit of Nine Evidence-Based Interventions for Substance Use Disorders Within HIV Service Organizations Across the United States: Results of a National Stakeholder-Engaged Real-Time Delphi Study

Garner, Bryan R. PhDa; Knudsen, Hannah K. PhDb; Zulkiewicz, Brittany A. MPHa; Tueller, Stephen J. PhDa; Gotham, Heather J. PhDc; Martin, Erika G. PhDd; Donohoe, Tom MPHe; Toro, Alyssa K. BSa; Loyd, Katie BAa; Gordon, Theodore MSf

Author Information
JAIDS Journal of Acquired Immune Deficiency Syndromes: July 1, 2022 - Volume 90 - Issue S1 - p S206-S214
doi: 10.1097/QAI.0000000000002981


  • Evidence-based innovations: The 9 evidence-based innovations for addressing substance use disorders are as follows: (1) acamprosate, (2) disulfiram, (3) oral naltrexone, (4) injectable naltrexone, (5) oral buprenorphine, (6) injectable buprenorphine, (7) contingency management, (8) motivational interviewing, and (9) cognitive behavioral therapy.
  • Innovation recipients: People with a substance use disorder.
  • Setting: Clinical and nonclinical HIV service organizations in the United States.
  • Implementation gap: There are many different evidence-based innovations for addressing substance use disorders, yet generalizable knowledge regarding the extent to which these evidence-based innovations are a good fit (ie, fundable, implementable, retainable, sustainable, scalable, and timely) for HIV service settings in the United States is lacking.
  • Primary research goal: To identify evidence-based innovations for addressing substance use disorders that are currently a good fit (ie, fundable, implementable, retainable, sustainable, scalable, and timely) for HIV service settings across the United States.


Launched in early 2019, Ending the HIV Epidemic (EHE) is an initiative seeking to reduce the number of new HIV infections in the United States by 90 percentage by 2030.1 If successful, EHE will help avert 250,000 new HIV infections.2 As part of a special supplement in The Journal of Acquired Immune Deficiency Syndromes, Eisinger et al (2019)3 noted “Implementation science will be essential to the successful achievement of the goals of this new initiative by translating evidence-based interventions, resulting from discovery and innovation science, into real-world practice” (p. S172). However, the design and conduct of rigorous implementation science trials for HIV prevention and treatment is challenging, which Hargreaves and colleagues (2019)4 highlighted as part of the same special supplement. HIV settings in the United States vary in the services that they deliver because some are clinically oriented for providing medical care, whereas others focus on social services to support people with HIV. Nevertheless, implementation research is urgently needed to support improved integration of substance use disorder (SUD) services into HIV settings5,6 because such services are important for helping with the EHE's treatment and prevention pillars.

Regarding efforts to improve the integration of SUD services within HIV settings, the Buprenorphine-HIV Evaluation and Support (BHIVES) Collaborative, conducted between 2004 and 2009, was one of the first and largest efforts.7-9 Focused on the integration of buprenorphine/naloxone within 10 clinical HIV service organizations (HSOs), the BHIVES initiative demonstrated that implementation of buprenorphine/naloxone in these settings was feasible and showed a positive effect on patient outcomes, including initiation of antiretroviral therapy, improvements in CD4 lymphocyte counts,10 and decreased opioid use.10–12 However, a systematic review by Haldane et al (2017)5 on integrating HIV and substance use services suggests that significant room for improvement remains more than a decade after BHIVES because buprenorphine/naloxone and other evidence-based SUD treatments remained limited within HSO settings.

Another large-scale effort to integrate SUD services within HSO settings was the Substance Abuse Treatment to HIV Care (SAT2HIV) Project, which was completed in 2020.13 A dual-randomized type 2 implementation-effectiveness hybrid trial,14 the SAT2HIV Project simultaneously tested 2 strategies for helping HSOs integrate a motivational interviewing-based brief intervention for SUDs15 and tested the effectiveness of the motivational interviewing-based intervention as an adjunct to usual care.15 Relative to usual strategies (ie, training, ongoing performance feedback, and ongoing intervention consultation), the Implementation and Sustainment Facilitation Strategy significantly improved implementation effectiveness (ie, the consistency and quality of implementation) and intervention effectiveness (ie, the effectiveness of the intervention for reducing substance use).13

In 2018, the National Institute on Drug Abuse funded the Substance Treatment Strategies for HIV Care (STS4HIV) Project, which is the focus of this study. As seen in the Implementation Research Logic Model for the STS4HIV Project (see Fig. 1),16 Aim 1 is to empirically identify stakeholder-driven recommendations regarding the most promising strategies to improve the integration of SUD services within HSO settings. To accomplish Aim 1, 3 stakeholder-engaged real-time Delphi (SE-RTD) studies were conducted to: (1) better understand the need for SUD treatment among people with HIV (ie, the patient needs construct of the outer setting domain); (2) assess the setting-intervention fit (SIF) of 9 evidence-based SUD treatment interventions and explore the extent to which SIF scores are predicted by characteristics of HSO representatives and inner setting characteristics; and (3) measure the setting-strategies fit of 10 strategies an AIDS Education and Training Center might use to help HSOs integrate treatment interventions for HIV and SUD. Aim 2 of the STS4HIV Project will compare the effectiveness of external facilitation as an adjunct to usual dissemination for helping HSOs advance 1 or more evidence-based SUD interventions along the Exploration-Preparation-Implementation-Sustainment framework's continuum17 to usual dissemination strategies (eg, mail, email, and website content).

Implementation research logic model for the STS4HIV project.

Results from our first SE-RTD study identified high levels of needs for SUD treatment services among people with HIV (PWH).18 After reading the DSM-5 criteria for SUD, staff from HSOs were asked to indicate the percentage of PWH who met the DSM-5 criteria for 5 substance use disorders. HSO staff reported that, on average, approximately 25.1% of PWH met criteria for past-year cocaine use disorder, 29.5% for methamphetamine use disorder, 33.3% for opioid use disorder, 40.1% for cannabis use disorder, and 41.8% for alcohol use disorder. Moreover, there were significant differences by geographic region of the United States, with generally lower perceived prevalence of alcohol, cannabis, cocaine, and opioid use disorders and higher perceived prevalence of methamphetamine use disorder in the West compared with those in other regions.

This study focused on the second SE-RTD study that assessed the fit between HSO settings and 9 evidence-based SUD treatment interventions, 6 of which were pharmacological interventions (acamprosate, disulfiram, oral naltrexone, injectable naltrexone, sublingual buprenorphine, and injectable buprenorphine) and 3 of which were psychosocial interventions (cognitive behavioral therapy, motivational interviewing, and contingency management). The 9 interventions were selected by the study's guiding coalition of stakeholders, which included experts from the fields of SUD treatment and HIV, based on reviews of effective treatments for SUDs19,20 and recommendations from the National HIV Curriculum.21

Similar to compatibility in Diffusion of Innovations Theory,22 innovation-values fit is a construct introduced by Klein and Sorra (1996)23 as a key implementation determinant within manufacturing settings, which Helfrich et al (2007)24 adapted for healthcare settings. Notably, the assessment of fit between the setting(s) of interest and the available evidence-based intervention(s) is an overlooked step in the implementation process,25 with a lack of appropriate measures being a key barrier.26,27 Thus, consistent with the recommendation by Allen and colleagues' (2017)27 for research to develop new measures for assisting with intervention selection, adoption, and implementation, we developed a 6-item measure for assessing the extent to which each specific HSO setting perceives each specific SUD treatment intervention to be: (1) fundable (to what extent is there funding available to train or hire a staff to offer the treatment intervention to individuals in need), (2) implementable (to what extent would a qualified staff have the necessary time and support to implement this treatment intervention with individuals in need), (3) retainable (once a qualified staff was trained or hired to offer this treatment intervention, to what extent would it be possible to keep the staff for at least 1 year after they have been hired or trained), (4) sustainable (after turnover of a staff qualified to offer this treatment intervention, to what extent would it be possible for a replacement staff to be hired or trained), (5) scalable (if there were an increase in client need, to what extent would it be possible to hire or train more staff to offer the treatment intervention), and (6) timely (to what extent is having a qualified staff available to offer this treatment intervention within this HIV service organization/site both needed and desired) for the setting of interest. Our development of these 6 criterion measures was guided by the Consolidated Framework for Implementation Research,28 which highlights the importance of the characteristics of interventions domain, and the Interactive Systems Framework,29 which highlights the importance of innovation-specific capacity (ie, the delivery system's organizational staff have the knowledge and skill for implementing the intervention with clients in need). We posit that assessment of these criterion measures is most useful when obtained before a setting making the formal decision to adopt a new intervention (ie, advance from the exploration phase to the preparation phase of the Exploration-Preparation-Implementation-Sustainment continuum),17 which aligns with insights from Proctor et al (2011).30 A common design is to make such assessments after spending a significant amount of time and money on training staff in the intervention.31–34 However, by measuring these factors before other strategies are deployed, the SIF index may help avoid wasting effort on interventions where implementation success is unlikely.

Beyond using the SIF index for assessing the absolute and relative fit of 9 evidence-based SUD treatment interventions, this study explored the extent to which SIF scores can be predicted by 1 or more measures from the characteristics of the individual domain (eg, HSO respondent's age, sex, and race) and/or the inner setting domain (eg, HSO's region of the United States, number of staff, and current availability of SUD services). All analyses were considered exploratory; therefore, no specific a priori hypotheses were posited.


Study Design

Expanding on the RTD design developed by Gordon and Pease (2006)35 and the traditional Delphi design,36 we developed an innovative web-based SE-RTD design. The RTD design encourages each participant to anonymously share the reasoning behind their rating selection for each item in open-ended text fields. They can then view other participants' ratings and reasonings and change their own rating based on the information shared by other participants. All these interactions happen almost instantaneously, so participants can interact with the most recent information in real time. Most RTDs include a relatively small number of experts, but the SE-RTD design engages a large number of stakeholders. Our design also included infographics about each treatment intervention, so participants would have the same basic knowledge available to inform their ratings. All study procedures were approved by RTI International's institutional review board.

Setting and Participants

For a previous SE-RTD design conducted as part of this study, research staff developed a database of HSOs in the United States through multiple publicly available resources. In early 2020, research staff updated this database through web searches and contacted each HSO to confirm they were still operating and to obtain current contact information. In April 2020, staff from all HSOs in this database were emailed invitations to complete an online informed consent and brief online screener form. To be eligible to participate, HSO respondents were required to be aged at least 18 years and currently employed by an HSO in the United States. Selected participants were invited to complete the SE-RTD design over a 2-week period in May 2020. HSO respondents received a $100 gift card for their participation in the SE-RTD design.

Data Sources and Variables

In April 2020, characteristics of the HSO respondent (eg, age, race, ethnicity, and sex) and characteristics of their HSO (eg, number of staff, HSO type, and services provided) were collected online through the screening process. In May 2020, the SE-RTD design was deployed. After watching a brief video clip describing each intervention, participants were asked to think about their HSO before the COVID-19 pandemic and to rate the extent to which each intervention was fundable, implementable, retainable, sustainable, scalable, and timely on a 4-point scale (0 = not at all, 1 = to a small extent, 2 = to a moderate extent, and 3 = to a great extent); these items were then summed into the SIF index score.

Statistical Analysis

First, we calculated the mean values for each SIF criterion measure and the SIF index. There were high levels of internal consistency among the SIF index criterion measures, with coefficient alpha scores that ranged from 0.87 for contingency management to 0.97 for oral buprenorphine. Next, we conducted pairwise t tests to test for significant (P < 0.05) differences in the mean values between interventions in the SIF score. Then, both bivariate and multiple regression analyses were conducted to examine the extent to which characteristics of the individual HSO respondent and characteristics of the HSO were significant predictors of each respective SIF score. SIF scores for pharmacological interventions were positively skewed with an excess of 0 values. This was dealt with by dichotomizing these scores to be 0 vs. nonzero SIF scores and modeled using logistic regression. The psychosocial interventions had normally distributed SIF scores and were modeled using linear regression. In multiple regression models, an average of 7% of individuals had missing data on 1 or more model variables, and these cases were dropped from the regression models.



As shown in Figure 2, of the 271 organizations screened, 228 HSOs were invited to participate in the SE-RTD study of whom two hundred two (89% participation rate) completed the 6 criterion measures for at least 1 intervention. Of the two hundred two HSOs included in our analyses, 26.9% were located in the Northeast, 12.9% in the Midwest, 30.4% in the South, and 29.9% in the West. HSOs were mostly nonclinical (60.4%) and served 100 or more clients (79.4%). Regarding services provided, 79.2% reported providing medical case management, 69.3% reported providing mental health counseling, 58.4% reported providing substance use services, 45.1% reported providing medications, and 37.1% reported providing ambulatory/outpatient medical care. Approximately half had 22 or more staff members (48.8%). Most of the participants (62.9%) identified as female and were younger than 45 years (54.7%). Most of the participants self-identified as White (61.4%), 32.2% were Black, and 6.4% were another race, and 20.8% of participants identified as Hispanic or Latino.

Participant flow.

Setting-Intervention Fit

Figure 3 presents the overall unadjusted average SIF score for each of the 9 evidence-based SUD treatment interventions, including the average contribution of the 6 criterion measures that make up the SIF index. The average SIF score for motivational interviewing (11.42) was the only 1 higher than the measure's midpoint of 9.5. The average SIF scores for cognitive behavioral therapy (9.50) and contingency management (8.35) were at and slightly below the midpoint, respectively. SIF scores were low for all 6 pharmacological interventions. When pairs of SIF scores were compared, the SIF scores for the 3 psychosocial interventions and oral buprenorphine were significantly different relative to each other and to all other interventions. Many of the comparisons between pairs of medications (except for oral buprenorphine) were not statistically significant.

Unadjusted setting-intervention fit index scores and dimension contributions.

Table 1 summarizes the mean and SD of the SIF scores for different subsets of the sample. For motivational interviewing, the SIF scores were above the midpoint (9.5) for both clinical (11.51) and nonclinical HSOs (11.36). For nonclinical HSOs, none of the other interventions were above the midpoint. For clinical HSOs, the average SIF scores were above the midpoint for CBT (10.97) and oral buprenorphine (9.51).

TABLE 1. - Setting-Intervention Fit Index Scores of Each Treatment Intervention, by Subgroup of the Sample
Pharmacological Treatment Interventions Psychosocial Treatment Interventions
Acamprosate (n = 188) Disulfiram (n = 185) Injectable
Buprenorphine (n = 184)
Buprenorphine (n = 185)=
Naltrexone (n = 185)
Naltrexone (n = 185)
Therapy (n = 188)
Management (n = 199)
Interviewing (n = 193)
Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD) Mean (SD)
 Northeast 5.19 (4.95) 4.75 (4.66) 5.08 (5.06) 8.35 (6.28) 5.96 (5.81) 5.81 (5.04) 9.88 (4.01) 9.07 (4.16) 11.98 (3.28)
 Midwest 3.16 (3.75) 3.64 (4.28) 3.44 (4.18) 3.84 (5.15) 3.40 (4.68) 3.52 (3.71) 8.48 (4.90) 7.73 (4.75) 9.56 (4.16)
 South 5.32 (5.27) 5.52 (5.59) 4.89 (5.50) 5.87 (6.02) 6.13 (5.88) 6.65 (5.94) 9.82 (3.74) 8.69 (3.80) 11.74 (3.93)
 West 3.17 (4.28) 3.53 (4.91) 3.15 (4.32) 5.13 (6.09) 3.23 (4.94) 4.19 (5.37) 9.09 (4.62) 7.37 (4.09) 11.28 (4.72)
Organization type
 Nonclinical 2.34 (3.38) 2.31 (3.58) 2.44 (4.25) 3.90 (5.73) 2.36 (4.24) 3.14 (4.63) 8.55 (4.42) 7.83 (4.35) 11.36 (4.17)
 Clinical 7.61 (4.98) 7.82 (5.07) 7.00 (4.57) 9.51 (5.32) 8.74 (5.18) 8.66 (4.82) 10.97 (3.61) 9.10 (3.83) 11.51 (4.05)
No. of staff (binary)
 <22 3.45 (4.50) 3.46 (4.45) 2.98 (4.30) 4.75 (5.70) 3.59 (4.90) 4.25 (4.93) 8.73 (4.30) 7.89 (4.35) 10.92 (4.20)
 22 or more 5.35 (4.98) 5.53 (5.38) 5.55 (5.16) 7.51 (6.42) 6.13 (5.93) 6.34 (5.69) 10.23 (4.17) 8.80 (4.01) 11.94 (4.01)
Offers substance use services
 No 3.06 (4.31) 3.33 (4.44) 2.49 (3.99) 3.46 (4.78) 2.99 (4.30) 3.33 (4.41) 8.41 (4.55) 7.70 (4.21) 9.95 (4.31)
 Yes 5.39 (4.95) 5.37 (5.26) 5.55 (5.13) 8.10 (6.40) 6.32 (6.00) 6.81 (5.64) 10.25 (3.92) 8.79 (4.13) 12.50 (3.62)
 Younger than 45 4.23 (4.57) 4.23 (4.78) 4.45 (5.01) 6.07 (6.14) 4.90 (5.51) 5.14 (5.27) 9.55 (4.19) 8.55 (4.24) 11.68 (3.87)
 45 or older 4.49 (4.95) 4.74 (5.22) 3.93 (4.78) 6.15 (6.29) 4.84 (5.69) 5.54 (5.63) 9.40 (4.44) 8.08 (4.16) 11.12 (4.41)
 Male 4.11 (4.51) 3.89 (4.81) 3.81 (4.64) 5.57 (5.89) 4.43 (5.29) 4.40 (5.14) 9.59 (3.90) 7.87 (4.17) 11.33 (4.38)
 Female 4.57 (4.98) 4.84 (5.12) 4.48 (5.04) 6.44 (6.36) 5.16 (5.74) 5.84 (5.52) 9.44 (4.47) 8.59 (4.20) 11.46 (3.98)
 White 3.98 (4.54) 4.15 (4.80) 3.76 (4.45) 5.73 (6.02) 4.62 (5.45) 5.12 (5.25) 9.01 (4.35) 7.80 (4.00) 11.39 (4.18)
 Black 5.84 (5.32) 5.95 (5.37) 5.84 (5.57) 7.66 (6.46) 6.21 (5.91) 6.62 (5.71) 10.44 (4.20) 9.38 (4.58) 11.65 (4.08)
 Other 1.75 (3.02) 1.42 (3.48) 1.58 (3.99) 3.17 (5.41) 1.67 (3.70) 1.58 (3.70) 9.50 (3.34) 8.17 (2.92) 10.50 (3.85)
 Not Hispanic 4.68 (5.07) 4.86 (5.27) 4.43 (5.02) 6.27 (6.17) 5.18 (5.73) 5.56 (5.50) 9.11 (4.27) 8.08 (4.22) 11.24 (3.97)
 Hispanic 3.38 (3.58) 3.21 (3.74) 3.56 (4.47) 5.69 (6.36) 3.92 (4.97) 4.54 (5.11) 10.90 (4.06) 9.32 (3.97) 12.07 (4.60)

Predictors of Setting-Intervention Fit

Results of bivariate regression analyses are shown in Supplemental Digital Content 1, There were some significant differences by region. For example, HSOs in the West reported significantly lower SIF scores than HSOs in the Northeast for 5 of the 6 pharmacological interventions and for contingency management. HSOs in the Midwest also reported significantly lower SIF scores for oral buprenorphine and motivational interviewing when compared with HSOs in the Northeast. There were no significant differences in SIF scores between HSOs in the South and HSOs in the Northeast. There were significantly greater SIF scores for HSOs in the South when compared with HSOs in the West for acamprosate and disulfiram. HSOs in the South reported greater SIF scores for injectable naltrexone than HSOs in the West.

Regarding HSO size, compared with HSOs with less than 22 staff, HSOs with 22 or more staff reported significantly higher SIF scores for all but 2 interventions: motivational interviewing and contingency management. Finally, the HSOs currently offering any substance use services reported significantly higher SIF scores (relative to HSOs not currently offering substance use services) for all but 3 interventions: acamprosate, disulfiram, and contingency management. For all SUD interventions except motivational interviewing, clinical HSOs reported significantly greater SIF scores than nonclinical HSOs.

Table 2 summarizes the results of the multivariate regression analyses. Organization type continued to be significantly associated with all 6 pharmacological interventions and cognitive behavioral therapy, with clinical HSOs reporting greater SIF scores than nonclinical HSOs. However, organization type was no longer associated with the SIF score for contingency management. Once region, organization type, and currently offering any substance use services were controlled, number of HSO staff was only a significant predictor for 2 interventions (disulfiram and injectable buprenorphine), although it was a predictor for 7 interventions in the bivariate analyses. Similarly, the current availability of substance use services was a significant predictor of SIF scores for fewer interventions in the multivariate analyses. Nonetheless, even after controlling for other variables, HSOs currently offering any substance use services reported significantly greater SIF scores than HSOs without these services for the 2 buprenorphine formulations and motivational interviewing. Across the pharmacological interventions, the overall R2 ranged from a high of 0.44 for injectable naltrexone to a low of 0.29 for acamprosate, whereas the overall R2 was considerably lower for the psychosocial interventions.

TABLE 2. - Summary of Multivariate Regression Analyses
Pharmacological Treatment Interventions Psychosocial Treatment Interventions
Acamprosate Disulfiram Injectable
Est. (SE) Est. (SE) Est. (SE) Est. (SE) Est. (SE) Est. (SE) Est. (SE) Est. (SE) Est. (SE)
 Northeast (ref.)
  Midwest −0.66 (0.57) −0.42 (0.58) −0.28 (0.60) −1.15 (0.61) −0.79 (0.62) −0.58 (0.61) −0.75 (1.01) −1.16 (1.00) −2.48* (0.98)
  South −0.43 (0.49) −0.34 (0.50) −0.92 (0.51) −1.05* (0.51) −0.35 (0.51) −0.60 (0.52) −0.38 (0.81) −0.75 (0.79) −0.56 (0.78)
  West −0.96* (0.47) −0.94 (0.48) −0.30 (0.49) −0.81 (0.49) −0.94 (0.50) −0.72 (0.50) −0.40 (0.81) −1.45 (0.80) −0.60 (0.78)
Organization type
 Nonclinical (ref.)
  Clinical 1.78* (0.40) 1.93* (0.43) 1.92* (0.41) 1.83* (0.42) 2.71* (0.46) 2.32* (0.49) 1.93* (0.64) 0.71 (0.63) −0.71 (0.62)
No. of staff (binary)
 < 22 (ref.)
  22 or more 0.76* (0.36) 0.90* (0.37) 1.18* (0.38) 0.77* (0.37) 0.63 (0.39) 0.39 (0.39) 1.12 (0.63) 0.85 (0.62) 0.78 (0.60)
Offers substance use services
 No (ref.)
  Yes −0.05 (0.36) −0.19 (0.37) 0.87* (0.38) 0.87* (0.37) 0.28 (0.39) 0.52 (0.38) 0.83 (0.64) 0.43 (0.63) 2.36* (0.61)
 Younger than 45 (ref.)
  45 or older 0.09 (0.35) 0.27 (0.37) 0.25 (0.38) 0.09 (0.37) 0.07 (0.39) 0.26 (0.38) 0.21 (0.62) −0.17 (0.61) −0.37 (0.60)
 Male (ref.)
  Female −0.02 (0.37) 0.07 (0.38) 0.34 (0.40) 0.01 (0.39) 0.32 (0.40) 0.54 (0.39) −0.16 (0.64) 0.59 (0.62) 0.30 (0.62)
 White (ref.)
  Black 0.52 (0.42) 0.33 (0.43) 0.47 (0.44) 0.00 (0.44) 0.03 (0.45) 0.20 (0.45) 1.62* (0.70) 1.57* (0.68) 0.11 (0.68)
  Other −1.11 (0.75) −2.42* (0.90) −2.26* (0.88) −1.51* (0.75) −1.78* (0.89) −2.25* (0.82) 0.46 (1.25) 0.39 (1.26) −1.02 (1.22)
 Not Hispanic (ref.)
  Hispanic 0.02 (0.45) 0.12 (0.47) 0.22 (0.47) −0.31 (0.47) −0.08 (0.49) 0.24 (0.49) 1.90* (0.78) 1.40 (0.78) 0.03 (0.75)
  Overall model R2† 0.29 0.34 0.41 0.34 0.44 0.37 0.16 0.10 0.014
Note: Logistic regression models were used for pharmacological interventions (setting-intervention fit score > 0 vs. setting-intervention fit score = 0) and linear regression was used for the psychosocial interventions. For the pharmacological interventions, we used Nagelkerke R2. For the psychosocial interventions, R2 is from linear regression.
*Indicates the estimate is significant at P < 0.05.
For the pharmacological interventions, we used Nagelkerke R2. For the psychosocial interventions, R2 is from linear regression.


Improving the integration of SUD treatment interventions within HSOs across the United States is needed to help the EHE initiative achieve its goals.37 As part of the STS4HIV Project, a large-scale SE-RTD study was used to empirically explore the fit between HSO settings across the United States and 9 evidence-based interventions for addressing SUDs and to explore the extent to which SIF index scores were predicted by characteristics of the individual HSO respondent and the inner setting characteristics of the HSO. Our findings contribute to knowledge regarding the integration of SUD interventions within HSO settings and implementation research more broadly, given the systematic review recently completed by Lewis et al (2021),38 which “confirms that intervention characteristics are understudied in behavioral health implementation research” (p. 13).

A decade after the BHIVES initiative concluded that integration of buprenorphine/naltrexone treatment into HIV care was “acceptable to providers and feasible in a variety of practice settings”8 (p. S68), we found oral buprenorphine to be perceived as a good fit for clinical HSOs, with an average SIF score of 9.51. Important to note, however, is that for each of the 4 pharmacological interventions, the indicator for timely (ie, having a qualified staff available to offer this treatment intervention was both needed and desired) was the highest rated of the 6 criterion measures. However, scores for fundable (ie, availability of funding to train or hire a staff) was almost always the lowest rated criterion measure. Thus, it seems that many of the financing, workforce, and training issues Finkelstein et al (2011)10 identified as part of the BHIVES initiative still persist. We therefore echo their recommendations regarding changing financing and reimbursement policies and recommend such changes include the flexibility needed to accelerate implementation of new evidence-based interventions that become available.

The average SIF score for each of the 3 psychosocial interventions, which may be used to address any SUD, was significantly higher than each of the 6 pharmacological interventions, which may be used only for alcohol and/or opioid use disorders. In addition, each of the 3 psychosocial interventions was significantly different from 1 other. However, motivational interviewing was the only intervention that on average was a good fit as indicated by a mean above the midpoint of the SIF index for both clinical and nonclinical HSOs. It was also encouraging that the good fit between HSOs and motivational interviewing did not significantly differ by any of the characteristics of the individual HSO respondent or the size of the HSO. As such, key stakeholders seeking to help improve the integration of motivational interviewing for substance use or SUDs within HSOs are encouraged to explore the 2 motivational interviewing–based interventions shown to be effective within HSO settings.13,39 In contrast to motivational interviewing, the average SIF score was at or slightly below the midpoint for cognitive behavioral therapy and contingency management. Nonetheless, cognitive behavioral therapy did seem to be a good fit for clinical HSOs, where the SIF index score for cognitive behavioral therapy was above the midpoint and significantly greater than SIF scores reported by nonclinical HSOs.


In addition to having important strengths (eg, a large sample, a diverse sample for both geographic region of the United States and type of HSO, and a high response rate), our study has several limitations. First, our fit-related assessment was limited to the 6 criterion measures used to create the SIF index measure that we developed for this study. Other intervention characteristics (eg, adaptability) may have been useful for assessing fit. Our hope, however, is that the SIF index will be helpful in addressing the measurement-related gaps identified by Allen et al27 as part of their systematic review. Another key study limitation was our inability to examine the predictive validity of the SIF index, which we will seek to examine in our future research.


Despite being highlighted as part of key theories,22–24 models,40 and frameworks,28,41 research assessing fit between a setting of interest and available evidence-based interventions remains limited.25,27 Addressing this research gap, this study assessed the extent to which 9 evidence-based SUD interventions were fundable, implementable, retainable, sustainable, scalable, and timely for HSOs across the United States. Based on our current findings, motivational interviewing seems to be the evidence-based SUD intervention with the greatest perceived fit for most HSOs (ie, both clinical and nonclinical). However, cognitive behavioral therapy and oral buprenorphine also seem to be a good fit for clinical HSOs. Ideally, these findings will help guide national dissemination and implementation efforts to improve the integration of SUD services into HIV care.


1. Fauci AS, Redfield RR, Sigounas G, et al. Ending the HIV epidemic: a plan for the United States. JAMA. 2019;321:844–845. doi:
2. U.S. Department of Health and Human Services. What is Ending the HIV Epidemic in the U.S.? Office of Infectious Disease and HIV/AIDS Policy. HHS. 2021. Available at: Accessed June 9, 2021.
3. Eisinger RW, Dieffenbach CW, Fauci AS. Role of implementation science: linking fundamental discovery science and innovation science to ending the HIV epidemic at the community level. J Acquir Immune Defic Syndr. 2019;82(suppl 3):S171–S172.
4. Hargreaves JR, Hassan S, Schellenberg J, et al. Five challenges in the design and conduct of IS trials for HIV prevention and treatment. J Acquir Immune Defic Syndr. 2019;82(suppl 3):S261–S270.
5. Haldane V, Cervero-Liceras F, Chuah FL, et al. Integrating HIV and substance use services: a systematic review. J Int AIDS Soc. 2017;20:21585.
6. Hitch AE, Gause NK, Brown JL. Substance use screening in HIV care settings: a review and critique of the literature. Curr HIV/AIDS Rep. 2019;16:7–16.
7. Cheever LW, Kresina TF, Cajina A, et al. A model federal collaborative to increase patient access to buprenorphine treatment in HIV primary care. J Acquir Immune Defic Syndr. 2011;56(suppl 1):S3–S6.
8. Weiss L, Netherland J, Egan JE, et al. Integration of buprenorphine/naloxone treatment into HIV clinical care: lessons from the BHIVES collaborative. J Acquir Immune Defic Syndr. 2011;56(suppl 1):S68–S75. doi:
9. Weiss L, Egan JE, Botsko M, et al. The BHIVES collaborative: organization and evaluation of a multisite demonstration of integrated buprenorphine/naloxone and HIV treatment. J Acquir Immune Defic Syndr. 2011;56(suppl 1):S7–S13.
10. Finkelstein R, Netherland J, Sylla L, et al. Policy implications of integrating buprenorphine/naloxone treatment and HIV care. J Acquir Immune Defic Syndr. 2011;56(suppl 1):S98–S104.
11. Schackman BR, Leff JA, Botsko M, et al. The cost of integrated HIV care and buprenorphine/naloxone treatment: results of a cross-site evaluation. J Acquir Immune Defic Syndr. 2011;56(suppl 1):S76–S82.
12. Lucas GM, Chaudhry A, Hsu J, et al. Clinic-based treatment of opioid-dependent HIV-infected patients versus referral to an opioid treatment program: a randomized trial. Ann Intern Med. 2010;152:704–711.
13. Garner BR, Gotham HJ, Chaple M, et al. The implementation and sustainment facilitation strategy improved implementation effectiveness and intervention effectiveness: results from a cluster-randomized, type 2 hybrid trial. Implementation Res Pract. 2020;1:1–23.
14. Landes SJ, McBain SA, Curran GM. Reprint of: an introduction to effectiveness-implementation hybrid designs. Psychiatry Res. 2020;283:112630.
15. Garner BR, Gotham HJ, Tueller SJ, et al. Testing the effectiveness of a motivational interviewing-based brief intervention for substance use as an adjunct to usual care in community-based AIDS service organizations: study protocol for a multisite randomized controlled trial. Addict Sci Clin Pract. 2017;12:31.
16. Smith JD, Li DH, Rafferty MR. The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects. Implement Sci. 2020;15:84.
17. Aarons GA, Hurlburt M, Horwitz SM. Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Pol Ment Health. 2011;38:4–23.
18. Garner BR, Gotham HJ, Knudsen HK, et al. The prevalence and negative impacts of substance use disorders among people with HIV in the United States: a real-time Delphi survey of key stakeholders. AIDS Behav. 2021;26:1183-1196
19. Volkow ND. Principles of Drug Addiction Treatment: A Research-Based Guide. Vol 12. 2nd ed. Collingdale, PA: DIANE Publishing; 2011.
20. U.S. Department. Facing addiction in America: the Surgeon General's report on alcohol, drugs, and health. In: Of health and human services (HHS), office of the surgeon general. Washington, DC: HHS; 2016.
21. Corcorran MA, Merrill JO. National HIV Curriculum: Substance Use Disorders. U.S. Department of Health and Human Services. 2021. Available at: Accessed October 7, 2021.
22. Rogers EM. Diffusion of Innovations. New York, NY: Free Press; 2003.
23. Klein KJ, Sorra JS. The challenge of innovation implementation. Acad Manage Rev. 1996;21:1055–1080.
24. Helfrich CD, Weiner BJ, McKinney MM, et al. Determinants of implementation effectiveness: adapting a framework for complex innovations. Med Care Res Rev. 2007;64:279–303.
25. Blanchard CM, Livet M. Ensuring intervention success: assessing fit as an overlooked step of the implementation process. Pharm Pract (Granada). 2020;18:2235.
26. Martinez RG, Lewis CC, Weiner BJ. Instrumentation issues in implementation science. Implement Sci. 2014;9:118.
27. Allen JD, Towne SD Jr., Maxwell AE, et al. Measures of organizational characteristics associated with adoption and/or implementation of innovations: a systematic review. BMC Health Serv Res. 2017;17:591.
28. Damschroder LJ, Aron DC, Keith RE, et al. Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci. 2009;4:50.
29. Wandersman A, Chien VH, Katz J. Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement. Am J Community Psychol. 2012;50:445–459.
30. Proctor E, Silmere H, Raghavan R, et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Pol Ment Health. 2011;38:65–76.
31. Nelson TD, Steele RG. Predictors of practitioner self-reported use of evidence-based practices: practitioner training, clinical setting, and attitudes toward research. Adm Pol Ment Health. 2007;34:319–330.
32. Seidman D, Carlson K, Weber S, et al. United States family planning providers' knowledge of and attitudes towards preexposure prophylaxis for HIV prevention: a national survey. Contraception. 2016;93:463–469.
33. Barnett M, Brookman-Frazee L, Regan J, et al. How intervention and implementation characteristics relate to community therapists' attitudes toward evidence-based practices: a mixed methods study. Adm Pol Ment Health. 2017;44:824–837.
34. Reding ME, Guan K, Regan J, et al. Implementation in a changing landscape: provider experiences during rapid scaling of use of evidence-based treatments. Cogn Behav Pract. 2018;25:185–198.
35. Gordon T, Pease A, RT. Delphi: an efficient, "round-less" almost real time Delphi method. Technol Forecast Soc Change. 2006;73:321–333.
36. Linstone HA, Turoff M. Introduction to the Delphi method: techniques and applications. In: Linstone HA, Turoff M, eds. Delphi Method: Techniques and Applications. Boston, MA: Addison Wesley Publishing Company; 1975:3–12.
37. Volkow ND. Research at the Intersection of HIV with Substance Use Disorders amidst the COVID-19 Pandemic. Bethesda, MD: HHS; 2020. Updated July 28. Availabe at: Accessed August 5, 2020.
38. Lewis CC, Mettert K, Lyon AR. Determining the influence of intervention characteristics on implementation success requires reliable and valid measures: results from a systematic review. Implementation Res Pract. 2021;2:2633489521994197.
39. Aharonovich E, Sarvet A, Stohl M, et al. Reducing non-injection drug use in HIV primary care: a randomized trial of brief motivational interviewing, with and without HealthCall, a technology-based enhancement. J Subst Abuse Treat. 2017;74:71–79.
40. Proctor EK, Landsverk J, Aarons G, et al. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Pol Ment Health. 2009;36:24–34.
41. Glasgow RE, Harden SM, Gaglio B, et al. RE-AIM planning and evaluation framework: adapting to new science and practice with a 20-year review. Front Public Health. 2019;7:64.

HIV; implementation science; innovation-values fit

Supplemental Digital Content

Copyright © 2022 Wolters Kluwer Health, Inc. All rights reserved.