Sexually transmitted diseases (STDs) remain a considerable public health concern, with approximately 20 million annual incident infections and a lifetime estimated direct medical cost of $15.6 billion.1,2 Although effective behavioral interventions exist to reduce the risk of acquiring STDs, many are complex (e.g., require multiple sessions and are too time-consuming) or expensive in high-volume clinical settings. Brief interventions that can be integrated easily into routine practice are desirable. Patients visiting STD clinics often spend more than 20 minutes in the waiting room,3 thereby providing a unique opportunity to provide health messages via interventions, such as video, at a low cost.
Safe in the City (SITC) was a 23-minute theory-based video intervention that displayed young couples negotiating proper condom use. Posters in the waiting room directed attention to the video and reinforced key messages. The effectiveness of the video was assessed among patients visiting a participating STD clinic in Denver, Long Beach, or San Francisco, from December 2003 to August 2005. A controlled trial design was used where clinics alternated the intervention and control condition in 4-week intervals (i.e., one 4-week period of the intervention, followed by one 4-week period of the control) for a period of 20 months. Participants in the control group received the standard waiting room experience without videos or posters, but condoms and educational pamphlets on STD prevention were made available to patients in both the treatment and control groups. The trial demonstrated that a short video with targeted prevention messages in a waiting room setting effectively reduced new infections after the initial clinic visit.4 The intervention was later disseminated by the Centers for Disease Control and Prevention's Diffusion of Effectiveness Behavioral Interventions project and was made available for adoption by publicly funded STD clinics across the United States.5 Our previous findings estimated the program costs of the SITC intervention from the original trial.6 Because most costs of the video production were incurred upfront,6 expanding this type of intervention to more sites nationwide could prove to be cost-effective. In this analysis, we use evidence from SITC to estimate the program costs and benefits of developing a video-based intervention and implementing it in more clinics with a larger patient population.
MATERIALS AND METHODS
Cost-effectiveness was examined from a health sector perspective. We calculated direct program costs and averted lifetime direct medical expenditures; however, we omitted indirect costs and benefits of the program. Calculated intervention costs and effects were in addition to the costs/effects of usual STD clinic care with no video intervention. Data on intervention effectiveness were drawn from the original trial of SITC, details of which are described elsewhere.4 Incident STDs in patients were measured using clinic diagnoses or reported cases for chlamydia, gonorrhea, syphilis, trichomoniasis, and HIV. During a mean of 14.8 months of observation, 2418 new laboratory-confirmed STDs were diagnosed for 2042 patients. Across all infection types, the intervention was found to reduce the number of new infections in patients attending any of the 3 STD clinics by 9% overall among those receiving versus not receiving the intervention.4
We then applied the intervention effect observed in the trial to a larger patient population represented in 40 STD clinics across 12 geographic areas in the United States from 2010 to 2011. These clinics, which make up the Sexually Transmitted Disease Surveillance Network (SSuN), receive support from the Centers for Disease Control and Prevention to collect enhanced surveillance data. Sexually Transmitted Disease Surveillance Network sites recorded 608,536 clinic visits from 363,607 unique patients and diagnosed 212,765 infections from 2010 to 2011.7 In our baseline scenario, we assumed that each patient received the video intervention during his/her index visit and that infections decreased by 9% during subsequent visits. The previous study found that 40% of the clinic visits in the SSuN data were repeat visits. To estimate the number of cases diagnosed during these visits, we assumed the same decrease in positivity from index to repeat visit as was observed in Warner et al.4: 16% of patients tested positive for at least one infection during the index visit and 6% tested positive during repeat visits (in the control group). Thus, we assumed that positivity from repeat visits in the SSuN data was 38% of positivity from index visits. Given the proportion of visits in the SSuN data that were repeat visits, we applied the intervention effect to 20% of the 80,261 cases of chlamydia, gonorrhea, syphilis, and trichomoniasis. Cases classified as late syphilis were omitted, as they were most likely acquired before the index visit. Although they were included in the original trial, HIV cases were excluded from most scenarios because of the small number of diagnoses during the intervention period and complexities in assigning benefit values to averted cases. However, as a robustness check, we calculated cost-effectiveness with HIV also included in the analysis.
Next, we used lifetime medical cost data,2 adjusted to 2017 dollars using the medical care portion of the consumer price index, to associate averted cases with avoided direct medical expenditures. Costs were aggregated over infection type, by sex, to calculate the overall costs averted from the intervention.
To derive program costs, we first adjusted for inflation the program cost estimates from previous findings into 2017 dollars.6 We assumed web-based distribution of video content for all clinics, as opposed to DVDs, which were used in the initial dissemination. Video production costs were fixed, but overall program costs varied by the number of clinics and the length (in years) of the intervention. The costs of televisions, their installation, and video operation labor varied by the number of clinics running the intervention. The cost of video web-hosting and video operation labor varied by the length of the intervention.
Total program costs were calculated by adding fixed program costs to variable costs. For example, to estimate the total cost when operating the intervention in 40 clinics over the span of 1 year, we carried out the following calculation:
We estimated costs and benefits of implementing the video intervention in the 40 SSuN clinics for 1 year. Pathela et al.7 did not report on the timing of index and repeat visits, so we assumed that all index visits occur during the first year, and we therefore calculate program costs as if it operated continuously for 1 year. We calculated 2 measures of cost-effectiveness: the net cost of the intervention and the net cost per case averted. Cases averted were estimated by multiplying the estimated number of repeat visit cases of chlamydia, gonorrhea, syphilis, and trichomoniasis in the SSuN data by the same 9% intervention effect size. Averted costs were estimated using data on lifetime health costs per case and the estimated number of cases averted. The net cost of the program was calculated as the difference between the program cost and averted costs. Negative net costs indicate that the intervention was cost saving. Dividing net costs by estimated cases averted yielded the cost per case averted.
Probabilistic Sensitivity Analysis
We performed a Monte Carlo simulation analysis to account for uncertainty in the model parameters and to derive 95% confidence intervals surrounding our cost-effectiveness estimates. For most model inputs, we took 10,000 random draws from Beta-PERT distributions defined by the parameter means and ranges in Table 1.9 For the intervention effect parameters, draws were taken from a normal distribution fit using the mean and 95% confidence interval.4 For example, the normal distribution for the overall effect size used a mean of 0.09 and an SD of 0.04 [(mean − lower bound of range)/1.96], so that 95% of the draws from the distribution lay within the 95% confidence interval. The values from each draw were then used to calculate the cost-effectiveness measures. The lowest and highest 2.5% of these calculated measures form the lower and upper confidence interval bounds, respectively.
The assumptions used for our baseline estimates may have significant impacts on the cost-effectiveness results. To better understand the potential impacts of the video intervention, we assessed cost-effectiveness under 5 alternative scenarios. Two of these scenarios expanded the reach of the intervention. The first assumed the intervention was implemented in twice as many (80) STD clinics for the same length of time. This scenario implicitly assumed that the patient population in the 40 additional clinics was identical to those in the baseline intervention, so expanding the intervention led to a linear increase in cases and costs averted. The second scenario doubled the length of time the intervention operated, from 1 to 2 years. Again, we assumed that cases, and therefore costs, averted increased linearly with the length of the intervention. Although these assumptions may be violated for reasons discussed in the limitations section, this exercise allowed us to estimate the relative returns to scale of expanding the intervention across clinics and time.
One potentially large benefit of the intervention omitted from the baseline analysis was the prevention of cases of HIV. The original SITC study found differences in diagnoses of HIV in the treatment (4 cases) and control groups (10 cases), but the overall number of cases was small, and the difference was not statistically significant (P = 0.12). Recent efforts to increase viral suppression among those living with HIV and use pre-exposure prophylaxis to prevent HIV acquisition.10 Wide adoption of these medications might decrease the effectiveness of video interventions for preventing HIV because they can prevent transmission in the absence of condom use. Furthermore, because HIV is not curable like the other 4 STDs included in our analysis, the effect of the intervention might be more appropriately measured in terms of delayed HIV acquisition. As a robustness check, we estimated the cost-effectiveness of the 9% intervention effect with an assumed 5-year delay of HIV for 9% of repeat cases, based on an estimated benefit of delayed acquisition.8 The benefits were calculated as the difference in discounted lifetime medical spending between HIV-infected and HIV-uninfected individuals who are at high risk of infection. The estimated costs averted from delaying acquisition of HIV by 5 years was $56,727 in 2017 dollars. Lower and upper bounds of this cost were derived assuming a 1-year delay and lifelong prevention, respectively (Table 1). Increased use of costly pre-exposure prophylaxis by high-risk uninfected individuals has decreased the difference in medical expenditures between these 2 groups, which therefore lowers the estimated benefit of HIV prevention or delayed acquisition.
Our next alternative scenario estimated the cost-effectiveness of the video intervention, assuming that it only had an effect on male cases. HIV was again omitted from this scenario. Warner et al.4 found a larger effect in male (13%) than in female patients (not significantly different from zero). The patient volume data from the SSuN study do not provide the number of cases by sex for syphilis and trichomoniasis, so overall cases were allocated using the proportion of male and female patients in the SSuN data and male-to-female ratios of rates (syphilis) or prevalence (trichomoniasis).11,12 Playing the video continuously for long periods may be distracting for clinic workers.5 As a final robustness check, we estimated the cost-effectiveness of alternative scenarios in which the intervention runs only 50% of the time for 2 years. In this scenario, video operation labor costs and estimated cases averted per year were halved.
A cumulative 80,261 chlamydia, gonorrhea, syphilis, and trichomoniasis infections were diagnosed in SSuN clinics from 2010 to 2011. Table 1 presents the estimated number of cases diagnosed during repeat visits by disease type, averted health cost inputs, and components of the program cost.
Table 2 presents results from the baseline and alternative scenarios. In the baseline scenario, the video intervention operated continuously in 40 clinics for 1 year. Total program costs were $432,742, and an estimated 1445 cases were averted. Costs saved from the averted cases amounted to $324,728, generating a net cost of $108,015 for the baseline scenario. However, because of uncertainty surrounding the intervention effect size and the lifetime medical costs associated with each infection, the 95% confidence intervals on averted costs, and therefore net cost, are wide. Specifically, we found that the 95% confidence interval for net cost ranges from −$194,894 to $403,751. The negative lower bound here indicates that the program is potentially cost saving. The net cost per case averted in the baseline scenario is $75 (95% confidence interval, −$91 to $1328).
The uncertainty in the baseline analysis is further illustrated in Figure 1. The figure shows combinations of costs averted and program costs from each of 1000 random draws from the Monte Carlo simulation. The simulated range in costs averted is much wider than the range in the program costs, largely due to relatively high uncertainty in the effect size estimate. The dotted line shows where costs averted were equal to program costs. Above this line, costs averted are higher than program costs, and the net cost is therefore negative (cost saving). Most points lie below this region, but the few points above the dotted line illustrate why the 95% confidence intervals have negative lower bounds.
The first 2 alternative scenarios expanded the reach of the intervention by doubling the number of clinics (80 clinics for 1 year) or doubling the length of the intervention (40 clinics for 2 years). Incremental program costs (costs in addition to the baseline scenario) were very similar between these scenarios at approximately $54,000 and $47,000, respectively. The incremental costs of expanding the program, either in length or in the number of clinics, are relatively small compared with the upfront program costs. Because we assumed that costs averted grew linearly in both of these scenarios, the incremental cases and costs averted were equivalent, so the program costs drove the small differences in incremental net cost for these 2 scenarios. The negative net costs for these 2 scenarios demonstrate that the video intervention is likely to be cost saving as it is expanded.
Table 2 presents results from 3 additional scenarios using some alternative assumptions. First, benefits from averted HIV costs were considered. Because the costs of HIV are high, its inclusion dramatically increased the estimated costs averted, even when assuming delayed, instead of permanently prevented, HIV acquisition. The costs averted from running the intervention in 40 clinics for 1 year are over $2.3 million greater than in the baseline scenario, and the net costs are large and negative because program costs are unchanged. The next scenario assumes that the intervention only runs 50% of the time over the 2-year period. Program and net costs do not change substantially when using this assumption. The final scenario uses the 13% effect size found for male patients in the original study and assumes no effect for female patients. In this scenario, program costs are unchanged, but we no longer calculate costs averted from female cases. The net cost in this scenario, assuming a 40 clinic run continuously for 1 year, increases by approximately $134,000.
This analysis demonstrates that brief video-based interventions can be cost-effective tools to prevent new infections in STD clinic settings. Our findings provide further support for the development of such brief interventions, especially in settings where provider time is limited or patients are not motivated to attend longer, more comprehensive STD prevention programs. The estimated net cost is positive in the baseline scenario; however, net cost was negative (cost saving) in 3 of the alternative scenarios, and the 95% confidence intervals contained zero for all net cost estimates. The wide confidence intervals are primarily due to the large range of the estimated effect sizes. For example, the lower bound on the 9% effect was only 1%. With a small intervention effect, even extending the length of the program or the number of clinics treated does not lead to positive net benefits because the variable program costs outweigh the small additional benefits. For example, in the 40 clinic, 2-year scenario, an effect size of approximately 7% would balance program costs and benefits, leading to zero net costs. In the HIV scenario, however, the intervention would be cost saving even at 1.5% effectiveness.
Estimated program costs did not vary substantially between the baseline and alternative scenarios. This reflects the fact that the added cost of expanding the program to additional clinics or extending the length of the intervention is negligible relative to the fixed costs of the program, that is, the video production costs. Therefore, video interventions are more cost-effective if they are broadly adopted and implemented. Interestingly, the variable costs of expanding the number of clinics and extending the length of the intervention are very similar.
The SSuN data included 363,607 index clinic visits, so the baseline program cost per visit was just $1.19. For comparison, this cost was slightly higher than previous condom distribution interventions, which have been estimated to cost $0.17 to $0.26 per condom distributed, in 2017 dollars.13,14 However the costs and benefits of these intervention are not directly comparable because condoms may still have been used in the absence of the distribution programs, multiple condoms may be distributed to a single person, and the SITC intervention had the added benefit of teaching and promoting proper condom usage. The low cost per patient visit demonstrates that video interventions can inexpensively reach patients and quickly be scaled up. Similar evidence on behavioral interventions broadly focused on STD and HIV prevention is scarce, but Project RESPECT, an evaluation of individually focused counseling interventions, found that behavioral counseling was effective at increasing condom use and reducing STDs.15 The incremental societal cost per STD case prevented of 2 counseling sessions, compared with usual care, was found to be $132 (adjusted to 2017 dollars).16,17 This is considerably higher than the $75 net cost we find for the baseline intervention, although the 2-session intervention was still considered to be cost-effective compared with usual care. Interventions such as RESPECT also require substantially more clinic staff effort than an intervention such as SITC.
Our analysis attempted not to overestimate the benefits of the intervention. The baseline results in Table 2 excluded benefits from HIV reductions, but the estimated net benefits of the intervention increase dramatically when HIV costs are considered. Lifetime medical costs of gonorrhea and chlamydia are higher in female than in male patients because of their associated sequelae.2 As a sensitivity analysis, we estimate the cost-effectiveness of the video intervention when assuming an effect on male patients only.
Our analysis is subject to several limitations. First, the analysis was conducted from a health sector perspective instead of a societal perspective. This omits potential nonmonetary benefits of the intervention, such as quality of life improvements from averted infections. Omitted intervention costs, such as patient transportation and time spent in the waiting rooms, were minimal because patients would already spend time in the clinics in the absence of the intervention. Late syphilis cases were omitted from the analysis, but cases classified as “unknown latent” and “unspecified/other” were included. Some of these cases may not have been acquired between the index and follow-up visits. Furthermore, the potential benefits of prevented HIV cases are not included in the baseline estimates. When HIV is included in the model, averted direct medical costs of delayed acquisition are calculated, but other indirect benefits may also be important. For example, delaying infection by a year also prevents secondary transmission of HIV during that period.18 Similarly, bacterial STDs may be transmitted to partners and also may increase risk of HIV transmission,19 but these downstream benefits of STD prevention are not considered in our analysis.
Next, the costs of producing a video have likely changed since the original intervention produced nearly 2 decades ago, in ways not captured by the inflation adjustment. Furthermore, the effectiveness of a video intervention may also be different today, as patients are increasingly focused on their smartphone devices, a modality that also lends itself to video interventions. However, a recent study in HIV clinics suggests that waiting room video interventions may be effective at improving treatment initiation and achieving viral suppression in patients.20 Clinics that adopt the SITC intervention may also adapt its implementation,5 potentially altering its effectiveness.
The effectiveness of the program is limited to the original study population and not necessarily generalizable to other samples. For example, fewer cases will be averted among a population with lower positivity. In addition, the number of cases averted may not increase linearly as the intervention is expanded. For example, increasing the number of clinics may include clinics with lower patient positivity and fewer cases. Increasing the length of the intervention may have diminishing effectiveness as the same population is repeatedly exposed to the intervention, or it may lead to finding more eligible cases as the window for observing repeat visits is extended. Finally, alternative assumptions could have been made regarding the uncertainty surrounding the input parameters. The sensitivity analysis further assumed the covariance between parameters to be zero, which affects the estimated 95% confidence intervals.
Under a variety of scenarios, developing and expanding video interventions with effectiveness similar to that of the SITC intervention to the SSuN patient population proved to be cost-effective. Updated video interventions that account for the changing STD landscape in the United States could play an important role in addressing the recent increases in STDs.
1. Satterwhite CL, Torrone E, Meites E, et al. Sexually transmitted infections among US women and men: Prevalence and incidence estimates, 2008. Sex Transm Dis 2013; 40:187–193.
2. Owusu-Edusei K Jr., Chesson H, Gift TL, et al. The estimated direct medical cost of selected sexually transmitted infections in the United States, 2008. Sex Transm Dis 2013; 40:197–201.
3. Myint-U A, Bull S, Greenwood GL, et al. Safe in the City: Developing an effective video-based intervention for STD clinic waiting rooms. Health Promot Pract 2010; 11:408–417.
4. Warner L, Klausner JD, Rietmeijer CA, et al. Effect of a brief video intervention on incident infection among patients attending sexually transmitted disease clinics. PLoS Med 2008; 5:e135.
5. Harshbarger CL, O'Donnell LN, Warner L, et al. Safe in the City: Effective prevention interventions for human immunodeficiency virus and sexually transmitted infections. Am J Prev Med 2012; 42:468–472.
6. Gift TL, O'Donnell LN, Rietmeijer CA, et al. The program cost of a brief video intervention shown in sexually transmitted disease clinic waiting rooms. Sex Transm Dis 2016; 43:61–64.
7. Pathela P, Klingler EJ, Guerry SL, et al. Sexually transmitted infection clinics as safety net providers: Exploring the role of categorical sexually transmitted infection clinics in an era of health care reform. Sex Transm Dis 2015; 42:286–293.
8. Schackman BR, Fleishman JA, Su AE, et al. The lifetime medical cost savings from preventing HIV in the United States. Med Care 2015; 53:293–301.
9. Vose D. Risk Analysis: A Quantitative Guide. New York, NY: John Wiley & Sons, 2008.
10. Mermin J, Fenton KA. The future of HIV prevention in the United States. JAMA 2012; 308:347–348.
11. Centers for Disease Control and Prevention. Sexually Transmitted Disease Surveillance 2017. Atlanta: U.S. Department of Health and Human Services, 2018.
12. Patel EU, Gaydos CA, Packman ZR, et al. Prevalence and correlates of Trichomonas vaginalis
infection among men and women in the United States. Clin Infect Dis 2018; 67:211–217.
13. Bedimo AL, Pinkerton SD, Cohen DA, et al. Condom distribution: A cost-utility analysis. Int J STD AIDS 2002; 13:384–392.
14. Renaud TC, Bocour A, Irvine MK, et al. The free condom initiative: Promoting condom availability and use in New York City. Public Health Rep 2009; 124:481–489.
15. Kamb ML, Fishbein M, Douglas JM, et al. Efficacy of risk-reduction counseling to prevent human immunodeficiency virus and sexually transmitted diseases: A randomized controlled trial. Project RESPECT Study Group. JAMA 1998; 280:1161–1167.
16. Varghese B, Kassler WJ, Kamb ML, et al. HIV prevention interventions—Which gives the most bang for the buck? Cost-effectiveness analysis—Project RESPECT [abstract]. Int J STD AIDS 2001; 12(Suppl 2):79.
17. Gift TL, Marrazzo J. Cost-Effectiveness Analysis. Behavioral Interventions for Prevention and Control of Sexually Transmitted Diseases. Boston, MA: Springer, 2007:482–499.
18. Pinkerton SD. HIV transmission rate modeling: A primer, review, and extension. AIDS Behav 2012; 16:791–796.
19. Jones J, Weiss K, Mermin J, et al. Proportion of incident human immunodeficiency virus cases among men who have sex with men attributable to gonorrhea and chlamydia: A modeling analysis. Sex Transm Dis 2019; 46:357–363.
20. Neumann MS, Plant A, Margolis AD, et al. Effects of a brief video intervention on treatment initiation and adherence among patients attending human immunodeficiency virus treatment clinics. PLoS One 2018; 13:e0204599.