Evidence suggests that individual stand-alone behavioural interventions targeted to men who have sex with men (MSM) are insufficient to reduce HIV acquisition and transmission [1,2]. Effective means to reduce transmission among this population require strategies that combine behavioural and biomedical interventions with structural changes to reduce barriers to access services, such as stigma and discrimination [1,3]. A combination prevention programme (CPP) is defined as a series of evidence-based strategies from all three of these spheres – biomedical, behavioural, and structural – that target the most affected populations and rely on community-based programmes [4,5].
Despite wide support for the implementation of CPP since 2008, little rigorous evidence exists as to CPPs’ impact and cost-effectiveness . Nevertheless, the need to implement a rapid response to the HIV epidemic in some countries has led to the investment of large amounts of resources and scaling up of CPP with no evidence or lessons learned documented [7,8]. The reasons behind this evidence gap include challenges undertaking experimental or quasi-experimental designs to measure impact, adapting strategies to the epidemiological and cultural context, efficiently providing services in coordination with communities to reduce costs, and accessing appropriate sample size [2,6,9].
Although some prevention packages for MSM have recently been implemented in South Africa, in the Americas (Brazil, Peru, and some cities in the United States), and in China , evidence on their effectiveness is not currently available. Results from an agent-based model suggest that a combination prevention package in South Africa could lower HIV incidence by 34% over 5 years . A recent CCP programme in Central America shows positive results in condom use and HIV testing .
In Mexico, the HIV epidemic is concentrated among MSM, transwomen, and injecting drug users . A recent nationally representative survey among MSM revealed 17% HIV prevalence . Until 2010, the national response to HIV was mostly focused on providing free access to antiretroviral therapy (ART) to eligible HIV-positive individuals [14,15]. In contrast, prevention efforts lagged behind in terms of resources and attention , and were primarily information campaigns for key population groups and condom distribution for the general population . Furthermore, despite the increased access to and uptake of ART, the national treatment effort had challenges in terms of reaching all in need. For example, results from one clinic in Mexico City showed that between 2000 and 2010, 61% of patients started treatment late because of late testing or late presentation to health services after diagnosis [18,19], and between 2008 and 2012, around 60% of all documented deaths among HIV patients in the public system occurred within the first 6 months of starting treatment .
In this context, in 2010, Mexico successfully applied to the Global Fund to Fight Aids Tuberculosis and Malaria (Global Fund) for a grant programme which ran for 3 years to implement a package of behavioural, biomedical, and structural interventions targeted to MSM in 44 cities that concentrate 72% of the cumulative AIDS cases in the country .
The aims of this study were: to document coverage and quality process indicators of that MSM-targeted CPP initiative (hereafter referred to as ‘the programme’); to estimate the impact of the programme on access to HIV testing, awareness of HIV status, sexual behaviour, and perception of stigma and discrimination; and to assess the programme's cost-effectiveness.
Original plan: quasi-experimental design
In light of the programmatic needs and constraints, a quasi-experimental design was planned before the CPP started. A subset of 15 cities was purposely selected into an early treatment group (starting in September 2011) and the rest of the cities were included in a later treatment group, starting at least 6 months later. For impact evaluation, 12 cities were randomly selected from the early-treatment group, and each one was pair-matched to a city in the late-treatment group. Given the lack of information on sexual practices among MSM at the time of the evaluation design, we used available information for male youth to match cities that were similar on knowledge of prevention behaviours. The following variables were used to match cities: estimated number of MSM by city , percentage of youth 15–19 years old who had not initiated sex, and percentage of 15–19 year olds who are aware that condom use prevents sexually transmitted diseases from the National Nutrition and Health Surveys.
Actual implementation of the programme
In the evaluation design, the late-treatment cities were scheduled to start at least 6 months later. However in practice, the roll-out of the programme was staggered from September 2011 through July 2012, so there was not a 6-month gap between early and late-treatment cities. Fig. 1 describes the actual, staggered implementation process of the CPP between September 2011 and July 2013 in both groups of cities.
The combination prevention programme
The CCP was a 3-year initiative (2011–2014) targeted to MSM. The programme included the following interventions in all cities: a massive and intense delivery of prevention kits (condoms, lubricants, and prevention information) in meeting places along with brief educational messages; HIV testing and counselling (HTC) offered in meeting places; four behavioural interventions in total, including a peer individual outreach programme, an individual brief motivational interview, and two group interventions (one for MSM and one for male sex workers and transgender women); and stigma and discrimination workshops for public servants (healthcare providers and police officers). Three large nongovernmental organizations (NGOs) were responsible for delivering the programme, which reached 1.3 million MSM over 3 years.
The CCP was targeted to MSM and most of the interventions were implemented in venues where they meet. We collected information on male individuals aged 18 or older, gathering in venues, who had been living in the country for at least 12 months and who had not participated in the same survey in another venue. We excluded individuals who were not MSM (10% of the sample), defining MSM as a man who had sex with another man in the past 12 months.
Between September and November 2011, we conducted a baseline survey in the 24 cities selected for the impact evaluation. The design of this survey has been described in detail elsewhere . The follow-up survey was conducted between September and November 2013 in the same 24 cities. We first updated the meeting places by including new venues opened after 2011 and by dropping places closed in the same period. The survey was applied to a random sample of the updated list of meeting places (including 60% of the original places in 2011).
The same questionnaire and survey procedures were applied in both surveys . All participants signed informed consent forms. The Ethics Committee at the National Institute of Public Health in Mexico (IPF Code 3627801) approved the full protocol to collect and analyse the baseline and follow-up surveys.
A sample size of 6000 MSM was estimated for the follow-up survey to identify (with 90% certainty) changes in HIV prevalence greater than 3% and changes in condom use larger than 5%. A total of 8503 MSM completed the baseline survey  and 6938 completed the follow-up survey.
The administrative data come from the monitoring and evaluation information system designed by the principal recipient of the Global Fund grant. The principal and the subrecipients were responsible for collecting and verifying the information provided by the NGOs that implemented the programme in each city. Subrecipients directly supervised in the field all the information collected by the NGOs and were then responsible for entering all data in the system by qualified personnel. The principal recipient was responsible for verifying the information sent by the subrecipients with random visits to verify the quality of the data. Data included: number of persons receiving HIV prevention kits, number of persons who received testing and counselling, size of the population covered by the four behavioural interventions, and cost data for each intervention.
Impact evaluation analysis
We estimate the effect of the programme on the following outcomes: access to HIV testing in the past 12 months, awareness of HIV status among those who were HIV-positive, percentage of HIV-positive individuals on ART, having more than one partner in the last month, condom use in the last sexual intercourse, and perception of stigma and discrimination from health personnel.
Given the differences between the original design and the actual implementation, for impact evaluation we followed two different econometric approaches. We compared outcomes between individuals from early versus late-treatment cities, using a difference-in-difference estimation for a between-group comparison. This approach takes advantage of the matching process designed to find the most comparable pairs of cities from each treatment group. Using the step-wise implementation of the programme, the second strategy took a dose–response approach taking advantage of the phased implementation of the programme across cities (Fig. 1), which provided variation in exposure to the CPP. We defined ‘dose’ at the city level as the time of exposure from the date the programme started to the date of the survey. This approach tested whether changes in outcomes between baseline and follow-up that were higher in cities in which individuals had more time of exposure to the programme. The dose–response model takes advantage of the variability in time of exposure because of the current roll-out of the programme. The variable time of exposure, expressed in years, is zero at baseline and equals the time that elapsed since the programme started in each city to the date the follow-up information was collected.
For both empirical strategies, we use fixed effects models for longitudinal data clustered at the city level to adjust for time invariant observable and nonobservable factors, particularly those associated with characteristics of the cities or the NGO that implemented the programme.
For each outcome, we derived the percentage increase with respect to baseline per additional year of exposure to the programme using the marginal effects from the model as follows: (marginal effect/baseline proportion) × 100.
Awareness of HIV status measures whether participants with HIV-positive results from rapid tests were aware of their status (i.e. self-reported being HIV positive in the questionnaire). HIV testing measures whether the respondent was tested in the past 12 months. HIV positive on treatment measures whether HIV-positive individuals are on ART. Behavioural indicators include having had more than one sexual partner in the last month and having used a condom in the last sexual intercourse. Finally, we analyzed the impact on self-reported perception of stigma and discrimination from healthcare personnel.
The two statistical model specifications are adjusted by the following covariates at the individual level: age; speaking an indigenous language; last school grade completed; whether the participant reported exchanging money for sex as a stable source of income; and number of sexual partners except when the outcome is sexual partners.
The total costs of staff, recurrent inputs (HIV tests, condoms, pamphlets), and recurrent services (utilities) were estimated for each intervention for the entire period of implementation. Costs reported in the study are ‘financial costs’, that is, as recorded by the implementing organizations and excluding donations and volunteer work. The perspective adopted is that of the funding agency.
We estimated the costs per person reached by the programme as the ratio between the total costs of the programme over 3 years and the total number of MSM reached by all the interventions during that period. The cost per individual tested was estimated as the ratio between the total cost of the programme and the total number of people tested, and the cost per HIV-positive individual detected was estimated by multiplying the number of people tested by the prevalence of HIV found in the baseline survey: 17% . Finally, as a rough proxy of cost per infection averted, we estimated the cost per person not exposed to an HIV-positive individual who is not on treatment, by dividing the total costs of the programme by the estimated number of partners of HIV-positive individuals who started ART because of the programme (4.5% increase). The number of partners over a year was estimated by multiplying by 12 the mean number of sexual partners in the past month reported in the baseline survey by HIV-positive individuals on ART.
MSM in the early and late-treatment groups were statistically similar in all outcomes except for number of sexual partners, percentage of HIV-positive individuals who reported being on ART, and education (Table 1).
A total of 40% of the participants reported having been offered an HIV test at meeting venues in the past year, on an average three times (Table 2). A total of 48% were tested during the past year, an increase of about 3% compared with the 2011 baseline survey. A similar increase was observed in the proportion of individuals reporting having received posttest counselling: from 77.5 to 80.5%. About 73% of participants had received prevention kits to prevent sexually transmitted diseases, on an average four packages, and among those who received them, 69% also received a brief educational message and 51% reported the package had all expected materials (condoms, lubricants, and educational tools).
Table 3 shows the impact evaluation of the CPP by identification strategy. For the comparison between the two groups of cities (column 2), we found only a slight positive change in self-reported condom use with this strategy: a 5.4 higher probability of using a condom. When we modelled impact using the dose–response approach (column 3), we found that an additional year of exposure to the programme was associated with a 3.4% higher probability of being tested for HIV (45% had tested for HIV at baseline), 2.0% increased probability of knowing their HIV status (31.7% knew their status at baseline), 4.7% increase of being on ART among HIV-positive individuals (69.9% at baseline), and 0.7% decrease in the probability of perceiving stigma or discrimination from healthcare personnel (10% at baseline).
The cost per MSM reached by the programme was US$14, the cost per MSM tested was US$98, and the cost per HIV-positive person detected was US$577 (Table 4). Finally, the cost per person not exposed to an untreated HIV-positive individual was US$400.90.
The study is one of the few documenting the impact of a CPP. Despite the advantage of having originally a quasi-experimental design that matched early and late-treatment cities to make them comparable and to allow for between-group comparison, in practice the programme was implemented gradually over time so that early treatment cities did not start at the same time and the original plan to start in the late-treatment cities 6 months later was not accomplished. Therefore, the comparison between early versus late-treatment cities is less appropriate than the dose–response model. In the absence of a control group, the dose–response tests if a higher exposure to the programme based on the current roll-out of the programme had an effect. With the difference-in-difference approach, we only found a significant increase in condom use. We next summarize the results of the evaluation based on the dose–response model.
We found more than 70% coverage of prevention kits delivered and 40% of HTC offered in meeting venues. Because of these levels of coverage, we found a 3.4% increase in HIV testing per additional year exposed to the programme, which implies a 7.5% increase in testing coverage with respect to baseline. Similarly, the marginal effect of 2.0% in awareness of status among HIV-positive individuals is equivalent to a 6.3% increase with respect to the baseline level. The increase of 4.7% in the proportion of HIV-positive individuals on ART translates into a 6.7% increase relative to baseline. Finally, the 0.7% reduction in the perception of stigma and discrimination from healthcare personnel per additional year exposed is equivalent to 7% reduction with respect to baseline levels.
Our results are consistent with recent findings from a CPP in Central America showing that those exposed to behavioural interventions were more likely to have been tested for HIV in the past year . The study also found impacts on condom and lubricant use, which differs from our findings in behaviour indicators: we found no impact on condom use or in number of sexual partners. That was expected for several reasons. First, the activities of the CPP in Mexico were heavily focused on HTC, which was mostly offered at venues with only a relatively small component of behavioural change activities. Although the programme included a component of condom distribution, behavioural activities included only educational materials and brief talks. Given existing evidence on the very limited effectiveness of interventions based on information, education, and communication to motivate changes in sexual behaviour, it would have been surprising to find something different . Second, the period of exposure evaluated was relatively short – just 2 years, which even in modelling exercises is too short to observe a change, even under the most optimistic assumptions of effectiveness [22,23]. Third, the time difference between early and late treatment was very small (3 months on an average), which drastically limited the exogenous variation in treatment between the two groups.
The impact estimates we found on HIV testing, HIV status awareness among HIV-positive individuals, and perception of stigma and discrimination are reassuring in two senses. First, given the content of the CPP, those are the most likely outcomes in which one would have had expected to observe an effect. Second, these outcomes are important in terms of their potential effect on preventing HIV infections.
It is now commonly acknowledged that HIV testing is a key component of any HIV treatment and prevention strategy . On the treatment side, it represents an opportunity for HIV-positive individuals to start ART and thus improve radically their chances of a longer and better life . On the prevention side, starting HIV-positive people on treatment as early as possible increases the potential of ‘treatment-as-prevention’ . Furthermore, even in the absence of treatment, self-awareness of HIV status, being positive or negative, has the potential impact of reducing sexual risk behaviour .
We found an overall 40% coverage of HTC services in meeting places. Although we lack baseline information on coverage by previous HTC campaigns for this population, anecdotal evidence on programme coverage suggests that prior to 2011, there were no articulated efforts to reach this population at this scale.
On the cost side, we found that the average cost per MSM reached by the CPP was US$14. The average cost per person tested of US$98 is higher than estimates from other countries because we included the total costs of the CPP instead of only the costs of the HTC component, which represents just 9% of total costs. The cost per HIV-positive MSM individual detected of US$577 is in the range of costs estimated in a recent study of efficiency in four countries in Africa (US$142–1367) . Our estimations also allowed us to estimate the cost per person not exposed to an untreated HIV-positive individual (US$400) based on the impact of the programme of increasing the proportion of individuals who self-reported being HIV positive and on ART. Given that this final metric is the closest we can be of infections averted, this result suggests that the approach of the CPP used in Mexico is very likely a cost-effective one.
Stratifications by region showed that the programme had larger effects on HIV testing in the Central West and Southern regions compared with the Central East and Northwest. For stigma and discrimination, we found significant effects in the Central West and Southern regions. For other outcomes, the sample size was too low to capture differences by region. Future studies should be conducted to understand the differential effects by region and to explore other sources of heterogeneity.
The study has some limitations. The models do not adjust for any other programme implemented simultaneously, although to our knowledge, there were not.
Although it was encouraging to find an increase in the number of HIV-positive individuals on ART, we acknowledge that this was a self-reported variable. It is important to note that the three NGO supplementing the programme reported significant efforts to link HIV-positive individuals to treatment services. However, we do not have any supporting evidence of the specific strategies followed.
Findings on perception of stigma and discrimination from healthcare personnel should be interpreted with caution as the survey asked whether the participants had experienced stigma or discrimination any time during their lifetime.
Results of this study are representative of MSM who gather in meeting places, which was the population targeted by the CPP. The programme did not reach MSM who find sexual partners through internet sites or mobile applications exclusively. Although there are no estimations available in the country on the size of this group, the use of these services is growing fast in Mexico.
Despite the limitations, the study provides evidence of the impact and cost-effectiveness of a CPP on outcomes along the HIV treatment cascade that should be used to design, scaleup, and improve existing HIV prevention efforts in Mexico and in other countries in the region with concentrated epidemics.
M.A.C., S.B.A. conceived and designed the study. M.A.C., S.B.A., S.G.S.R., and M.R.R. conceived the impact evaluation design. M.A.C. and M.A.C.O. had full access to the data in the study and are responsible for the statistical analysis. J.S. reviewed the cost data and analysis. M.A.C. and S.B.A. were responsible for writing the first draft of the paper. M.A.C., S.B.A., M.R.M., .J.S., S.G.S.R., P.U. reviewed and commented the manuscript.
We are very grateful to all the nongovernment organizations (NGOs) that worked with us to map venues in each city that served as the sampling frame for the baseline and follow-up surveys and their company and support during field work. We are enormously grateful to all the participants for their inclination and ability to complete the surveys. We hope the results will be of benefit to their communities. We would like to thank the commitment and excellent work from all enumerators coordinated by Aurora Franco. Finally, we would like to thank the support we received during the follow-up survey from Blanca Rico from FUNSALUD.
Source of funding: The HIV seroprevalence surveys were funded by the Global Fund to Fight AIDS, Malaria and Tuberculosis (Global Fund) through the principal recipient for Mexico's one Global Fund grant: the Mexican Health Foundation (known in the original Spanish as Fundación Mexicana para la Salud, or FUNSALUD). The funder had no role on what cities were selected for the survey, study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Conflicts of interest
Two co-authors worked at Mexican Health Foundation (the principal recipient of the Global Fund Grant): P.U. as the director and J.S. head of the monitoring and evaluation department. Neither of them had any influence on the design of the impact evaluation as it was commissioned to the National Institute of Public Health as an independent group.
1. Sullivan PS, Carballo-Dieguez A, Coates T, Goodreau SM, McGowan I, Sanders EJ, et al. Successes and challenges of HIV prevention in men who have sex with men
2. Kurth AE, Celum C, Baeten JM, Vermund SH, Wasserheit JN. Combination HIV prevention: significance, challenges, and opportunities
. Curr HIV/AIDS Rep
3. Beyrer C, Sullivan PS, Sanchez J, Dowdy D, Altman D, Trapence G, et al. A call to action for comprehensive HIV services for men who have sex with men
4. Hankins CA, de Zalduondo BO. Combination prevention: a deeper understanding of effective HIV prevention
2010; 24 (Suppl 4):S70–S80.
5. Wilson D, Halperin DT. ‘Know your epidemic, know your response’: a useful approach, if we get it right
6. Padian NS, McCoy SI, Manian S, Wilson D, Schwartlander B, Bertozzi SM. Evaluation of large-scale combination HIV prevention programs: essential issues
. J Acquir Immune Defic Syndr
7. Patel M, Allen K, Keatley R, Jonsson U. Introduction
. Eval Program Plann
8. Peersman G, Rugg D. Intervention research and program evaluation: the need to move beyond monitoring
. New Dir Eval
9. Schwartlander B, Stover J, Hallett T, Atun R, Avila C, Gouws E, et al. Towards an improved investment approach for an effective response to HIV/AIDS
10. Brookmeyer R, Boren D, Baral SD, Bekker LG, Phaswana-Mafuya N, Beyrer C, et al. Combination HIV prevention among MSM in South Africa: results from agent-based modeling
. PloS one
11. Firestone R, Rivas J, Lungo S, Cabrera A, Ruether S, Wheeler J, et al. Effectiveness of a combination prevention strategy for HIV risk reduction with men who have sex with men in Central America: a mid-term evaluation
. BMC Public Health
12. Centro Nacional para la Prevención y el Control del VIH/SIDA. El VIH/SIDA en México 2010
, 1st edition. México City: Centro Nacional para la Prevención y el Control del VIH/SIDA; 2010.
13. Bautista-Arredondo S, Colchero MA, Romero M, Conde-Glez CJ, Sosa-Rubi SG. Is the HIV epidemic stable among MSM in Mexico? HIV prevalence and risk behavior results from a nationally representative survey among men who have sex with men
. PloS one
14. Bautista-Arredondo S, Dmytraczenko T, Kombe G, Bertozzi SM. Costing of scaling up HIV/AIDS treatment in Mexico
. Salud Publica Mex
2008; 50 (Suppl 4):S437–S444.
15. Secretaría de Salud. Programa Nacional de Salud 2001–2006. Ciudad de México: 2001.
16. Magis-Rodriguez C, Rivera-Reyes M, R.F CP, Gutierrez J. HIV/AIDS care and prevention expenditures in Mexico: trends and estimations 1997–2002
. Salud Publica Mex
Coordinating Mechanism. Strenthening the National Response to HIV
for MSM and male and female IDU in Mexico Mexico
: The Global Fund to fight AIDS, Tuberculosis and Malaria; 2010 [Accessed 3 November 2014].
18. Crabtree-Ramirez B, Caro-Vega Y, Belaunzaran-Zamudio F, Sierra-Madero J. High prevalence of late diagnosis of HIV in Mexico during the HAART era
. Salud Publica Mex
19. Crabtree-Ramirez B, Caro-Vega Y, Shepherd BE, Wehbe F, Cesar C, Cortes C, et al. Cross-sectional analysis of late HAART initiation in Latin America and the Caribbean: late testers and late presenters
. PloS one
20. Silverman-Retana O, Bautista-Arredondo S, Serván-Mori E, Lozano R. AIDS-related early mortality in Mexico between 2008 and 2012
. Salud Publica Mex
2015; 57 (Suppl 2):S119–S126.
21. Fundación Mexicana para la Salud A.C. Mexico
/AIDS, Round 9 2014 [Accessed 3 November 2014].
22. Stover J, Walker N, Garnett GP, Salomon JA, Stanecki KA, Ghys PD, et al. Can we reverse the HIV/AIDS pandemic with an expanded response?
23. Granich RM, Gilks CF, Dye C, De Cock KM, Williams BG. Universal voluntary HIV testing with immediate antiretroviral therapy as a strategy for elimination of HIV transmission: a mathematical model
24. World Health Organization. HIV
testing and counselling: the gateway to treatment, care and support 2003 [Accessed 21 January 2015].
25. Siegfried N, Uthman OA, Rutherford GW. Optimal time for initiation of antiretroviral therapy in asymptomatic, HIV-infected, treatment-naive adults
. Cochrane Database Syst Rev
26. Cohen MS, Chen YQ, McCauley M, Gamble T, Hosseinipour MC, Kumarasamy N, et al. Prevention of HIV-1 infection with early antiretroviral therapy
. N Engl J Med
27. Landis SE, Earp JL, Koch GG. Impact of HIV testing and counseling on subsequent sexual behavior
. AIDS Educ Prev
28. Bautista-Arredondo S, Sosa-Rubi SG, Opuni M, Contreras-Loya D, Azamar-Alonso A, Ochoa-Moreno I, et al. HIV prevention costs in Kenya, Rwanda
. South Africa and Zambia: findings from the ORPHEA study (unpublished work)