With respect to the global HIV pandemic, men who have sex with men (MSM) comprise most new HIV infections reported in Australia,1 Canada,2 Mexico,3 and the United States4 as well as a substantial proportion of newly diagnosed HIV infections in many countries of Western Europe.5 Furthermore, MSM constitute a relatively small proportion of the overall population in the United States,6 yet they account for nearly half of the cumulative AIDS caseload.4 In addition to being disproportionately affected by the HIV pandemic, recent global trends for an increased incidence of gonorrhea7-9 and syphilis10,11 among MSM suggest that increased risk taking is eclipsing the safer sex practices promoted in the 1980s and early 1990s.12-17 This resurgence of sexually transmitted diseases (STDs) raises another serious concern, because STDs have been implicated in increased efficiency of HIV transmission and more rapid HIV disease progression.18-20 Behavioral interventions that effectively produce reductions in risky sexual behavior and promote maintenance for safer sexual practice remain an important tool in fighting HIV.21 Numerous interventions targeting risk reduction among MSM have been conducted in recent years, and because of the wide diversity of methodologies, interventions, and findings, it is important to conduct ongoing systematic reviews of the empiric literature to attain an integrated understanding of effective intervention components and to make evidence-based recommendations.
Several reviews22-25 that provide qualitative descriptions of studies have been helpful in this endeavor. Together, these reviews suggest that effective interventions for MSM include small-group or community-level skills training to motivate and maintain sex risk behavior change. A meta-analysis examining the effects of 9 behavioral interventions conducted in the United States and reported on in the literature before June 1998 reported a 26% reduction in the proportion of men engaging in unprotected anal sex.26 A related meta-analysis that included an additional 3 interventions conducted outside the United States and reported on before June 1998 resulted in similar findings.27 Both analyses associated overall intervention efficacy with interpersonal skills training, community-level delivery, and a focus on younger populations. Numerous outcome evaluations of HIV interventions targeting MSM, especially those conducted outside the United States, have been reported recently. The integration of international evidence can benefit the HIV field globally by highlighting the elements of efficacious HIV interventions that emerge even across different cultural contexts.28
We conducted a meta-analysis of the available international literature on behavioral interventions that aim to reduce risky sexual behaviors of MSM. Our specific goals include assessing the overall success of interventions in reducing risky sexual behaviors that potentially lead to HIV transmission and infection and, more importantly, identifying factors (ie, population characteristics, study design, intervention features) or constellations of factors that are associated with intervention efficacy.
Database and Search Strategy
As part of the Centers for Disease Control and Prevention (CDC) HIV/AIDS Prevention Research Synthesis (PRS) project, we developed a database system to accumulate the HIV, AIDS, and STD prevention literature. For this review, the PRS project database was updated through July 2003 using a comprehensive search strategy that consisted of automated and manual searches. The specifications of the automated search involved subject headings (focused and inclusive) and key words (in title or abstract only) that were cross-referenced in 4 areas: (1) HIV, AIDS, or STD prevention; (2) prevention research methods; (3) sexual risk behaviors and biologic outcomes; and (4) target population. The automated search strategy was tailored for the following on-line databases: AIDSLINE (1988 to discontinuation in December 2000); MEDLINE, PsycINFO, EMBASE, and the National Institutes of Health (NIH) Web of Science (1988 to November 2002); and SocioFile and ERIC (1988 to June 2003). All automated searches were limited to English-language reports. The full search strategy is available from the authors.
To reduce publication bias and gaps in the automated search, we implemented 5 manual search strategies to identify additional relevant studies. First, we searched the published conference abstracts from recent international AIDS conferences, the on-line Cochrane Controlled Trials Register, the Current Controlled Trials Register, and the Computer Retrieval of Information on Scientific Projects database (http://crisp.cit.nih.gov). Second, we requested the EPPI Centre in London to search their database of effectiveness reviews to identify additional published and unpublished health promotion interventions targeting MSM. Third, we conducted a hand search of 40 journals published between June 2002 and July 2003. Fourth, we contacted principal investigators of published studies and other experts to obtain recommendations of current and ongoing research. Finally, we reviewed the reference lists of all relevant studies for additional citations.
Included studies had to evaluate an HIV, AIDS, or STD behavioral intervention targeting or focusing on MSM (including homosexual, bisexual, or non-gay-identified men); include only men, with 85% or more reporting same-sex behavior; assess the same group prospectively over time or compare multiple groups receiving an intervention with a control or comparison group; assess outcomes before and after completion of the intervention; and report outcome data necessary to calculate effect size estimates for at least 1 sex behavior or biologic outcome measure. For reports that did not report sufficient information to calculate effect size estimates, we attempted to contact authors for clarification or additional information. Interventions were excluded if they did not meet these criteria.
All coding instruments were revised since a prior PRS project review26 to include more detailed data collection. Also new to this review, we identified linkages among citations to ensure that multiple reports describing a single study or intervention program were included in the coding of study sample, intervention description, and design. All coding and data abstraction were performed from January through July 2003 by pairs of independent researchers; disagreements were resolved by discussion. The average interrater agreements (κ) for the study sample and/or intervention and design variables were 0.79 and 0.76, respectively.
We assessed the methodologic quality of the studies by coding the following study design elements: evaluation design (eg, 1-group vs. multiple-group), assignment method (random vs. nonrandom), type of control group (wait list vs. treatment), time of follow-up assessments, and overall and differential retention rates.29 We have chosen to address study quality with analytic strategies as opposed to using a composite quality scale because of the potential problems in interpretation of these scales.30 Thus, we examined study heterogeneity and implemented stratified or restricted analyses to assess efficacy based on study quality.
In addition to quality assessment, data were coded for study information (eg, setting), target population characteristics (eg, sexual orientation), participant characteristics (eg, race/ethnicity), and intervention features (eg, number of sessions). For effect size calculations, we abstracted baseline and follow-up data for the most frequently reported sex behavior outcomes (ie, insertive or receptive unprotected anal intercourse [UAI], insertive or receptive unprotected oral intercourse [UOI], number of sex partners, and condom use during anal sex) and biologic outcomes (ie, diagnosis of STD or HIV).
Quantitative Data Synthesis
The statistical analyses included a meta-analysis of the overall effect of intervention studies for each outcome measure, stratification and sensitivity analyses, examination of potential publication bias, and analysis of time trends. For each study, we calculated the odds ratio (OR) to represent the estimated effect size for individual studies. The OR for a 1-group intervention represents the proportion of participants who changed behavior after the intervention relative to baseline. For multiple-group interventions with a treatment comparison, the OR estimates the relative effect or difference between 2 interventions. The OR for multiple-group interventions with a wait list control represents the absolute effect of the intervention. For unprotected sexual behavior and number of sex partners, ORs less than 1.0 favor the intervention group and indicate a protective effect. For condom use, an OR greater than 1.0 favors the intervention group.
ORs were converted to the natural logarithm of the OR (lnOR), and each lnOR was weighted by the inverse of its variance when combining to estimate the population effect. We calculated a homogeneity statistic, Q, which has an approximate χ2 distribution with k−1 degrees of freedom (k = number of studies). We used the more conservative random effects model to calculate an overall weighted effect for all studies.31 We used the following rules to guide effect size calculations:
- To meet the independence of effect size assumption, we selected the intervention group that was theorized to be most effective, usually as specified by the authors, when studies had more than 1 intervention and a single comparison group.
- For studies reporting separate outcomes of the same construct (eg, insertive and receptive UAI), we chose the outcome with the highest risk of disease transmission to estimate the intervention effect. Insertive UAI confers the greatest risk for HIV-positive MSM to transmit HIV, whereas receptive UAI confers the greatest risk for HIV-negative MSM to contract HIV.
- A hierarchic approach was used to ascertain effect sizes. If studies reported an adjusted OR and 95% confidence interval (CI), these values were retained for meta-analysis as baseline differences and potential confounding variables are typically controlled in adjusted models. Otherwise, we calculated ORs and 95% CIs for the follow-up outcome data and adjusted for baseline differences.32
- ORs and 95% CIs were calculated for each outcome reported as categoric data (eg, percentage of UAI) using standard formulas32; continuous outcomes (eg, mean UAI occasions) were calculated as standardized mean differences and then converted to lnOR.33
- For studies that reported results for more than 1 follow-up, the follow-up closest to 3 months was selected for the overall effect size estimate to evaluate intervention effects on behavioral change.
- For studies with nested designs that used community- or group-level assignment but employed individual-level analyses, 95% CIs were adjusted to compensate for the underestimate of the variance by using estimates of intraclass correlation.34 All analyses were also performed without this adjustment. Because the results did not differ, we report the adjusted CIs in this article.
Stratified analyses used random-effects models to examine whether overall effects differed across subgroups of studies based on characteristics of the samples, intervention features, study design, and length of follow-up assessment. We assessed the likelihood of subgroup differences using the between-groups heterogeneity statistic, QB.31 Sensitivity analyses compared the estimated effect size for all studies, with the effect size obtained after excluding a study (or set of studies) that might influence the overall intervention effect. We also assessed whether the overall results were sensitive to the method used for effect size calculation. Publication bias was ascertained by inspection of a funnel plot.32
As shown in Figure 1, among the 1891 citations identified through the comprehensive search strategy, 65 citations representing 39 intervention studies were eligible for the review. Six studies had insufficient data for the computation of effect sizes and were excluded.35-40Table 1 shows details of the 33 included studies.
Eighteen studies evaluated an intervention with a delayed treatment or wait list control group.40a,41-57 10 studies included a treatment comparison group,58-67 and 5 studies were 1-group interventions without a control or comparison group.68-72
Twenty-two studies were conducted in the United States, with Seattle and San Francisco being the most prevalent sites, and 11 took place in Australia, Brazil, Bulgaria, Canada, Mexico, New Zealand, Russia, and the United Kingdom. Most studies (k = 22) were reported between 1996 and 2003, with 13 reported since 1998. One study was from an unpublished report, and 2 were from conference proceedings.
In terms of subgroups of MSM, 3 studies focused on adolescents or youth, 2 on substance users, 2 on HIV-positive individuals, 1 on sex workers, and 1 on non-gay-identified MSM. Interestingly, both studies of HIV-positive individuals were conducted within the first decade of the epidemic. The median age of the study samples was 31.7 years (range: 19.3-36.6 years), and 3 studies (all conducted in the United States) focused on ethnic groups (African Americans, Asian and Pacific Islanders, and Puerto Ricans) using culturally tailored interventions.
Twenty studies were randomized controlled trials (RCTs), 2 used matching or alternation techniques to minimize assignment bias, and the remaining 6 studies employed assignment methods that allowed self-selection or assignment based on capacity or convenience factors. Six studies administered interventions to communities, whereas the remainder administered interventions to individuals or groups. The most common recruitment settings were commercial (67%) and public area and/or community (58%) venues; many studies (45%) failed to report the setting of intervention delivery. Most studies evaluated interventions with follow-up longer than 3 months (67%) and achieved greater than 70% retention of participants (56%).
The 33 studies consisted of 55 intervention groups. Regarding mechanisms used to promote behavior change, most interventions provided facts and information to increase HIV knowledge (82%) or sought to change cognitive factors such as attitudes and beliefs (73%). The most common delivery method was group discussion (55%), and the most common deliverers were peers (35%) or counselors (33%). Technical skills (eg, correct use of condoms [20%]), personal skills (eg, eroticizing safer sex and coping strategies [31%]), or interpersonal skills (eg, safer sex negotiation and assertiveness [44%]) were frequently modeled, demonstrated, or practiced. The mean number of sessions was 2 (range: 1-48 sessions), the median amount of time to deliver the intervention was 8 hours (range: 1-96 hours), and the total time span of the intervention ranged from 1 day (15 studies) to 3 years. With regard to type of outcome measure, 29 studies reported data on UAI, 10 on the number of sex partners, 9 on condom use, and 4 on UOI. Only 1 study reported incident STD diagnosis.
Quantitative Data Synthesis
Effects on Sex-Risk Outcomes
The aggregated effect size of the 29 studies that reported intervention effects on UAI was protective and significant (OR = 0.81, 95% CI: 0.67-0.98, N = 7170). Because 1-group study designs are unable to rule out potential confounding variables that may bias results (eg, maturation or historical biases), the 5 studies using this type of design were excluded from subsequent analyses.73 Focusing on the remaining 24 multiple-group studies, the aggregated effect size remained significant (OR = 0.77, 95% CI: 0.65-0.92, N = 6080), suggesting a 23% reduction in odds of UAI among study participants. Figure 2 presents a plot of the 24 multiple-group studies and the summary effect size estimate. Most of the ORs are in the protective range. Five studies have ORs greater than 1.0, and 2 of these studies52,54 were identified as significant outliers using standard analytic procedures.74 There was little evidence of publication bias, because examination of a funnel plot (not shown) revealed no asymmetry among the studies.
For condom use with anal intercourse, the 9 studies included in the analysis40a,42,44,45,50,53,56,60,65 resulted in a significant effect (OR = 1.61, 95% CI: 1.16-2.22, N = 2339). Although the effect was heterogeneous (Q = 18.9, df = 8, P = 0.015), sensitivity analyses did not reveal subsets of studies that moderated the significant effect. Among 10 studies41,42,44,47,50,51,53,56,58,60 that reported intervention effects on the number of sex partners, a nonsignificant effect was observed (OR = 0.81, 95% CI: 0.64-1.04, N = 2581). Sensitivity analysis suggested that 1 international study51 was a significant outlier, contributing a large portion of the variance. Exclusion of this study revealed a significant reduction in the number of sex partners (OR = 0.76, 95% CI: 0.61-0.94). The analysis of 4 studies42,50,53,56 reporting UOI was not significant (OR = 0.86, 95% CI: 0.55-1.34, N = 615).
Stratified Analyses of Studies Reporting Unprotected Anal Intercourse
For the 24 multiple-group studies reporting UAI, the intervention effect was heterogeneous (Q = 38.7, df = 23, P = 0.021). We initially considered study design characteristics as potential sources of heterogeneity among these studies. The 20 studies that used an RCT or non-RCT with minimal assignment bias exhibited a significant protective effect (OR = 0.68, 95% CI: 0.58-0.79, N = 3929), whereas a nonsignificant effect was obtained among the 4 studies52,54,55,57 that used convenience factors in assignment, which can potentially introduce bias (OR = 1.05, 95% CI: 0.69-1.61, N = 2151). The between-groups statistic indicated that the 20 studies using an RCT or method of assignment associated with minimal bias were associated with significantly greater reductions in UAI than the 4 studies using assignment methods associated with potential bias (QB = 11.4, df = 1, P = 0.0007). Because potentially biased assignment methods can distort study results, subsequent analyses were restricted to the 20 studies that were RCTs or non-RCTs with minimal assignment bias.
Next, we examined moderators of intervention efficacy on UAI (Table 2). Because the 20 studies are relatively homogeneous, only 1 significant between-group difference was found. Interventions that reported a theoretic basis were associated with significantly greater reductions in UAI than interventions not reporting a theory (QB = 3.02, df = 1, P < 0.05). Greater reductions in UAI risk in the intervention group relative to the comparison group were seen in interventions studies that are based on the diffusion of innovations theory (OR = 0.62, 95% CI: 0.47-0.83, 3 studies) or the model of relapse prevention (OR = 0.61, 95% CI: 0.47-0.78, 8 studies) as a theoretic basis. There were no significant differences on the reduction of UAI between the intervention and comparison groups in intervention studies that are based on the social learning theory (OR = 0.60, 95% CI: 0.33-1.09, 3 studies) or other behavioral theories (OR = 0.73, 95% CI: 0.45-1.20, 3 studies).
Because of the small number of studies and apparent lack of statistical power to detect significant between-group differences, it was informative to examine patterns of significance among potential moderating variables. Interventions were similarly efficacious for each subgroup of the following stratification variables: unit of assignment, type of comparison group, length of follow-up, method of effect size calculation, use of incentives, age, education, and self-reported baseline UAI. Interventions conducted before and after 1996, the year highly active antiretroviral therapy (HAART) was introduced, were similarly efficacious in reducing UAI.
Because the OR for studies with a wait list control group (k = 11) represents the absolute effect of the intervention and the OR for studies with a comparison group receiving an HIV-related intervention (k = 9) represents the relative effect of the intervention, we conducted separate stratified analyses of intervention content for these study designs (Table 3). Despite differences in interpreting intervention effects, the subgroups that showed consistent evidence of efficacy across both types of study design included group discussions, 4 or more delivery methods, and interpersonal skills building. Skills training delivered by role plays or practice and by lectures (among wait list control studies) were associated with significant reductions in UAI. Further, efficacy was associated with interventions having greater intervention exposure complexity, defined as more than 1 session, a duration lasting 4 or more hours, and a time span of at least 3 weeks.
Analysis of Time Trends
The length and number of follow-ups varied across studies, allowing us to examine whether reductions in UAI are found to be consistent at different follow-up time points. Although our primary analysis included the follow-up assessment closest to 3 months, we used further stratified analyses to estimate intervention effects at each of the following 4 follow-up times: immediately after the intervention (k = 6), 3 months (k = 7), 5 to 6 months (k = 6), and 8 to 12 months (k = 7). Effect sizes associated with significant reductions in UAI risk were found immediately after the intervention (OR = 0.65, 95% CI: 0.48-0.87), at 5 to 6 months (OR = 0.64, 95% CI: 0.49-0.85), and at 8 to 12 months (OR = 0.57, 95% CI: 0.44-0.75). Although the effect size computed at 3 months suggested protection, it was not statistically significant (OR = 0.86, 95% CI: 0.65-1.15).
The findings of this review support the view that behavior change interventions for MSM work. Not only do behavioral interventions reduce rates of UAI, decrease numbers of sexual partners, and increase condom use during anal sex; they also support behavioral risk reductions up to 12 months after interventions. Although it has been speculated that the introduction of HAART might lead to increases in risky sexual behavior75 and less efficacious interventions,76 it is interesting to note that intervention studies conducted after the introduction of HAART achieved the same reductions in risk behavior as those reported before the introduction of these life-saving medications. The effect sizes associated with these behavioral interventions are substantial: the interventions resulted in a 23%, reduction in odds of UAI and a 61% increase in odds of condom use during anal sex. When translated into final health outcomes (eg, quality-adjusted life-years gained), effect sizes of this magnitude are well within the range of those considered to be cost-effective. In fact, several cost-effectiveness analyses77-82 have shown that many interventions targeting MSM are indeed cost-saving.83
Aside from overall efficacy, the question of which components of interventions are associated with the greatest risk reduction is also of great interest (Table 4). This review identifies an entirely new set of factors associated with efficacy beyond those originally suggested in prior qualitative22-25 and quantitative26,27 reviews of HIV prevention for MSM. Interventions that reported a basis in behavioral theory were associated with efficacy. HIV prevention for MSM would be well served by developing interventions based on the behavioral mechanisms proposed by these theories, which provide a useful framework for explaining behavior change84 and for developing intervention strategies to target relevant mediators of behavior change. Although no behavioral theory in particular was associated with significantly better intervention efficacy than any other theory, interventions based on the diffusion of innovations theory85 and the model of relapse prevention86 showed greater point estimates for reductions in risk behavior in the intervention group relative to the comparison group. Diffusion of innovations posits that popular people who endorse innovations (eg, HIV risk reduction) can help to refine behavioral norms and standards.44 Relapse prevention approaches aid at-risk individuals to identify situational risk factors (eg, unprotected sex) and develop cognitive coping skills to resist lapses.53 It is important for future research to explore exactly how guiding theories are translated into intervention activities and how carefully interventions must follow the principles of their guiding theories to achieve successful risk reduction.
Additional intervention components suggestive of intervention efficacy include interpersonal skills training; skills training delivered by role plays or lectures; multiple delivery methods; and greater intervention exposure complexity, consisting of number of sessions, duration, and time span. Although we are not claiming that these components are necessary or sufficient for behavioral interventions for MSM to work, we recommend that prevention providers consider these components when developing intervention programs for this population.
Although we now have a body of literature showing that HIV prevention can and does work among MSM, careful empiric tests that demonstrate these same effects across subgroups of MSM (eg, non-gay-identified and substance-using MSM) would also be welcome. In the meantime, adapting and tailoring effective interventions for subgroups of MSM may be an efficient strategy.87 Evaluations of the effects of behavioral interventions designed specifically for subgroups of MSM and the effects of interventions adapted and tailored for these subgroups should be accorded a high priority in the behavioral intervention research portfolio.
Future intervention efforts also ought to consider newly identified predictors of increased sexual risk taking among MSM, including HIV treatment optimism,88 social norms regarding intentional unsafe sex,89 and use of the Internet to solicit sex partners.90 There is also a crucial need for intervention research to meet the needs of emerging MSM communities throughout the world. Although a few researchers have taken on this daunting task,66,72 the overwhelming need for proven models of HIV prevention for MSM in developing world settings is self-evident.
Only one intervention trial included in this review examined biologic endpoints in addition to self-reported sex behavior change.63 Despite finding a significant decline in self-reported UAI, the trial showed that the intervention was associated with a significant increase in incident STDs. Although the results of this review demonstrate that HIV behavioral interventions are efficacious in changing self-reported risk behaviors among MSM, it remains uncertain how large an effect would be required to lead to significant reductions in STD or HIV incidence. This question becomes even more complex in an era of “HIV serosorting,” where men of like serostatus select each other for sexual contact, sometimes without benefit of protection.91 Future intervention studies should complement self-reported behavioral measures with more objective biologic outcomes in predictions of intervention efficacy and collect information on partner selection.
The methodology of meta-analysis reflects all the weaknesses of the original literature that it reviews as well as the weaknesses of meta-analysis itself. One major limitation is reliance on self-reports of sexual risk behavior, which is open to recall bias and socially desirable responding.92 Many studies in our review took several actions to ensure the confidentiality of data, including asking participants to answer paper-and-pencil questionnaires without the presence of an interviewer. Furthermore, several studies asked respondents to self-report sex behaviors using short recall durations, ranging from 1 to 6 months, which have been shown to maximize self-report accuracy.93 Because direct methods of sex behavior assessment are unethical and impractical, future intervention evaluations are advised to consider using new technology (eg, computer-assisted assessments) to improve the veridicality of self-reported sex behavior.94,95 Regarding the meta-analysis, we can examine the effects of moderators only on the variables that were reported in the studies. In fact, these analyses are only meaningful when variables are reported in most studies, thus resulting in a small amount of missing data. A lack of reporting on other potentially important variables (eg, gay community involvement, culture, religion, partner selection) in the primary studies also limits our ability to have a closer examination of the intervention effects. More clear and transparent reporting of key elements in intervention studies would improve the quality of meta-analysis in the future.96,97 Another limitation of our meta-analysis is that the clustering of many intervention characteristics across interventions did not allow us to use more sophisticated meta-analytic procedures, such as meta-regression, to disentangle the independent effects or interactions among study characteristics. We highly recommend that future intervention studies use factorial designs so that particular characteristics can be independently evaluated.
Although initial empiric research has indicated that moving science-based behavioral interventions into community-based practice can be successfully accomplished,98 it is not without challenges.99 In response to this situation, the CDC has embarked on a national program not only to reproduce scientifically proven interventions but to set up a technology transfer support program to ensure the faithful replication and successful adaptation of proven interventions in the field.100-103 The ongoing challenge of the reproduction of efficacious interventions for the front lines of prevention work raises the need for an ongoing operational research agenda that identifies the factors promoting effectiveness of behavioral interventions in real-world settings.
Finally, some might question the wisdom of investing substantial resources to field interventions that only reduce odds by approximately one-quarter. We note that this field is not static; the body of research described herein reports risk reductions obtained in a set of pioneering studies. Continued progress in reducing residual levels of risk should be expected in the next generation of HIV prevention research. Such research would be particularly welcome in the context of stigmatized and marginalized populations such as gay men. It is clear that there is a high probability that we are not likely to have an effective HIV vaccine for many years to come and that any eventual vaccine may only be partially effective.104 Thus, HIV prevention efforts need to rely on a combination of behavioral and biomedical risk reduction strategies. This may be especially important for populations with high prevalence rates of HIV infection, such as MSM. Although the development of a body of proven HIV behavioral risk reduction strategies for MSM is good news indeed in the global fight against AIDS, there still remains a need for ongoing research and innovative interventions for continued reductions in sexual risk behaviors among MSM.
The following HIV/AIDS PRS team members contributed to this review (listed alphabetically): Tanesha Griffin, Angela Hutchinson, Linda Kay, Angela Kim, Paola Marrero-Gonzalez, Mary Mullins, Jocelyn Patterson, Sima Rama, and Sekhar Thadiparthi. The authors acknowledge the former PRS team members, consultants, and contractors who contributed to the previous PRS MSM review, particularly Wayne Johnson. They extend special thanks to those principal investigators who provided additional information and data regarding their interventions. They also acknowledge the thoughtful comments of 2 peer reviewers.
1. National Centre in HIV Epidemiology and Clinical Research (NCHECR). HIV/AIDS and related diseases in Australia: annual surveillance report
. Sydney: NCHECR; 1999.
2. Health Canada. HIV and AIDS in Canada. Surveillance report to December 31, 2002
. Ottawa, Ontario: Centre for Infectious Disease Prevention and Control, Health Canada; 2003.
3. Pan American Health Organization. Update on HIV/AIDS surveillance in the Americas. Epidemiol Bull
4. Centers for Disease Control and Prevention. HIV/AIDS surveillance report
. Atlanta: Centers for Disease Control and Prevention; 2001:13.
5. European Centre for the Epidemiological Monitoring of AIDS. HIV/AIDS surveillance in Europe. Mid-year report 2003, no. 69
. Saint-Maurice: Institut de Veille Sanitaire; 2003.
6. Binson D, Michaels S, Stall R, et al. Prevalence and social distribution of men who have sex with men
: United States and its urban centers. J Sex Res
7. Fennema JSA, Cairo I, Coutinho RA. Substantial increase in gonorrhoea and syphilis among attenders of the Amsterdam sexually transmitted diseases clinic. Ned Tijdschr Geneeskd
8. Martin IMC, Ison CA. London Gonococcal Working Group. Rise in gonorrhoea in London, UK. Lancet
9. Fox KK, del Rio C, Holmes KK, et al. Gonorrhea in the HIV era: a reversal in trends among men who have sex with men
. Am J Public Health
10. Centers for Disease Control and Prevention. Outbreak of syphilis among men who have sex with men
-Southern California, 2000. MMWR Morb Mortal Wkly Rep
11. Centers for Disease Control and Prevention. Primary and secondary syphilis among men who have sex with men
-New York City, 2001. MMWR Morb Mortal Wkly Rep
12. Stall RD, Hays RB, Waldo CR, et al. The Gay 90's: a review of research in the 1990s on sexual behavior and HIV risk among men who have sex with men
13. Wolitski RJ, Valdiserri RO, Denning PH, et al. Are we headed for a resurgence of the HIV epidemic among men who have sex with men
? Am J Public Health
14. UNAIDS. AIDS Epidemic Update
. Geneva: UNAIDS; 2002.
15. Stolte IG, Coutinho RA. Risk behavior and sexually transmitted diseases are on the rise in gay men, but what is happening with HIV? Curr Opin Infect Dis
16. Quinn TC. The global HIV/AIDS pandemic 2002: a status report. Hopkins HIV Rep
17. Nicoll A, Hamers FF. Are trends in HIV, gonorrhoea, and syphilis worsening in Western Europe? BMJ
18. Fleming DT, Wasserheit JN. From epidemiological synergy to public health policy and practice: the contribution of other sexually transmitted diseases to sexual transmission of HIV infection. Sex Transm Dis
19. Ciesielski CA. Sexually transmitted diseases in men who have sex with men
: an epidemiologic review. Curr Infect Dis Rep
20. Lynn WA, Lightman S. Syphilis and HIV: a dangerous combination. Lancet Infect Dis
21. Ross MW, Kelly JA. Interventions to reduce HIV transmission in homosexual men. In: Peterson JL, DiClemente R, eds. Handbook of HIV Prevention
. New York: Kluwer Academic/Plenum Publishers; 2000:201-216.
22. Choi KH, Coates TJ. Prevention of HIV infection. AIDS
23. Oakley A, Oliver S, Peersman G, et al. Review of Effectiveness of Health Promotion Interventions for Men Who Have Sex With Men
. London: Institute of Education, Social Science Research Unit, EPPI Centre; 1996.
24. Kegeles SM, Hart GJ. Recent HIV-prevention interventions for gay men: individual, small-group and community-based studies. AIDS
25. Ellis S, Barnett-Page E, Morgan A, et al. HIV Prevention: A Review of Reviews Assessing the Effectiveness of Interventions to Reduce the Sexual Risk of HIV Transmission
. London: Health Development Agency; 2003.
26. Johnson WD, Hedges LV, Ramirez G, et al. HIV prevention research for men who have sex with men
: a systematic review and meta-analysis
. J Acquir Immune Defic Syndr
27. Johnson WD, Hedges LV, Diaz RM. Interventions to modify sexual risk behaviors for preventing HIV infection in men who have sex with men
. In: The Cochrane Library
. Chichester: John Wiley & Sons; 2004.
28. Elford J, Hart G. If HIV prevention works, why are rates of high-risk sexual behavior increasing among MSM? AIDS Educ Prev
29. Flores SA, Crepaz N. Quality of study methods in individual- and group-level HIV intervention research: critical reporting elements. AIDS Educ Prev
30. Juni P, Altman DG, Egger M. Assessing the quality of randomized controlled trials. In: Egger M, Smith GD, Altman DG, eds. Systematic Reviews in Health Care: Meta-Analysis in Context
. London: BMJ
31. Hedges LV, Vevea JL. Fixed and random effects models in meta-analysis
. Psychol Methods
32. Cooper H, Hedges LV, editors. The Handbook of Research Synthesis
. New York: Russell Sage Foundation; 1994.
33. Johnson WD, Semaan S, Hedges LV, et al. A protocol for the analytical aspects of a systematic review of HIV prevention research. J Acquir Immune Defic Syndr
34. Rooney BL, Murray DM. A meta-analysis
of smoking prevention programs after adjustment for errors in the unit of analysis. Am J Epidemiol
35. Kelly JA, St. Lawrence JS, Betts R, et al. A skills-training group intervention model to assist persons in reducing risk behaviors for HIV infection. AIDS Educ Prev
36. Rotheram-Borus MJ, Reid H, Rosario M. Factors mediating changes in sexual HIV risk behaviors among gay and bisexual male adolescents. Am J Public Health
37. Wanigaratne S, Billington A, Williams M. Initiating and maintaining safer sex: evaluation of group work with gay men. In: Catalan J, Sherr L, eds. The Impact of AIDS: Psychological and Social Aspects of HIV Infection
. Amsterdam: Hardwood Academic Publishers; 1997:27-41.
38. Hospers HJ, Debets W, Ross MW, et al. Evaluation of an HIV prevention intervention for men who have sex with men
at cruising areas in the Netherlands. AIDS Behav
39. Ziersch A, Gaffney J, Tomlinson DR. STI prevention and the male sex industry in London: evaluating a pilot peer education programme. Sex Transm Infect
40. McOwan A, Gilleece Y, Chislett L, et al. Can targeted HIV testing campaigns alter health-seeking behaviour? AIDS Care
40a. Goldbaum GM, Johnson W, Wolitski RJ, et al. Sexual behavior change among non gay-identified men who have sex with men
: Response to a community-level intervention. 1999, unpublished.
41. Coates TJ, McKusick L, Kuno R, et al. Stress reduction training changed number of sexual partners but not immune function in men with HIV. Am J Public Health
42. Kelly JA, St. Lawrence JS, Hood HV, et al. Behavioral intervention to reduce AIDS risk activities. J Consult Clin Psychol
43. Rosser BRS. Evaluation of the efficacy of AIDS education interventions for homosexually active men. Health Educ Res
44. Kelly JA, St. Lawrence JS, Diaz YE, et al. HIV risk behavior reduction following intervention with key opinion leaders of population: an experimental analysis. Am J Public Health
45. Tudiver F, Myers T, Kurtz RG, et al. The Talking Sex Project: results of a randomized controlled trial of small-group AIDS education for 612 gay and bisexual men. Eval Health Prof
46. Kelly JA, Murphy DA, Bahr GR, et al. Outcome of cognitive-behavioral and support group brief therapies for depressed, HIV-infected persons. Am J Psychiatry
47. Choi KH, Lew S, Vittinghoff E, et al. The efficacy of brief group counseling in HIV risk reduction among homosexual Asian and Pacific Islander men. AIDS
48. Kegeles SM, Hays RB, Coates TJ. The Mpowerment Project: a community-level HIV prevention intervention for young gay men. Am J Public Health
49. Peterson JL, Coates TJ, Catania J, et al. Evaluation of an HIV risk reduction intervention among African-American homosexual and bisexual men. AIDS
50. Roffman RA, Picciano JF, Ryan R, et al. HIV-prevention group counseling delivered by telephone: an efficacy trial with gay and bisexual men. AIDS Behav
51. Zimmerman MA, Ramirez-Valles J, Suarez E, et al. An HIV/AIDS prevention
project for Mexican homosexual men: an empowerment approach. Health Educ Behav
52. Miller RL, Klotz D, Eckholdt HM. HIV prevention with male prostitutes and patrons of hustler bars: replication of an HIV preventive intervention. Am J Community Psychol
53. Roffman RA, Stephens RS, Curtin L, et al. Relapse prevention as an interventive model for HIV risk reduction in gay and bisexual men. AIDS Educ Prev
54. Elford J, Graham B, Sherr L. Peer education has no significant impact on HIV risk behaviours among gay men in London. AIDS
55. Flowers P, Hart GJ, Williamson LM, et al. Does bar-based, peer-led sexual health promotion have a community-level effect amongst gay men in Scotland? Int J STD AIDS
56. Picciano J, Roffman R, Kalichman S, et al. A telephone based brief intervention using motivational enhancement to facilitate HIV risk reduction among MSM: a pilot study. AIDS Behav
57. Kegeles SM, Rebchook GM, Hays RB, et al. Staving off increases in young gay/bisexual men's risk behavior in the HAART era. Presented at: XIV International AIDS Conference; 2002; Barcelona.
58. Valdiserri RO, Lyter DW, Leviton LC, et al. AIDS prevention in homosexual and bisexual men: results of a randomized trial evaluating two risk reduction interventions. AIDS
59. Gold RS, Rosenthal DA. Preventing unprotected anal intercourse in gay men: a comparison of two intervention techniques. Int J STD AIDS
60. Kelly JA, Murphy DA, Sikkema KJ, et al. Randomised, controlled, community-level HIV-prevention intervention for sexual-risk behaviour among homosexual men in US cities. Lancet
61. Gold RS, Rosenthal DA. Examining self-justifications for unsafe sex as a technique of AIDS education: the importance of personal relevance. Int J STD AIDS
62. Stall RD, Paul JP, Barrett DC, et al. An outcome evaluation to measure changes in sexual risk-taking among gay men undergoing substance use disorder treatment. J Stud Alcohol
63. Imrie J, Stephenson JM, Cowan FM, et al. A cognitive behavioural intervention to reduce sexually transmitted infections among gay men: randomised trial. BMJ
64. Dilley JW, Woods WJ, Sabatino J, et al. Changing sexual behavior among gay male repeat testers for HIV: a randomized, controlled trial of a single-session intervention. J Acquir Immune Defic Syndr
65. Rosser BRS, Bockting WO, Rugg DL, et al. A randomized controlled intervention trial of a sexual health approach to long-term HIV risk reduction for men who have sex with men
: effects of the intervention on unsafe sexual behavior. AIDS Educ Prev
66. Sampaio M, Brites C, Stall R, et al. Reducing AIDS risk among men who have sex with men
in Salvador, Brazil. AIDS Behav
67. Shoptaw S, Reback CJ, Peck J, et al. Behavioral treatment approaches for methamphetamine dependence and HIV-related sexual risk behaviors among urban gay and bisexual men. Drug Alcohol Depend
. (In press).
68. Rosser BRS. The effects of using fear in public AIDS education on the behaviour of homosexually active men. J Psychol Human Sex
69. Remafedi G. Cognitive and behavioral adaptations to HIV/AIDS among gay and bisexual adolescents. J Adolesc Health
70. Miller RL. Assisting gay men to maintain safer sex: an evaluation of an AIDS service organization's safer sex maintenance program. AIDS Educ Prev
71. Toro-Alfonso J, Varas-Diaz N, Andujar-Bello I. Evaluation of an HIV/AIDS prevention
intervention targeting Latino gay men and men who have sex with men
in Puerto Rico. AIDS Educ Prev
72. Amirkhanian YA, Kelly JA, Kabakchieva E, et al. Evaluation of a social network HIV prevention intervention program for young men who have sex with men
in Russia and Bulgaria. AIDS Educ Prev
73. Bonell C, Bennett R, Oakley A. Sexual health interventions should be subject to experimental evaluation. In: Stephenson JM, Imrie J, Bonell C, eds. Effective Sexual Health Interventions: Issues in Experimental Evaluation
. Oxford: Oxford University Press; 2003:3-16.
74. Hedges LV, Olkin I. Statistical Methods for Meta-Analysis
. Orlando: Academic Press; 1985.
75. Crepaz N, Hart TA, Marks G. Highly active antiretroviral therapy and sexual risk behavior: a meta-analytic review. JAMA
76. Valdiserri RO. Mapping the roots of HIV/AIDS complacency: implications for program and policy development. AIDS Educ Prev
77. Holtgrave DR, Kelly JA. Cost-effectiveness of an HIV/AIDS prevention
intervention for gay men. AIDS Behav
78. Pinkerton SD, Holtgrave DR, Valdiserri RO. Cost-effectiveness of HIV-prevention skills training for men who have sex with men
79. Pinkerton SD, Holtgrave DR, DiFranceisco WJ, et al. Cost-effectiveness of a community-level HIV risk reduction intervention. Am J Public Health
80. Kahn JG, Haynes-Sanstad KC. The role of cost-effectiveness analysis in assessing HIV-prevention interventions. AIDS Public Policy J
81. Tao G, Remafedi G. Economic evaluation of an HIV prevention intervention for gay and bisexual male adolescents. J Acquir Immune Defic Syndr Hum Retrovirol
82. Kahn JG, Kegeles SM, Hays R, et al. Cost-effectiveness of the Mpowerment Project, a community-level intervention for young gay men. J Acquir Immune Defic Syndr
83. Pinkerton SD, Johnson-Masotti AP, Holtgrave DR, et al. A review of the cost-effectiveness of interventions to prevent sexual transmission of HIV in the United States. AIDS Behav
84. Bonell C, Imrie J. Behavioural interventions to prevent HIV infection: rapid evolution, increasing rigour, moderate success. Br Med Bull
85. Rogers E. Diffusion of Innovations
. New York: Free Press; 1983.
86. Marlatt GA, Gordon JR. Relapse Prevention: Maintenance Strategies in the Treatment of Addictive Behaviors
. New York: Guilford Press; 1985.
87. Kelly JA, Sogolow ED, Neumann MS. Future directions and emerging issues in technology transfer between HIV prevention researchers and community-based service providers. AIDS Educ Prev
. 2000;12(Suppl A):126-141.
88. Elford J, Sherr L, Adam P, et al. HIV treatment optimism among gay men: an international perspective. J Acquir Immune Defic Syndr
89. Halkitis PN, Parsons JT, Wilton L. Barebacking among gay and bisexual men in New York City: explanations for the emergence of intentional unsafe behavior. Arch Sex Behav
90. Centers for Disease Control and Prevention. Internet use and early syphilis infection among men who have sex with men
-San Francisco, California, 1999-2003. MMWR Morb Mortal Wkly Rep
91. Truong HM, McFarland W, Kellogg T, et al. Increases in ‘serosorting’ may prevent further expansion of the HIV epidemic among MSM in San Francisco. Presented at: 11th Conference on Retroviruses and Opportunistic Infections; 2004; San Francisco.
92. Weinhardt LS, Forsyth AD, Carey MP, et al. Reliability and validity of self-report measures of HIV-related sexual behavior: progress since 1990 and recommendations for research and practice. Arch Sex Behav
93. Jaccard J, McDonald R, Wan CK, et al. The accuracy of self-reports of condom use and sexual behavior. J Appl Soc Psychol
94. Schroeder KE, Carey MP, Vanable PA. Methodological challenges in research on sexual risk behavior: II. Accuracy of self-reports. Ann Behav Med
95. Newman JC, Des Jarlais DC, Turner CF, et al. The differential effects of face-to-face and computer interview methods. Am J Public Health
96. Moher D, Schulz KF, Altman DG. The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomised trials. Lancet
97. Des Jarlais DC, Lyles CM, Crepaz N, and the TREND Group. Improving the reporting quality of nonrandomized evaluation of behavioral and public health interventions: the TREND statement. Am J Public Health
98. Kelly JA, Somlai AM, DiFranceisco WJ, et al. Bridging the gap between the science and service of HIV prevention: transferring effective research-based HIV prevention interventions to community AIDS service providers. Am J Public Health
99. Neumann MS, Sogolow ED, Holtgrave DR. Supporting the transfer of HIV prevention behavioral research to public health practice. AIDS Educ Prev
100. Neumann MS, Sogolow E. Replicating effective programs: HIV/AIDS prevention
technology transfer. AIDS Educ Prev
101. Kelly JA, Heckman TG, Stevenson LY, et al. Transfer of research-based HIV prevention interventions to community service providers: fidelity and adaptation. AIDS Educ Prev
102. Kegeles SM, Rebchook GM, Hays RB, et al. From science to application: the development of an intervention package. AIDS Educ Prev
103. Collins CB, Cleveland J, Harshbarger C, et al. Diffusion of effective behavioral interventions
for HIV prevention through a public health prevention service delivery system. Presented at: National HIV Prevention Conference; 2003; Atlanta.
104. Hu DJ, Vitek CR, Bartholow B, et al. Key issues for a potential human immunodeficiency virus vaccine. Clin Infect Dis