Share this article on:

Comparison of HIV Behavioral Indicators Among Men Who Have Sex With Men Across Two Survey Methodologies, San Francisco, 2004 and 2008

Kellogg, Timothy A. MA*; Hecht, Jennifer MPH; Bernstein, Kyle PhD*; McFarland, Willi MD, PhD*; Connors, Andy BA, ASN; Perloff, Leah BA; Raymond, H. Fisher DrPH*

doi: 10.1097/01.olq.0000431354.96087.50
Original Study

Background Our goal was to examine whether community-based behavioral surveys can augment data collected for the National HIV Behavioral Surveillance System (NHBS) among men who have sex with men (MSM) in San Francisco.

Methods We compared estimates of sexual risk behaviors among MSM using data from two cycles of NHBS (2004 and 2008) and outreach surveys conducted by STOP AIDS Project (SAP) during the same years. We compared estimates of unprotected anal intercourse (UAI) and other indicators to assess concordance of estimates across methodologies.

Results Of the 3248 interviews conducted, the NHBS sample included more nonwhite and older MSM, more self-reported HIV positive, and less sexually active men than the SAP sample. Estimates of UAI in the last 6 months were slightly higher in the NHBS survey than in the SAP surveys (2004: 40% vs. 36%, P = 0.03; 2008: 44% vs. 38%, P = 0.08). In 2008, where respondent-partner HIV-discordant status could be measured, estimates of UAI with a potentially discordant partner were similar (12% vs. 12%, P = 0.87). Also, the NHBS and SAP surveys observed similar estimates of UAI by high-risk positioning with potentially discordant partners (HIV-positive men reporting insertive UAI with a potentially HIV-negative partner: 13% vs. 11%, P = 0.45; HIV-negative men reporting receptive UAI with a potentially HIV-positive person: 5% vs. 4%, P = 0.85).

Conclusions Behavioral estimates drawn from convenience sampling methods can provide informative surveillance estimates of key behavioral indictors that can augment data from more rigorous national HIV behavioral surveillance surveys.

A comparison of 2 survey methods of men who have sex with men in San Francisco found similar estimates of unprotected anal intercourse suggesting that community-based surveys can be useful in behavioral surveillance.

From the *San Francisco Department of Public Health, and †The STOP AIDS Project, San Francisco, CA

Conflict of interest: The findings and conclusions in this manuscript are those of the authors. All authors claim no conflict of interest.

The San Francisco Department of Public Health is supported by the Centers of Disease Control (2004 U62/CCU923549 and 2008 U62/PS000961) to conduct National HIV Behavioral Surveillance surveys.

Correspondence: Timothy A. Kellogg, MA, San Francisco Department of Public Health, 25 Van Ness Ave, Suite 500, San Francisco, CA 94102. E-mail:

Received for publication February 19, 2013, and accepted April 30, 2013.

During the last decade, surveillance methods used to track the HIV/AIDS epidemic in the United States and elsewhere have changed profoundly.1,2 In addition to traditional HIV/AIDS case reporting, surveillance systems have added programs to collect data designed to monitor behaviors in populations at risk for HIV. Tracking behavioral data has the advantage of identifying potential short-term changes in the epidemic before increases or decreases in HIV cases are detected in traditional case-finding activities. Expansion of behavioral surveillance system corresponded to a resurgence of the HIV epidemic among men who have sex with men (MSM) in North America.3,4 Data from behavioral surveys are useful in assessing behavioral factors associated with increases in HIV infection among MSM.

To standardize behavioral surveillance in the United States, the Centers for Disease Control and Prevention (CDC) has released a framework for conducting a National HIV Behavioral Surveillance (NHBS) system that provides population-based estimates of behavioral indicators every 3 years among MSM.5 In San Francisco, the San Francisco Department of Public Health (SFDPH) implemented NHBS in 2004 and 2008, during which representative samples of MSM were recruited using time-location sampling to estimate a variety of behavioral indicators.6 The NHBS platform has become the gold standard for systematic behavioral surveillance in the United States. However, its rigor and scale make it resource intensive, and the MSM cycle occurs only every 3 years, preventing the collection of potentially important data during years when NHBS is not conducted.

In addition to the NHBS surveys, the SFDPH uses other behavioral data collected through ongoing outreach activities conducted regularly by STOP AIDS Project (SAP), a community-based, nonprofit organization in San Francisco. These data have been used locally to measure trends in the prevalence of high-risk sexual behaviors such as unprotected anal intercourse (UAI).7 This methodology differs from NBHS in that it uses convenience sampling methods to recruit respondents and thus may be prone to sampling bias. However, SAP surveys are conducted continuously, and the available data can be used to supplement behavioral surveillance between cycles of NHBS. Together, the NHBS and SAP data are principal sources for behavioral surveillance in San Francisco and the SFDPH leverages SAP’s ongoing surveys during the periods when NHBS-MSM is not implemented. However, it is important to investigate whether estimates of key indicators are comparable and to corroborate trends when estimates are either increasing or decreasing.

In this study, we compared sample characteristics and estimates of several indicators among MSM who were recruited from the 2004 and 2008 NHBS surveys and SAP outreach surveys and evaluated the extent to which data from community-based outreach surveys can be used to augment national behavioral surveillance data.

Back to Top | Article Outline


Participant Recruitment and Sampling

National HIV Behavioral Surveillance System

Two waves of NHBS surveys were conducted among MSM in San Francisco in 2004 and 2008. Methods for the NHBS surveys are described elsewhere.6,8,9 Briefly, NHBS used time-location sampling to recruit and survey men at public venues, street locations, and other venues where MSM were known to congregate. A formative research stage preceded sampling where key informants assisted in identifying venues and peak times of participant concentration. A roster of all possible venue-day-time periods was produced for each sampling venue. Sampling venues included cafes, bookstores, dance clubs, bars, social organizations, public parks and beaches, fitness clubs/gyms, and street locations in gay-identified neighborhoods. From this roster, a random sample of venues was drawn each month, and then from each venue, a venue-day-time was randomly selected and scheduled for sampling. Once in the field, staff adhered to standard operating procedures to systematically intercept men entering the venue or crossing a predetermined point in the sampling area. Verbal consent from eligible participants (men at least 18 years or older and residing in the 9-county Bay Area) was obtained during the sampling event. Participation in NHBS was anonymous.

Back to Top | Article Outline

SAP Outreach Sampling

STOP AIDS Project is a San Francisco-based nonprofit organization that develops and implements HIV prevention intervention programs designed to reduce risk behaviors in gay, bisexual, and transgender men at high risk for HIV. Since 1994, SAP has conducted serial cross-sectional surveys of MSM as part of the organization’s outreach efforts to develop and implement focused intervention campaigns. Details of SAP survey methods have been described elsewhere.7,10–12 Briefly, risk behavior surveys were conducted through short street-based intercept interviews among men recruited at gay-oriented community events (eg, street fairs and the annual Gay Pride Festival, etc), sex clubs, and, like the NHBS surveys, at gay-identified clubs and bars, gyms, and at street locations in gay-identified neighborhoods. A roster of outreach venues is organized by SAP staff, updated periodically to ensure that operating hours and venue locations are current. During outreach events, demographic and risk behavior data were collected using a standardized risk assessment questionnaire. Participant recruitment and enrollment were conducted by trained volunteers and staff. Sampling of outreach events occurred throughout the year. Data collection occurred within the context of a risk reduction conversation, using a standard survey instrument as a guide. Outreach contacts were told about the confidential nature of data collection but did not provide verbal consent because this conversation is part of program activities and data were not collected for research purposes. Participation in SAP surveys was anonymous.

Back to Top | Article Outline

Data Collection and Sexual Indicators

Although two separate instruments were used for each survey, questions about specific risk behaviors were similar in meaning and therefore comparable across surveys and survey years. One notable difference between surveys: the NHBS survey defined sex as oral and/or anal, whereas the SAP survey included anything they considered sex, such as mutual masturbation.

In both the NHBS and SAP surveys, respondents were asked their age, race and ethnic background, sexual orientation, city of current residence, prior HIV testing, dates of prior HIV testing, and the results of their last HIV test. Respondents were also asked questions about sex behaviors with male partners in the past 6 months including if they engaged in UAI with any of their partners, and whether they engaged in insertive or receptive UAI with any partner. Respondents were also asked the HIV status of their UAI partners. Together with the respondent’s self-reported status, we were able to determine if participants engaged in UAI with a potentially discordant partner (defined as a partner whose HIV status was different from or unknown to the respondent).

In addition to cumulative partner data within the past six months, the NHBS and SAP surveys collected partner-by-partner assessment of behaviors engaged with each partner (up to 3) along with the HIV status of the partner (if known). From partnership data, we determined the percent of respondents who reported high-risk UAI positioning behaviors with potentially discordant partners, behaviors that increased the risk of HIV transmission. High risk UAI was defined as HIV-positive respondents reporting insertive UAI with a partner who is HIV negative or whose status is unknown and HIV-negative respondents who reporting receptive UAI with a partner who is HIV positive or whose status is unknown. Estimates of high-risk UAI with potentially discordant partners provide a more precise indicator of potential HIV transmission.

Back to Top | Article Outline

Sample Population and Data Analysis

For this analysis, we included respondents who were 18 years or older, resided in San Francisco, and either identified as homosexual, gay, or bisexual or reported at least 1 male sex partner in the last 12 months before the survey, regardless of sexual identification. The samples included participants recruited from gay-identified clubs and bars, gyms, street locations in gay-identified neighborhoods (both NHBS and SAP), cafes, bookstores, churches, social organizations, public parks and beaches (NHBS only), and street fairs and the annual Gay Pride Festival (SAP only). We excluded respondents who were recruited from “sex clubs” from the SAP survey because these venues were not randomly selected in the NHBS surveys.

Between- and within-survey comparisons were assessed. Between-survey comparisons test the assumption that if survey methodologies are similar, both would detect similar changes in magnitude and direction of estimates across years. Ninety-five percent confidence intervals are calculated for all estimates based on a Poisson distribution. Comparison of categorical variables between surveys was assessed using a global χ2 test. Differences in median variables were assessed using the Wilcoxon ranks sums test. Comparisons of behavioral indicators by survey type and year were assessed using log-binomial regression adjusting for differences in demographic distributions (age, race, and sexual orientation). Statistical Analysis Software (SAS version 9.2) was used for all analyses.

Back to Top | Article Outline


A total of 3248 interviews were completed for the NHBS (n = 1624) and SAP (n = 1624) surveys combined. Table 1 shows demographic characteristics of each survey by year. Most respondents for both surveys were white, non-Hispanic, although more nonwhite respondents were recruited in the NHBS survey than the SAP survey (2004: 42% vs. 31%, P < 0.0001; 2008: 45% vs. 42%, P < 0.0001). In 2004, respondents recruited by the NHBS were slightly older (median age, 36 years vs. 34 years; P < 0.001), although in 2008, both surveys had similar age distributions. Most respondents from both surveys identified as gay or homosexual, with smaller proportions identifying as bisexual, heterosexual or other sexual preference.



Table 2 shows estimates of prior HIV testing and behavioral indicators. Prior HIV testing was consistently high for both surveys ranging from 88% to 97%, although the 2008 NHBS survey showed higher lifetime HIV testing than SAP (97% vs. 88%, P < 0.0001). The NHBS surveys reported lower recent testing (in the past 12 months) than the SAP surveys for both years (2004: 59% vs. 70%, P < 0.0001; 2008: 60% vs. 68%, P = 0.02). Fewer respondents from the 2004 NHBS survey self-reported their status as HIV negative than SAP (77% vs. 83%, P = 0.009), but in 2008, self-reported HIV status was similar for both surveys. Number of sex partners reported in the past 6 months differed between surveys for both years, with men recruited by NHBS less sexually active than men in the SAP surveys (2004: median partners, 2 vs. 5 [P < 0.0001]; 2008 median partners, 2 vs. 4 [P < 0.0001]). Roughly 15% to 17% of NHBS respondents reported abstinence in the past 6 months compared with 4% to 5% of respondents in the SAP surveys (both 2004 and 2008: P < 0.0001). Whereas less than 20% of respondents in NHBS in 2004 and 2008 reported 6 or more partners in the past 6 months compared with approximately 40% of SAP respondents in both years (both 2004 and 2008: P < 0.0001).



Estimates of any UAI (either insertive or receptive) with at least 1 partner in the past 6 months was significantly higher in the NHBS survey than in SAP survey in 2004 (40% vs. 36%, P = 0.03) and marginally significant in 2008 (44% vs. 38.0%, P = 0.08). Estimates of insertive UAI were similar for both surveys for 2004 (31% vs. 28%, P = 0.22) and 2008 (35% vs. 33%, P = 0.60). Nonsignificant differences for receptive UAI were found in 2004 (27% vs. 23%, P = 0.06), but in 2008, the NHBS survey had a significantly higher estimate of receptive UAI (29% vs. 18%, P < 0.0001).

In 2008, when HIV discordancy among partners could be assessed in both surveys, estimates of UAI (either insertive or receptive) with a potentially HIV-discordant partner were similar (12% vs. 12%, P = 0.87). Among HIV-positive respondents, the NHBS survey showed a higher percent of men reporting UAI with a potentially discordant partner (28% vs. 17%, P = 0.03). Estimates of insertive UAI with a potentially discordant partner for both surveys were low and similar (NHBS: 13% vs. 11%, P = 0.45). Among HIV-negative respondents, estimates of UAI with a potentially discordant partner were similar between surveys (10% vs. 12%, P = 0.34), as were estimates of receptive UAI with a potentially positive partner (5% vs. 4%, P = 0.85).

Table 3 shows results of within-survey comparisons by year. For the NHBS survey, all indicators between 2004 and 2008 were similar, with insertive UAI slightly higher in 2008 (31% vs. 35.0%, P = 0.04). We did note differences in several indicators between 2004 and 2008 in the SAP surveys. Lifetime HIV testing was significantly lower in 2008 (97%–88%, P < 0.0001), as was the percent of men who self-reported as HIV negative (83%–77%, P = 0.02) and the percent of men who reported receptive UAI with a potentially discordant partner (23%–18%, P = 0.02).



Back to Top | Article Outline


This comparison of two behavioral surveys of MSM, one a probability sample and the other a community-based survey using convenience sampling, found broadly similar estimates of several indicators of UAI, suggesting that data drawn from community-based outreach surveys can be useful in generating plausible estimates of sexual risk behaviors among MSM in San Francisco. Both surveys had comparable estimates of any UAI (range, 36%–44%), insertive UAI (28%–35%), and UAI with a potential discordant partner (both roughly 12%). There was also agreement of indicators measuring high-risk UAI with potentially discordant partners, with 11% to 13% of HIV-positive men in these surveys reporting insertive UAI with a potentially discordant partner and between 4% and 5% of HIV-negative men engaging in receptive UAI with a potentially discordant partner.

These estimates are similar to findings obtained from a 2001 population-based telephone survey conducted in California where researchers found that 38% of statewide MSM reported any type of UAI and 11% of men reported UAI with a serodiscordant partner.13 Also, our estimate of insertive UAI is consistent with a meta-analysis that showed approximately 13% of HIV-diagnosed men reported UAI with an at-risk partner.14 Measures of UAI with discordant partners are important in understanding the potential for HIV transmission, especially when considering the growing evidence for increasing use of seroadaptive behaviors among MSM.15,16

These data suggest that the level of potentially risky sex among MSM has not changed considerably since 2001, although reductions in new HIV diagnoses have been documented in San Francisco during this time.17 Such a paradox of stable behavioral risk during a time of decreasing infections suggests that structural level changes in how San Francisco manages its HIV epidemic, such as a reduction of community viral load through increasing access to antiretroviral therapy, may have had a positive impact in new diagnoses, although we note that there is considerable debate on measuring the population effects of antiretroviral therapy.18,19 Moreover, we recognize that some indicators may not precisely measure HIV transmission events because of the imprecision of how individual data are collected (eg, correct knowledge of HIV status of partners). Therefore, the lack of correlation between stability of high-risk behaviors and the reductions of HIV infections should be interpreted with caution.

We do note that there were differences in some estimates in this comparison. Although prior HIV testing was high in both surveys, recent HIV testing was higher for respondents in the SAP surveys. Moreover, the level of sexual activity differed by surveys. SAP respondents reported greater sexual activity although the differences in how sex was defined between the surveys may account for this difference. Although receptive UAI was found to be significantly lower among SAP respondents, all other UAI indicators in 2008, especially those measuring potentially discordant partnerships, did not differ statistically.

The similarity of these key behavioral estimates, in our view, provides a method for survey validation and corroboration, with each survey methodology having specific strengths and weaknesses. One strength of the NHBS methodology is its sampling rigor and its ability to generate representative estimates that may be more stable over time than community surveys. Another strength is that the NHBS protocol included HIV testing (SAP does not). Together with self-reported status, results from HIV test can be used to estimate the number of respondents with unrecognized infection. For example, the 2004 San Francisco NHBS survey found that 16% of HIV-positive MSM were unaware of their infection.20 A principal asset of community-based surveys is that data can be collected continuously. In San Francisco, the SAP surveys are conducted annually so data can be analyzed quickly and used to inform programs of changes in community behavior in a timely fashion.7 Rapid assessment of behavioral surveillance using community data can be useful at the local level to assist in prevention planning activities such as prevention-for-positive programs21 and treatment-based prevention interventions such as preexposure prophylaxis.22 Moreover, community surveys can also be used to triangulate with other data to inform other surveillance programs, such as STD Control Branch to understand syphilis screening coverage among MSM in San Francisco (especially in the context of increasing syphilis infections).23

We recognize several limitations in our assessment. First, this comparison used unweighted estimates because the SAP survey is not a probability sample and sampling weights are not known; therefore, estimates drawn from this type of sampling are often biased. Second, the SAP survey relies heavily on community events to recruit participants, such as the annual Gay Pride celebration, and such events may be prone to variations in attendance because of weather, economic conditions, or other factors. Third, the variance of estimates drawn from convenience sampling cannot be adjusted to take into account the survey design; thus, standard errors are often inflated. Moreover, the NHBS survey has the advantage of being a national survey from which samples can be pooled and used for national estimates of behavioral indicators.2 Fourth, because the SAP survey is integrated with a risk reduction counseling session (including condom and lube distribution), participants may be influenced by prevention messages and behaviors elicited from these interviews may be biased. Lastly, we recognize that San Francisco is well suited to conduct community surveys because of the size and integration of the community. Municipalities with small or highly stigmatized populations, and those with few venues where MSM congregate may have challenges in implementing community surveys. However, in many cities in the United States and elsewhere, community outreach has become an important model by which HIV prevention and care programs are monitored and evaluated, thus providing an opportunity to expand behavioral surveillance (especially in areas that do conduct NHBS).

In conclusion, community surveys offer several advantages, and estimates drawn from such surveys can be useful to augment routine behavioral surveillance activities, especially between cycles of the population-based NHBS. Such integration of community-level behavioral data has been successful in gauging the epidemic in San Francisco.

Back to Top | Article Outline


1. Garrett GP, Garcia-Calleja JM, Rehle T. Behavioral data as an adjunct to HIV surveillance data. Sex Transm Infect 2006; 82 (suppl 1): i57–i62.
2. Gallagher KM, Gallagher KM, Lanksy A, et al. Behavioral surveillance among people at risk for infection in the U.S.: The National HIV Behavioral Surveillance System. Public Health Rep 2007; 122 (suppl 1): 33–38.
3. Jaffe HW, Valdiserri RO, De Cock KM, et al. The reemerging HIV/AIDS epidemic in men who have sex with men. JAMA 2007; 298: 2412–1414.
4. Sullivan P, Hamouda O, Delpech V. Reemergence of the HIV epidemic among men who have sex with men in North America, Western Europe, and Australia, 2006–2005. Ann Epidemiol 2009; 19: 423–431.
5. Lanksy A, Sullivan PS, Gallagher KM, et al. HIV behavioral surveillance in the U.S.: A conceptual framework. Public Health Rep 2007; 122: 16–23.
6. MacKellar D, Gallagher K, Finlayson T, et al. Surveillance of HIV risk and prevention behaviors of men who have sex with men—A national application of venue-based, time-space sampling. Public Health Rep 2007; 122 (supp 1): 39–47.
7. Scheer S, Kellogg TA, Klausner JD, et al. HIV is hyperendemic among men who have sex with men in San Francisco: 10-Year trends in HIV incidence, HIV prevalence, sexually transmitted infections and sexual risk behavior. Sex Transm Infect 2008; 84: 493–498.
8. Ferreira LOC, Oliveira ES, Raymond HF, et al. Use of time-location sampling for systematic behavioral surveillance of truck drivers in Brazil. AIDS Behav 2008; 12: S32–S38.
9. Raymond HF, McFarland W. Racial mixing and HIV risk among men who have sex with men. AIDS Behav 2009; 13: 630–637.
10. Chen SY, Gibson S, Weide D, et al. Unprotected anal intercourse between potentially HIV-serodiscordant men who have sex with men, San Francisco. J Acquir Immune Defic Syndr Hum Retrovirol 2003; 33: 166–170.
11. Katz MH, Schwarcz SK, Kellogg TA. Impact of highly active treatment on HIV seroincidence among men who have sex with men: San Francisco. Am J Public Health 2002; 92: 388–394.
12. Vaudrey J, Raymond HF, Chen S, et al. Indicators of use of methamphetamine and other substances among men who have sex with men, San Francisco, 2003–2006. Drug Alcohol Depend 2007; 90: 97–100.
13. Xia Q, Osmond D, Tholandi M, et al. HIV prevalence and sexual risk behaviors among men who have sex with men: Results from a statewide population-based survey in California. J Acquir Immune Defic Syndr 2006; 41: 238–245.
14. Crepaz N, Marks G, Liau A, et al. Prevalence of unprotected anal intercourse among HIV-diagnosed MSM in the United States: A meta-analysis. AIDS 2009; 23: 1617–1629.
15. Golden MR, Stekler J, Hughes JP. HIV serosorting in men who have sex with men: Is it safe? J Acquir Immune Defic Syndr Hum Retrovirol 2008; 49: 212–218.
16. McConnell JJ, Bragg L, Shiboski S. Sexual seroadaptation: Lessons for prevention and sex research from a cohort of HIV-positive men who have sex with men. PLoS One 2010; 5: e8831.
17. Das M, Chu PL, Santos G-M, et al. Decreases in community viral load are accompanied by reductions in new HIV infections in San Francisco. PLoS One 2010; 5: e11068.
18. Smith MK, Powers KA, Muessig KE, et al. HIV treatment as prevention: The utility and limitations of ecological observation. PLoS Med 2012; 9: e1001260.
19. Cohen MS, Muessig KE, Smith MK, et al. Antiviral agents and HIV prevention controversies, conflicts, and consensus. AIDS 2012; 26: 1585–1598.
20. Raymond HF, Bingham T, McFarland W. Locating unrecognized infections among men who have sex with men: San Francisco and Los Angeles. AIDS Educ Prev 2008; 20: 408–419.
21. Rose CD, Courtenay-Quirk C, Knight K, et al. HIV interventions for providers study: A randomized controlled trial of a clinician-delivered HIV risk-reduction for HIV-positive people. J Acquir Immune Defic Syndr 2010; 55: 572–581.
22. Koblin BA, Mansergh G, Frye V, et al. Condom-use decision making in the context of hypothetical pre-exposure efficacy among substance-using men who have sex with men: Project MIX. J Acquir Immune Defic Syndr 2011; 58: 319–327.
23. Zetola NM, Klausner JD. Syphilis and HIV infection: An update. Clin Infect Dis 2007; 44: 1222–1228.
© Copyright 2013 American Sexually Transmitted Diseases Association