Secondary Logo

Journal Logo

Editorial

Evaluation of the Center for Disease Control and Prevention’s HIV Behavioral Surveillance of Men Who Have Sex With Men: Sampling Issues

Pollack, Lance M. PhD*; Osmond, Dennis H. PhD; Paul, Jay P. PhD*; Catania, Joseph A. PhD*

Author Information
doi: 10.1097/01.olq.0000175419.02839.d6
  • Free

DESPITE PREVENTION EFFORTS AND SIGNIFICANT reductions in risk behavior, men who have sex with men (MSM) have experienced a continuing HIV epidemic in the United States, with HIV incidence rates estimated at 1% to 2% over the past 20 years.1,2 In 1996, HIV prevalence in MSM urban communities was estimated at 22%,1 and this increased to approximately 28% by 2003.3 In this regard, there has been a call for increased monitoring of HIV risk behaviors and HIV prevalence/incidence among MSM populations in the United States.2

The Centers for Disease Control and Prevention (CDC) has responded with a number of ongoing monitoring initiatives in collaboration with state health departments, including a major effort to use biannual venue-based time–space sampling surveys that will monitor HIV prevalence and risk behavior among adult MSM in the United States.4 This effort addresses the challenge of obtaining representative samples of MSM, a rare population from a sampling perspective, at relatively less cost compared with other methods (e.g., see2,5).

Because this monitoring effort rests on methodology that has not been fully examined, particularly about issues of representativeness, we examine here data from a recently conducted household probability survey of MSM in San Francisco that tests the efficacy of sampling from “gay venues” for varying time periods. Community-based household probability samples provide relatively unbiased data on HIV estimates at the level of urban communities in particular1,6,7 and, therefore, offer a valid point of comparison. However, community-based household probability surveys are more costly than venue-based studies, so there is good reason to consider this alternative approach. What is unclear is if the CDC’s recommended protocol for venue-based sampling is the most efficacious and cost-effective approach.

The protocol guidelines for the National HIV Behavioral Surveillance of men who have sex with men (NHBS-MSM) specify that eligible venues are those attended by MSM for purposes other than receiving health or social services or HIV/sexually transmitted disease (STD) testing. Eligible venues can include bars, clubs, and other retail establishments as well as social and religious organizations, high-traffic street locations and other public areas where MSM may congregate, and special events like festivals, raves, and circuit parties. Given the large number of cities to be involved, the lengthy sampling period proposed (12 months initially and then 9 months thereafter) and the wide range of venues to be sampled, it is clear that a great deal of time and resources will be devoted to this effort.

The sampling strategy advocated by the NHBS-MSM is grounded in empirical research on sampling rare populations using a location approach.5 This approach uses time–space sampling methodology. Recent developments in time–space sampling have sought to systematize the sampling process to obtain reproducible random samples of venue attendees that can be weighted to obviate some sampling biases. This approach involves a multiple step sampling procedure.8,9 In brief, the universe of potential venues is identified from which individual venues are randomly selected. Within each venue, day–time periods are identified and surveyed for level of traffic and then are randomly selected for inclusion in the sampling frame, often stratified by level of traffic. Within the selected day–time periods, systematic sampling techniques are used to determine which venue attendees to approach about participation in the study. This approach has been used primarily in large urban areas to obtain samples of young and Latino MSM.10–12

Among the issues to be addressed in the current study are: 1) the proportion of the adult MSM population potentially covered by the NHBS-MSM following the protocol as written, 2) whether the proposed 9-month sampling period is too long, 3) whether the full panoply of venues specified in the protocol (e.g., bars, clubs, public areas, and special events) is necessary to accomplish the CDC’s goals, and 4) whether the resulting sample of venue attendees is actually representative of the adult MSM population in terms of HIV prevalence, risk behavior, and other characteristics.

Methods

Procedures

The 2002 San Francisco cohort of the Urban Men’s Health Study (UMHS 2002) is a household probability sample of adult MSM obtained using a random-digit dial (RDD) sampling approach. The data are weighted to adjust for probability of selection differences. A detailed sampling report is available from the first author. The sampling methodology was based on prior work.1,6,13,14 Between May 2002 and January 2003, we sampled from 13 zip codes in San Francisco where the majority of adult MSM reside (estimated 87% in 1996) and completed 879 interviews (73.7% of eligible households). Sample development and interviewing were conducted by Westat Corporation. Participants were provided informed consent with procedures approved by Institutional Review Boards at the University of California San Francisco and Westat, Inc. Advance letters were sent to sampled households with obtainable addresses (49% of the RDD sample). The letter provided a toll-free number for respondents to call if they had questions. On request, Westat provided a mailed letter with information about the study accompanied by California’s experimental subject’s bill of rights and a one-page memo explaining the scope of the certificate of confidentiality. Men who reported same-gender sex since age 14 and/or who self-identify as gay or bisexual were eligible for the study, an approach that yields closeted and less gay-identified men.1,7 If a household had more than one eligible, one was randomly selected for interview. Both the screener and the interview were offered in English and Spanish. Interviews averaged 39 minutes and included assessments of a wide range of HIV-relevant topics. Additional data were collected by a paper-and-pencil mail survey, which was completed by 81% of the respondents.

Measures

The study instruments are available from the first author. The telephone survey topics included HIV and STD testing, visitation of venues, sexual activity in the past 12 months, sexual beliefs and attitudes, childhood sexual abuse, emotional well-being, demographics, and migration history. Mail survey topics included participation in clubs or organizations, use of support services, degree of “outness” about sexual orientation, sexual beliefs and attitudes, alcohol and substance use, and emotional well-being. Measures for this article are from the telephone interview unless indicated otherwise.

Demographics

Demographic measures included self-reported age, race/ethnicity, education, income, employment status, and whether the respondent had a male domestic partner/spouse. In the mail survey, respondents were asked how many family members, friends, and neighbors they were “out” to about their sexual orientation on a five-point scale ranging from all to none.

Migration

Respondents were asked to report how many years total they had lived in San Francisco, how many places they had lived since first having same-gender sex after their 12th birthday, and how many moves they had made since turning 18. Moves within the same city or town were not counted as a move or new place.

Venue Attendance

Four separate questions asked the number of times in the past 12 months the respondent visited a: 1) bar or club that primarily serves a gay crowd (including dance or nightclubs); 2) bathhouse or sex club with primarily a gay clientele; 3) t-room, adult bookstore, X-rated theater, park, beach, or other cruising area; and 4) gay “circuit party” in the United States or some other country. After a nonzero response, respondents were also asked the number of months since their last visit to that venue.

HIV and Sexually Transmitted Disease Testing

Respondents were asked to report the month and year of their first and most recent HIV antibody test. Research has shown that self-reported HIV serostatus by adult MSM is extremely accurate.15 They were also asked if in the past 12 months a healthcare worker had told them they have an STD other than HIV.

Sexual Development

Age when respondent first had same-gender sex, when subtracted from current age, yields the number of years since same-gender sexual debut. Respondents were also asked if they had experienced childhood sexual abuse, i.e., whether they were “forced or frightened by someone into doing something sexually that you did not want to do” before age 18.16

Sexual Behavior

For the period covering the past 12 months, respondents are asked to report who they have sex with (men or women or both), how many male sex partners they had, and the number of male sex partners with which they engaged in specific sexual behaviors, including insertive and receptive anal sex without a condom. Respondents were also asked detailed questions about their four most recent male sex partners in the past year, including sexual behavior in the past 12 months (number of times engaged in insertive and receptive anal intercourse both overall and while using a condom), the partner’s HIV serostatus and demographic characteristics, and which one of the partners, if any, is the respondent’s primary partner, i.e., a man “you are currently in love with or feel a special commitment to.” Based on the partner-specific data, high-risk sex is operationalized as an HIV-positive respondent who has unprotected insertive anal intercourse with an HIV-negative or sero-unknown partner or a nonpositive respondent who has unprotected receptive anal intercourse with an HIV-positive or sero-unknown partner. The partner-specific data can also be combined with data from previous broader questions about sexual behavior to define unprotected anal intercourse with a nonprimary partner, i.e., reporting having unprotected insertive or receptive anal intercourse with more than one male sex partner or with a male sex partner who is not the current primary partner.

Substance Use

If a respondent reported ever injecting recreational drugs, he was asked the month and year he last injected. In the mail survey, respondents reported the number of times in the past 6 months they had used each of 13 types of substances: “downers,” marijuana/hashish, poppers, crack cocaine, cocaine other than crack, methamphetamines, other “uppers,” ecstasy, Ketamine, GHB/GBL, other “club drugs,” psychedelics/hallucinogens including LSD and PCP, and heroin or other opiates. Nonintravenous drug use was defined as the maximum of these 13 reports. Also in the self-administered questionnaire, respondents were asked how often they had a drink containing alcohol in the past 6 months and usual number of drinks in a sitting in the past 30 days. Alcohol use was defined as infrequent/occasional if the respondent did drink but less than once a week, as frequent light drinking if he drank at least once a week but had fewer than five drinks at a sitting, and as frequent heavy drinking if he drank at least once a week and had five or more drinks at a sitting.17

Mental Health

Depression was assessed using a shortened eight-item version of the Center for Epidemiologic Studies Depression Scale (CES-D).18 A separate item asked respondents to rate their overall life satisfaction on a six-point scale ranging from very satisfied to very dissatisfied.

Statistical Analysis

The data analysis is designed to evaluate how well the adult MSM population of San Francisco (operationalized as the total UMHS 2002 sample) can be represented by a venue-based sample (operationalized as respondents who report attending selected venues). Because the focus is on sample performance rather than statistical inference, evaluation of the underlying sample distributions are performed with unweighted data. To evaluate whether subgroups of adult MSM would be over- or underrepresented in a venue sample, statistical relationships between venue attendance and other variables are assessed by computing a chi-squared statistic. How representative a given venue sample would be of the adult MSM population is assessed by comparing point estimates for the subsample of venue attendees with point estimates for the total sample in which the latter is treated as the expected proportion and a Z-score is computed. For variables composed of more than two groups (e.g., age, race/ethnicity, etc.), critical P values for determining the significance of each Z-score were calculated using a modified Bonferroni procedure.19

Results

Coverage

Table 1 presents data on the percent of the total sample that would be covered by sampling each venue. Sampling bar/club venues even for 1 month would potentially cover over 60% of the adult MSM population and would potentially reach almost 82% if sampling occurred for 12 months. Coverage in all other venues is quite poor even after 12 months of fielding (<35%). Assuming all four venues are sampled, the initial 12-month sample period suggested by the CDC would result in nearly 88% coverage, whereas the subsequent 9-month sample periods would achieve 86% coverage.

T1-11
TABLE 1:
Cumulative Percent Coverage of Total Sample by Length of Sample Period

Sample Period

If all four venues are sampled, maximal coverage appears to be reached by a 6-month sample period (see Table 1). Extending fielding to a 9-month or 12-month period increases coverage by only 1.3% and 2.6%, respectively. Moreover, HIV and risk behavior prevalence estimates do not change appreciably as the sample period lengthens beyond 6 months (see Table 2). Regardless of the sampling period used, venue attendees are significantly more likely to report risk behavior in the past year, but are not more likely to be HIV-positive, than nonattendees. Sample periods of 1 to 2 months show attendees less likely to be HIV-positive, whereas longer sample periods show them more likely, but in every case, the difference is statistically nonsignificant.

T2-11
TABLE 2:
Prevalence of HIV and Risk Behavior by Venue Attendance for Varying Sample Periods

Range of Venues

Relative to a bar/club-only sample, combining venues does not provide more than an average 6% gain in coverage for any given sample period (see Table 1). Like the combined venue sample, in bar/club venues coverage is little improved by sampling beyond 6 months. Only another 1.0% is gained by extending fielding to 9 months and 2.5% if extended to 12 months. Furthermore, comparisons of men who would be covered by a bar/club sample alone with the additional sample that would be obtained if the other three venue types were included (assuming a 6-month sample period) show no significant differences (all P values >0.10) in HIV prevalence or prevalence of sexual risk behavior (not shown).

Representativeness

We then constrained the venue sample to bars/clubs only using a 6-month sampling period and examined its representativeness by comparing it with the total UMHS 2002 sample (see note in Table 3). As might be expected, a bar/club-only sample would overestimate the level of drinking. Furthermore, because the bar/club sample overrepresents men under age 50, there are associated overestimates of the proportion of men who are recent in-migrants and men who more recently had their same-gender sexual debut (P <0.0125). Retired/disabled MSM are also underrepresented in a bar/club sample (P <0.01). There is no difference between the venue sample and the total sample in HIV prevalence or prevalence of high-risk sexual behavior (P >0.05). However, bar/club sampling would result in a significant overestimation of the prevalence of risk behaviors based on less restrictive definitions (i.e., unprotected insertive or receptive anal intercourse) as well as the proportion of men with male primary sexual partners (P <0.05). When the analysis is restricted to MSM age 18 to 49 years, differences between the total sample and the 6-month bar/club sample in prevalence of risk behavior are no longer statistically significant (see note in Table 4).

T3-11
TABLE 3:
Comparison of Attendees With Nonattendees of Gay Bars/Nightclubs/Dance Clubs Based on a 6-Month Sample Period and With the Total Sample
T3A-11
TABLE 3:
(Continued)
T4-11
TABLE 4:
Comparison of Attendees With Nonattendees of Gay Bars/Nightclubs/Dance Clubs Based on a 6-Month Sample Period Among Respondents Under Age 50

As suggested by Berry et al.,20 we also compared 6-month bar/club attendees with nonattendees to identify subgroups of MSM that would be over- or underrepresented by such a venue sample (see Table 3). We found no significant difference in HIV prevalence between attendees and nonattendees, but attendees are more likely to report having an STD in the last 12 months. Attendees are significantly more likely to report sexual risk behavior regardless of the definition of risk. These differences persist even when the analysis is restricted to MSM under age 50 (see Table 4). Attendees are more likely to be substance users but less likely to inject drugs. Attendees are somewhat younger and better educated, are more likely to report full-time employment, and have higher incomes. Attendees are more recent residents of the city but have not been more mobile (across cities). In general, attendees are temporally more proximal to their same-gender sexual debut, more sexually active, and more likely to be in a primary sexual relationship and have both primary and secondary sexual partners. Attendees are also more likely to be “out” to friends and neighbors about their sexual orientation. Attendees do not differ from nonattendees in reporting a history of childhood sexual trauma, and although they do not report more depressive symptoms, they do express higher life satisfaction.

Discussion

The current study examined the CDC’s NHBS-MSM protocol, which uses a time–space sampling methodology as a basis for generating HIV/behavioral monitoring surveys for MSM in the United States. The current study suggests that the NHBS-MSM protocol may be satisfactory for sampling urban MSM within defined limits, but could be conducted at significantly less cost without diminishing representativeness by constraining the sample to bar/club venues and 6 months of fielding work.

Because the Census does not collect data on sexual orientation, we used a household probability survey of MSM as the basis for judging the representativeness of a venue-based sample. Past work indicates that household probability surveys of MSM provide accurate HIV prevalence estimates at the community level and reflect the demographic realities of urban MSM.1,6,7 Such surveys have proved costly, particularly when oversamples of minority MSM are sought. Thus, it is important to consider more cost-effective methods like a venue sampling approach. However, the NHBS-MSM protocol assumes that the gap between the venue-based sample and the community-based sample will be closed by sampling diverse venues (e.g.,10,11). The current work suggests that sampling circuit parties, sex clubs, and cruising areas will not necessarily improve on data obtained from sampling MSM bars/clubs alone. The CDC protocol also intends to sample MSM-oriented retail businesses and special events and perform street intercepts. It is an open empirical question as to whether these venues will make the obtained sample more closely approximate the adult MSM population.

Other possible limitations and caveats concerning the NHBS-MSM protocol should also be noted. Given that a 6-month bar/club sample accounts for 79% of the total UMHS 2002 sample, few differences are found when comparing the venue sample with the total community-based sample. Importantly, we find no significant differences in estimated prevalence of HIV infection or HIV-related risk behavior. However, in considering the differences in estimates, statistical significance should not be the only criterion. It is also relevant to consider the magnitude of the differences in terms of the numbers of people that may be misidentified. For example, with an estimated adult MSM population of 46,000 in San Francisco,6 the 1.3% point difference in the estimated prevalence of high-risk behavior between the total sample and the bar/club sample would result in an overestimation of some 600 men. Extrapolating to the state of California, with an estimated adult MSM population somewhere between 841,000 and 868,000,21 the overestimation of high-risk sex would exceed 11,000 cases. If one assumes that program emphases and funding allocations will be based on the observed prevalence of risk behavior obtained from venue-based samples, the fiscal impact of the overestimations could be significant, even if the absolute size of the overestimation is not.

The NHBS-MSM protocol also was developed to obtain samples of younger MSM (ages 18–20), older MSM (age 40+), nonwhite MSM, and closeted MSM. Our data indicate venue sampling may underrepresent older age groups that continue to be significant factors in the HIV epidemic.22 They may also underrepresent closeted men. However, research to date does not suggest that closeted men contribute significantly to HIV and risk behavior estimates at the community level,1 although they could contribute in unspecified ways to particular subpopulations of MSM (e.g., black men).

It is important to note that venue attendance was unrelated to race/ethnicity. However, the fact that in-migration patterns differ between the total sample and the 6-month bar/club sample suggests caution. In-migration substantially affects the demographics of MSM urban communities by infusing a large white population into the indigenous MSM community.1,6 Differences in venue attendance by recent in-migrants over time could distort race/ethnic representativeness if the underlying pattern of race/ethnic in-migration begins to shift. That is, the recent in-migrants for some period of time may be less representative of the community as a whole until a new equilibrium is reached.

Moreover, other factors that also can affect the distribution of HIV were found to differ across sample frames. Bar/club attendees, in addition to being younger, more recent urban in-migrants, and less closeted, are also better educated and more likely to be employed than nonattendees. Furthermore, attendees differ from nonattendees in terms of mental health and sexual development factors and about types of sexual relationships. These are all factors that may impact sexual risk behavior, which also differed significantly between attendees and nonattendees. The possibility that these differences may grow or shrink over time argues for the need to provide alternative methods of validating the trends identified using the NHBS-MSM protocol.

Thus, to safeguard against bias in monitoring time trends (e.g., from changes over time in venue attendance, population migration changes, and other factors that may affect venue coverage), we recommend adding periodic household probability-based community-level surveys to the protocol. To ameliorate cost, we propose that these validation studies be conducted at 5-year intervals in a subsample of urban areas to test the time trends developed from the venue-based sample program. Even with the addition of validating procedures, it will be incumbent on everyone reporting data obtained using the NHBS-MSM protocol to acknowledge that the sample is of venue attendees, which is a subset of the adult MSM population.

The current study has several limitations. First, UMHS 2002 does not examine the efficacy of sampling from street intercepts and MSM-oriented organizations and businesses, but it does provide the ability to assess how a well-executed, well-funded time–space sample based on “typical” gay venues would perform. Furthermore, the present findings may not be generalizable beyond San Francisco. However, data from the 1997 Urban Men’s Health Study1 show that prevalence of bar/club attendance in the past year was comparable in Chicago (90.3%), San Francisco (85.6%), New York (83.4%), and Los Angeles (82.3), which suggests our current data may be generalizable to other large urban areas. UMHS 1997 data also show similar comparability in attendance of health and social clubs and religious organizations (Chicago 66.4%, New York 61.2%, Los Angeles 60.5%, San Francisco 58.1%), indicating that the addition of service/retail venues to the sample frame would not affect generalizability. Finally, prior mapping work of MSM communities in 26 major urban centers of the United States13 identified only one city (Detroit) that would not have the physical and cultural clustering of MSM needed for proper execution of time–space sampling.8,9 Nevertheless, to the extent that smaller cities have MSM populations that maintain a relatively low public profile, venue sampling may yield data that is less representative of that population.

On a related note, we evaluated differences between the population (represented by the total UMHS 2002 sample) and the venue sample with recruitment procedures held constant. That is, because data for the venue sample are from the subsample of respondents reporting venue attendance, by definition, the “venue” sample has the same variations in recruitment as the “community” sample. Comparison of a community-based sample with an actual venue-based sample would include such recruitment-related variations that may result in larger differences than are observed in this study.

Factors that can negatively affect recruitment of venue samples include not having a complete list of venues, not sampling a large enough subset of venues, not sampling a sufficiently diverse set of venues, having too short a sampling period, not sampling enough day–time periods at each selected venue, and not sampling a sufficiently diverse (in terms of level of traffic) set of day–time periods at each venue. These factors may combine to yield a biased sample of venue attendees primarily because infrequent attendees of those venues are less likely to be included in the sample. Previous research indicates that infrequent attendees of public sex environments are less likely to engage in risky sexual behavior,23 and that “closeted” MSM (i.e., MSM who do not reveal their sexual orientation to others and typically do not frequent venues associated with MSM) are less likely to be HIV-positive.1 Thus, venue samples of MSM that exclude infrequent attendees may yield inflated estimates of HIV prevalence (whereas the current analysis indicates a well-executed sample will yield an accurate estimation) and even greater overestimations for prevalence of risk behavior.

The current study also does not address the issue of total survey error,24 which is important to consider with regard to sampling protocols and recruiting procedures developed over the course of the AIDS epidemic but which have never been thoroughly assessed. Total survey error (sampling error plus response error) is crucial to consider given the varied results to date on venue-based sampling of MSM. For instance, past studies have reported screening rates that have varied from 61% to 88% and survey completion rates varying from 59% to 88%.8,10,12 Future evaluations should examine this key issue in understanding the quality of the NHBS-MSM estimates. One advantage of following the recommendations noted previously is that the resources conserved could be devoted to increasing cooperation rates and survey response. Moreover, sampling fewer types of venues allows one to increase the number of day–time periods sampled, which in turn may increase the representativeness of the sample data through increased coverage.

Although issues of efficiency (e.g., sample period, choice of venues, response rates) vary by research protocol, the issue of representativeness is one that generalizes to the use of any time–space sampling approach. A fully supported and realized sample allows one to make statistical inferences about the population of venue attendees from which the sample was drawn. However, in the absence of data from probability samples, which is often the case with rare and hard-to-reach populations like MSM, researchers will seek to extrapolate from the sample to a larger population of whom the venue attendees are a subset. Thus, whether a sample of adult MSM who attend certain venues is representative of the population of adult MSM (either in a particular city or urban MSM generally) is a significant concern. In the case of MSM, assessing representativeness is even more difficult because U.S. Census information is not available and previous research indicates extensive migration by subgroups of adult MSM.1 However, even when demographic information is available for a given population, such descriptive information is necessary but not sufficient to evaluate representativeness. All the factors that may affect the health or policy issue in question must be investigated. Until validation with Census data, surveillance information, or probability-based sample data can be achieved, extrapolations from venue samples to larger populations must be viewed very cautiously, perhaps even skeptically. Thus, anyone planning to use venue sampling for this extrapolative purpose should strongly consider whether such validation will be possible, and if not, whether a venue-based approach would yield results with any use.

In summary, the current investigation argues for pursuing the NHBS-MSM protocol with validity checks based on household probability samples conducted at infrequent intervals. Reducing the types of venues and the sampling window has practical implications for the quality of the NHBS-MSM monitoring system. Future work needs to address the issues of total study error about the NHBS-MSM protocol.

References

1.Catania JA, Osmond D, Stall RD, et al. The continuing HIV epidemic among men who have sex with men. Am J Public Health 2001; 91:907–914.
2.Catania JA, Morin SF, Canchola J, Pollack L, Chang J, Coates TJ. US priorities—HIV prevention. Science 2000; 290:717.
3.Catania JA, Osmond D. HIV prevalence and risk behavior among men who have sex with men in the California Health Interview MSM survey and the Urban Men’s Health surveys I and III. Paper presented at the State Office on AIDS; January 2004; Sacramento, CA.
4.Centers for Disease Control and Prevention, DoHAP-SaE, Behavioral and Clinical Surveillance Branch. National Surveillance of HIV Risk and Prevention Behaviors of Men Who Have Sex With Other Men: Protocol Guidelines. Atlanta: Centers for Disease Control and Prevention, 2003.
5.Kalton G. Sampling considerations in research on HIV risk and illness. In: Ostrow DG, Kessler RC, eds. Methodological Issues in AIDS Behavioral Research. New York: Plenum Press, 1993:53–74.
6.Catania JA, Canchola JA, Pollack LM, Chang J. Understanding survey sample demographic characteristics of men who have sex with men. American Statistical Association Proceedings, 2001.
7.Mills TC, Stall R, Pollack L, Binson D, Canchola J, Catania JA. Health-related characteristics of men who have sex with men: A comparison of those living in ‘gay ghettos’ with those living elsewhere. Am J Public Health 2001; 91:980–983.
8.Muhib FB, Lin LS, Stueve A, et al. A venue-based method for sampling hard-to-reach populations. Public Health Rep. 2001; 116(suppl 1):216–222.
9.MacKellar D, Valleroy L, Karon J, Lemp G, Janssen R. The Young Men’s Survey: Methods for estimating HIV seroprevalence and risk factors among young men who have sex with men. Public Health Rep 1996; 111(suppl 1):138–144.
10.Valleroy LA, MacKellar DA, Karon JM, et al. HIV prevalence and associated risks in young men who have sex with men. Young Men’s Survey Study Group. JAMA 2000; 284:198–204.
11.Stueve A, O’Donnell LN, Duran R, San Doval A, Blome J. Time–space sampling in minority communities: Results with young Latino men who have sex with men. Am J Public Health 2001; 91:922–926.
12.Diaz RM, Ayala G, Bein E, Henne J, Marin BV. The impact of homophobia, poverty, and racism on the mental health of gay and bisexual Latino men: Findings from 3 US cities. Am J Public Health 2001; 91:927–932.
13.Binson D, Moskowitz JT, Mills TC, et al. Sampling men who have sex with men: Strategies for a telephone survey in urban areas in the United States. Proceedings of the Section on Survey Research Methods American Statistical Association, August 4–8 1996; 1:68–72.
14.Blair J. A probability sample of gay urban males: The use of two-phase adaptive sampling. Journal of Sex Research 1999; 36:39–44.
15.Osmond DH, Catania JA, Pollack L, et al. Obtaining HIV test results with a home collection test kit in a community telephone sample. J Acquir Immun Defic Syndr Hum Retrovirol 2000; 24:363–368.
16.Paul JP, Catania JA, Pollack L, Stall R. Understanding childhood sexual abuse as a predictor of sexual risk-taking among men who have sex with men: The Urban Men’s Health Study. Child Abuse Negl 2001; 25:557–584.
17.Cahalan D. Problem Drinkers: A National Survey. San Francisco: Jossey-Bass, 1970.
18.Radloff L. The CES-D Scale: A self-report depression scale for research in the general population. Applied Psychological Measurement 1977; 1:385–401.
19.Jaccard J, Wan CK. LISREL Approaches to Interaction Effects in Multiple Regression. Thousand Oaks, CA: Sage Publications, 1996.
20.Berry S, Duan N, Kanouse DE. Use of probability versus convenience samples of street prostitutes for research in sexually transmitted diseases and HIV risk behaviors: How much does it matter? In: Warnecke R, ed. Health Survey Research Methods Conference Proceedings. Hyattsville, MD: Department of Public Health and Human Services, 1996:93–97.
21.Catania JA, Pollack LM, Canchola JA. Using survey data to estimate the population size and distribution of MSM. California Department of Health Services–Office of AIDS. Available at: http://www.dhs.ca.gov/AIDS/Reports/PDF/SOAEstMSM1102.pdf. Accessed November 2002.
22.Dolcini MM, Catania JA, Stall RD, Pollack L. The HIV epidemic among older men who have sex with men. J Acquir Immun Defic Syndr 2003; 33:S115–S121.
23.Binson D, Woods WJ, Pollack L, Paul JP, Stall R, Catania JA. Differential HIV risk in bathhouses and public cruising areas. Am J Public Health 2001; 91:1482–1486.
24.Biemer P. Nonresponse bias and measurement bias in a comparison of face to face and telephone interviewing. Journal of Official Statistics 2001; 17:295–320.
© Copyright 2005 American Sexually Transmitted Diseases Association