Villarroel, Maria A. MA*†; Turner, Charles F. PhD*‡; Rogers, Susan M. PhD*; Roman, Anthony M. MA§; Cooley, Phillip C. MS∥; Steinberg, Allyna B. MPH¶; Eggleston, Elizabeth DrPH*; Chromy, James R. PhD#
Click on the links below to access all the ArticlePlus for this article.
Please note that ArticlePlus files may launch a viewer application outside of your web browser.
IN THE ABSENCE OF POPULATION surveys that collect and test biologic specimens,1–3 estimates of the burden of sexually transmitted diseases (STDs) in a population must rely upon either counts of diagnosed infections reported to local health departments or respondents’ self-reports of STD symptoms and diagnoses provided in surveys. Both of these sources underestimate the true burden of infection in a population. STD cases are well known to be underreported to health departments.4 The extent of this undercount bias is not thought to be uniform over time or localities, and inferences about trends over time or variations across jurisdictions in diagnosed infections are subject to nontrivial inaccuracy. Similarly, survey respondents are often hesitant to admit stigmatized, illicit, or embarrassing behaviors to a human interviewer5–8 —a concept often attributed to “social desirability” bias.9
Eliminating the requirement that respondents disclose STD-related and other sensitive information to human interviewers substantially increases the likelihood that this information will be reported in population surveys and clinical measurements.10–19 Until recently, telephone surveys could not offer a fully private mode of questioning comparable to the paper self-administered questionnaires and audio computer-assisted self-interviewing (audio-CASI) technology used for in-person surveys.
In 1995, we adapted an audio-CASI system to support complex telephone surveys.20–22 A randomized experiment embedded in the 1996 to 1998 Urban Men’s Health Survey compared results obtained with this Telephone audio-CASI (T-ACASI) system with those obtained by human interviewers. This experiment found that T-ACASI respondents were more likely to report use of a range of illegal drugs, concern about their current drug use, and exchange of money or drugs for sex.23 Subsequent work by our group and collaborators has found that T-ACASI increased reporting of sensitive and stigmatized behaviors, including illicit (but not licit) drug use, same-gender and heterosexual sexual behaviors and “unpopular attitudes” in general population samples, and teen smoking in a regional sample.22,24–27 A growing body of evidence amassed by other researchers has yielded generally supportive evidence of the positive impact of T-ACASI on reporting of sensitive sexual and STD-related behaviors.28–30
The National STD and Behavior Measurement Experiment
The National STD and Behavior Measurement Experiment (NSBME) was designed to assess the impact of T-ACASI on reporting of STDs and a range of other sensitive characteristics and behaviors. In this article, we report NSBME findings for survey questions asking about respondents’ history of STD symptoms and diagnoses, health care for STD symptoms, and interactions with their main partner about STDs.
The protocol for this research was approved and supervised by Institutional Review Boards at the Research Triangle Institute and the University of Massachusetts at Boston, MA.
Sample and Experimental Design
As described more completely in prior publications,25,31,32 the NSBME experiment was embedded in a telephone survey of a probability sample of women and men aged 18 to 45 years residing in US households with working land line telephones. The survey was conducted between September 1999 and April 2000. Two sample strata were recruited for this survey measurement experiment: (a) a sample of the telephone-accessible US household population aged 18 to 45 (national stratum), and (b) a parallel sample of the telephone-accessible population of the city of Baltimore, MD (Baltimore stratum). The Baltimore stratum was included to allow a comparison of results from the NSBME survey3 with those from a similar in-person survey conducted between 1997 and 1998.
For the national stratum, 14,250 telephone numbers were generated, and 12,322 telephone numbers (86.5%) were successfully screened for eligibility. Of these screened telephone numbers 2183 were found to be residential numbers with 1 or more eligible English-speaking respondents aged 18 to 45. One eligible household member of these households was randomly selected for participation in the survey (without substitution). Of the 2183 target respondents in the national strata 1452 (66.5%) completed interviews (66.5%) and 91 (4.2%) completed partial interviews that included at least 1 substantive questionnaire section. Up to a maximum of 91 calls (mean = 11.3) were made to screen households and complete an interview in the national stratum.
The second sample stratum was drawn to represent the adult population of Baltimore, MD. For this stratum, 7498 telephone numbers were generated and 6326 (84.4%) were successfully screened for eligibility. Screening identified 1072 households with an eligible respondent, and 697 of these eligible respondents completed interviews (65.0%). An additional 47 respondents (4.4%) completed partial interviews. Up to a maximum of 82 calls (mean = 12.3) were made to screen households and complete an interview in the Baltimore stratum.
Telephone numbers were randomly assigned to the telephone interviewer-administered questioning (T-IAQ) or T-ACASI conditions before their release to the telephone survey unit. After screening and recruitment into the study, telephone interviewers at the Center for Survey Research (University of Massachusetts, Boston, MA) conducted the survey either by asking questions of the respondent and recording their answers (T-IAQ condition) or by transferring the respondent to the T-ACASI system.
The NSBM experiment included 35 questions that asked about communication with partner(s) about sexual behaviors and STD prevention, recognition of both actual and fictitious STDs, incidence of STD symptoms and health care treatment for these symptoms, if any, and STD diagnoses. Survey questions were adapted from a range of widely cited national surveys from the United States and Britain,33–37 including Tanfer’s National Survey of Men, Catania et al.’s National AIDS Behavior Survey, Johnson et al.’s National Survey of Sexual Attitudes and Lifestyles, Sonenstein et al.’s National Survey of Adolescent Males, and Laumann et al.’s National Health and Social Life Survey. Symptoms and diseases covered included dysuria, genital sores or blisters, genital discharge, genital warts, gonorrhea, chlamydia, genital phlemoria (a fictitious STD), pelvic inflammatory disease (PID), and any STD “such as herpes, gonorrhea, or syphilis.” Although the descriptions in this article make use of formal terms such as “dysuria,” our survey questions used common English phraseology, e.g., “Have you EVER had a burning feeling when you urinate?”
Typical sequences of STD questions asked were (a) whether respondents had ever heard of a specific STD, e.g., chlamydia; (b) whether they had ever been diagnosed with that STD; (c) recency of their last diagnosis; and (d) total number of lifetime diagnoses with this STD. Respondents who did not recognize an STD were not asked the subsequent 3 questions about that STD; similarly persons with no history of that STD were not asked the 2 follow-up questions about recency and number of infections. On average, sexually experienced respondents answered 16 of the 35 STD-related questions included in the NSBME. Wordings for all of these survey questions can be found in both an on-line publication38 and also in STD’s Article Plus repository.
Our analyses are intended to answer 2 major questions:
* Compared with human telephone interviewers, does T-ACASI change the likelihood that respondents will report STD symptoms, infections and treatment, discussion of STDs with their sexual partner, partner’s STDs, and avoidance of sex because of concerns about STD transmission?
* Is the magnitude and direction of the T-ACASI effect, if any, homogeneous across subpopulations defined by social and demographic factors, such as gender, age, and education?
The NSBME also included 2 questions on a fictitious STD to provide a crude indicator of the extent, if any, to which T-ACASI might encourage false claims of STD knowledge or diagnoses.
To answer our major questions, we combine the NSBME’s national and Baltimore sample strata. These combined sample strata are treated as a population that has been randomly allocated to 1 of 2 experimental conditions: T-ACASI or T-IAQ interview mode. Data in these analyses are unweighted, and our statistical analyses assess the likelihood that observed fluctuations in survey responses across the 2 interview modes arose by chance from the random allocation of respondents to 1 of the 2 experimental groups. When this null hypothesis is rejected, we conclude that the mode of interview had an effect on survey response.
To assess the magnitude of this effect, we compute the crude odds ratios (ORs) for our STD measurements comparing responses obtained by T-ACASI and T-IAQ. Because the availability of touchtone phones and differential sample loss in the 2 conditions could introduce undetected divergence in the sociodemographic composition of the respondents assigned to the 2 modes, we also report adjusted ORs controlling for the impact of deviations in sample composition. Adjusted ORs are calculated using multivariate logistic regression models that control for sample strata (national and Baltimore), gender, age group (18–25, 26–35, 36–45), educational level (<high school, high school, some college, ≥4-year college degree), marital status (married, cohabiting, divorced or widowed or separated, unmarried), race or ethnicity (non-Hispanic black, non-Hispanic white, non-Hispanic other, Hispanic), and 2 levels of urbanization [21 largest metropolitan statistical areas vs. other locales; see Ref. 25 for a list of metropolitan statistical areas].
Tests of the homogeneity of estimated T-ACASI mode effects were conducted by first obtaining maximum-likelihood estimates of logit models that predicted each STD measurement as a function of interview mode and the social or demographic variables of interest: gender, sample strata (the United States or Baltimore), years of education, age, race (black vs. nonblack), and marital status (married or living with partner vs. other). The fit of this model was then compared using a likelihood ratio test to the fit of a model that added an additional term representing the mode-by-demographic factor interaction. Our tests of homogeneity were targeted on the most important mode effects—in particular, those that were (a) statistically significant, (b) reasonably large (ORs >2.0 or ORs <0.50), and (c) involved questions answered by a substantial majority of respondents (N > 1700). In addition to the foregoing analyses, we accommodated a reviewer’s concerns by conducting similar analyses checking for mode-by-stratum interactions for all of the STD measurements reported in Tables 1 and 2.
In our final analysis, estimates of STD burden for the United States and Baltimore populations were calculated separately by survey mode using sampling and poststratification weights39 to project the sample data to the national and Baltimore city populations. All statistical analyses were carried out using Stata, versions 6.0 and 8.0.40
Previously published analyses tested the equivalence of T-IAQ and T-ACASI samples by gender, age, marital status, education, race or ethnicity, region, urbanization, and sample strata. No comparison produced evidence of nonequivalence with a P value less than 0.28.25,31
Partner’s STD Status and Communication
Over half of all sexually experienced respondents reported never avoiding sex to prevent STD transmission (Table 1). Reports of never avoiding sex were greater in T-ACASI than in T-IAQ (59.0% vs. 53.9%, OR = 1.2, P <0.05), but the difference was of borderline significance when adjusted for demographic characteristics (OR = 1.2, P = 0.08). Table 1 also presents results for questions about the respondent’s “main sex partner during the past year.” The odds that respondents assigned to the T-ACASI condition reported that their main partner in the past year had a history of STDs were 2.3 times higher than for respondents in the T-IAQ condition (7.9% vs. 3.5%, P <0.001) and 1.3 times higher for reporting that they had never talked to their partner about protecting themselves against STDs (48.2% in T-ACASI vs. 41.6% in T-IAQ, P <0.05). Among those respondents who reported talking about STD protection, there was no difference in reporting that it occurred before rather than after first engaging in sexual activity (82.0% vs. 83.0%, NS). T-ACASI respondents were, however, more likely to report more frequent discussions of their sex life with their main partner. Thus, 47.6% of T-ACASI respondents reported having such discussions weekly compared with 40.1% of respondents in the T-IAQ condition (P <0.01), whereas 5.5% of T-ACASI respondents reported never having such discussions compared with 9.6% or T-IAQ respondents (P <0.001 for linear trend across 5 categories of frequency of discussion).
STI Symptoms and Disease
The top panel of Table 2 compares the reported incidence in the past year for 4 STI-related symptoms: dysuria, genital sores, genital discharge, and genital warts by interview mode. In each instance, T-ACASI increased the odds that these symptoms were reported by factors of 1.5 to 2.8. With the exception of genital warts—for which reported incidences were less than 1%—all of these results are statistically reliable (P <0.001). The second panel of Table 2 tests for differences in the likelihood that symptomatic respondents would report that they had sought treatment for their symptom. For all 4 symptoms, T-ACASI respondents were more likely than T-IAQ respondents to report that they did not seek medical treatment for their symptoms (ORs = 1.2–1.9). These results are statistically reliable or borderline for reporting of dysuria and genital discharge but not for the other 2 symptoms that were reported by fewer than 50 respondents in the T-IAQ condition.
The third panel of Table 2 examines reported recognition of gonorrhea, chlamydia, PID, and 1 fictitious STD (genital phlemoria). Although the majority of respondents had heard of gonorrhea, significantly fewer respondents in T-ACASI reported knowing of the disease (96.1% vs. 92.6%; adjusted OR = 0.5, P <0.001). T-ACASI respondents were, however, more likely to claim knowledge of the fictitious disease, genital phlemoria (adjusted OR = 1.5, P <0.001). No statistically reliable difference was found for reported recognition of chlamydia or PID.
The final 2 panels of Table 2 examine the reported incidence in the past year of these same STDs for all respondents and PID for females. Respondents were substantially more likely to report chlamydial and gonococcal infections in the past year when interviewed by a T-ACASI computer rather than a human interviewer (adjusted OR = 6.1, P <0.01 and adjusted OR = 3.6, P <0.10, respectively). No statistically reliable differences were found between interview modes in reporting of PID or the fictitious disease, genital phlemoria. (Only 4 respondents reported being diagnosed with genital phlemoria: 1 in the T-IAQ and 3 in the T-ACASI condition.)
Homogeneity of Interview Mode Effects
To test the homogeneity of the T-ACASI effect, we selected measurements that met the following criteria: (a) the unadjusted estimate of the mode effect had an OR outside the range 0.49 ≥ OR ≥ 1.99; (b) the sample size for the T-ACASI versus T-IAQ mode comparison included most of the NSBME sample (N > 1700); and (c) the statistical reliability of estimates of both the unadjusted and adjusted mode effects were significant with P < 0.05. Four measurements from Tables 1 and 2 meet these criteria: reporting that your partner ever had an STD; and reporting that you had genital blisters, genital discharge, or a chlamydia diagnosis during the past year.
Estimates of the T-ACASI mode effect for each of these measurements were tested for homogeneity across subpopulations defined by gender, years of education, age, race (black vs. nonblack), marital status (married or living with partner vs. other), and sample strata (United States vs. Baltimore).
Our analysis identified 6 statistically reliable or suggestive instances of heterogeneity across population subgroups in the estimated impact of T-ACASI. The pattern of results for these 7 instances is shown in Table 3. It can be seen that
* as respondents’ educational level increased, the impact of T-ACASI on reporting of genital sores (P = 0.13) in the past year decreased;
* as respondents’ age increased, the impact of T-ACASI decreased for reporting of genital sores in the past year (P = 0.05) and partner ever having an STD (P = 0.05);
* the impact of T-ACASI on reporting of genital sores in the past year was greater for black than for nonblack respondents (P = 0.03); and
* the impact of T-ACASI on reporting of genital discharge in the past year was stronger for respondents who were married or living with a partner (P = 0.05) than for other respondents, but the reverse pattern was found for reporting of genital sores or blisters (P = 0.06).
To accommodate a reviewer’s concern, we conducted parallel tests for mode-by-strata interactions for the 21 crude ORs reported in Tables 1 and 2. (These analyses are designed to detect variation in the magnitude or direction of the impact of T-ACASI in the Baltimore vs. national strata.) In no instance could we reject the null hypothesis that the mode effect was homogeneous across the 2 sample strata. In 18 cases, the P values for the interaction test exceeded 0.40; in 2 cases they were in the range 0.17 to 0.18; and in 1 case the test could not be conducted because of sparse cell counts.
Table 4 presents separate T-ACASI and T-IAQ estimates of the burden of STDs in the populations of the United States and Baltimore city. Unlike preceding tables, this table is derived from an analysis that used sampling and poststratification weights to project sample results to the populations from which they were drawn, i.e., adults aged 18 to 45 who resided in households in the United States and in Baltimore, MD, who were accessible by landline telephones. The impact of T-ACASI on population estimates of STD incidence in the population is substantial. Estimates of the annual incidence of self-reported gonococcal and chlamydial infections increase by factors of 2.4 to 9.7.
In considering these results, it should be borne in mind that the purpose of this analysis is not to test the null hypothesis that T-ACASI increases reporting of incident STD infections. This impact of T-ACASI has been demonstrated in Tables 1 and 2 using unweighted data from respondents in both sample strata. Rather, this analysis is intended to provide readers with an appreciation of the likely understatement of populationwide STD burdens that occurs when T-IAQ surveys are used. Because this analysis does not combine sample strata, the smaller sample sizes for each strata (N`s = 1543 and 744 vs. 2287 combined) decrease statistical precision, resulting in wide 95% confidence intervals for the calculated T-ACASI versus T-IAQ ORs within each sample strata.
It can be seen from Table 4 that the impact of T-ACASI upon population estimates of lifetime exposure to these infections is attenuated but consistently positive (ORs = 1.2–1.8). Furthermore, this table illustrates that use of T-ACASI can unmask important regional differences in STD incidence. Thus, comparison of T-IAQ-derived estimates of annual chlamydial incidence in Baltimore with those for the United States failed to reject the conclusion that they were equivalent (0.34% vs. 0.39%, P >0.5). T-ACASI increased reporting of chlamydial infections in both populations, and it revealed that Baltimore has a substantially higher (self-reported) annual incidence of chlamydia than that in the United States (3.7% vs. 0.8%, P = 0.017).
Our results indicate that the answer to our first research question is consistently positive. T-ACASI substantially increases the likelihood respondents will report their own STD symptoms and diagnosed infections as well as their partner’s history of infection. The magnitude of the observed effects of T-ACASI was often dramatic. The adjusted odds that respondents would report chlamydial and gonococcal infections during the past year, for example, increased by factors of 6.1 and 3.6 when the measurements were made by T-ACASI rather than by a human interviewer.
T-ACASI also increased the likelihood respondents would report that they did not seek treatment for STD symptoms, never talked to their partner about protecting themselves against STDs, and never avoided sex because of concerns about STD transmission. In each of these cases, it could be argued that T-ACASI reduced the “social desirability” bias contaminating our T-IAQ measurements—although other interpretations are plausible. One could, among many alternatives, speculate that the T-ACASI interview mode reduced respondents’ embarrassment, which led to more complete reporting.
There was one exception to the general pattern of T-ACASI measurements producing more frequent reports of what we presumed to be socially undesirable or embarrassing facts. Respondents interviewed by T-ACASI reported more frequent discussions of their “sex life” with their partner. We found this result puzzling because we assumed that frequent communication between partners about their sex lives would be more socially desirable and less embarrassing than reporting never talking to one’s partner about sex. It is unclear whether this is an anomalous result or merely evidence of an erroneous assumption on our part. It is possible, for example, that respondents viewed reporting frequent discussions of sex with their partners as suggesting that they were having sexual problems or that they were “talking dirty” to stimulate each other.
We found one additional anomaly of a different sort. Respondents were more likely to recognize our fictitious disease, genital phlemoria, when responding to a T-ACASI computer rather than to a human interviewer. Because we had only one such question in the NSBME, it is unclear whether generalization is warranted. (As one reviewer observed, this might have been a spurious result. Because Tables 1 and 2 report 21 comparisons, one might have expected one spurious result at the 0.05 level in 21 tests, assuming the tests were independent.) It is worth emphasizing, however, that although 27% of T-ACASI respondents and 21% of T-IAQ respondents had “heard” of genital phlemoria, only 4 respondents reported being diagnosed with this fictitious disease (1 T-IAQ respondent and 3 T-ACASI respondents). Clearly, however, additional research on this phenomenon is needed.
Overall, our results are consistent with a growing body of studies that find that T-ACASI increases reporting of stigmatized and sensitive sexual behaviors, illicit drug use, teen smoking, and unpopular social attitudes.22–30 It is reasonable to infer that the privacy afforded by T-ACASI is responsible for this increased reporting. Our respondents’ evaluations of their experience support this interpretation. After the main survey was completed, T-ACASI respondents were asked a brief series of questions about their participation. When asked which survey method most people would prefer for asking “questions on sensitive topics such as anal sex and STDs,” 87% of respondents reported thinking that most people would prefer T-ACASI; 72% thought T-ACASI was “best for protecting privacy,” and 80% thought T-ACASI was “best for getting the most honest answers.”
Our T-ACASI results indicate considerably higher estimated annual and lifetime incidence of gonococcal and chlamydial infection in Baltimore, MD, than in the rest of the United States. For our T-ACASI measurements, ORs were 3.1 and 4.8, respectively, for estimated annual incidence in Baltimore versus the United States; and 2.9 and 2.1 for lifetime incidence (Table 4). These results are consistent with reporting to Baltimore and other US Health Departments of diagnosed gonococcal and chlamydial infections. For the year 2000, Centers for Disease Control and Prevention reported an annual incidence of 886 reported gonococcal infections per 100,000 in Baltimore versus only 132 per 100,000 in the United States as a whole. Similarly for that year, Centers for Disease Control and Prevention41 reported an annual incidence of 859 reported chlamydial infections per 100,000 in Baltimore versus 258 per 100,000 in the United States. The (incomplete) congruence of our T-ACASI measurements with these results provides some confidence that the T-ACASI measurements are measuring valid population differences in infection incidence. Although human interviewing also yields differences in the same direction in 3 of 4 comparisons, we note that the estimated annual incidence of chlamydial infection is virtually identical in Baltimore and the United States (OR = 1.2, NS) when human telephone interviewers—rather than T-ACASI—are used to collect these data.
The answer to our second research question is less clear cut. There was ample evidence that the magnitude of the T-ACASI mode effect varied across subpopulations, but in no instance was the variation found for all 4 STD measurements we studied. In addition, 2 apparently similar STD measurements (genital discharge and genital sores in the past year) evidenced significant but opposite patterns of variation in the estimated impact of T-ACASI for married or cohabiting respondents (vs. others). We do, however, have 3 logically consistent results for education and age. The impact of T-ACASI declined with increasing age and increasing education. This result mirrors the recent report of a parallel variation by educational level and age in the estimated impact of T-ACASI on respondents’ willingness to express “unpopular” opinions.27 Younger and less-educated respondents seem to be most affected by the privacy afforded by T-ACASI.
Finally, it is clear from our population projections for the United States and Baltimore city that STD measurements made by human interviewers substantially understate the burden of sexually transmitted infections in the population. When respondents are afforded the privacy of a T-ACASI interview, the estimated annual incidence of diagnosed gonococcal and chlamydial infections rose by factors of 2.4 to 9.7. Reporting bias in T-IAQ surveys apparently depresses reporting of recent STD diagnoses, leading to a substantial underestimation of the burden of these STDs in the population.
1. Ku L, St Louis M, Black C, et al. Risk behaviors, medical care and chlamydial infection among young men in the United States. Am J Public Health 2002; 92:1140–1143.
2. Fenton KA, Korovessis C, Johnson AM, et al. Sexual behaviour in Britain: Reported sexually transmitted infections and prevalent genital Chlamydia trachomatis infection. Lancet 2001; 358:1851–1854.
3. Turner CF, Rogers SM, Miller HG, et al. Untreated gonococcal and chlamydial infection in a probability sample of adults. JAMA 2002; 287:726–733.
4. Eng TR, Butler WP, eds. Hidden Epidemic: Confronting Sexually Transmitted Diseases. Report of the Committee on Prevention and Control of Sexually Transmitted Diseases, Institute of Medicine. Washington, DC: National Academy Press, 1997.
5. Jones EF, Forrest JD. Underreporting of abortion in surveys of U.S. women: 1976 to 1988. Demography 1992; 29:113–126.
6. Miller HG, Gribble JN, Mazade LC, et al. The association between self-reports of abortion and breast cancer risk: Fact or artifact? In: Stone A, Turkkan J, Bachrach C, et al., eds. The Science of Self Report. Mahwah, NJ: Lawrence Erlbaum Associates, 1999.
7. Turner CF, Lessler JT, Devore J. Effects of mode of administration and wording on reporting of drug use. In: Turner CF, Lessler JT, Gfroerer JD, eds. Survey Measurement of Drug Use: Methodological Issues. Washington, DC: Government Printing Office, 1992. DHHS Publication Number 92–1929.
8. Turner CF, Forsyth BH, O’Reilly J, et al. Automated self-interviewing and the survey measurement of sensitive behaviors. In: Couper M, Baker R, Bethlehem J, et al., eds. Computer-Assisted Survey Information Collection. New York: Wiley, 1998.
9. DeMaio TJ. Social desirability and survey measurements: A review. In: Turner CF, Martin E, eds. Surveying Subjective Phenomena, vol 2. New York: Russell Sage, 1984:257–282.
10. Turner CF, Lessler J, Gfroerer J. Future directions for research and practice. In: Turner C, Lessler J, Gfroerer J, eds. Survey Measurement of Drug Use: Methodological Issues. Washington, DC: Government Printing Office, 1992: chap. 12.
11. Turner CF, Shoen A. Behavioral studies relevant to vaccine trial preparation. AIDS Res Hum Retroviruses 1994; 10:S273–S276.
12. Turner CF, Ku L, Rogers SM, et al. Adolescent sexual behavior, drug use, and violence: Increased reporting with computer survey technology. Science 1998; 280:867–873.
13. Tourangeau R, Smith T. Asking sensitive questions: The impact of data collection mode, question format, and question context. Public Opin Q 1996; 60:275–304.
14. Al-Tayyib AA, Rogers SM, Gribble JG, et al. Effect of low medical literacy on health survey measurements. Am J Public Health 2002; 92:5–8.
15. Rogers SM, Willis G, Al-Tayyib A, et al. Audio computer-assisted interviewing to measure HIV risk behaviors in a clinic population. Sex Transm Infect 2005; 81:501–507.
16. Johnston G. Demonstration of computer-administered audio survey technology. Presented at: the National Center for Health Statistics; January 28, 1992; Hyattsville, MD.
17. O’Reilly J, Hubbard M, Lessler J, et al. Audio and video computer-assisted self-interviewing: Preliminary tests of new technologies for data collection. J Off Stat 1994; 10:197–214.
18. Johnson AM, Copas AJ, Erens B, et al. Effect of computer assisted self-interviews on reporting of sexual HIV risk behaviours in a general population sample: A methodological experiment. AIDS 2001; 15:111–115.
19. Metzger D, Koblin B, Turner CF, et al. A randomized controlled trial of audio computer-assisted self interviewing: Utility and acceptability in longitudinal studies. Am J Epidemiol 2000; 152:99–106.
20. Cooley PC, Turner CF. Implementing audio-CASI on Windows platforms. Comput Hum Behav 1998; 14:195–207.
21. Cooley PC, Miller HG, Gribble JN, et al Automating telephone surveys: Using T-ACASI to obtain data on sensitive topics. Comput Hum Behav 2000; 16:1–11.
22. Turner CF, Miller HG, Smith TK, et al. Telephone audio computer-assisted self-interviewing (T-ACASI) and survey measurements of sensitive behaviors: Preliminary results. In: Banks R, Fairgrieve J, Gerrard L, eds. Survey and Statistical Computing. Buckinghamshire, UK: Association for Survey Computing, 1996.
23. Gribble JN, Miller HG, Catania JA, et al. The impact of T-ACASI interviewing on reported drug use among men who have sex with men. Subst Use Misuse 2000; 35:869–890.
24. Turner CF, Villarroel MA, Rogers SM, et al. Reducing bias in telephone survey estimates of the prevalence of drug use: A randomized trial of telephone audio-CASI. Addiction 2005; 100:1432–1444.
25. Villarroel MA, Turner CF, Eggleston EE, et al. Same-gender sex in the USA: Impact of T-ACASI on prevalence estimates. Public Opin Q 2006; 70:166–196.
26. Currivan D, Nyman AL, Turner CF, et al. Does telephone audio computer-assisted self-interviewing improve the accuracy of prevalence estimates of youth smoking? Public Opin Q 2004; 68:542–564.
27. Harmon T, Turner CF, Rogers SM, et al. Impact of T-ACASI on survey measurements of subjective phenomena. Washington, DC: Program in Health and Behavior Measurement, Research Triangle Institute, 2007. Technical papers in health and behavior measurement, No. 77.
28. Lau JTF, Tsui HY, Wang QS. Effects of two telephone survey methods on the level of reported risk behaviours. Sex Transm Infect 2003; 79:325–331.
29. Corkey R, Parkinson L. A comparison of four computer-based telephone interviewing methods: Getting answers to sensitive questions. Behav Res Methods Instrum Comput 2002; 34:354–363.
30. Moskowitz JM. Assessment of cigarette smoking and smoking susceptibility among youth: Telephone computer-assisted self-interviews versus computer-assisted telephone interviews. Public Opin Q 2004; 68:565–587.
31. Villarroel MA, Turner CF, Eggleston, et al. Sample design and demographic characteristics of respondents in the 1999–2000 National STD and Behavior Measurement Experiment (NSBME). Washington, DC: Program in Health and Behavior Measurement, Research Triangle Institute, 2006. Technical papers in health and behavior measurement, No. 69. Available at: soc.qc.edu/turner/Reprints/TechPDFs/69_NSBMEsample.pdf
32. Roman A. Survey of AIDS and other social issues: Field report [Unpublished manuscript]. Boston, MA: Center for Survey Research, University of Massachusetts, 2000.
33. Tanfer K. National survey of men: Design and execution. Int Fam Plan Perspect 1993; 25:83–86.
34. Catania JA, Coates TJ, Stall R, et al. Prevalence of AIDS-related risk factors and condom use in the United States. Science 1992; 258:1101–1106.
35. Johnson AM, Wadsworth J, Wellings K, et al. Sexual Attitudes and Lifestyles. Oxford: Blackwell Scientific Publications, 1994.
36. Sonenstein FL, Ku L, Lindberg L, et al. Changes in sexual behavior and condom use among teenage men: 1988 to 1995. Am J Public Health 1998; 6:956–959.
37. Laumann E, Gagnon J, Michael R, et al. The Social Organization of Sexuality. Chicago, IL: University of Chicago Press, 1994.
38. Villarroel MA, Turner CF, Rogers SM, et al. Supplemental materials for article: T-ACASI reduces bias in STD measurements—The National STD and Behavior Measurement Experiment (NSBME). Washington, DC: Program in Health and Behavior Measurement, Research Triangle Institute, 2007. Technical papers in health and behavior measurement, No. 77. Available at: soc.qc.edu/turner/Reprints/TechPDFs/77_STDQuestions.pdf
39. Turner CF, Villarroel MA, Gordek H, et al. 2000 NSBME: Final weighting strategy. Washington, DC: Program in Health and Behavior Measurement, Research Triangle Institute, 2004. Technical papers on health and behavior measurement, No. 61.
40. StataCorp. STATA statistical software for Windows, versions 6.0 & 8.0. College Station, TX: STATACorp, 2000–2003.
41. Centers for Disease Control and Prevention. Sexually Transmitted Disease Surveillance, 2000. Atlanta, GA: Centers for Disease Control and Prevention, 2001: Tables 1, 7, and 16.