MEASUREMENT OF SEXUAL behaviors in the adolescent population has become increasingly important because of the high prevalence of sexually transmitted diseases (STD) and the presence of human immunodeficiency virus (HIV) infection.1,2 Validity of measurement (the extent to which any measuring instrument measures what it is intended to measure) is central to all research.3 Assessment of an instrument's validity becomes difficult when investigators must rely on self-report. Observation of certain private behaviors (such as sexual activity or use of illicit substances) is not possible.4 There is additional concern when the behavior (e.g., sexual activity or substance use) may carry social prohibitions.4–7 Unlike studies of self-reported substance use in which investigators may measure the substance in biologic fluids, there is no corresponding biologic indicator of sexual activity. Studies assessing the validity of self-reported measures of sexual behaviors are largely absent from the recent literature.8,9
In contrast, investigators have addressed the reliability of such measures.4,10–18 Reliability refers to the extent to which a particular measure (instrument or test) yields the same results on repeated trials.3 Several issues influence the reliability of the results, including the format of obtaining the information, the period of recall, and the wording of the questions.10,12,18–21 Among adults, self-administered questionnaires (SAQ) may yield more information about illicit substance use and “stigmatized” sexual behaviors compared to face-to-face interviews.7,10,19,21 Reliability of the reports of sexual behaviors appears to be increased when the period of recall is shorter.4,13,14,22 Recent reports suggest that computer-assisted interviews with audio delivery of the questions increase the amount of information obtained compared with SAQ.6,7,23,24 Surprisingly, little research about the reliability of measures of sexual behaviors has been conducted among adolescents, whose responses may be prone to additional error because of limited sexual vocabulary, misunderstanding, exaggeration, or even deliberate falsification.4,17,25,26
The purpose of this study is to examine the criterion-related validity3 of alternative approaches to measurement of sexual intercourse using infection with an STD as a biomarker (the criterion). In a sample of sexually active adolescent women at high risk for STD, a group was identified who were free of infection after treatment and who returned for examination 3 months later. This permitted us to examine relationships among specific questions about sexual activity (i.e., occurrence of sexual intercourse, the number of sexual partners, the number of episodes of sexual intercourse [coitions], and the presence of infection). Knowing that an individual cannot acquire an STD without having engaged in interim sexual intercourse permits one to assess the relationship between the absence of infection and whether intercourse occurred as measured by a single question and multiple questions. In other words, to what extent are the reports of not having sexual intercourse over a specific time interval valid? It is important to keep in mind that the presence of an STD is an imperfect (but clearly measurable criterion) marker of sexual intercourse; although an individual cannot acquire any of the STDs without having at least one interim coition, not all sexual encounters result in the acquisition of an STD, even if the partner is infected.
The subjects were adolescent women 15 through 19 years of age receiving reproductive health care at four adolescent clinics and an STD clinic who were participating in a larger study of STD detection and treatment. Entry into the study depended on the documentation of genitourinary tract infection with Neisseria gonorrhoeae, Chlamydia trachomatis, or Trichomonas vaginalis identified by standard culture techniques27,28 and supervised treatment with appropriate single-dose antibiotics. Sensitivity of tissue culture to detect C. trachomatis in endocervical samples was 94.7% using two DNA amplification techniques (ligase chain reaction and polymerase chain reaction; R. B. Jones, personal communication).
After written informed consent was obtained from the adolescents, subjects completed an SAQ requesting information about sexual behaviors and demographic characteristics. The subjects were requested to return in 2 to 4 weeks for a test-of-treatment (TOT) culture and in 3 months for culture and ascertainment of interim sexual behaviors. The 255 subjects who returned at 3 months comprise the study group for this research. Subjects received $15.00 at enrollment and on returning at 3 months. The study was approved by the Institutional Review Board for Human Subjects of Indiana University Purdue University at Indianapolis.
The SAQ included questions about sexual behaviors. The wording for these questions reflected the recommendations of 24 sexually active adolescent women selected from clinic attendees who had participated in focus groups before the initiation of the study. Specifically, “regular” was chosen as a modifier of intercourse because the focus group participants believed this would differentiate vaginal intercourse from other forms of intercourse. Subjects were asked to respond with Yes or No to the following questions: “Have you had regular sexual intercourse (vaginal sex) during the LAST 3 MONTHS? Have you had oral (mouth) sex during the LAST 3 MONTHS? Have you had anal (rectal or butt) sex during the LAST 3 MONTHS?” To the following questions, subjects were instructed to indicate the number of occurrences or answer “0” if there were no occurrences: “How many sex partners have you had in the LAST 3 MONTHS? In the past 3 months, HOW MANY TIMES did you have sex? How many sex partners have you had in your entire life?” Finally, they were asked: “How old were you the first time you had sexual intercourse?”
Our analytic plan was to examine the relationships among the various reports of sexual behaviors (sexual intercourse, frequency of coitions, and number of partners) and the presence of subsequent infection, and then to determine the demographic characteristics of subjects who were “consistent” and “inconsistent” in their responses. “Consistent” responders were those whose answers about sexual behaviors were in logical agreement across questions; individuals whose answers were logically in conflict were classified “inconsistent” responders. For example, an individual who reported that she had not had regular intercourse (vaginal sex), oral sex, and anal sex during the previous 3 months but reported more than 0 sex partners or more than 0 times having had sex during the same interval was classified as inconsistent. An additional variable, “had sexual intercourse,” was created in which “No” reflected responses of “no” and “0” to all questions about intercourse during the 3 months between enrollment and follow-up. Continuous-level variables were analyzed using t test; chi-square test was used for nominal and ordinal level data. The Wilcoxon rank sum test was used to test for differences in nonnormally distributed variables. P < 0.05 was considered statistically significant.
Table 1 shows the demographic and reported sexual behaviors (at enrollment and 3 months) of the 255 subjects. Their median age was 17 years. They had been sexually active nearly 3 years and reported a median of four lifetime sexual partners, and one sexual partner and six coitions in the preceding 3-month period. In the interval between enrollment and follow-up, they had a median of one sexual partner and five coitions.
Two hundred fourteen of the 255 adolescents had returned for a TOT culture 2 to 4 weeks after enrollment; 186 (73%) were not infected at the TOT visit. Of these 186 women with a negative TOT culture, 56 (30%) were infected at the 3-month visit. Based on responses obtained at the 3-month visit, 26 (14%) of the 186 adolescents denied having had regular intercourse (vaginal sex) between enrollment and the 3-month visit. Twelve of these 26 (46%) were classified as consistent responders because they also reported 0 sexual partners and 0 coitions. The other 14 were inconsistent; 2 reported at least 1 sexual partner but 0 coitions, 1 failed to answer the question about number of partners, but reported at least one coition, and 11 reported both sexual partners and coitions. None of these 26 adolescents who denied regular intercourse (vaginal sex) reported anal or oral sex during this period, whereas 28 of the 160 who reported regular intercourse (vaginal sex) also reported oral sex, 2 reported anal sex, and 3 indicated having both anal and oral sex. One of the 28 adolescents with an STD at the TOT visit was classified as inconsistent because she reported no regular sexual intercourse, no anal and no oral sex, but indicated one interim sexual partner. Overall, 15 of the 214 subjects who returned for a TOT culture were classified as inconsistent responders based on responses in the SAQ obtained at the 3-month visit.
The prevalences of STDs at 3 months by reported sexual behaviors and TOT status are shown in Table 2. Among individuals who returned for TOT and were found to be free of infection (the group for whom data are most easily interpretable), individuals who answered “no” to regular intercourse (vaginal sex) were as likely to be infected at 3 months as those who answered “yes” (23% vs. 31%). In contrast, the questions about the number of sexual partners, number of coitions, and the reclassified group “sexual intercourse” discriminated better with respect to infection status at 3 months-no adolescent who answered “no” to these questions had an STD at 3 months. Overall, the newly created variable, “sexual intercourse,” performed better in that there were no missing data and all subjects who were classified as not having had intercourse in the 3-month interval were free of STD. The number of adolescents who failed to indicate the number of coitions was greater than the numbers who failed to answer other questions; 21% to 25% of those who failed to include the number of coitions had an STD at 3 months. Inspection of the remainder of the data in Table 2 shows that the questions performed similarly across all subgroups, including those who failed to return for a TOT.
Table 3 displays the reported sexual behaviors of the 214 subjects with TOT visits based on the consistency of their reports of sexual activity during the interval period. Those with inconsistent responses were similar to the consistent responders who reported interim intercourse with respect to age, age of first intercourse, duration of sexual activity, number of lifetime sexual partners, and incident rate of STD; they reported fewer coitions in the intervening 3 months, but there was no significant difference in the number of sexual partners (P = 0.07). The 15 adolescents who had not had regular sexual intercourse (vaginal sex) and were consistent in their responses were younger and had been sexually active for shorter period of time then the other two groups.
The presence of a new genitourinary tract infection with N. gonorrhoeae, C. trachomatis, or T. vaginalis among a group of adolescents previously demonstrated to be free of these infections very strongly suggests the occurrence of interval vaginal intercourse. Even for chlamydial infection, adequate antibiotic treatment almost certainly eradicates the infection.29 Thus it is unlikely that infection 3 months after treatment represents persistent infection. Among the group who were free of infection at TOT, no adolescent woman who reported 0 sexual partners or 0 episodes of intercourse acquired an STD over the 3-month period; over 30% of those indicating interval intercourse became infected. Reinfection was common among those who failed to answer either of these two questions.
The question asking about regular intercourse (vaginal sex) failed to discriminate infection status. Despite the recommendations of the focus groups conducted before the research, this question was not a valid reflection of interval sexual behavior; 14 of 26 of those who answered “no” also reported more than 0 sexual partners or episodes of intercourse, and 23% (6/26) were infected. These “inconsistent” responders reported a median of four interval episodes of coition (significantly fewer than the “consistent” group, who reported regular intercourse) and one sexual partner between TOT and return at 3 months. This would suggest that, as the focus groups had indicated, regular did reflect vaginal intercourse (none reported interval anal or oral intercourse), but also reflected the periodicity of sexual activity. Perhaps providing definitions in the questionnaire of the specific behaviors would reduce the number of inconsistent responses.
Research on STD often requires information about the frequency of intercourse (exposure) during a specific time interval. Because measurement error increases with the length of the recall period, investigators have frequently limited the recall period to no more than 3 months.4,18,22 It has also become common to calculate the proportion of protected coitions based on the number of times intercourse took place and the number of times that a condom was used,8,30,31 rather than relying on self-reported estimates of the proportion of coitions protected by condom use. Among our sample of sexually experienced adolescent women at high risk for STD, the question about the number of coitions over a 3-month period generated the most missing data (36 of 255 subjects failed to answer this question); 25% of the subjects who failed to answer this question acquired a subsequent STD. Although subjects were not requested to report the number of times a condom was used, the large amount of missing data and incident infection in this group suggest that calculation of protected coitions among adolescents may be both difficult and inaccurate.18
There are limitations to our study. The subjects consisted of adolescent women recruited from clinics because of infection, most of whom were African American and of lower socioeconomic status. It is unclear the extent to which our findings would be similar among adolescents at lower risk for these STDs. Such a study would be expensive because of the large numbers required to obtain adequate statistical power. It is also possible that the subjects were influenced by the nature of the study because they were aware that cultures would be obtained. In this respect, our study is similar to clinical care in which subjects are routinely questioned about sexual behaviors and then examined for infection. The incident rate of STD (30%) is similar to other reports,8,32–34 suggesting that we have not studied an overly compliant population; all subjects had been encouraged to use condoms at every sexual encounter.
Clearly, we have examined only one side of the issue of validity of self-report—incident infection and reports of not having had sexual intercourse. We were not able to estimate the extent to which adolescents may have erred in reporting the occurrence of vaginal intercourse when this had not occurred. We do not believe error in this direction would be likely in this population.
Our data suggest that adolescents can accurately report whether they have engaged in vaginal intercourse. The validity of the measure used to obtain this information appears sensitive to the exact wording of the questions; multiple questions appear to be a more accurate reflection of sexual behaviors than single items. Based on biomarkers as the criterion outcome, it does not appear that adolescent women falsify reports of interval sexual activity when assessed on multiple questions in a SAQ. We may have failed to detect a few infections with C. trachomatis because tissue culture was only 95% sensitive. Additional research with other populations with specific attention to time intervals and incorporating DNA amplification techniques to detect infection is needed.
1. Sweeney P, Lindegren ML, Buehler JW, Onorato IM, Janssen RS. Teenagers at risk of human immunodeficiency virus type 1 infection: Results from seroprevalence surveys in the United States. Arch Pediatr Adolesc Med 1995; 149:521–528.
2. Division of STD Prevention. Sexually Transmitted Disease Surveillance, 1994. Atlanta: Centers for Disease Control and Prevention, 1995.
3. Carmines EG, Zeller RA. In: Sullivan JL, ed. Reliability and Validity Assessment. Newbury Park, CA: Sage Publications, 1979.
4. Catania JA, Chitwood DD, Gibson DR, Coates TJ. Methodological problems in AIDS behavioral research: Influences on measurement error and participation bias in studies of sexual behavior. Psychol Bull 1990; 108:339–362.
5. Adolescent Health and Social Behavior. In: Irwin CE, ed. New Directions in Child Development. San Francisco: Jossey-Bass, 1987:1–125.
6. Turner CF, Danella RD, Rogers SM. Sexual behavior in the United States, 1930–1990: trends and methodological problems. Sex Transm Dis 1995; 22:173–190.
7. Turner CF, Lessler JT, Devore JW. Effects of mode of administration and wording on reporting of drug use. In: Turner CF, Lessler JT, Gfroerer JG, eds. Survey Measurement of Drug Use: Methodological Studies. Washington, DC: Government Printing Office, 1992:177–220.
8. Zenilman JM, Weisman CS, Rompalo AM, et al. Condom use to prevent incident STDs: The validity of self-reported condom use. Sex Transm Dis 1995; 22:15–21.
9. Udry J, Morris N. A method for validation of reported sexual data. Journal of Marriage and the Family 1967; August:442–446.
10. Keller SE, Schleifer SJ, Bartlett JA, Johnson RL. The sexual behavior of adolescents and risk of AIDS. JAMA 1988; 260:3586.
11. Padian NS, Aral S, Vranizan K, Bolan G. Reliability of sexual histories in heterosexual couples. Sex Transm Dis 1995; 22:169–172.
12. Hornsby PP, Wilcox AJ. Validity of questionnaire information on frequency of coitus. Am J Epidemiol 1989; 130:94–99.
13. Coates RA, Calzavara LM, Soskolne CL, et al. Validity of sexual histories in a prospective study of male sexual contacts of men with AIDS or an AIDS-related condition. Am J Epidemiol 1986; 128:719–728.
14. Saltzman SP, Stoddard AM, McCusker J, Moon MW, Mayer KH. Reliability of self-reported sexual behavior risk factors for HIV infection in homosexual men. Public Health Rep 1987; 102:692–697.
15. Metzler CW, Noell J, Biglan A. The validation of a construct of high-risk sexual behavior in heterosexual adolescents. J Adolesc Res 1992; 7:233–249.
16. Coates RA, Soskolne CL, Calzavara L, et al. The reliability of sexual histories in AIDS-related research: Evaluation of an interview-administered questionnaire. Can J Public Health 1986; 77:343–348.
17. Alexander CS, Somerfield MR, Ensminger ME, Johnson KE, Kim YJ. Consistency of adolescents' self-report of sexual behavior in a longitudinal study. J Youth Adoles 1993; 22:455–471.
18. Fortenberry JD, Zimet GD, Orr DP. Concordance between diary and self-report questionnaires of sexual behaviors of adolescent women. In: Bancroft J, ed. Methodologic Issues in Sexual Behavior Research. Bloomington, IN: Indiana University Press, 1996 (in press).
19. Williams BL, Suen S. A methodological comparison of survey techniques in obtaining self-reports of condom-related behaviors. Psychol Rep 1994; 75:1531–1537.
20. Ford K, Norris A. Methodological considerations for survey research on sexual behavior: Urban African American and Hispanic youth. J Sex Res 1991; 28:539–555.
21. Catania J, McDermott L, Pollack L. Questionnaire response bias and face-to-face interview sample bias in sexuality research. J Sex Res 1986; 22:52–72.
22. Krauth MR, St Lawrence JS, Kelly JA. Reliability of retrospective assessments of sexual HIV risk behavior. AIDS Educ Prev 1991; 3:207–214.
23. O'Reilly JM, Hubbard ML, Lessler JT, Biemer PP, Turner CF. Audio and video computer-assisted self interviewing: Preliminary tests of new technologies for data collection. J Offic Stat 1992; 10:197–214.
24. O'Reilly J, Hubbard M, Lessler J, Biemer P. Audio computer assisted self-interviewing: New technology for data collection on sensitive issues and special populations. Presented at the American Statistical Association Annual Meeting, Boston, August 12, 1992.
25. Hovell MF, Hillman ER, Blumberg E, et al. A behavioral-ecological model for adolescent sexual development: A template for AIDS prevention. J Sex Res 1994; 31:267–281.
26. Rodgers J. The rescission of behaviors: Inconsistent responses in adolescent sexuality data. Soc Sci Res 1982; 11:280–296.
27. Jones RB, Katz BP, Van Der Pol B, Caine VA, Batteiger BE, Newhall WJ. Effects of blind passage and multiple sampling on recovery of Chlamydia trachomatis
from urogenital specimens. J Clin Microbiol 1986; 24:1029–1033.
28. Krieger JN, Verdon M, Siegel N, Critchlow C, Holmer KK. Risk assessment and laboratory diagnosis of trichomoniasis in men. J Infect Dis 1992; 166:1362–1366.
29. Workowski KA, Lampe MF, Wong KG, Watts MG, Stamm WE. Long-term eradication of Chlamydia trachomatis
genital infection after antimicrobial therapy: Evidence against persistent infection. JAMA 1993; 270:2071–2075.
30. Rotheram-Borus M, Koopman C, Haignere C, Davies M. Reducing HIV sexual risk behaviors among runaway adolescents. JAMA 1991; 266:1237–1241.
31. DiClemente RJ, Wingood GM. A randomized controlled trial for an HIV sexual risk-reduction intervention for young African-American women. JAMA 1995; 274:1271–1276.
32. Blythe MJ, Katz BP, Batteiger BE, Ganser JA, Jones RB. Recurrent genitourinary chlamydial infections in sexually active female adolescents. J Pediatr 1992; 121:487–493.
33. Kinghorn GR, Pryce D, Morton RS. Repeated gonorrhea in Sheffield: The size of the problem, epidemiologic significance, and personal characteristics of repeaters. Ses Transm Dis 1982; 9:165–169.
34. Noble RC, Kirk NM, Slagel WA, Vance BJ, Somes GW. Recidivism among patients with gonococcal infection presenting to a venereal disease clinic. Sex Transm Dis 1977; 4:39–43.