Introduction: The aim of this study was to evaluate the reliability of self-reported HIV among men who have sex with men (MSM) in Brazil.
Methods: MSM 18 years of age or older were recruited to a multicenter study using respondent-driven sampling. We compared self-report of the HIV test with a rapid HIV test using the kappa coefficient.
Results: A total of 3859 MSM were recruited, and 39% reported ever having an HIV test; their results were reported and they agreed to a new test. Agreement between self-report and the test was very good (kappa = 0.88).
Conclusion: Our results suggest that self-report of HIV infection is a reliable indicator among MSM.
From *Universidade Federal do Ceará, Saúde Comunitária, Fortaleza, Brazil; †Tulane University School of Public Health and Tropical Medicine, Center for Global Health Equity, New Orleans, LA; ‡Fundação Oswaldo Cruz, CICT, Rio de Janeiro, Brazil; §US Centers for Disease Control and Prevention, Atlanta, GA; ‖Universidade Federal da Bahia, Instituto de Saúde Coletiva, Salvador, Brazil; ¶Universidade Federal de Minas Gerais, Medicina Preventiva, Belo Horizonte, Brazil; **Fundação Oswaldo Cruz, Centro de Pesquisas Aggeu Magalhães, Reicife, Brazil; ††Universidade Federal do Rio de Janeiro, NESC, Rio de Janeiro, Brazil; ‡‡Secretaria de Saúde de Santos, Santos, Brazil; §§Fundação Alfredo da Mata, Manaus, Brazil; ‖‖Secretaria de Saúde de Curitiba, Curitiba, Brazil; ¶¶Secretaria de Saúde de Itajaí, Itajaí, Brazil; ***Universidade de Brasília, Saúde Coletiva, Brasília, Brazil; †††Secretaria de Saúde de Campo Grande, Campo Grande, Brazil; and ‡‡‡University California San Francisco, Institute for Global Health, San Francisco, Brazil.
This work was carried out by the Federal University of Ceará with technical and financial support of the Ministry of Health/Secretariat of Health Surveillance/Department of STD, AIDS and Viral Hepatitis through the Project of International Technical Cooperation AD/BRA/03/H34 between the Brazilian Government and the United Nations Office on Drugs and Crime-UNODC.
The authors have no conflicts of interest to disclose.
Correspondence to: Ligia Regina Franco Sansigolo Kerr, MD, PhD, Departamento de Saúde Comunitária, Rua Prof. Costa Mendes 1608-5, Andar, CEP: 60.430-971, Fortaleza-CE, Brazil (e-mail: email@example.com; firstname.lastname@example.org).
To date, relatively few studies have examined the reliability of self-report of HIV infection in populations at risk for HIV infection. In injection drug users, McCusker1 found that 96.7% reported their results correctly, and later studies reported specificity above 99%.2-4 A multicenter study in the United States,5 evaluating the accuracy of self-reports of HIV testing among men who have sex with men (MSM) 50 years old and older, showed that all the men who had positive results on screening tests self-reported as HIV-positive and 99% of those with negative tests accurately reported themselves as negative. However, in the 2007 Kenya AIDS Indicator Survey (KAIS), 63% of largely heterosexual participants who had reported having tested negative before tested positive in the survey (negative predictive value, 37%); these were felt, in large part, to be interim seroconversions.6
Brazil has invested heavily in HIV behavioral surveillance (but not biological), especially in populations considered more vulnerable to infection. The reluctance to incorporate testing in surveillance is the result of concerns voiced by representatives of these most at-risk populations, including MSM. Many MSM expressed concern that reports of seroprevalence in the community would increase stigma. This was found in a recent study reporting results in Brazil.7 Precisely because testing is such a charged topic, every surveillance activity proposed debates the effect of including or requiring testing. For example, in the baseline national study of biological and behavioral surveillance, the National Program adopted the design of making testing voluntary and enrolling subjects who refused to test into the study. It was felt that this study would allow us to gauge the effect of requiring testing and help interpret seroprevalence in the current and future rounds of surveillance. Thus, this study aims to investigate the reliability of self-reported HIV test results in a multicenter study of MSM in Brazil.
MATERIAL AND METHODS
This was a multicenter study using respondent-driven sampling conducted in 10 cities: Manaus, Recife, Salvador, Campo Grande, Brasilia, Belo Horizonte, Rio de Janeiro, Santos, Curitiba, and Itajaí. Samples were collected to provide independent estimates for each city. The survey was conducted among 3859 men older than 18 years of age reporting sex with another man in the past year. The cities were selected by the Department of STD, AIDS and Viral Hepatitis to represent regional, socioeconomic, and cultural diversity. Details of the methods for this study are reported elsewhere.7
Our outcome variable was HIV infection as measured by finger stick rapid HIV test. Whole blood was used in all the diagnostic tests, and the study followed the national algorithm for rapid testing8: first the Rapid Check HIV-1 & 2 (Núcleo de Doenças Infecciosas [NDI], Vitória, Espírito Santo, Brazil) and Bio-Manguinhos HIV-1 & 2 (Instituto de Tecnologia em imunobiologicos, Bio-Manguinhos, FIOCRUZ, Rio de Janeiro, Brazil) at the same time followed by a third test in case the previous two tests did not match. Our principal predictor variable was self-report of HIV serostatus as measured by two questions: “Have you been tested for AIDS at any time?” and “Could you tell me the result of your last test?”
We offered rapid testing to participants after counseling,9 and we required written consent. Finger prick blood was drawn and results provided in less than 1 hour with counseling. Infected participants were referred to public centers to follow-up clinical and laboratory diagnosis of infection.8
We examined concordance between self-reported HIV infection and positive rapid test results using the kappa coefficient10 adjusted for probability and bias (PABAK).11,12 We also calculated the positive predictive value, negative predictive value (NPV), and false-positive and false-negative rates. Positive predictive value was calculated as the proportion who self-reported a prior positive test who tested positive and the NPV as the proportion who self-reported a prior negative test who tested negative. We also examined the NPV by municipality. For data analysis, we used Landmann Szwarcwald weights (published in this volume) that take into account individual social network size and the proportion of MSM in each city.13 After this calculation, we exported individual weights and calculated estimates of prevalence and prevalence ratios with their 95% confidence intervals (CIs) using Stata 11 (STATA Corp, College Station, TX). The association analysis was calculated using the likelihood ratio test.
The study was approved by the Brazilian National Research Ethics Committee (CONEP #14494).
Of a total of 3859 MSM surveyed, 51.5% reported having at least one test before the survey (Fig.1). Of these, 89.8% were willing to report their results; 0.7% did not report and 9.6% did not returned to collect their results. From those who were willing to report their results, 13.2% reported that they were HIV-infected. For the 1802 MSM who reported previous testing, 89.3% were tested in this study. Of those who self-reported as positive in the last test, 17.9% refused testing. One third of these refused because of the previous test. Of those who self-reported HIV-negative, 9.5% did not test. Of these, almost half (47.5%) claimed a previous result as a reason to not test. All these calculated percentages were weighted. The final sample of nonweighted percentage of participants in this study was 38.7% (1493 of 3859).
Of those who self-reported as positive in the last test, 17.9% refused testing. One third of these refused because of the previous test. Of those who self-reported HIV-negative, 9.5% did not test. Of these, almost half (47.5%) claimed a previous result as a reason to not test. A significant difference (P = 0.018) was observed in the proportion of self-reporting seropositive and seronegative men who refused to test (17.9% vs 9.5%, respectively) (prevalence ratio = 1.8) (Fig. 1).
Concordance of self-report and test was excellent (κ = 0.88 [± 0.01]). Among MSM who self-reported positive, the probability of testing positive was 100% (0.0% false-positive rate) (Table 1). Among those self-reporting negative, 92.9% (95% CI, 88.9-95.5) tested negative. Concordance ranged from 84.8% (95% CI, 67.9-93.6) in Itajaí to 99.4% (95% CI, 95.6-99.9) in Santos. The confidence interval for NPV and false-negative rate did not overlap for all cities.
DISCUSSION AND CONCLUSIONS
We found a high degree of concordance between self-report of prior HIV test results and test results produced in our study. Differences occurred in self-reported negative results. This result was consistent with Dolcino's5 findings in which discordance was only found among those who self-reported that they were uninfected. Other studies1-4 in injection drug users report similar results but with sensitivity lower than 100%. An obvious reason for this is seroconversion in the interval between the self-reported test and the study test. In KAIS, participants whose self-reports and actual test results were discordant were significantly more likely to have CD4 counts above 500 cells/μL than those who self-reported that they were infected, suggesting recent infection.6
More than half of respondents self-reported having had at least one HIV test before, and many of these participants did not test in this study. In the case of positive self-report, the selection bias introduced in estimates by the exclusion of self-reported results may be greater than the information bias introduced by their inclusion. To minimize information bias, it is important to ensure that respondents in intervention programs feel that their results will be confidential. The findings that the cities presented different NPV for HIV-and proportions of false-negative MSM indicate that prevalence estimates based on results from self-report may also need adjusting by municipality considering the differing socioeconomic and cultural contexts.
There are several potential limitations. More HIV-uninfected than HIV-infected men were tested in the study. There are obvious reasons for this selection bias but no reason to assume that if other self-reported seropositive individuals agreed to be tested the result would be any different. A second issue is the lack of a time-limited recall period for testing. For example, self-reported uninfected individuals may have had a longer time period since testing, making comparison difficult. A third and more important limitation is that we can say relatively little about the individuals who reported never testing, those who tested and did not report, and those who did not returned to collect their results. Although testing has increased in the last several years, it was not a routine part of surveillance in Brazil. The 13 MSM who refused to report their test result may very well be positive, but we cannot interpret the behavior of those 158 who did not return to collect their test. However, counseling could play an important role among those individuals increasing the chance of their return to collect their results. A population-based survey in Brazil showed that 55% of the total studied population, including MSM, reported having not received counseling.14 Both absence of routine testing for surveillance and absence of counseling could explain the low rate of testing and refusal to collect the results. Nonetheless, what this study clearly shows is that testing is still a controversial issue among the populations most affected by AIDS in Brazil and that the real prevalence among this population could be even higher than we observed in this study.
Our findings bolster our confidence in the reliability of self-reported measures of infection. Persons who are mistaken about being HIV-uninfected by virtue of a previous negative test may account for a disproportionate amount of transmission either by virtue of high viral loads early in infection or because their assumption of seronegativity leads them to have other HIV-negative partners. Persons with undiagnosed infection are also those who need to be linked to HIV care programs. Data on self-reported serostatus, coupled with serologic testing, may therefore be used to efficiently guide prevention and care efforts. As a general conclusion, we are comfortable with the idea that self-reported seropositivity be treated as HIV infection for purposes of surveillance.
The authors thank the Department of STD, AIDS and Viral Hepatitis (DSAVH) of the Ministry of Health, which funded the research (CSV 234/07), Center for Global Health Equity of the Tulane University for technical support, Institute for Global Health of the University of San Francisco California where the project was conceived, and Coordenação de Aperfeiçoamento de Pessoal de Nível Superior (CAPES) BEX 3495/06-0 for supporting L.R.F.S.K.'s postdoctoral. The authors also thank Dr Márcio Antônio de Sá and Dr. Gerson Pereira for coordinating the Project in the 10 cities on behalf of the DSAVH; Dr Ana Roberta Pati Pascom (from DSAVH) and Dr Aristides Barbosa (from the Centers of Diseases Control and Prevention/GAP Brazil) for their technical and implementation assistance in the respondent-driven sampling methodology in Brazil; and Prof Dr Gabriela Calazans, Dr Ligia Rivero Pupo, Prof Dr Fernando Seffner, and Prof Dr Alexandre Grangeiro for their suggestions.
1. McCusker J, Stoddard AM, McCarthy E. The validity of self-reported HIV antibody test results. Am J Public Health
2. Ross MW, Loxley W, Wodak A, et al. Drug users' self-reported false-positive HIV status. Am J Public Health
3. McCusker J, Stoddard AM. McCusker and Stoddard respond. Am J Public Health
4. Fisher DG, Reynolds GL, Jaffe A, et al. Reliability, sensitivity and specificity of self-report of HIV test results. AIDS
5. Dolcini MM, Catania JA, Stall RD, et al. The HIV epidemic among older men who have sex with men. J Acquir Immune Defic Syndr
7. Kerr LRFS, Pinho A, Mello M, et al. MSM in Brazil: baseline national data for prevalence of HIV. XVIII International AIDS Conference; Vienna; July 18-23, 2010.
10. Cohen J. A Coefficient of agreement for nominal scales. Educ Psychol Meas
11. Cicchetti DV, Feinstein AR. High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol
12. Feinstein AR, Cicchetti DV. High agreement but low kappa: I. The problems of two paradoxes. J Clin Epidemiol
13. Szwarcwald CL, Souza PRB Jr, Damacena GN, et al. Analysis of data collected by RDS among sex workers in 10 Brazilian cities, 2009: Estimation of the prevalence of HIV, variance and design effect. 2011;57(suppl 8):S129-S135.
14. França Júnior I, Calazans G, Zucchi EM. Mudanças no âmbito da testagem anti-HIV no Brasil entre 1998 e 2005. Rev Saude Publica