Objectives: To define the performance characteristics of the Focus ELISA HSV-1 and HSV-2 assay among 100 university students.
Study Design: HSV-1 and HSV-2 Focus ELISA and Western Blot assays were performed on sera from university students who reported no history of genital herpes.
Results: HSV-2 and HSV-1 seroprevalence by Western Blot were 3.4% and 48%, respectively. In this population, the positive predictive value of the Focus HSV-2 ELISA was 37.5%, the sensitivity was 100%, and specificity was 94.1%. The PPV of the Focus HSV-1 ELISA was 96.7%, the sensitivity was 69.0%, and the specificity was 97.8%.
Conclusions: In this low-prevalence population, the positive predictive value of the Focus HSV-2 ELISA test was low. This finding, together with those reported elsewhere, indicates that caution is warranted when recommending HSV screening in low-prevalence or heterogeneous populations. Consideration should be given to raising the cutoff index value for defining a positive test result.
A study of university students without a reported history of genital herpes who were tested for herpes simplex virus types 1 and 2 by Focus ELISA using Western Blot as the reference standard, found low positive predictive values.
From the *Department of Community and Public Health Nursing, Johns Hopkins University School of Nursing; †Department of Population, Family and Reproductive Health, Johns Hopkins University School of Public Health; and ‡Department of International Health, Johns Hopkins University School of Medicine, Baltimore, Maryland
This study was supported by a grant from NINR 5RO3NR9064.
Correspondence: Hayley Mark, PhD, Johns Hopkins University School of Nursing, 525 North Wolfe Street, Baltimore, MD 21205-2110. E-mail: email@example.com.
Received for publication September 18, 2006, and accepted December 18, 2006.
INFECTIONS CAUSED BY THE HERPES simplex virus types 1 (HSV-1) and 2 (HSV-2) are highly prevalent.1 They are associated with substantial morbidity and transmission and acquisition of the human immunodeficiency virus (HIV).2 In the reported literature, approximately 17% of adults in the United States have antibodies to HSV-2 and 58% have antibodies to HSV-1.1 Over two-thirds are unaware of their infections, and the majority of infections are transmitted by these individuals.3 While HSV-1 is primarily the cause oral-labial herpes and HSV-2 causes genital infection, HSV-1 accounts for increasing proportions of newly diagnosed primary genital herpes.4–7
In the last decade, type-specific serological assays that detect HSV-1 and HSV-2 antibodies have become commercially available, prompting debate about their use as a screening test. Some experts argue that identifying people with unrecognized HSV-2 may result in a decreased risk of transmission from such persons to others.8–10 Others express concerns about the accuracy of the tests, the burden on healthcare practitioners to provide counseling, and the psychosocial distress that may accompany a diagnosis.11–13 Because of the low associated morbidity and lack of associated social stigma, until recently serological testing for HSV-1 had not been considered valuable by most experts and had not been requested by patients. However, interest in HSV-1 testing has increased concomitantly with the awareness of HSV-1 as a cause of genital herpes and the affect of HSV type on the prognosis and subsequent counseling related to an infection. In addition, experts have recently suggested that an awareness of HSV-1 antibody status can assist in the interpretation of a serological diagnosis of HSV-2.14
In the Prevention Agenda for Genital Herpes, the Centers for Disease Control and Prevention called for studies that describe the “real world” performance of type-specific assays for herpes virus infections.15 HerpeSelect HSV-1 and HerpeSelect HSV-2 enzyme-linked immunosorbent assays (ELISA) (Focus Diagnostics, Cypress, CA) are among the most commonly used tests for serodiagnosis of HSV infection16 and have previously been shown to be sensitive and specific for the diagnosis of HSV-2 in high-prevalence populations.17–19 However, the prevalence of a disease in a population affects screening test performance. Positive predictive value (PPV) refers to the proportion of patients with positive test results who actually have the disease (i.e., true positives), and in low-prevalence settings, even tests with high sensitivity and specificity can have poor positive predictive values. Several recent studies found that false-positive HSV results are not unlikely in select patient groups tested by ELISAs.14,20 We embarked on a study of university students with no history of genital herpes to develop appropriate counseling strategies for HSV testing and identified problems with the positive predictive value of certain tests.
Study Subjects and Procedures
Participants from 1 urban university were recruited by flyers and announcements in classrooms as part of a broader study examining the impact of serological testing on students without a history of genital herpes or genital sores. Eligible subjects were: (a) 18–40 years of age; (b) full- or part-time students; (c) sexually active (i.e., self-reported having oral, anal, or vaginal sex in the past 6 months) and (d) without a known history of genital sores or genital herpes.
After obtaining written informed consent, data collection consisted of self-administered questionnaires that assessed demographic characteristics, herpes knowledge, depression, anxiety, and sexual behavior. Blood samples were collected and participants were provided with HSV test results by phone approximately 2 weeks after enrollment. Participants were compensated $20.00 for questionnaire completion. Subjects who tested HSV-2 positive were asked to return to the clinic within 1 week to meet with the study staff to ensure that all questions were answered. In addition, all subjects were offered free counseling from the school clinic and extra meetings with the study staff to review results and answer questions. The research protocol was reviewed and approved by the University Institutional Review Board.
Samples were initially tested by HerpeSelect HSV-1 ELISA IgG and by HerpeSelect HSV-2 ELISA IgG (Focus Diagnostics) at the Zenilman laboratory and then stored. ELISA tests were performed according to the manufacturer's instructions. The results of Herpe Select HSV-1 and HSV-2 ELISAs were defined as follows: index values <0.9, negative; >1.1, positive; and inclusive for values between 0.9 and 1.1.
In response to a test result with a low index value, subsequent difficulty with interpretation, and new concerns about the PPV of HSV ELISA tests, Western blot (WB) testing was added to the protocol for all collected samples. All samples were retested for HSV-1 and HSV-2 by WB at the University of Washington Diagnostic Virology Laboratory (UWDVL).
After study enrollment was nearly complete, Focus Diagnostics released a second generation HerpeSelect ELISA test. All samples which had positive first generation HSV-2 Focus ELISA results were reassessed with the second-generation test (n = 8). To further evaluate the samples with discordant results, second-generation HSV-2 Focus ELISA results that were discordant with WB results were sent to Focus Diagnostics for retesting using the HSV-2 inhibition assay as described by Hoegrefe et al.21 (N = 3). Briefly, the patient's serum was diluted in diluents, 1 containing purified HSV-1 viral lysate and the second containing purified HSV-2 viral lysate. The samples were then tested per the Focus HSV-2 procedure using the HerpeSelect gG-2 coated microtiter plates. The absorbance of the HSV-2 lysate sample was divided by the absorbance HSV-1 lysate sample to calculate the percent inhibition caused by the addition of HSV-2 lysate. A percent inhibition of 60% or greater confirmed the presence of HSV-2 antibodies. The HSV-1 lysate diluent served as a control for any effects of nongG2 material present in the viral lysate.
Sensitivity, specificity, positive predictive value, and Fisher exact methods for 95% CI of the Focus-HSV tests were calculated using the University of Washington WB results as the reference standard. Frequency distributions of selected demographic and behavioral measures were computed using SPSS.
Study participants were, on average, 24.5 years old (range 18–39). Almost two-thirds were female (64%) and more than two-thirds (69%) were white, reflecting the ethnic composition of university student body. Two out of five participants were enrolled in graduate programs (Table 1).
Sexual Orientation, Behavior, and Condom Use
The majority of the participants were self-reported heterosexuals (86%). Three-quarters of the participants reported 1 main sex partner in the past 3 months and one quarter reported at least 1 casual sex partner during the same time period. Almost half (47%) of the participants reported using a condom most of the time or always with a main sex partner during the past 3 months. Among those who reported having casual sex partners, 45% reported using condoms always during sex and 12% reported never using condoms during sex with their casual partners (Table 1).
HSV-2 gG ELISA Performance
One hundred subjects enrolled in the study. Because WB testing was initiated after the start of the project, several of the first samples (n = 8) were not sent for WB testing and were excluded from analysis. In addition, 4 inconclusive results for HSV-1 and 3 inconclusive results for HSV-2 were excluded from the analysis.
Eighty-nine samples were tested for HSV-2 by Focus ELISA and WB. Three of the 89 samples were HSV-2 positive by WB (3.4% prevalence). Eight of the 89 samples were HSV-2 positive by the Focus ELISA tests (9% prevalence). Compared with HSV-2 Western blot analysis, sensitivity of the HSV-2 Focus ELISA was 100% (95% CI 30.9–100%), and specificity was 94.1% (95% CI 86.3–97.8). Positive and negative predictive values (PPV, NPV) of the Focus ELISA were 37.5% (CI 10.2–74.1) and 100.0% (CI 94.3–100.0), respectively (Table 2).
Because we tested only the 8 HSV-2 positive samples with the second-generation Focus ELISA test, we were unable to accurately calculate the sensitivity, specificity, and predictive value of the new kit. However, of the 8 participants who had a positive HSV-2 result with the first generation Focus ELISA test, 6 tested positive when the second-generation test was used. Three of these six participants also tested positive by WB. Two participants who tested negative on the second-generation Focus ELISA test were also found to be negative by WB (Table 3).
As mentioned previously, the 3 samples that had discordant second generation Focus ELISA and WB results were sent to Focus Diagnostics for inhibition assays. Index values were reassessed there before inhibition assays were run. One sample had a negative index value when tested by Focus Diagnostics, and therefore, an inhibition test was not completed on this sample. The 2 remaining samples had inhibition assays >90% and were interpreted as positive for HSV-2 by the inhibition assay criteria.
HSV-1 gG ELISA Performance
Eighty-eight samples were tested for HSV-1 by Focus ELISA and WB. By WB analysis, HSV-1 prevalence was 48%. Concordant results for HSV-1 were obtained for all but 11 specimens. With the exception of one sample, all of the HSV-1 discordant samples were Focus negative, but WB positive. Compared with HSV-1 WB, sensitivity of the HSV-1 Focus ELISA was 69.0% (95% CI 52.7–81.9%), and specificity was 97.8% (95% CI 87.0–99.9). Positive and negative predictive values of the Focus ELISA were 96.7% (CI 80.9–99.8) and 77.6% (CI 64.4–87.1), respectively (Table 2).
Concern about the performance of commercially available type-specific serological testing for HSV-2 has increased the need for studies in low-prevalence populations. In our study, among university students with a 3.4% seroprevalence of HSV-2 by WB, the positive predictive value of the Focus HSV-2 ELISA test was 37.5% (95% CI 10.2–74.1). Even though our sample size was small, this rate is unlikely to be acceptable to clinicians or patients.
Other recent investigations have raised concerns about the PPV of ELISA tests for HSV-2 in low-prevalence populations. In a recent analysis of sera from 108 men seeking clinical care at the University of Washington STD clinic, a subset of a group with a 13% prevalence of HSV-2 by ELISA, Golden et al. report a PPV of 85%.14 In an analysis of 1238 sera from non-HIV infected Vietnamese housewives with a seroprevalence of 8.7% by ELISA, only 25 out of 108 samples that were positive by ELISA were confirmed positive by WB.22 This corresponds with a PPV of 23.1%. Other investigators have used data from higher prevalence populations to estimate the PPV in lower-prevalence groups. Ashley-Morrow and colleagues estimated a PPV of 61.9% in a 10% seroprevalence population using data from 2 populations in the United States.23 Turner and colleagues estimated a PPV of 66% in a 10% prevalence population of low-income women in California.19
This and other studies highlight a major issue interpreting positive HSV-2 results in low-prevalence populations. Low positive predictive values reflect frequent positive results in individuals who do not actually have the disease (i.e., false positives). Although most guidelines do not recommend HSV-2 serological screening in the general population or in pregnant women,24–26 there are growing calls from experts to expand screening programs to these populations.27,28 As illustrated in this study, false positives are not unlikely in low-prevalence populations. Incorrectly labeling people as infected with HSV may have significant consequences for their well-being and perceived health.
In an effort to improve the accuracy of the ELISA tests for HSV, some authors have suggested selecting a higher index cutoff value to define positivity.14,21,29 We found that discordant ELISA and Western Blot results in this study were more common in samples with low index values. In the current study, increasing the cut off index value defining positivity to 3.0 would have increased the PPV to 75.0%. Clinicians using this test in low-prevalence populations should consider using a higher index value to define positivity.
This study also illustrates current challenges in determining the true HSV-2 status of an individual using a serological test. The Western Blot is currently considered the gold standard for detecting HSV, and it is the test by which the performance characteristics of most HSV ELISAs are determined. However, in a recent study of patients with culture-documented first episodes of genital herpes, HSV seroconversion was detected significantly faster by HerpeSelect than by WB.30 In our study, 2 of the samples that tested HSV-2 positive by WB tested HSV-2 negative by both Focus ELISA and inhibition assays. We did not collect data on exposure to HSV, nor on the development of symptoms after testing, but it is possible that these 2 results reveal early HSV-2 seroconversions and the positive Focus ELISA result reflects the actual disease status, rather than the negative WB. Test accuracy may vary by prevalence of the disease, timing of the test, and the presence or absence of symptoms. More studies are needed to evaluate the limitations of serological tests for HSV.
HSV-1 results were concordant in 86% of the samples tested by both ELISA HSV-1 and WB. All but 1 of the discordant results were ELISA HSV-1 negative but WB positive. This rate corresponds to a PPV of 96.7%, but a sensitivity of only 69%. Ashley-Morrow and colleagues recently presented an analysis of the performance characteristics of Focus-HSV-1 in 969 sera from 8 countries and found a sensitivity of 99% and specificity of 78%.20 Performance characteristics varied by country and suggested a geographic difference in test performance. Significant differences in the sites and HSV-1 prevalence (93–99% in the study of Ashley-Morrow et al., 48% in our study) might explain the different performance characteristics in these 2 studies. Further investigation is warranted to determine the performance characteristics of the Focus HSV-1 tests.
There are several limitations to this study. First this was a small sample which was not sufficient to narrowly define the sensitivity and specificity of the tests. More studies are clearly needed to determine the performance characteristics of these tests in low-prevalence populations. However, our findings are in range of what others have found in populations with a low HSV-2 prevalence. Second, as mentioned previously, Focus Diagnostics has released a second-generation HerpeSelect ELISA. We evaluated the 8 HSV-2 positive samples with this new test, and of the 5 samples that had discordant ELISA and WB HSV-2 results using the first generation test, 2 were no longer discordant when using the second-generation kit. Thus, it appears that the second-generation Focus ELISA may be somewhat more specific than the first-generation test. Finally, we used a convenient sample of students at 1 university. These findings can not necessarily be generalized to other populations or even students at other universities.
In summary, these data and those reported elsewhere suggest that the ELISA may have an unacceptably low PPV among populations with a relatively low prevalence of HSV-2. Clinicians using the test among low-prevalence populations should be prepared to inform patients of the possibility of false-positive results and should consider using a higher index value to define positivity. Rates of HSV vary considerably by age, gender, race, and other risk factors. Serological testing for HSV may play an important role in prevention programs in high-risk target populations, but these results emphasize the risks of using 1 currently available test to screen for HSV-2 in low-prevalence populations.
1. Xu F, Sternberg MR, Kottiri BJ, et al. Trends in herpes simplex virus type 1 and type 2 seroprevalence in the United States. JAMA 2006; 296:964–973.
2. Freeman EE, Weiss HA, Glynn JR, et al. Herpes simplex virus 2 infection increases HIV acquisition in men and women: Systemic review and meta-analysis of longitudinal studies. AIDS 2006; 20:73–83.
3. Brugha R, Keersmaekers K, Renton A, et al. Genital herpes infection: A review. Int J Epidemiol 1997; 26:698–709.
4. Schillinger JA, Xu F, Sternberg MR, et al. National seroprevalence and trends in herpes simplex virus type 1 in the United States, 1976–1994. Sex Transm Dis 2004; 31:753–760.
5. Lamey PJ, Hyland PL. Changing epidemiology of herpes simplex virus type 1 infections. Herpes 1999; 6:20–24.
6. Coyle PV, O'Neill HJ, Wyatt DE, et al. Emergence of herpes simplex type 1 as the main cause of recurrent genital ulcerative disease in women in Northern Ireland. J Clin Virol 2003; 27:22–29.
7. Nieuwenhuis RF, van Doorum GJ, Mulder PG, et al. Importance of herpes simplex virus type-1 (HSV-1) in primary genital herpes. Acta Derm Venereol 2006; 86:129–134.
8. Wald A, Krantz R, Selke S, et al. Knowledge of partners' genital herpes protects against herpes simplex virus type 2 acquisition. J Infect Dis 2006; 194:42–25.
9. Corey L, Handsfield HH. Genital herpes and public health: Addressing a global problem. JAMA 2000; 283:791–794.
10. Brown ZA. HSV-2 specific serology should be offered routinely to antenatal patients. Rev Med Virol 2000; 10:141–144.
11. Brugha R, Brown D, Meheus A, et al. Should we be screening for asymptomatic HSV infections? Sex Transm Infect 1999; 75:142–144.
12. Wilkinson D, Barton S, Cowen F. HSV-2 specific serology should not be offered routinely to antenatal patients. Rev Med Virol 2000; 10:145–153.
13. Mindel A, Taylor J. Debate: The argument against. Should every STD clinic patient be considered for type-specific serological screening for HSV? Herpes 2002; 9:35–37.
14. Golden MR, Morrow RA, Swenson P, et al. Herpes simplex virus type 2 (HSV-2) western blot confirmatory testing among men testing positive for HSV-2 using the focus enzyme-linked immunosorbent assay in sexually transmitted disease clinic. Sex Transm Dis 2005; 32:771–777.
15. Handsfield HH, Stone KM, Wasserheit JN. Prevention agenda for genital herpes. Sex Transm Dis 1999; 26:228–231.
16. Ashley-Morrow R, Brown ZA. Common use of inaccurate antibody assays to identify infection status with herpes simplex virus type 2. Am J Obstet Gynecol 2005; 193:261–262.
17. Ashley RL, Eagleton M. Type specific antibodies to HSV-1 and HSV-2: Review of methodology. Herpes 1998; 5:33–38.
18. Prince HE, Ernst CE, Hogrefe WR. Evaluation of an enzyme immunoassay for measuring herpes simplex virus (HSV) type 1 and HSV type 2 specific IgG antibodies. J Clin Lab Anal 2000; 14:14–16.
19. Turner KR, Wong E, Kent CK, et al. Serologic herpes testing in the real world. Validation of new type-specific serologic herpes simplex virus tests in a public health laboratory. Sex Transm Dis 2002; 29:422–425.
20. Ashley-Morrow R, Nollkamper J, Robinson NJ, et al. Performance of Focus ELISA tests for herpes simplex virus type 1 (HSV-1) and HSV-2 antibodies among women in ten diverse geographical locations. Clin Microbiol Infect 2004; 10:530–536.
21. Hoegrefe W, Su X, Song J, et al. Detection of herpes simplex virus type 2 specific immunoglobulin G antibodies in African sera by using recombinant gG2, western blotting and gG2 inhibition. J Clin Microbiol 2002; 40:3635–3640.
22. Ngo TH, Laeyendecker OB, La H, et al. Comparison between Sure-Vue Rapid HSV-2 Test, Focus and Kalon enzyme-linked assays (ELISA) for antibodies to herpes simplex virus type 2 glycoprotein G in a low-risk population using Western Blot as the “Gold Standard”. Poster presentation. 2006 National STD Conference, Jacksonville, FL, May 2006.
23. Morrow RA, Friedrich D, Meier A, et al. Use of biokit HSV-2 rapid assay to improve the positive predictive value of Focus HerpeSelect HSV-2 ELISA. BMC Infect Dis 2005; 5:1–7.
24. US Preventive Services Task Force (USPSTF). Screening for genital herpes: Recommendation statement. Rockville, MD: Agency for Healthcare Research and Quality (AHRQ); 2005:11.
25. CDC. STD treatment guidelines, 2006. MMWR 2006; 55:1–94.
26. American College of Obstetricians and Gynecologists. Management of herpes in pregnancy: Clinical management guidelines for obstetrician-gynecologists. Int J Gynaecol Obstet 2000; 68:165–173.
27. Hook EW, Leone P. Time to translate new knowledge into practice: A call for a national genital herpes control program. J Infect Dis 2006; 194:6–7.
28. Brown ZA, Gardella C, Wald A, et al. Genital herpes complicating pregnancy. Obstet Gynecol 2005; 106:845–856.
29. Laeyendecker O, Henson C, Gray RH, et al. Performance of a commercial, type-specific enzyme lined immunosorbent assay for detection of herpes simplex virus type 2 specific antibodies in Ugandans. J Clin Microbiol 2004; 42:1794–1796.
30. Ashley-Morrow R, Krantz E, Wald A. Time course to seroconversion by HerpeSelect ELISA after acquisition of genital herpes type 1 (HSV-1) or HSV-2. Sex Transm Dis 2003; 30:310–314.