Herpes simplex virus type 2 (HSV-2) infection is the most common cause of genital ulcer disease worldwide,1,2 and is characterized by asymptomatic periods during which subclinical genital shedding may frequently occur.3,4 Accurate serological tests are therefore needed to determine HSV-2 infection status rather than relying on presence of symptoms. HSV-2 is associated with increased risk of HIV acquisition and infectiousness,5 and epidemiologic studies of the association with HIV rely on accurate laboratory tests to identify HSV-2 serostatus. For example, several recent randomized controlled trials have evaluated the potential for HSV suppressive therapy to reduce HIV acquisition, infectivity, and transmission6–13 and rely on high test specificity to avoid enrolment of HSV-2 uninfected individuals. In addition, studies using HSV-2 as a biologic marker for risky sexual behavior require accurate tests for HSV-2 infection.14,15 Accurate tests for HSV-2 are also important for determining prevalence or incidence in the general population as might be the case in studies evaluating risk factors for HIV or in studies evaluating a vaccine for HSV-2, if this were available.
Gold standard noncommercial tests for HSV-2 include the immunodot enzyme assay (IEA) developed at Emory University, Atlanta, GA; the Western blot test developed at the University of Washington (UW-WB) and the monoclonal antibody-blocking enzyme immunoassay (MAb EIA) developed by the Central Public Health Laboratory in the United Kingdom.16 These tests are used in their respective reference laboratories but are not feasible to replicate in many settings, thereby limiting their suitability for large-scale epidemiologic studies. The UW-WB test has been used as a gold standard in several studies, including evaluation of commercial serological assays required for clearance by the Food and Drug Administration (United States), and in the evaluation of the performance of other gold standard tests.16,17 The IEA uses immunoaffinity purified glycoprotein gG-2, which is specific for HSV-2.18,19 When evaluating IEA in symptomatic patients with culture confirmed shedding of HSV-2, it showed a sensitivity of 98% and specificity of 99%18 and has been used in large national studies to document epidemiologic trends of HSV-2 in the United States.18,19 The MAb EIA is the main reference test for HSV-2 in the United Kingdom.16 The performance of MAb EIA was found to be comparable to UW-WB among samples from England and Wales (98% concordant results among 100 samples tested by both assays)20 and it performed well when compared against UW-WB in African samples (sensitivity = 98%, specificity = 97%).17
Several type-specific serological tests for HSV-2 are commercially available. These tests detect the HSV-2 specific glycoprotein gG-2, and therefore, can distinguish between HSV-1 (gG-1) and HSV-2.16 This distinction is essential given the almost universal prevalence of HSV-1 infection in many settings.21 These commercial tests are easier, faster, and more cost-effective to perform than Western blot (WB). Studies from industrialized countries indicate good overall performance,22 but poor specificity has been noted in several studies from sub-Saharan Africa.17,22,23 We conducted a systematic review of studies evaluating the most commonly used commercial HSV-2 serologic tests against accepted gold standards in African populations and a meta-analysis of test performance, including effect-modification by HIV infection.
MATERIALS AND METHODS
We searched PubMed up to February 7, 2009, to identify studies evaluating the performance of HSV-2 serologic tests against accepted gold standards (WB or MAb-EIA) in sub-Saharan African populations. An initial exploratory search was done using the phrases “HSV-2 test performance” or “HSV-2 test validation” and following links for “related articles.” Reference lists of these articles were checked for other relevant studies. The MeSH terms of these relevant articles were then examined and a systematic review was performed using the following combinations of MeSH terms: (“Herpesvirus 2, Human” OR “Herpes Genitalis”) and (“Laboratory Techniques and Procedures” OR “Reagent Kits, Diagnostic”). The reference lists of identified studies were also searched for additional studies. In addition, we obtained known unpublished work and draft/in press articles from authors. The data abstracted were descriptive items (author name, journal, publication year), gold standard used, HSV-2 test, and cut-off used, and sensitivity and specificity. Further details were obtained from authors as necessary. Many publications included data of test performance stratified by HIV status, and these were used for separate analyses.
Studies were eligible for inclusion in the meta-analysis for the following reasons: (i) whether they evaluated performance of either Kalon or Focus using WB or MAb-EIA as the gold standard on blood samples from sub-Saharan Africa, (ii) whether they reported sensitivity and/or specificity or whether they provided sufficient data to calculate these, and (iii) whether they reported the cut-off optical density (OD) index value used.
To pool estimates of sensitivity and specificity across studies, we conducted a random-effects meta-analysis using Stata version 10.0. The primary analysis included only studies that assessed test performance using the manufacturer's recommended cut-off OD index value. Confidence intervals (95% CI) were estimated from the data using the Wilson score interval, which is suitable for probabilities approaching 1.24 A further analysis was restricted to studies that used OD cut-off values other than that recommended by the manufacturers. Possible heterogeneity due to HIV infection was explored using meta-regression. In 1 article, analyses were based on MAb EIA as the gold standard, because this comparison had more complete data and provided results similar to those of the UW-WB test.17
Of 882 abstracts identified through the literature search, 42 were deemed potentially relevant as the abstract indicated they were likely to fulfill the eligibility criteria. Full-text copies of these articles were reviewed, and 7 of these were found to be relevant.17,22,23,25–28 In addition, one recently published article was included that was identified during the initial exploratory search but did not appear in the final search as it had not yet been indexed,29 2 draft papers (now in press) were obtained from collaborating coauthors (Lingappa et al., personal communication, 2009).30 Data from one unpublished evaluation of the performance of Kalon conducted by the MRC/UVRI Uganda Research Unit on AIDS in the general population of Masaka district, Uganda, were also included. We excluded one article26 from further analyses as it used data from the same population from Rakai, Uganda, as a more recent study.23 Altogether these 10 articles contained 35 substudies from 18 study populations from 8 African countries.
The most commonly evaluated assays were the Focus HerpeSelect HSV-2 enzyme-linked immunosorbent assay (ELISA) (Focus Technologies, Cypress, CA) (21 substudies) and the Kalon gG2 ELISA (Kalon Biologicals Ltd, Guilford, United Kingdom) (12 substudies). In addition, 2 substudies evaluated the performance of Biokit Rapid Assay (BioKit USA Inc, Lexington, MA)23 (Lingappa et al., personal communication) (Table 1).
The pooled sensitivity and specificity of Focus and Kalon are shown in Figure 1. Performance of Focus using the manufacturer's recommended cut-off OD index value of 1.1 was assessed in 17 study populations17,22,23,25,27–30 (Lingappa et al., personal communication) adding up to 3387 samples. The random-effects summary sensitivity for Focus was 99% (95% CI: 99–100), but specificity was poor (69%, 95% CI: 59–80). There was significant between-study heterogeneity for specificity (P < 0.001). Performance of Kalon using the manufacturer's recommended cut-off OD index value of 1.1 was assessed in 9 study populations17,23,28–30 (Lingappa et al., personal communication) and in the unpublished evaluation in Masaka district, Uganda, adding to 2206 samples. The summary estimate of sensitivity was 95% (95% CI: 93–97) with specificity of 91% (95% CI: 86–95). Specificity results from 1 study (Hogrefe, Kenya B) were excluded, because there was only 1 negative result on UW-WB and no negative result on Focus. For Kalon, there was significant between-study heterogeneity for both sensitivity (P = 0.005) and specificity (P < 0.001).
Four studies evaluated Focus using a higher OD cut-off (between 2.2 and 3.5) totaling 1530 samples22,23,29 (Lingappa et al., personal communication) (Table 1). Amongst these studies, the sensitivity was 91% (95% CI: 85%–97%) and specificity 85% (95% CI: 79%–92%) (Fig. 2). Similarly, 3 studies evaluated Kalon using a higher OD cut-off for a total of 923 samples23,29 including the unpublished Masaka study (Table 1). The pooled estimate of sensitivity was 93% (95% CI: 90%–95%) and specificity 89% (95% CI: 83%–95%). At these higher cut-off ODs also, there was significant heterogeneity for both sensitivity (P < 0.001) and specificity (P = 0.004) for Focus, and for specificity of Kalon (P = 0.03).
Using the manufacturer's recommended cut-off OD index value of 1.1, sensitivity of both Focus and Kalon were high and similar in HIV-positive and HIV-negative samples (Fig. 3). In contrast, specificity of Focus was significantly lower among HIV-positive (54%, 95% CI: 40–68) than among HIV-negative samples (69%, 95% CI: 56–82; P = 0.02) (Fig. 4), although there was substantial between-study heterogeneity amongst HIV-negative samples (P < 0.001). There was a similar trend for Kalon, although the differences by HIV status were less pronounced. Among HIV-positive samples, specificity was 88% (95% CI: 75–100); among HIV-negative samples, specificity was 93% (95% CI: 88–98) (P = 0.7). However, there was substantial heterogeneity for specificity of Kalon among both the HIV-positive (P = 0.001) and HIV-negative samples (P < 0.001). Of note, few HIV-positive samples were also HSV-2 seronegative, so there were wide confidence intervals for specificity of the tests among HIV-positive samples.
The performance of Biokit was inconsistent in the 2 studies that evaluated it in Uganda. Biokit had a sensitivity of 95.8% and specificity of 56.1% in Rakai District, and a sensitivity of 86.4% and specificity of 97% in Kampala city (Lingappa et al., personal communication).23
Most studies of HSV-2 epidemiology in sub-Saharan Africa have used the Kalon or Focus assays. Our findings show that Focus tended to have low specificity (summary estimate 69%) compared with the gold standard when using the manufacturer's recommended cut-off (OD = 1.1). Kalon tends to perform better when compared with Focus in the same study populations, although performance also varied across studies. These results contrast with studies using North American samples, which have found specificity of Focus of 96% to 97% in antenatal and STI clinic populations.31 The study of Nigerian samples included in our review also evaluated samples from 6 countries in Asia and South/Central America, and found specificity of 91% to 95% in these non-African settings.22 In that study, sensitivity was also very high (98%–100%) in all but 1 site (91% in Songkla, Thailand).
Possible causes of discrepant results between Focus, Kalon, and UW-WB may include a low sensitivity of UW-WB and Kalon to detect either recent HSV-2 seroconversions,32 presence of African HSV-2 strain variants not detectable by the commercial tests, or selective cross-reactivity of antibodies to non-HSV-2 proteins in the Focus test. A study in Seattle found poor sensitivity of UW-WB to detect early HSV-2 seroconversions.33 In this study, Focus detected seroconversion earlier (21 days) than UW-WB (68 days) or Kalon (120 days).33 Similarly, in a study comparing performance of Focus and Kalon directly among genital ulcer disease patients in Ghana and the Central African Republic, Focus detected significantly more HSV-2 seroconverters among patients with proven first episode genital HSV-2 at Day 14 following presentation (77% vs. 23%; P = 0.01).34 Authors of 2 studies have proposed the use of an HSV-2 recombinant gG ELISA inhibition assay as an alternative gold standard to the WB.25,29 This assay measures antibody binding to multiple epitopes of HSV-2 gG2 present in cell culture lysates to inhibit the binding of gG2-specific antibodies to recombinant gG2.25,29 However, the assay has not been evaluated exhaustively and will require further study. Sensitivity may also be lower of Kalon and UW-WB then Focus, as a result of cross-reactivity with HSV-1 antibodies is also a possibility. HSV-1 in many African settings is more than 90% compared with 50% to 70% in North America.21
The lower specificity of Focus observed in these African populations may be in part because of cross-reactivity with unidentified antibodies that might be common in African populations but are uncommon elsewhere. It may also be due to nucleotide polymorphism in the gG2 sequence that are specific to African populations. Although a European study35 found that the gG2 epitope was highly conserved, HSV-2 sequences in African populations have not yet been characterized in the GenBank database.36 Sequencing of HSV-2 strains in African populations may also help explain some of the geographical differences in test performance. HIV infection appears to cause a reduction in specificity. The reasons for this are not clear, although it is plausible that immune responses associated with HIV infection, such as immunoglobulin G polyclonal immune stimulation, may cause cross-reactions in the test.37 Finally, operator error may cause low specificity, for example whether there is contamination of samples by splashing during dilution of sera or during transfer of diluted sera to test plates. Ashley-Morrow et al. showed that as little as 0.5 μl of added positive serum could convert a Focus negative to a low positive.22 It should be noted that this low specificity of Focus was not observed in all African populations. Two studies among HIV negative participants in Zimbabwe did not find lower specificity.
Biokit rapid assay results were highly variable across 2 studies. This could be due to inherent differences in the populations studied or, more likely, to variability in reading criteria for designating a test as positive.30
The studies reviewed show that in these sub-Saharan African populations a useful strategy to improve performance is to use an index cut-off higher than that recommended by the manufacturer. Optimal performance was obtained when the cut-off was increased to 1.5 for Kalon, and to 3.4 or 3.5 for Focus. Studies have evaluated algorithms using Focus as the initial screening test, at either manufacturer's or higher cut-off OD index values, followed by confirmatory testing of all positives with Kalon, but these strategies have not proven to be advantageous over using Kalon alone23,29 with some exceptions.30
Our review has some limitations. Heterogeneity was observed in most analyses, implying that the performance of the tests may have differed by study population. The summary estimates should therefore be interpreted with caution. Also, the studies that were included in the meta-analysis evaluating the effect of increasing the cut-off OD did not always use the same cut-off OD. This may limit the comparability of these studies. However, in 5 of 7 studies, the assays were optimized for the study population and the most appropriate higher cut-off OD was used, thereby maximizing performance of the tests for these populations.
To summarize, the specificity of commercially available HSV-2 serological tests using African samples appear generally inferior to that reported from industrialized countries. There is a large variation in performance depending on geographical location and characteristics of the study populations. Therefore, studies using HSV-2 testing would benefit from an evaluation of test performance in the proposed study population, bearing in mind the aims of testing, for example, estimation of prevalence, establishing etiology of genital ulcers, estimating the effect of HSV-2 on risk of HIV acquisition, or infectivity. Different assays could be used for different purposes, either requiring high sensitivity or high specificity. Increasing the cut-off OD to 1.5 for Kalon and between 2.2 and 3.5 may improve test performance. An assay that is well suited for African populations is needed.
1. Malkin JE. Epidemiology of genital herpes simplex virus infection in developed countries. Herpes 2004; 11(suppl 1):2A–23A.
2. Weiss H. Epidemiology of herpes simplex virus type 2 infection in the developing world. Herpes 2004; 11:24A–35A.
3. Mark KE, Wald A, Magaret AS, et al. Rapidly cleared episodes of herpes simplex virus reactivation in immunocompetent adults. J Infect Dis 2008; 198:1141–1149.
4. Mertz GJ. Asymptomatic shedding of herpes simplex virus 1 and 2: Implications for prevention of transmission. J Infect Dis 2008; 198:1098–1100.
5. Corey L, Wald A, Celum CL, et al. The effects of herpes simplex virus-2 on HIV-1 acquisition and transmission: A review of two overlapping epidemics. J Acquir Immune Defic Syndr 2004; 35:435–445.
6. Watson-Jones D, Weiss HA, Rusizoka M, et al. Effect of herpes simplex suppression on incidence of HIV among women in Tanzania. N Engl J Med 2008; 358:1560–1571.
7. Celum C, Wald A, Hughes J, et al. Effect of aciclovir on HIV-1 acquisition in herpes simplex virus 2 seropositive women and men who have sex with men: A randomised, double-blind, placebo-controlled trial. Lancet 2008; 371:2109–2119.
8. Nagot N, Ouédraogo A, Foulongne V, et al. Reduction of HIV-1 RNA levels with therapy to suppress herpes simplex virus. N Engl J Med 2007; 356:790–799.
9. Zuckerman RA, Lucchetti A, Whittington WL, et al. Herpes simplex virus (HSV) suppression with valacyclovir reduces rectal and blood plasma HIV-1 levels in HIV-1/HSV-2 seropositive men: A randomized, double-blind, placebo-controlled crossover trial. J Infect Dis 2007; 196:1500–1508.
10. Baeten JM, Strick LB, Lucchetti A, et al. Herpes simplex virus (HSV)-suppressive therapy decreases plasma and genital HIV-1 levels in HSV-2/HIV-1 coinfected women: A randomized, placebo-controlled, cross-over trial. J Infect Dis 2008; 198:1804–1808.
11. Dunne EF, Whitehead S, Sternberg M, et al. Suppressive acyclovir therapy reduces HIV cervicovaginal shedding in HIV- and HSV-2-infected women, Chiang Rai, Thailand. J Acquir Immune Defic Syndr 2008; 49:77–83.
12. Delany S, Mlaba N, Clayton T, et al. Impact of HSV-2 suppressive therapy on genital and plasma HIV-1 RNA in HIV-1 and HSV-2-seropositive women not taking anti-retroviral therapy: A randomized, placebo-controlled trial in Johannesburg, South Africa. AIDS 2009; 23:461–469.
13. Celum C, Wald A, Lingappa J, et al. Twice-daily acyclovir to reduce HIV-1 transmission from HIV-1 / HSV-2 co-infected persons within HIV-1 serodiscordant couples: A randomized, double-blind, placebo-controlled trial 5th IAS Conference on HIV Pathogenesis Treatment and Prevention. Cape Town, South Africa. Available at: http://www.ias2009.org/pag/Abstracts.aspx?SID=2436&AID=3699,2009
14. Cowan FM, Johnson AM, Ashley R, et al. Antibody to herpes simplex virus type 2 as serological marker of sexual lifestyle in populations. BMJ 1994; 309:1325–1329.
15. Obasi A, Mosha F, Quigley M, et al. Antibody to herpes simplex virus type 2 as a marker of sexual risk behavior in rural Tanzania. J Infect Dis 1999; 179:16–24.
16. Ashley RL. Sorting out the new HSV type specific antibody tests. Sex Transm Infect 2001; 77:232–237.
17. van Dyck E, Buvé A, Weiss HA, et al. Performance of commercially available enzyme immunoassays for detection of antibodies against herpes simplex virus type 2 in African populations. J Clin Microbiol 2004; 42:2961–2965.
18. Fleming DT, McQuillan GM, Johnson RE, et al. Herpes simplex virus type 2 in the United States, 1976 to 1994. N Engl J Med 1997; 337:1105–1111.
19. Xu F, Sternberg MR, Kottiri BJ, et al. Trends in herpes simplex virus type 1 and type 2 seroprevalence in the United States. JAMA 2006; 296:964–973.
20. Vyse AJ, Gay NJ, Slomka MJ, et al. The burden of infection with HSV-1 and HSV-2 in England and Wales: Implications for the changing epidemiology of genital herpes. Sex Transm Infect 2000; 76:183–187.
21. Smith JS, Robinson NJ. Age-specific prevalence of infection with herpes simplex virus types 2 and 1: A global review. J Infect Dis 2002; 186(suppl 1):S3–S28.
22. Ashley-Morrow R, Nollkamper J, Robinson NJ, et al. Performance of Focus ELISA tests for herpes simplex virus type 1 (HSV-1) and HSV-2 antibodies among women in ten diverse geographical locations. Clin Microbiol Infect 2004; 10:530–536.
23. Gamiel JL, Tobian AA, Laeyendecker OB, et al. Improved performance of enzyme-linked immunosorbent assays and the effect of human immunodeficiency virus coinfection on the serologic detection of herpes simplex virus type 2 in Rakai, Uganda. Clin Vaccine Immunol 2008; 15:888–890.
24. Wilson EB. Probable inference, the law of succession, and statistical inference. J Am Stat Assoc 1927; 22:209–212.
25. Hogrefe W, Su X, Song J, et al. Detection of herpes simplex virus type 2-specific immunoglobulin G antibodies in African sera by using recombinant gG2, Western blotting, and gG2 inhibition. J Clin Microbiol 2002; 40:3635–3640.
26. Laeyendecker O, Henson C, Gray RH, et al. Performance of a commercial, type-specific enzyme-linked immunosorbent assay for detection of herpes simplex virus type 2-specific antibodies in Ugandans. J Clin Microbiol 2004; 42:1794–1796.
27. Gorander S, Mbwana J, Lyamuya E, et al. Mature glycoprotein g presents high performance in diagnosing herpes simplex virus type 2 infection in sera of different tanzanian cohorts. Clin Vaccine Immunol 2006; 13:633–639.
28. Ng'ayo MO, Bukusi E, Morrow RA, et al. Sexual and demographic determinants for herpes simplex virus type 2 among fishermen along Lake Victoria, Kenya. Sex Transm Infect 2008; 84:140–142.
29. Smith JS, Bailey RC, Westreich DJ, et al. Herpes simplex virus-type 2 antibody detection performance in Kisumu, Kenya, using the Herpeselect ELISA, Kalon ELISA, Western Blot and inhibition testing. Sex Transm Infect 2009; 85:92–96.
30. Delany S, Jentsch U, Weiss H, et al. Comparison of Focus HerpesSelect and Kalon HSV-2 gG2 ELISA serological assays to detect herpes simplex virus type 2 (HSV-2) antibodies in a South African population. Sex Transm Infect 2010; 86:46–50.
31. Ashley RL. Performance and use of HSV type-specific serology test kits. Herpes 2002; 9:38–45.
32. Ashley-Morrow R, Krantz E, Wald A. Time course of seroconversion by HerpeSelect ELISA after acquisition of genital herpes simplex virus type 1 (HSV-1) or HSV-2. Sex Transm Dis 2003; 30:310–314.
33. Morrow RA, Friedrich D, Krantz E. Performance of the Focus and Kalon enzyme-linked immunosorbent assays for antibodies to herpes simplex virus type 2 glycoprotein G in culture-documented cases of genital herpes. J Clin Microbiol 2003; 41:5212–5214.
34. LeGoff J, Mayaud P, Gresenguet G, et al. Performance of HerpeSelect and Kalon assays in detection of antibodies to herpes simplex virus type 2. J Clin Microbiol 2008; 46:1914–1918.
35. Liljeqvist JA, Svennerholm B, Bergstrom T. Typing of clinical herpes simplex virus type 1 and type 2 isolates with monoclonal antibodies. J Clin Microbiol 1999; 37:2717–2718.
36. Benson DA, Karsch-Mizrachi I, Lipman DJ, et al. GenBank. Nucleic Acids Res 2007; 35:D21–D25.
37. Nascimento MC, Ferreira S, Sabino E, et al. Performance of the HerpeSelect (Focus) and Kalon enzyme-linked immunosorbent assays for detection of antibodies against herpes simplex virus type 2 by use of monoclonal antibody-blocking enzyme immunoassay and clinicovirological reference standards in Brazil. J Clin Microbiol 2007; 45:2309–2311.