Validation of rapid point-of-care tests for herpes simplex virus-type 2 in rural Tanzania

Abstract

Background

Herpes simplex virus-type 2 (HSV-2) affects nearly 500 million people worldwide. Up to 70% of women in Tanzania are infected by age 30. Point-of-care (POC) tests to detect HSV-2 antibodies are commercially available. We validated the performance of two HSV-2 POC tests in women in rural Tanzania.

Methods

Blood from 56 women was tested for HSV-2 antibodies by Biogate Labs and OnSite Duo rapid tests and Kalon HSV-2 IgG ELISA. Results obtained from the POC tests were compared with those from the ELISA to determine sensitivity, specificity, positive and negative predictive values (PPV and NPV).

Results

The Biogate had a sensitivity and specificity of 30.3% [15.6%–48.7%] and 100% [85.2%–100%], respectively. The OnSite Duo had a sensitivity and specificity of 69.7% [51.3%–84.4%] and 87.0% [66.4%–97.2%], respectively. The PPV and NPV for the Biogate was 100% [69.2%–100%] and 50% [44.4%–55.6%], respectively. The OnSite Duo had a PPV and NPV of 88.5% [72.3%–95.8%] and 66.7% [53.8%–77.5%], respectively.

Conclusion

These tests had different performance characteristics, with the Biogate having a poor sensitivity and excellent specificity, and the OnSite Duo having a slightly better sensitivity though lower specificity. Validation of POC tests is important, as they can be valuable diagnostic tools in low-resource settings.

Keywords

HSV (Herpes Simplex Virus)Africa women equipment diagnosis

Introduction

Herpes simplex virus-type 2 (HSV-2), the leading cause of genital ulcer disease and genital herpes, affects an estimated 520 million people aged 15–49 years worldwide.¹ HSV-2 poses the greatest burden in sub-Saharan Africa, where seroprevalence is estimated at 37%,² and it infects women at a rate two times higher than men.¹ In rural areas in Tanzania, up to 70% of women are infected by age of 30 years.³ HSV-2 is a lifelong infection that not only causes pain and stigma, but can also cause severe complications such as disseminated infection and neonatal herpes when transmitted vertically.¹ Moreover, HSV-2 increases the risk of acquisition and transmission of HIV, especially in women.^1,3 Although antiviral drugs do not cure HSV-2 infection, treatment decreases the duration of ulcer episodes and severity, thus improving quality of life and minimizing both vertical and horizontal transmission. Furthermore, diagnosis of HSV-2 in individuals presenting with genital ulcer disease helps to minimize use of empiric treatment, potentially reducing overuse of antimicrobials.^1,4

The detection of antibodies to HSV-2 is often done using Enzyme-Linked Immunosorbent Assays (ELISAs), which require laboratories with highly trained personnel, equipment, and power supply, and preclude provision of same-day results. This limits access to HSV-2 testing in resource limited settings. Use of point of care (POC) rapid diagnostic tests can address limitations of ELISA and improve access to HSV-2 testing. Simple and rapid HSV-2 diagnostic tests would allow clinicians to identify patients who have HSV-2 infection among those presenting with genital ulcer diseases and make appropriate, same-day management decisions. There has been an increase in the availability of commercial POC rapid diagnostic tests for the detection of HSV-2 antibodies. However, it is important to assess the performance of these tests in the setting where they will be used and confirm the performance reported by manufacturers.^5,6 For example, the Biokit HSV-2 Rapid test was reported to have a sensitivity of 91%,⁷ but the test had a sensitivity of 66% when it was used to test for HSV-2 antibodies among fishermen in Kenya.⁶

Therefore, we conducted a validation of two POC tests, Biogate Labs and OnSite Duo to determine their performance characteristics in order to select a test to screen for HSV-2 in our ongoing Female Genital Schistosomiasis (FGS) study.

Methods

Study procedures

A cohort study is being conducted in Itilima District in northwestern, Tanzania to evaluate the effect of S. haematobium infection on genital immune cells, epithelial cell barrier, and susceptibility to other genital infections such as HSV-2 as well as the impact of schistosome treatment on these parameters among women of reproductive age (18–50 years). At screening, all women were tested for S. haematobium infection by urine filtration and serum antigen, HIV following the national guidelines, and HSV-2 using a rapid POC test. Eligible women, who were non-pregnant and not living with HIV, underwent gynecologic examinations that included a vaginal swab for rapid detection Trichomonas vaginalis antigen (OSOM, Sekisui Diagnostics, CA, USA) that was performed in the field as well as an endocervical swab for detection of Neisseria gonorrhoeae and Chlamydia trachomatis by PCR (Artus CT/NG QS-RGQ, Qiagen, Hilden, Germany) that was performed at the National Institute for Medical Research (NIMR) laboratory in Mwanza.

HSV testing

During validation of the tests, we tested for HSV-2 antibodies in the field using two different HSV-2 POC rapid diagnostic tests, the Biogate Labs HSV-2 IgG/IgM Rapid Test (Burnaby, Canada) and the OnSite Duo HSV-1/2 IgG/IgM Rapid Test (CTK Biotech, Poway, CA), in blood samples collected from 56 participants. The same blood samples were also tested for HSV-2 antibodies by the Kalon HSV-2 IgG ELISA (Kalon Biological Ltd, UK) at the NIMR laboratory. Based on the manufacturer’s instructions, samples with an optical density of <0.49 in the ELISA were categorized as negative, those with optical density of >0.6 were categorized as positive, and samples with optical density ≥0.49 and ≤0.6 were categorized as indeterminate and had testing repeated. The laboratory staff performing the Kalon HSV-2 IgG ELISA were blinded to the test results for the two rapid POC tests. The results for the two POC tests were compared with the results of the Kalon IgG ELISA. Based on the results of this validation, the OnSite Duo HSV-1/2 IgG/IgM Rapid Test (CTK Biotech, Poway, CA) is being used for HSV-2 screening in the field for participants who are being enrolled in the study and confirmed with the Kalon IgG ELISA.

Statistical analysis

Statistical analyses were performed using STATA/IC Version 15.0 (Stata Corp. College Station, Texas, USA). Variables were summarized by frequencies and medians/interquartile ranges [IQR]. Sensitivity and specificity and their respective 95% confidence intervals (CIs) of both rapid tests were calculated using the Kalon IgG ELISA result as the gold standard. Positive predictive value (PPV) and negative predictive value (NPV) were also calculated. To assess the association between Kalon optical density result and performance of the rapid tests, a Wilcoxon rank-sum test was used.

Results

A total of 56 women, who had a median age of 37.0 [IQR, 29.5–46.5] years (Table 1, Appendix) were included in the validation of the two POC tests. Food insecurity was reported by 42.9% of participants, and 19.6% had never attended school. Nearly two-thirds reported that their husband had children outside their marriage, and 23.2% had more than one sexual partner in the last year. Over half of participants reported to have experienced genital ulcer disease symptoms in the past year, 7.1% had chlamydia infection, and 25.0% had trichomoniasis.

Ten out of 56 (17.9%) blood samples were positive for HSV-2 antibodies by both Biogate and Kalon IgG ELISA and 23 (41.1%) out of 56 blood samples were negative by both Biogate and Kalon IgG ELISA, giving a sensitivity of 30.3% [95% CI: 15.6%–48.7%] and a specificity of 100% [85.2%–100%] for the Biogate Labs test (Table 1). The PPV and NPV for the Biogate Labs test was 100% [69.2%–100%] and 50% [44.4%–55.6%] respectively (Table 1). For the OnSite Duo test, 23 (41.1%) out of 56 blood samples had concordant positive results with the Kalon IgG ELISA, and 20 (35.7%) had a concordant negative result, giving a sensitivity of 69.7% [51.3%–84.4%] and a specificity of 87.0% [66.4%–97.2%] (Table 2). The OnSite Duo test had a PPV and NPV of 88.5% [72.3%–95.8%] and 66.7% [53.8%–77.5%], respectively (Table 2). A positive Kalon IgG ELISA with a lower optical density value was associated with a false negative OnSite Duo test result compared to a true positive Onsite Duo test result (1.0 [0.8–1.8] vs 2.7 [2.2–3.2], p < 0.001) (Figure 1). Three samples with false positive OnSite Duo tests were negative on Kalon IgG ELISA with optical densities well below the negative cut-off value (0.06, 0.094, 0.095).

Table 1.

Sensitivity, specificity, and positive and negative predictive values of the Biogate Labs HSV-2 IgG/IgM Rapid Test when compared to the Kalon IgG ELISA.

		Kalon IgG ELISA
		Positive	Negative	Total
Biogate Labs HSV-2 IgG/IgM rapid test	Positive	10	0	10	PPV = 100%
	Negative	23	23	46	NPV = 50%
	Total	33	23	56
		Sensitivity = 30.3%	Specificity = 100%

Table 2.

Sensitivity, specificity, and positive and negative predictive values of the OnSite Duo HSV-1/2 IgG/IgM Rapid Test when compared to the Kalon IgG ELISA.

		Kalon IgG ELISA
		Positive	Negative	Total
OnSite Duo HSV-1/2 IgG/IgM rapid test	Positive	23	3	26	PPV = 88.5%
	Negative	10	20	30	NPV = 66.7%
	Total	33	23	56
		Sensitivity = 69.7%	Specificity = 87.0%

Figure 1.

Biogate and OnSite Duo Test Result compared to Kalon Optical Density Result. Comparison of Kalon ELISA optical densities, shown on the y-axis, with Biogate and Onsite Duo rapid test results. Lower Kalon ELISA optical densities were associated with false negative results on the OnSite Duo rapid test, whereas optical densities were not associated with false negative results in the Biogate rapid test.

Discussion

Women who participated in this validation reported high risk sexual behavior and lived in a rural setting with limited access to a reference test for HSV-2 and a high prevalence of HSV-2 seropositivity at 59%. This population therefore represents an important target population whose care could be impacted by a high-performing HSV-2 point-of-care test.

The results of this study demonstrate the importance of conducting a validation of a new test before its use. Both Biogate Labs and OnSite Duo rapid tests showed lower sensitivity than that reported by manufacturers. False negative results on the OnSite Duo test were significantly associated with lower Kalon ELISA optical density values compared with true positive results. This suggests that the lowest detection limit for the OnSite Duo test is higher than that of the Kalon IgG ELISA, and therefore samples with low HSV-2 antibody levels may not be detected by the OnSite Duo test but detected by the Kalon IgG ELISA. This may be a marker of more recent acquisition of the virus in individuals who have not yet developed high antibody titers.⁸ It is also possible that some transmitted strains of HSV-2 may be less immunogenic or generate lower levels of systemic antibody responses,⁹ thereby leading to false negative results in rapid tests. Although the specificity for the Biogate Labs test was excellent, its sensitivity was poor. Thus, we opted to use the OnSite Duo test which had a slightly better sensitivity for screening for HSV-2 and to confirm our results with Kalon IgG ELISA for enrollment in our FGS cohort study.

Previous studies evaluating the performance of POC tests for HSV-2 suggested that their performance could be affected by other viral infections and N. gonorrhoeae infection.^10,11 It is unlikely that the performance of these two tests were affected by HIV infection because people living with HIV were excluded from this study. Similarly, we did not find an association between having gonorrhea and false negative results for POC tests, though we were likely underpowered to detect association because we only had three women with gonorrhea.

This study had several limitations. First, the University of Washington Western blot test is recognized as the gold standard for HSV-2 diagnosis. However, because of its high cost and limited feasibility, we opted to use the Kalon IgG ELISA as the confirmatory test. The use of a less than perfect comparison test for diagnosis of HSV-2 could have underestimated the performance characteristics of the two POC tests. However, Kalon IgG ELISA has been shown to have high concordance with Western blot in sub-Saharan African populations,⁶ so it seems unlikely that its use in this study would have affected the determination of performance of the two POC tests. Second, we performed this study in a population of females in Tanzania, and results may not be broadly generalizable. Third, interpreter error is a concern for semi-quantitative tests that require human interpretation. To minimize this effect, two study team members read all test results independently, and used a third reader as a tie breaker when the results from two readers were discordant.

Point-of-care tests can be valuable diagnostic tools in complementing syndromic management of genital tract infections in resource limited settings. However, our study underscores the importance of validating the performance of point-of-care rapid HSV-2 tests with a reference test in different settings before their widespread use. To improve accessibility of HSV-2 testing, further work is needed to optimize the performance of POC tests for reliable use in diverse real-world settings globally.

Footnotes

Acknowledgments

We would like to thank the laboratory team at the National Institute for Medical Research and the field team at Itilima in Bariadi for performing the tests that contributed to the data used in this paper. We also appreciate the women who participated in the study.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institutes of Health / National Institute of Allergy and Infectious Diseases (R01 AI 168306 and K24 AI 182638) and by the Fogarty International Center (D43 TW 011826).

Ethical statement

The study was approved by the Medical Research Coordinating Committee (MRCC) of the National Institute for Medical Research (NIMR/HQ/R.8a/Vol.IX/3625) and the Weill Cornell Medicine Institutional Review Board in New York (20–10022745). All women provided written informed consent prior to participation. The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

ORCID iD

Kristen Riley

Appendix

Table A1 Table A1.

Demographic and clinical characteristics of overall study population (n = 56).

Characteristic	Number (percent) or median (interquartile range)
Age in years	37.0 [29.5–46.5]
Body mass index (kg/m²)	22.4 [19.8–24.2]
Reported missing lunch or dinner in the past month due to shortage of food	24 (42.9%)
Schooling obtained
None	11 (19.6%)
Primary or secondary	45 (80.4%)
Age in years at first sex	17.0 [16.0–18.0]
Number of children	5 [3–7]
Husband has children outside the marriage	35 (63.6%)
Number of sex partners in the past year
One	43 (76.8%)
More than one	13 (23.2%)
Self-report of any prior sexually transmitted infection	10 (17.9%)
Self-report of genital ulcers in the past year	29 (51.8%)
Current Trichomonas vaginalis infection	14 (25.0%)
Current Chlamydia trachomatis infection	4 (7.1%)
Current Neisseria gonorrhoeae infection	3 (5.6%)
Current Schistosoma haematobium infection	12 (21.4%)

References

World Health Organization . Herpes simplex virus. https://www.who.int/news-room/fact-sheets/detail/herpes-simplex-virus (2024, accessed 1 October 2024).

Harfouche

Abu-Hijleh

James

, et al. Epidemiology of herpes simplex virus type 2 in sub-Saharan Africa: systematic review, meta-analyses, and meta-regressions. EClinicalMedicine 2021; 35: 100876.

Pujades Rodríguez

MDM

Obasi

Mosha

, et al. Herpes simplex virus type 2 infection increases HIV incidence: a prospective study in rural Tanzania. AIDS 2002; 16: 451–462.

Workowski

Bachmann

Chan

, et al. Sexually transmitted infections treatment guidelines, 2021: morbidity and mortality weekly report. 2021.

Biraro

Mayaud

Morrow

, et al. Performance of commercial herpes simplex virus type-2 antibody tests using serum samples from Sub-Saharan Africa: a systematic review and meta-analysis. Sex Transm Dis 2011; 38: 140–147.

Ng’ayo

Friedrich

Holmes

, et al. Performance of HSV-2 type specific serological tests in men in Kenya. J Virol Methods 2010; 163: 276–281.

Morrow

Friedrich

Meier

, et al. Use of ‘biokit HSV-2 Rapid Assay’ to improve the positive predictive value of Focus HerpeSelect HSV-2 ELISA. BMC Infect Dis 2005; 5: 1–7.

Legoff

Péré

Bélec

. Diagnosis of genital herpes simplex virus infection in the clinical laboratory. Virol J 2014; 11: 1.

Daikoku

. Novel deletion in glycoprotein G forms a cluster and causes epidemiologic spread of herpes simplex virus type 2 infection. J Med Virol 2013; 85: 1818–1828.

10.

Summerton

Riedesel

Laeyendecker

, et al. Effect of sexually transmitted disease (STD) coinfections on performance of three commercially available immunosorbent assays used for detection of herpes simplex virus type 2-specific antibody in men attending Baltimore, Maryland, STD clinics. Clin Vaccine Immunol 2007; 14: 1545–1549.

11.

Laderman

Whitworth

Dumaual

, et al. Rapid, sensitive, and specific lateral-flow immunochromatographic point-of-care device for detection of herpes simplex virus type 2-specific immunoglobulin G antibodies in serum and whole blood. Clin Vaccine Immunol 2008; 15: 159–163.