The false-positive and false-negative predictive value of HIV antibody test in the Chinese population

Abstract

Objectives

To analyse the relationship between the false-positive/false-negative predictive value (FPPV/FNPV) of the HIV-antibody (HIV-Ab) test and prevalence in different Chinese population groups.

Methods

HIV prevalence among different population groups was obtained by a screening survey of blood donors and the national HIV/AIDS surveillance programme in China. Given the sensitivity and specificity of a test kit and the prevalence of HIV infection, the estimated values of FPPV/FNPV were calculated using Bayes' formula. The actual value of FPPV of blood donors was obtained by screening 1,195,286 blood donors.

Results

This study indicates that the FPPV of HIV-Ab enzyme-linked immunosorbent assay (ELISA) assays varies widely in different Chinese populations: about 99.5% in the blood donor population, but only 3.2% in the injecting-drug users in high-risk areas. In 1,195,286 sera specimens from the blood donors, 2439 specimens were HIV-Ab positive by third ELISA, and 11 HIV cases were confirmed by Western blot. The HIV prevalence of the blood donor population in this survey was 0.0009% (11/1,195,286), but the HIV-Ab positive rate of third ELISA is 0.2% (2439/1,195,286) and 222 times higher than the prevalence.

Conclusions

Evaluation of HIV prevalence through the HIV-Ab positive rate by third ELISA will significantly overestimate the true prevalence in a low-prevalence population. Individual HIV-infection status should be taken into consideration when analysing the results of HIV-Ab tests in a population with low infection.

INTRODUCTION

In the current surveillance of HIV infection in China, two new situations are emerging: one is the increasing number of national HIV/AIDS sentinel surveillance sites from 194 in 2003 to 329 in 2005,¹ resulting in more complete and representative data about HIV prevalence. Some studies show that the HIV prevalence has corresponding high variation with differences of prevalence between the general population and the high-risk population group (more than about 1000 times¹). The other development is the improved quality of the test kits, leading to higher accuracy. For example, the sensitivity and specificity of the double-antigen sandwich enzyme-linked immunosorbent assay (third-generation enzyme-linked immunosorbent assay, third ELISA) for HIV-Ab test, widely used in China, are both over 96%.^2–10

The studies from United States, Japan, India and Pakistan show that even high-accuracy test kits cannot prevent a high proportion of false-positive results, especially when testing individuals from a low-risk population, which will directly affect the researchers' judgements on the HIV-Ab positive results in practice.^11–17 Although many studies on the sensitivity and specificity of the test kits and the HIV prevalence of different population groups have been conducted in China,^1,3–10 studies that analyse the relationship between the false-positive/negative predictive value (FPPV/FNPV) of the HIV-Ab test and the HIV prevalence in different Chinese population groups are scarce.

In this paper, first, we characterized the factors that influence the FPPV/FNPV of HIV-Ab test among different Chinese population groups using data on HIV-Ab from the Jiangsu Provincial screening survey of blood donors, and from the national HIV/AIDS surveillance programme in China. Secondly, the relationship between the FPPV/FNPV and HIV prevalence in the population was interpreted and assessed according to the current possible variation range of sensitivity and specificity of third ELISA HIV-Ab test kits. Finally, we estimated the correct HIV-Ab ELISA test results in different populations.

METHODS

False-positive predictive value and false-negative predictive value

FPPV/FNPV may be defined as the probabilities of misclassified individuals with positive or negative test readings, respectively when a chosen test kit is applied to a population with a certain prevalence.¹⁶ Both the FPPV and FNPV can be written with probability notation as FPPV = Pr(D^–/T⁺) and FNPV = Pr(D⁺/T^–), where D⁺ and D^– are the presence and absence of the disease; T⁺ and T^– are the positive and negative test readings respectively; and Pr (A/B) is the conditional probability of event A given event B. The FPPV therefore gives an estimate of the proportion of misclassified individuals with positive tests, and the FNPV gives an estimate of the proportion of misclassified individuals with negative tests. We can calculate the probabilities of misclassification associated with the possible outcomes of a test by

where positive predictive value (PPV) is the probability of correctly classifying an individual if the individual tests positive, or, NPV (negative predictive value) if the individual tests negative. Obviously, FPPV and FNPV are the reverse events of PPV and NPV, respectively.

Given the sensitivity (Se) and specificity (Sp) of a test kit and the prevalence (P) of HIV infection, the false-positive and NPVs can be calculated using Bayes' formula.^16,18,19

HIV prevalence in different population groups

A total of 1,195,286 voluntary blood donors who visited the blood banks in Jiangsu Province, China, between April 2004 and April 2006 were screened for HIV type 1/type 2 by ELISA kits ([State Food and Drug Administration (SFDA)] of China approved), according to the strategy recommended by the Ministry of Health of People's Republic of China for blood banks.²⁰ All serum was first tested with two third ELISA assays (both based on different antigen preparation and made by different manufacturers). Any serum that is reactive on either ELISA test (on the first test and/or on the second test) was sent to the HIV Reference Laboratory of Jiangsu Provincial Center for Disease Control and Prevention (Jiangsu CDC) to undergo confirmation testing by Western blot (WB). WB results were interpreted according to WHO criteria.²¹

Serum that is non-reactive on both ELISA assays is considered to be HIV-Ab negative. Serum reactive on either ELISA test and on WB test is considered to be HIV-Ab positive. Serum reactive on either or both ELISA test but non-reactive on WB test is considered to be HIV-Ab false-positive. If WB results are inconclusive, the following procedures are followed: (a) if the blood donors are followed up successfully, use the results from follow-up were used; (b) if the blood donors have not been followed up successfully and were positive on both ELISAs, the subjects with an indeterminate WB were considered to be HIV-Ab positive; (c) if the blood donors have not been followed up successfully and the results were only weak positive on either ELISA, the subjects with an indeterminate WB were considered to be HIV-Ab negative; (d) if the blood donors have not been followed up successfully and results were strongly positive on either ELISA, the subjects with an indeterminate WB were considered to be HIV-Ab indeterminate and were not used in the calculations.

Besides the HIV prevalence of voluntary blood donors which was obtained by our retrospective survey, the HIV prevalence of other populations, including pregnant women (high-risk areas), sex workers, drug users and injecting-drug users (high-risk areas), were taken from the official report, 2005 Update on the HIV/AIDS Epidemic and Response in China, released by Ministry of Health of People's Republic of China, Joint United Nations Programme on HIV/AIDS, World Health Organization.¹

The sensitivity and specificity of HIV antibody test kits

The samples first tested by third ELISA and then confirmed by WB in the National AIDS Reference Laboratory or the provincial HIV reference laboratories in China were treated as the reference samples for calculating the sensitivity and specificity of the test kits. Both manual and computer retrieval methods were employed to search various published studies from January 2002 to December 2006, which evaluated the HIV-Ab test kits made in China for the estimation of the range of the sensitivity and specificity.

Statistical analysis

False-positive and NPV with the variation of sensitivity, specificity and prevalence were calculated by SAS 9.1 (SAS Institute Inc, Cary, NC, USA).

RESULTS

Sensitivity and specificity of the HIV antibody test kits

Eight articles, in which 19 evaluations of HIV test kits conformed with our criteria, including 1165 reference samples for the assessment of sensitivity and 2631 reference samples for the assessment of specificity, were obtained by manual and computer retrieval.^3–10 Some descriptive statistics of the sensitivity and specificity of the third ELISA HIV-Ab test kits are shown in Table 1.

Table 1

The sensitivity and specificity of third enzyme-linked immunosorbent assay test kits for HIV antibody

	Number of evaluations	Sample size	Median	Minimal value	Maximal value
Sensitivity (%)	19	1165	97.78	90.71	100.00
Specificity (%)	19	2631	96.81	91.72	99.22

False-positive predictive value and false-negative predictive value of the HIV antibody evaluated in different population groups

According to Equations (1) and (2), with median specificity of 96.81% and median sensitivity of 97.78% (see Table 1), evaluations of the FPPV/FNPV of the HIV-Ab in different population groups are shown in Table 2.

Table 2

The FPPV and FNPV value of HIV-antibody (HIV-Ab) tests of different population groups

Population groups	Prevalence (%)	FPPV (%)	FNPV (%)
Voluntary blood donation	0.001*	99.697^† (99.549^‡)	0.000^†
Pregnant women (high-risk areas)	0.260^§	92.601^†	0.006^†
Sex workers	0.930^§	77.605^†	0.022^†
Drug users	6.480^§	32.011^†	0.159^†
Injection drug users (high-risk areas)	50.000^§	3.159^†	2.242^†

FPPV, false-positive predictive value; FNPV, false-negative predictive value

*The prevalence data were from a survey of 1,195,286 voluntary blood donators in Jiangsu province, China from April 2004 to April 2006, all of samples were tested by third ELISA for HIV-Ab, and then all of positive samples were confirmed by WB with 11 true-positive and 2428 false-positive

^†The estimated values of FPPV/FNPV value were calculated based on the Equations (1) and (2)

^‡The actual value of FPPV ((2439 – 11)/2439) was obtained by survey of 1,195,286 blood donors

^§The prevalence of different groups can be found in the official report of 2005 update on the HIV/AIDS epidemic and response in China¹

As Table 2 shows, the FPPVs among different population groups differ significantly. The FPPV estimated based on Equation (1) can reach 99.697% in the blood donation population group. In other words, 99.697% of the HIV-Ab positive results may be false-positives in this group. Our surveys also show that, in 1,195,286 sera specimens from the blood donors, 2439 specimens were HIV-Ab positive by third ELISA, and 11 HIV cases were confirmed by WB. So the actual value of FPPV in the survey could be (2439 – 11)/2439 = 99.549% (actual value obtained by screening survey of voluntary blood donors), which is very close to the estimated value of 99.697% (the estimated value calculation based on Equation (1)). The HIV prevalence of the blood donor population in this survey was 0.0009% (11/1,195,286), but the HIV-Ab positive rate of third ELISA was 0.2% (2439/1,195,286), 222 times higher than the prevalence. Table 2 also shows that with the exception of the injecting-drug user group, the FPPV was significantly higher than the FNPV.

DISCUSSION

This study indicates that the FPPV of HIV-Ab ELISA assays varies widely in different Chinese populations. About 99.5% of the HIV-Ab tests may be false-positive in the blood donor population, whereas the FPPV is only 3.2% in the injecting-drug users in high-risk areas. Therefore, prevalence in the surveillance population should be taken into consideration when analysing the results of the HIV-Ab test. The routine disclosure of HIV serum screening results prior to WB confirmation should be avoided in very low-risk populations. However, in high-risk populations, the reactive results of HIV-Ab ELISA have important diagnostic value for HIV infection.

In addition, 1,195,286 specimens from the voluntary blood donors in this survey reveal that the HIV-Ab positive rate of third ELISA is 222 times higher than its true prevalence, which suggests that evaluating the prevalence of a population by the HIV-Ab positive rate (such method is usually adopted by researchers if the population prevalence is high and both the sensitivity and the specificity of the test kits are excellent) will significantly overestimate the true prevalence in a low-prevalence population. Thus when assessing the results of the HIV-Ab ELISA test, it is incorrect to think that a positive reading is very significant in practice without considering the population infection level.

The number of positive sera of voluntary blood donors by ELISA which were confirmed by WB in our study (11 of the 2439 samples, 0.45%) is significantly smaller than the number reported by Sheikh et al. and Zacharias et al. ^11,12 Our ELISA HIV-Ab false-positive rate was much higher than those of Scheikh and colleagues and Zacharia and colleagues. We believe that the reasons for this difference are our different screening strategy and screened populations. In our study, all serum were tested with two ELISA assays on stage I, and serum reactive on either assay was retested on stage II with a WB as a confirmation test. In Scheikh et al.'s study, only one rapid test was used on stage I, and any serum reactive on rapid test entered stage II and stage III for confirmation by ELISA assays. Our screening strategy will decrease the chance of HIV-Ab false-negatives (which is vital to ensure transfusion safety), and at the same time, increase the probability of HIV-Ab false-positives. WB as a confirmation test in our screening provides much more opportunity to find HIV-Ab false-positives than using ELISAs as confirmation test in Scheikh et al.'s study. Another factor causing more ELISA HIV-Ab false-positives in our study was the difference between the population of our voluntary blood donors and the population in the studies of Sheikh et al. and Zacharias et al. Zacharias et al.'s study subjects were gravidas delivering at a birthing centre. A history of multiple (≥5 lifetime) sexual partners was elicited in the majority of HIV-infected patients. The HIV prevalence in the study by Zacharias et al. ¹² was 0.71% (69/9781) and in Sheikh et al.'s²² study was 0.22% (11/5000),¹¹ but the HIV prevalence among our voluntary blood donors was only 0.0009% (11/1,195,286) which is somewhat higher than in the German voluntary blood donors (0.0006%). Voluntary blood donors in our study mainly consist of healthy university students, soldiers and other generally healthy population and their HIV prevalence is likely to be very low. Studies show that the lower the prevalence, the higher the proportion of false-positives.^16,19 So this is another potential reasonable explanation for the higher proportion of false-positives in our study in comparison with those of Schikh et al. and Zacharias et al.

Footnotes

ACKNOWLEDGEMENTS

The authors would like to thank all the researchers who contributed to the data collection and management. The study was supported by National Natural Science Foundation of China (30471501).

References

Ministry of Health of People's Republic of China, Joint United Nations Programme on HIV/AIDS, World Health Organization. 2005 Update on the HIV/AIDS Epidemic and Response in China. Beijing: National Center for AIDS/STD Prevention and Control of China CDC, 2006

Joon-Sup

, Jae-Beom

, Seung-Ho

, Evaluation of a new third-generation ELISA for the detection of HIV Infection. Ann Clin Lab Sci 2006;36:73–8

Shu-wu

, Wen-bo

, Yong-hui

, The analysis and assessment of the detection results of the HIV antibody primary screenings. Chin J Health Lab Technol (Chin) 2006;16:81–2

Ying-lai

, Tong-qin

, Xu-dong

. The quality assessment of the HIV antibody test kits of the domestic manufactures. Chin J Blood Transfus (Chin) 2004;17:267–8

Xiu-hua

, Ai-ying

, Shu-fang

, Evaluation of reagent kit for detection of HIV antibodies. Chin J Lab Med (Chin) 2003;26:136–8

Ai-ling

, Xiao-li

. The quality assessment of five HIV antibody specimens in primary screenings. Chin J Health Lab Technol (Chin) 2004;14:362

Wei-guo

, Zhe

, Xin-peng

, Quality assessments of five HIV antibody test kits for the clinical application. Lab Med (Chin) 2004;19:323, 326

Yong

, Hui-ping

, Heng

, Evaluation of HIV antibody ELISA kits used in Hubei province in 2004. Jiangxi Med J (Chin) 2005;40:623–4

ZI-chen

, Qing-neng

, Yi-le

. Comparison of the results from two diagnostic reagents for HIV antibody. Shanghai J Prev Med (Chin) 2003;15:529–30

10.

Xiao-qing

, Cheng-met

, Hong-jing

, Clinical quality evaluation of five ELISA kits for HIV screening test. Jiangsu Prev Med (Chin) 2003;14:10–11

11.

Sheikh

, Sheikh

, High frequency of false positive results in HIV screening in blood banks. J Pak Med Assoc 2006;56(Suppl 1):S72–5

12.

Zacharias

, Athanassaki

, Sangi-Haghpeykar

, High false-positive rate of human immunodeficiency virus rapid serum screening in a predominantly hispanic prenatal population. J Perinatol 2004;24:743–7

13.

Choudhury

, Ayagiri

, Ray

. True HIV seroprevalence in Indian blood donors. Transfus Med 2000;10:1–4

14.

Kleinman

, Busch

, Hall

, False-positive HIV-1 test results in a low-risk screening setting of voluntary blood donation. Retrovirus epidemiology donor study. JAMA 1998;280:1080–5

15.

Osato

, Matsubayashi

, Nagao

, False positives and false negatives seen in anti-HIV-1 antibody tests. Kansenshogaku Zasshi 1994;68:949–52

16.

, Litvak

, Pagano

. Issues in human immunodeficiency virus screening. Am J Epidemiol 1992:136:244–55

17.

Meyer

, Pauker

. Screening for HIV: can we afford the false positive rate? N Engl J Med 1987;317:238–41

18.

Zhou

, Obuchowski

, McClish

. Statistical Methods in Diagnostic Medicine. New York: John Wiley and Sons, Inc, 2002:56.

19.

Gupta

, Malina

. Group testing in presence of classification errors. Stat Med 1999;18:1049–68

20.

Ministry of Health of People's Republic of China. China Transfusion Technology Standardized Operation. Beijing: Ministry of Health of People's Republic of China, 1997

21.

UNAIDS. Operational Characteristics of Commercially Available Assays to Detect Antibodies to HIV-1 and/or HIV-2 in Human Sera, Report 11. Geneva: World Health Organization, 1999

22.

Offergeld

, Ritter

, Hamouda

, HIV, HCV, HBV and syphilis infections among blood donors in Germany 2005. Report from the Robert Koch Institute in accordance with Article 22 of the Transfusion Act. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz 2007;50:1221–31