How to Interpret Borderline HCV Antibody Test Results: A Comparative Study Investigating Four Different Anti-HCV Assays

Abstract

Anti-HCV testing is the first step to diagnose hepatitis C. Although anti-HCV assay performance improved during the last 2 decades, very high sensitivity required for screening may lead to limitations in specificity. Thus, there remains an uncertainty how to interpret anti-HCV test results with a borderline signal-to-cut-off ratio. Comparison was made of concordance and performance of four licensed anti-HCV assays in samples with borderline signal-to-cut-off ratios. Out of 12,090 consecutive samples tested for anti-HCV with the Abbott Architect Anti-HCV assay over a period of 29 months, 95 plasma samples with a signal-to-cut-off ratio between 0.5 and 2 were selected for this study. All samples were re-tested with the Enzygnost Anti-HCV version 4.0, the Ortho anti-HCV version 3.0, and the Monolisa anti–HCV-Plus version 2 assays. Discordant samples were classified by additional immunoblot testing. Overall, only 52% of the Architect borderline samples gave similar results in all four assays. Inter-assay concordance ranged between 58% and 80%. The highest discordance was observed between the Architect and the Monolisa assay (42%). In contrast, a high level of concordance was found between the Enzygnost and Ortho assays (80%). The Monolisa was best to identify negative samples (100%), while the Enzygnost correctly classified most of the positive samples (96%). Anti-HCV antibody assays show significant variation in classifying samples with low signal-to-cut-off ratios. Different performances may have cost and management implications, as false-positive results are not infrequent. However, sensitivities were good for all assays if indeterminate results are not considered as negative.

Introduction

Number of patients with detectable anti-HCV has been increasing over the last 20 years to more than 185 million people worldwide (7), including 8–11 million HCV infected subjects in Europe (3). Chronic HCV infection is a major contribution to the global prevalence of liver cirrhosis and hepatocellular carcinoma (5,13). Great improvements have been achieved in treating HCV infection. However, currently available triple therapies are still limited in terms of eligibility, safety, and efficacy at least for certain patient cohorts (8,12). Serological screening is one of the most important tools to counter HCV endemic, in particular since the majority of the infected individuals stay asymptomatic for quite a long time (5,13). Becoming aware of an infection allows the prevention of further viral transmission and the appropriate allocation of patients to antiviral treatment or specific surveillance programmes. The impact of HCV screening for the management of the endemic may become even more important in the upcoming years. Highly effective interferon-free treatment regimens are on the horizon, widely resolving the current problem of both limited eligibility and limited efficacy of antiviral treatment, which will allow curing most of the infected individuals (4,12).

At this stage, the identification of all infected subjects may remain the last hurdle to finally overwhelm the disease. The introduction of anti-HCV testing represents a milestone in medical history since it led to a significant reduction of transfusion related viral transmission. Over the last 20 years, full automatic tests were developed, and both sensitivity and specificity of these tests have been improved to more than 97% for most of the licensed assays (1,6,9). Up to now the detection of anti-HCV antibodies in the patients' blood is the most common way for HCV-screening, since this is a fast and cost-effective method (5,14). Despite the advantages that have been achieved, there is still a significant number of test results yielding borderline signal-to-cut-off (s/co) ratios. In these cases, interpretation of the result is uncertain since the reliability of anti-HCV assays in this borderline setting has rarely been investigated. Thus re-testing, immunoblot-assays, or real-time PCR becomes necessary but is time-consuming and expensive. However, it remains the only way to prevent false positive and false negative results.

In this study, we compared the concordance and performance of four licensed anti-HCV assays in samples with borderline s/co ratios obtained during routine clinical practice at a tertiary referral and transplant center.

Materials and Methods

Study design

Selection of samples

Over a period of 29 months starting in September 2009, 12,090 consecutive patient samples were tested with the Architect Anti-HCV assay (Abbott) in our department. A s/co ratio of ≥1 was considered as a positive test result as suggested in manufactures' recommendations. All samples were stored at −20°C. Of all consecutive samples, 171 (1.4%) yielded a borderline s/co ratio, which was defined as 0.5–2.0. Of these samples, 100 were randomly selected for this study (53 classified as positive and 47 classified as negative by the Architect Anti-HCV assay). In five of these selected samples, the remaining volume was not sufficient to perform re-testing with all assays. Unfortunately, the patients could not be re-contacted for additional blood drawings. These samples were therefore excluded from the study. Finally, 95 samples were studied (49 classified as positive and 46 classified as negative by the Architect Anti-HCV assay) (Fig. 1).

FIG. 1.

Algorithm for the selection of the study samples.

The 95 selected samples were derived from 86 patients, of which 22% had serological evidence for a previous or current hepatitis B virus (HBV) infection, and in 44% hepatitis A virus (HAV)-IgG was detectable (Table 1).

Table 1.

Characteristics of the Patients from Whom the Tested Serum Samples Were Derived

Number of patients	86
Gender
Male	60 (70%)
Female	26 (30%)
Age (mean±SD)	48.8 yrs (±18.7)
History of HBV¹	19 (22%)
Anti-HAV IgG positive	38 (44%)

Determined by the presence of either anti-HBc, HBsAg, or HBV DNA.

Re-testing

All 95 selected borderline samples were re-tested using the Enzygnost Anti-HCV 4.0 (Siemens Diagnostics), the Ortho anti-HCV 3.0 (Ortho Clinical Diagnostics) and the Monolisa anti–HCV-Plus version 2 (BioRad) assays. Re-testing was done according to the manufactures` standard protocols.

Immunoblot testing

For immunoblot testing, we used the Inno-Lia HCV Score (Innogenetics) and followed the manufactures' protocol. Test results with more than one reactive bands were considered to be positive.

Classification of samples

Samples were classified as anti-HCV positive or negative if the result was concordant in all four anti-HCV assays. In discordantly tested samples, final classification was based on result of the immunoblot. Eight discordant samples could no be classified due to an insufficient amount of serum for immunoblot testing.

Results

Overall inter-assay concordance

In only 52% of the selected samples, the test result of the Architect Anti-HCV was confirmed by all three other anti-HCV assays (Fig. 2). A higher level of concordance was observed for samples that were previously classified as negative by the Architect Anti-HCV (87%), while only 18% of the positive test results in the Architect Anti-HCV were confirmed by all the other three assays. In 27 samples (28%), only one of the assays showed a different result than the other three, and in 19 cases (20%) two assays were positive and two produced a negative result (Fig. 2). 26%–31% of the samples preselected by the Architect Anti-HCV also yielded a borderline s/co ratio (0.5–2) in one of the other three assays.

FIG. 2.

Overview of the amount of discordantly and concordantly tested samples. The upper boxes represent the portion of concordant results (bright positive and dark negative samples). The middle boxes include samples with only one discordant assay result, while the white box represents all samples, which lead to two positive as well as two negative results in the four assays.

Concordance between the single assays

There was a remarkable difference in the inter-assay concordance between the four assays ranging from 58% up to 80% when comparing two assays (Table 2). The highest level of discordance was documented between the Architect Anti-HCV and the Monolisa anti–HCV-Plus version 2 (42%) and between the Enzygnost Anti-HCV 4.0 and the Architect Anti-HCV (27%). In contrast, a higher level of concordance was found between the Enzygnost Anti-HCV 4.0 and Ortho anti-HCV 3.0 (80%) and the Enzygnost Anti-HCV 4.0 and the Monolisa anti–HCV-Plus version 2 (77%) (Table 2). Mean difference in s/co ratios measured in the same sample was the highest between the Enzygnost Anti-HCV 4.0 and the Architect Anti-HCV (0.79), whereas the smallest was found between the Enzygnost Anti-HCV 4.0 and the Monolisa anti–HCV-Plus version 2 (0.5).

Table 2.

Overview of the Interassay Concordance of All Tested Serum Samples

Assay	Enzygnost Anti-HCV 4.0 Assay	ArchitectAnti-HCV	Ortho HCV 3.0 SAVe	Monolisa anti–HCV-Plus v. 2
Enzygnost Anti-HCV 4.0 Assay	X	27%	20%	23%
Architect Anti-HCV	73%	X	26%	42%
Ortho HCV 3.0 SAVe	80%	74%	X	26%
Monolisa anti–HCV-Plus v. 2	77%	58%	74%	X
Concordant Results

The Architect Anti-HCV assay produced more positive results compared to the Enzygnost Anti-HCV 4.0 (52% vs. 33%). Consequently, 22 samples (23%) identified as anti-HCV positive in the Architect Anti-HCV assay were classified as negative using the Enzygnost Anti-HCV 4.0. In contrast, negative test results of the Architect Anti-HCV assay were widely confirmed by re-testing with Enzygnost Anti-HCV 4.0, except for four samples (4.2%) (Fig. 3a). A similar observation was made comparing the Architect Anti-HCV assay with the Ortho anti-HCV 3.0 (Fig. 3b). The Enzygnost Anti-HCV 4.0 and Ortho anti-HCV 3.0 produced comparable numbers of positive and negative results (Fig. 3c). In contrast, the Monolisa anti–HCV-Plus version 2 classified all of the discordant results as negative (Fig. 3d–f).

FIG. 3.

Direct comparison of the performance of two single assays. The graphs show the s/co ratios of each single serum sample in two different assays (x- and y-axes, respectively). The two compared assays are the Architect Anti-HCV and the Enzygnost Anti-HCV 4.0 (A), the Ortho anti-HCV 3.0 and the Architect Anti-HCV (B), the Ortho anti-HCV 3.0 and the Enzygnost Anti-HCV 4.0 (C), the Monolisa anti-HCV-Plus v.2 and the Architect Anti-HCV (D), the Monolisa anti-HCV-Plus v.2 and the Enzygnost Anti-HCV 4.0 (E) and the Monolisa anti-HCV-Plus v.2 and the Ortho anti-HCV 3.0 (F), respectively.

Assay performances in all borderline samples

Overall, 87 (92%) samples were classifiable with a high certainty either by using confirmatory immunoblot testing or due to concordant results through all four assays. 64 (74%) were identified as anti-HCV negative, and 23 (26%) as positive. According to this classification, specificity was the highest for the Monolisa anti–HCV-Plus version 2 (100%) and the lowest for the Architect Anti-HCV assay (64%). Specificities of the Enzygnost Anti-HCV 4.0 and the Ortho anti-HCV 3.0 were 91% and 84%, respectively. Sensitivity was good for the Architect Anti-HCV assay and the Ortho anti-HCV 3.0 (83%). The Enzygnost Anti-HCV 4.0 showed the best sensitivity (96%), while the Monolisa anti-HCV-Plus version 2 was the weakest (39%), missing more than half of the positive samples (Fig. 4). The lowest s/co ratio within the misclassified positive samples by the Architect Anti-HCV assay was 0.89 and 0.8 for the Enzygnost Anti-HCV 4.0, both close to the limit of a correct, positive detection. In contrast, the Ortho anti-HCV 3.0 and the Monolisa anti-HCV-Plus version 2 produced wrong results with far lower s/co ratios (0.15 and 0.22 respectively).

FIG. 4.

Overall performance of the four anti-HCV assays. The black bar indicates the sensitivity, while the gray bar shows the specificity (eight samples were not classifiable due to insufficient amount of serum for immunoblot testing and were therefore excluded from this analysis).

By decreasing the s/co cut-off by 25% to 0.75, no anti-HCV positive sample would have been missed with the Enzygnost Anti-HCV 4.0 and the Architect Anti-HCV assay, whereas there still would have been seven anti-HCV positive samples not identified by the Monolisa anti-HCV-Plus version 2 and three by the Ortho anti-HCV 3.0. On the other side, there would have been more false positive test results, twelve more with the Architect Anti-HCV, five additional with the Ortho anti-HCV 3.0 and three more with the Enzygnost Anti-HCV 4.0 compared to the suggested cut-off at one. Three false positive results would have occurred with the Monolisa anti-HCV-Plus version 2.

Discussion

Anti-HCV screening is the first step in the diagnostic algorithm for HCV infection (5,14). Therefore reliability of anti-HCV assays is essential. Major improvements in assay performance have been achieved over the last 2 decades, resulting in high sensitivities and specificities of currently used anti-HCV assays. Still, a great uncertainty remains about the correct interpretation of borderline test results. These borderline results are not infrequent. In our study 1.4% of all samples yielded a borderline s/co ratio after testing with the Architect Anti-HCV assay. In the corresponding patient cohort, we documented a high prevalence of anti-HBc, indicating previous contact to HBV. Due to similar ways of transmission, it can be assumed that this investigated patient cohort was at a particular risk for HCV infection, emphasizing the importance of reliable test results in the selected samples (11).

In previous studies comparing different anti-HCV assays, discordant results have rarely been observed affecting about 2% of the samples (2,16). In this study we could show that in samples with borderline s/co ratios discordance rates occurred of up to 38% between two assays and of almost 50% for all four investigated assays. Conclusively, not all anti-HCV assays produce reliable results in the here-defined borderline s/co setting.

In terms of anti-HCV assays, two different scenarios have to be considered: false positive as well as false negative results. Usually, a reactive anti-HCV result requires confirmatory testing with either immunoblot or HCV RNA PCR. However, both methods are costly and therefore false positive results produce additional unnecessary costs. In our study, the Monolisa anti-HCV-Plus version 2 did not mark a single negative sample as positive, avoiding unnecessary additional testing. In contrast, the Architect Anti-HCV assay produced the highest number of false positive test results and therefore the highest amount of unnecessary costs. However, it has to be highlighted that an obvious bias was introduced due to the applied approach for selecting the samples. Selecting only those samples that showed borderline results with the Architect Anti-HCV assay but in only about one-third of the other assays, certainly resulted in a disadvantage for the Architect Anti-HCV assay. Therefore, the observed performance here must not be compared directly with the other three assays. Future studies aiming to compare all four assays formally in an unbiased way should therefore select also borderline samples obtained by the other assays.

The second scenario concerning false negative test results may be seen as the more important characteristic for a screening test. While false positive results only cause additional costs, false negative test may result in a complete miss of an infection. Certainly not every patient with detectable HCV antibodies also suffers from an ongoing infection. In the majority of patients who cleared the virus either spontaneously or after successful antiviral treatment, HCV antibodies persist. Nevertheless, in a certain number of these patients the level of HCV-antibody decreases by time and may reach very low levels or antibodies may entirely disappear (10,15). As a result, some of these cases may also present with borderline results in anti-HCV assays. In our study, the Monolisa anti-HCV-Plus version 2, the assay that saved the most costs was also the assay that missed the highest number of positive samples according to the immunoblot results.

It is necessary to determine valid cut-offs until the results of these screening assays can safely be trusted. In our study we were not able to identify a valid cut-off for reliable positive results. False positive classifications were found with s/co ratios up to the upper limit of the defined borderline range (s/co ratio 2.0) in the Architect Anti-HCV assay and occurred with even higher s/co ratios in the Enzygnost Anti-HCV 4.0 and the Ortho anti-HCV 3.0 (maximum s/co ratio of a false positive result 4.86 and 3.21, respectively). This demonstrates that the range, in which false positive results must be expected, exceeds the borderline area we defined in our study. To determine a cut-off for false negative results, we calculated a scenario with a lower cut-off of s/co ratio of 0.75. No single positive sample would have been missed with the Enzygnost Anti-HCV 4.0 as well as with the Architect Anti-HCV assay, which indicates that, in samples with a s/co ratio below this modified cut-off in these two assays, no additional testing may be needed in our cohort. However, the here identified cut-off may not directly be transferred to other laboratories as the samples were preselected with the Architect Anti-HCV assay. Thus, future studies will be needed to determine the most cost-effective cut-off until additional immunoblotting or HCV RNA measurements are needed. Furthermore, due to the lack of HCV RNA testing for all patients, we were not able to distinguish between resolved and ongoing HCV infection in every patient. Therefore our study is not able to determine how many patients out of those with a false negative result would have required antiviral therapy.

We have to stress that this study did not determine formal sensitivity cut-offs of the respective assays. We aimed to investigate concordance and discordance between the assays in “real-world” samples from routine clinical testing and their performance in the borderline setting that came up during testing. Formal testing for assays sensitivities should include testing of different HCV genotypes, immunoblotting, and HCV RNA testing of all samples based on appropriate power calculations. HCV antibody test performance was not significantly influenced by the anti-HBc serostatus or HCV RNA detectability. However, other possible confounding factors would need to be investigated in future studies.

Conclusively, anti-HCV assays show different characteristics and performances in a borderline setting with a high level of inter-assay discordance. Still sensitivities of most assays are good as long as additional methods are applied for samples with s/co ratios very close to the limit of positive detection in particular in those, where a false negative result may have dramatic consequences.

Footnotes

Acknowledgment

Funding: The study has partly been funded by the IFB-Tx a project of the BMBF (German Federal Ministry of Education and Research). Siemens Healthcare Diagnostics GmbH has provided the tests.

Ethical approval: The analysis of stored serum samples and anonymous analysis of patient data from patient records was approved by the local ethic committee of Hannover Medical School.

Author Disclosure Statement

Siemens Healthcare Diagnostics GmbH has provided the tests.

HW received lecture fees from Siemens Diagnostics and Abbott Molecular. BM received consulting fees from Abbott Molecular.

Abbreviations Used

References

Alborino

, Burighel

, Tiller

, et al. Multicenter evaluation of a fully automated third-generation anti-HCV antibody screening test with excellent sensitivity and specificity. Med Microbiol Immunol, 2011; 200:77–83.

Berger

, Rabenau

, Allwinn

, and Doerr

. Evaluation of the new ARCHITECT anti-HCV screening test under routine laboratory conditions. J Clin Virol, 2008; 43:158–161.

Cornberg

, Razavi

, Alberti

, et al. A systematic review of hepatitis C virus epidemiology in Europe, Canada and Israel. Liver Int, 2011; 31:30–60.

Dusheiko

, and Wedemeyer

. New protease inhibitors and direct-acting antivirals for hepatitis C: Interferon's long goodbye. Gut, 2012;

EASL Clinical Practice Guidelines: Management of hepatitis C virus infection. J Hepatol, 2011; 55:245–264.

Echevarria

, Avellon

, Jonas

, et al. Sensitivity of a modified version of the ARCHITECT Anti-HCV test in detecting samples with immunoblot-confirmed, low-level antibody to hepatitis C virus. J Clin Virol, 2006; 35:368–372.

Mohd Hanafiah

, Groeger

, Flaxman

, and Wiersma

. Global epidemiology of hepatitis C virus infection: New estimates of age-specific antibody to hepatitis C virus seroprevalence. Hepatology, 2013; 57:1333–1342.

Hezode

, Fontaine

, Dorival

, et al. Triple therapy in treatment-experienced patients with HCV-cirrhosis in a multicentre cohort of the French Early Access Programme (anrs co20-cupic)-nct01514890. J Hepatol, 2013; 59:434–441.

Jonas

, Pelzer

, Beckert

, Hausmann

, and Kapprell

. Performance characteristics of the ARCHITECT anti-HCV assay. J Clin Virol, 2005; 34:97–103.

10.

Lefrere

, Girot

, Lefrere

, et al. Complete or partial seroreversion in immunocompetent individuals after self-limited HCV infection: Consequences for transfusion. Transfusion, 2004; 44:343–348.

11.

Lehmann

, Meyer

, Monazahian

, Tillmann

, Manns

, and Wedemeyer

. High rate of spontaneous clearance of acute hepatitis C virus genotype 3 infection. J Med Virol, 2004; 73:387–391.

12.

Maasoumy

, Port

, Markova

, et al. Eligibility and safety of triple therapy for hepatitis C: Lessons learned from the first experience in a real world setting. PLoS One, 2013; 8:e55285.

13.

Maasoumy

, and Wedemeyer

. Natural history of acute and chronic hepatitis C. Best Pract Res Clin Gastroenterol, 2012; 26:401–412.

14.

Sarrazin

, Berg

, Ross

, et al. 2010. Prophylaxis, diagnosis and therapy of hepatitis C virus (HCV) infection: The German guidelines on the management of HCV infection. Z Gastroenterol, 2010; 48:289–351.

15.

Takaki

, Wiese

, Maertens

, et al. Cellular immune responses persist and humoral responses decrease two decades after recovery from a single-source outbreak of hepatitis C. Nat Med, 2000; 6:578–582.

16.

Zachary

, Ullmann

, Djeddi

, et al. Evaluation of three commercially available hepatitis C virus antibody detection assays under the conditions of a clinical virology laboratory. J Clin Virol, 2005; 34:207–210; discussion 216–218.