Quality of different in-clinic test systems for feline immunodeficiency virus and feline leukaemia virus infection

Abstract

Many new diagnostic in-house tests for identification of feline immunodeficiency virus (FIV) and feline leukaemia virus (FeLV) infection have been licensed for use in veterinary practice, and the question of the relative merits of these kits has prompted comparative studies. This study was designed to define the strengths and weaknesses of seven FIV and eight FeLV tests that are commercially available. In this study, 536 serum samples from randomly selected cats were tested. Those samples reacting FIV-positive in at least one of the tests were confirmed by Western blot, and those reacting FeLV-positive were confirmed by virus isolation. In addition, a random selection of samples testing negative in all test systems was re-tested by Western blot (100 samples) and by virus isolation (81 samples). Specificity, sensitivity, positive and negative predictive values of each test and the quality of the results were compared.

Feline immunodeficiency virus (FIV) and feline leukaemia virus (FeLV) are two common retroviruses in cats that are associated with significant morbidity. The most important sequel of these infections is immunodeficiency followed by opportunistic infections, tumours, and haematological abnormalities, and clinical signs vary according to these related diseases. Thus, diagnosing FIV or FeLV infection clinically is impossible, and the underlying retrovirus infection is often overlooked (Barr 1996). All cats that are presented with clinical signs should be tested, because some FIV- or FeLV-positive cats are immunodeficient which may significantly alter the prognosis and influence the chosen treatment. In addition, the most effective method to guard against infection is to prevent exposure to FIV- and FeLV-infected cats. Testing to identify infected cats is the mainstay of preventing transmission of the viruses. Neither FIV nor FeLV vaccination should be considered a substitute for testing cats (Levy et al 2001). Therefore, reliable test systems are crucial.

Recommended screening tests for FIV and FeLV infection are enzyme-linked immunosorbent assays (ELISA) or other immunochromatographic tests (Levy et al 2001), which are available in several formats for use in veterinary practice or diagnostic laboratories. These tests detect the presence of antibodies against FIV proteins (usually against the p24 core protein or gp40 transmembrane protein) and the presence of soluble antigen of FeLV (usually the p27 core protein). However, the abilities and limitations of each test format and the prevalence of the infectious agent in question must be considered when interpreting test results (Barr 1996).

In recent years, many new in-house tests for diagnosis of FIV and FeLV infection in veterinary practice have been introduced to the market. The question of relative merits of each kit has prompted comparative studies. This study was designed to define the strengths and weaknesses of 11 commercially available tests and to assess their sensitivity, specificity, and predictive values.

Materials and methods

Serum samples

The study was performed at the Department of Small Animal Medicine, College of Veterinary Medicine, University of Georgia, Athens, USA. Serum samples were obtained from 536 randomly selected cats. These cats were examined and treated for a variety of diseases or tested routinely before vaccination at the Teaching Hospital of the College of Veterinary Medicine, University of Georgia, Athens, USA. Serum was separated within 30 min after blood sampling. Tests were performed either immediately or serum was frozen at −70°C and thawed directly before use. According to the manufacturers' instructions, haemolysis, freezing and thawing as well as storing (up to 3 days) in the refrigerator do not interfere with the results.

Samples reacting FIV-positive in one or more of the tests were evaluated by Western blot analysis; samples reacting FeLV-positive in one or more of the tests were evaluated by virus isolation. In addition, a random selection of samples testing negative in all test systems was re-tested by Western blot (100 samples) and by virus isolation (81 samples).

Test systems evaluated

Eleven commercially available test systems for the detection of FIV antibodies and/or FeLV soluble antigen were evaluated. Some of these tests combine detection of FIV antibodies and FeLV antigen in one test system, others either test for FIV antibodies or FeLV antigen.

The serum samples were tested for FIV antibodies with seven tests including Snap Combo Plus (IDEXX, USA), PetChek Plus Anti-FIV (IDEXX, USA), Duo Speed (Bio Veto Test, France), Fastest (MegaCor, Germany), Witness (Synbiotics, USA), Virachek FIV (Synbiotics, USA), and Mapic FIV (Biotech, USA). For testing of FeLV antigen, eight tests including Snap Combo Plus (IDEXX, USA), PetChek FeLV (IDEXX, USA), Duo Speed (Bio Veto Test, France), Fastest (MegaCor, Germany), Witness (Synbiotics, USA), Virachek FeLV (Synbiotics, USA), Mapic FeLV (Biotech, USA), and One-Step (EVL, The Netherlands) were used.

PetChek Plus Anti-FIV, Virachek FIV, PetChek FeLV, and Virachek FeLV are designed for use in laboratory settings as a high number of samples can be tested at once. They use microtitre plates, which make the testing procedure more difficult and time- and labour-intensive. All the other test systems are designed for private practice. They are comparable in costs, requirements in technical skills, and time consumption, as they are easy and fast to perform.

Snap Combo Plus, PetChek Plus Anti-FIV, Virachek FIV, PetChek FeLV and Virachek FeLV are ELISAs and they have to be stored refrigerated (+4°C). Virachek FIV, Virachek FeLV, and PetChek FeLV, contain a synthetic peptide of the FIV p24 antigen and monoclonal anti-FeLV p27 antibodies, respectively. Snap Combo Plus and PetChek Plus Anti-FIV contain FIV p24 and gp40 antigens and monoclonal anti-FeLV p27 antibodies. The other test systems are based on an immunochromatographic principle and kits can be stored at room temperature. They contain a synthetic peptide equivalent to the immunodominant peptide of the FIV gp40 transmembrane protein and monoclonal anti-FeLV p27 antibodies.

Confirmation tests

Serum samples reacting FIV-positive in one or more of the tests as well as 100 negative samples were confirmed by Western blot as described by Egberink et al (1991). The blots contained proteins of purified FIV and envelope protein fragments expressed in Escherichia coli (De Ronde et al 1994). They were regarded as FIV-positive in Western Blot if two or more bands were detected (at the region of the TM2, SU3 proteins and/or the region of the p24 and p15 protein). Serum samples reacting FeLV-positive in one or more of the tests as well as 81 negative samples were assessed for presence of FeLV by virus isolation as described by Jarrett and Ganiere (1996). All samples had been stored at −70°C and were shipped over night on dry ice.

Data analysis

Parameters used to compare the quality of the tests were the percentage of invalid test results, eg, missing control band or coloured background; the percentage of results that were difficult to interpret, eg, weak control band, spots or dirty result window; the diagnostic sensitivity (the proportion of positive test results in infected animals); the diagnostic specificity (the proportion of negative test results in uninfected animals); the positive predictive value (the probability that a test-positive animal is infected); and the negative predictive value (the probability that a test-negative animal is uninfected). In addition, the positive predictive values when combining two FIV and FeLV test systems were calculated.

The evaluation was based on partial verification which is a valid and efficient option when disease events are not prevalent in a population, and verification of all samples would be too costly and time-consuming (Greiner 1999). The test considered as gold standard (Western blot for FIV positivity and virus isolation for FeLV positivity) was performed on each sample that had yielded a positive result in at least one test system and, in addition, on 100 (Western blot) and 81 (FeLV virus isolation) negative samples, respectively. Calculation of the confidence intervals was performed using ‘confidence interval analysis’ (Altmann et al 2000) based on the method of Wilson (1927).

Results

The presence of antibodies against FIV was confirmed by Western blot in 55 of 536 samples (prevalence 10.3%). FeLV viraemia was confirmed by virus isolation in 39 of 528 samples (prevalence 7.4%). Tables 1 and 2 summarise the percentages of invalid test results, the percentages of results that were difficult to interpret, the diagnostic sensitivity and specificity, and the positive and negative predictive values of the different test kits. The test systems Mapic FIV and Mapic FeLV were excluded from the calculations of sensitivity, specificity, positive and negative predictive values, because of their high number of invalid tests (23.1% and 30.4%, respectively) and tests that were difficult to interpret (11.0% and 7.2%, respectively). Tables 3 and 4 show the positive predictive values when combining two FIV and two FeLV test systems, respectively.

Table 1

Comparison of seven FIV test systems in

Tests	Witness	Snap Combo Plus	Fastest	Duo Speed	Virachek FIV	PetChek Plus Anti-FIV	Mapic FIV
Companies	Synbiotics	IDEXX	MegaCor	Bio Veto Test	Synbiotics	IDEXX	Biotech
Countries	USA	USA	Germany	France	USA	USA	USA
Invalid tests (%)	0.4	1.1	0.6	1.1	0.6	0.2	23.1
Tests difficult to interpret (%)	0.8	0.6	0.6	0.8	1.5	0	11.1
Sensitivity (%)	94.5	100	96.4	96.3	92.6	94.5	nd
95% CI (sensitivity)	85.1–98.1	93.1–100	87.7–99.0	87.5–99.0	82.4–97.1	85.1–98.1	nd
Specificity (%)	99.4	99.6	99.2	98.9	99.8	100	nd
95% CI (specificity)	98.5–99.9	98.5–99.9	97.9–99.7	97.6–99.5	98.8–100	99.2–100	nd
Positive predictive value (%)	94.5	94.5	93.0	91.2	98.0	100	nd
Negative predictive value (%)	99.4	100	99.6	99.6	99.2	99.4	nd
n	535	535	535	535	535	535	402

CI=confidence interval; nd=not determined.

Table 2

Comparison of eight FeLV test systems in

Tests	Witness	Snap Combo Plus	Fastest	Duo Speed	Virachek FeLV	PetChek FeLV		One-Step Mapic FeLV
Companies	Synbiotics	IDEXX	MegaCor	Bio Veto Test	Synbiotics	IDEXX	Biotech	EVL
Countries	USA	USA	Germany	France	USA	USA	USA	Netherlands
Invalid tests (%)	1.3	0.6	0.2	1.9	0.2	0.4	30.4	13.3
Tests difficult to interpret (%)	13.8	0.4	1.3	1.5	1.5	1.4	7.2	8.4
Sensitivity (%)	92.1	92.3	94.7	94.7	94.9	92.1	nd	96.8
95% CI (sensitivity)	79.7–97.3	79.7–97.3	82.7–98.5	87.2–98.5	83.1–98.6	79.2–97.3	nd	83.9–100
Specificity (%)	97.5	97.3	98.8	99.2	98.4	99.8	nd	95.4
95% CI (specificity)	95.7–98.6	95.5–98.4	97.3–99.4	97.9–99.7	96.8–99.2	97.3–99.4	nd	93.2–96.8
Positive predictive value (%)	74.5	73.5	85.7	90.0	82.2	85.4	nd	62.0
Negative predictive value (%)	99.4	99.4	99.6	99.6	99.6	99.4	nd	99.7
n	528	528	528	528	528	528	378	517

CI=confidence interval; nd=not determined.

Table 3

Positive predictive values of a sample that reacts positive in two FIV test systems (positive predictive value of test combinations)

	Witness	Snap Combo Plus	Fastest	Duo Speed	Virachek FIV	PetChek Plus Anti-FIV
Witness		98.0	98.1	98.1	98.0	100
Snap Combo Plus			98.0	98.0	97.8	100
Fastest				92.9	98.0	100
Duo Speed					98.0	100
Virachek FIV						100
PetChek Plus Anti-FIV

See Table 1 for the manufacturers' contact information.

Table 4

Positive predictive values of a sample that reacts positive in two FeLV test systems (positive predictive value of test combinations)

	Witness	Snap Combo Plus	Fastest	Duo Speed	Virachek FeLV	PetChek FeLV	One-Step
Witness		81.4	91.9	94.6	83.3	87.2	93.5
Snap Combo Plus			87.5	92.3	81.8	85.0	90.6
Fastest				94.6	89.7	91.9	96.7
Duo Speed					92.3	92.1	94.0
Virachek FeLV						85.4	90.6
PetChek FeLV							90.3
One-Step

See Table 1 for the manufacturers' contact information.

Discussion

A valid study to assess reliability of diagnostic tests should fulfil certain criteria (Polzin et al 1999). Test systems to be evaluated should be compared with a reference ‘gold standard’. The confirmation testing should be performed blinded by an independent person. In addition, samples of the patients tested should be representative of those in which the test will be applied in clinical practice.

In this study, Western blot was chosen as the gold standard for verification of FIV infection. Western blot is generally accepted as the gold standard as confirmation test. However, Western blot analysis has some shortcomings. The requirement for a positive result proposed by Hosie and Jarrett (1990) is the demonstration of antibodies against gp120 or against at least three core proteins, whereas others (Barr et al 1991, Reid et al 1992) suggest that the presence of two virus-specific bands is indicative of a positive reaction. Thus, number and characteristics of the bands have not been clearly defined.

Virus isolation was chosen as the gold standard for verification of FeLV infection. There are some limitations to this confirmatory test. Virus isolation detects the presence of replication-competent virus while the evaluated tests detect soluble antigen in blood. Presence of virus, however, is not always consistent with the presence of soluble antigen. For these reasons, it could be that some true-positive results were misinterpreted as false-positive. In addition, transport and storage of samples can lead to false-negative results in virus isolation due to destruction of the virus (Jarrett et al 1982). In order to avoid this problem, serum samples were stored at −70°C and were shipped over night on dry ice. The decision to choose virus isolation as the confirmatory test in this study was based on the fact that there is no generally accepted gold standard, but in most studies virus isolation is used and in this way comparison with the results of older studies was facilitated. In addition, other tests have limitations, too. For example, the immunofluorescent assay (IFA) yields a high number of false-negative and false-positive results for a number of reasons (Jarrett 1995, Weijer and Van Herwijnen 1995). PCR to detect FeLV DNA in blood cells or FeLV RNA in saliva (Gomes-Keller et al 2006) could be used but PCR might detect latently infected cats and thus, it is not an ideal method as a gold standard for comparison of tests that detect soluble antigens. In a study of Hofmann-Lehmann et al (2001), a significant proportion of animals (10%) was negative for p27 antigen and FeLV-positive by PCR. The goal of the study, however, was the comparison between all FeLV test systems detecting soluble antigen routinely used in private practice and by commercial laboratories and limitations of the confirmation methods can be applied to all FeLV tests evaluated in this study.

The second requirement mentioned above (tests performed by an independent person) was fulfilled. To provide the highest level of standardisation, tests were always carried out by the same person, a veterinarian trained by representatives from the manufacturers. Not all samples in this study were confirmed with the reference methods, but all samples that gave a positive result in one or more of the test systems. In addition, a high number of negative samples (100 and 81, respectively) were subjected to the confirmatory tests. This method is called ‘partial verification’ which is considered a valid option for test comparisons (Greiner 1999).

The samples tested were representative for the patients in which the test will be applied in clinical practice as patients were randomly selected, originating from the ‘normal’ clinic population (diseased and healthy cats) for which FIV and FeLV tests were requested. Prevalence of FIV infection in the investigated population was 10.3%, prevalence of FeLV infection was 7.4%. This is representative for the cat population presented to the Veterinary Teaching Hospital of the University of Georgia, Athens, USA. In a recent study in the United States, 18,038 cats were tested at 345 veterinary practices and 145 animal shelters, 446 (2.5%) reacted positive for FIV antibodies and 409 (2.3%) for FeLV antigen (Levy et al 2006). The difference in prevalence can be explained by the fact that in the present study a high number of referred cases to the Teaching Hospital with unclear diseases were among the cats tested. Consequently, FIV and FeLV prevalence may be higher than the infection rate expected in private practice depending on the region and number of stray cats.

For the FIV diagnostic test systems, Mapic FIV test was unacceptable due to 23.1% invalid tests and 11.1% tests difficult to interpret. Therefore, this test was excluded from further evaluation and cannot be recommended for use in practice. All other in-clinic tests showed good overall performance. They were comparably high in sensitivity and specificity as well as in predictive values. Although only slightly different from the other test systems, Duo Speed had a lower sensitivity, positive predictive value, and slightly lower specificity. Witness showed better results than Duo Speed and Fastest. Snap Combo Plus had the overall best results among all FIV test systems and the best improvement when compared to the results of Hartmann et al (2001). It not only showed a much lower number of tests difficult to interpret, but also had increased sensitivity and positive predictive value. This may be due to the fact that an additional antigen was incorporated in the test between the study published in 2001 and this study. To the authors' knowledge, none of the other test systems changed their product composition. FIV microplate ELISA tests, Virachek FIV and PetChek Plus Anti-FIV, also performed very well. They had slightly lower sensitivity than Snap Combo Plus, but specificities and positive predictive values exceeded results obtained from Snap Combo Plus.

Comparing general performance of all FeLV test systems evaluated in this study, all of them, other than Mapic FeLV, showed acceptable results. Considering the high number of invalid test results, Mapic FeLV was completely unacceptable as almost one-third of the results obtained were invalid and an additional 7.2% were difficult to interpret. Therefore, this test was excluded from further evaluation and also cannot be recommended for use in practice. One-Step with 13.3% invalid tests and 8.4% tests difficult to interpret also showed a high number of unacceptable results. Sensitivity of One-Step, however, was very high concerning detection of FeLV. Therefore, One-Step cannot be recommended as a single test for screening, but could be used in combination with other tests to increase positive predictive values. Witness also had a high number of tests that were difficult to interpret (13.8%). Fastest had a low number of invalid tests and of tests difficult to interpret. Duo Speed showed the overall best performance and thus has to be considered as the best in-house test for FeLV testing. It had the highest specificity and the highest positive predictive value. It also had a low number of tests that were invalid or difficult to interpret. If the microplate ELISA tests, Virachek FeLV and PetChek FeLV, were compared to the in-clinic tests; those tests had a sensitivity comparable to Duo Speed but a lower specificity. This explains the lower positive predictive values.

Combinations of tests were evaluated to find out whether combining two different tests would provide higher positive predictive values (Tables 3 and 4). For the FIV test systems, positive predictive values of the microplate ELISA tests were already very high. For the in-clinic tests, highest positive predictive value of one single test was 94.5% (Snap Combo Plus and Witness). A positive predictive value of 98.0% could be reached when a combination of SNAP Combo Plus and Witness, Snap Combo Plus and Fastest, Snap Combo Plus and Duo Speed, and of 98.1% when Witness and Fastest or Witness and Duo Speed was used. While the highest positive predictive value of a single FeLV test was 90.0% (Duo Speed), a combination of different tests was able to reach a positive predictive value of 96.7%, if both tests reacted positive. Recommended test combinations are Duo Speed in combination with Witness, Duo Speed in combination with the Fastest, Duo Speed in combination with One-Step, and Fastest in combination with One-Step.

In conclusion, the following recommendations can be given. Most in-clinic tests in general are comparable to the microplate ELISA tests in their performance and can be recommended for use in private practice. For FIV testing, Snap Combo Plus is recommended as the best performing in-clinic test. The second test to increase the positive predictive value should be Fastest or Duo Speed. For FeLV testing, Duo Speed is the in-house test of choice. It has the highest positive predictive value. However, a positive result should be confirmed (especially in a healthy cat of a low risk environment). For second testing of a Duo Speed FeLV-positive sample, Witness or Fastest are recommended.

Footnotes

Acknowledgements

We thank all the companies who participated in this study and provided the tests at reasonable prizes, especially IDEXX, who financed most of the study.

References

Altmann

D.G.

Machin

Bryant

T.N.

Gardner

M.J.

Statistics with Confidence, 2nd edn, 2000, BMJ Books

Barr

M.C.

Pough

M.B.

Jacobson

R.H.

Scott

F.W.

Comparison and interpretation of diagnostic tests for feline immunodeficiency virus infection, Journal of the American Veterinary Medical Association 199, 1991, 1377–1391.

Barr

M.C.

FIV, FeLV, and FIPV: interpretation and misinterpretation of serological test results, Seminar of Veterinary Medicine and Surgery of Small Animals 11, 1996, 144–153.

De Ronde

Stam

J.G.

Boers

Langedijk

Meloen

Hesselink

Kerdermans

L.C.E.J.M.

Van Vviet

Verschoor

E.J.

Horzinek

M.C.

Egberink

H.F.

Antibody response in cats to the envelope proteins of feline immunodeficiency virus: identification of an immunodominant neutralization domain, Virology 198, 1994, 257–264.

Egberink

H.F.

Lutz

Horzinek

M.C.

Use of Western blot and radioimmunoprecipitation for diagnosis of feline leukemia and feline immunodeficiency virus infections, Journal of the American Veterinary Medical Association 199, 1991, 1339–1342.

Gomes-Keller

M.A.

Gönczi

Tandon

Riondato

Hofmann-Lehmann

Meli

M.L.

Lutz

Detection of feline leukemia virus RNA in saliva from naturally infected cats and correlation of PCR results with those of current diagnostic methods, Journal of Clinical Microbiology 44, 2006, 916–922.

Greiner

(1999) Current concepts in test validation and interpretation. In: Proceeding of the 9th Annual Congress of the European Society of Veterinary Internal Medicine (ESVIM), Perugia. pp 67–72.

Hartmann

Werner

R.M.

Egberink

Jarrett

Comparison of different in-house tests for rapid diagnosis of feline immunodeficiency and feline leukaemia virus infection, Veterinary Record 149, 2001, 317–320.

Hofmann-Lehmann

Huder

J.B.

Gruber

Boretti

Sigrist

Lutz

Feline leukaemia provirus load during the course of experimental infection and in naturally infected cats, Journal of General Virology 82, 2001, 1589–1596.

10.

Hosie

M.J.

Jarrett

Serological responses of cats to feline immunodeficiency virus, Acquired Immunodeficiency Syndrome 4, 1990, 263–278.

11.

Jarrett

Golder

M.C.

Stewart

M.F.

Detection of transient and persistent feline leukaemia virus infections, Veterinary Record 110, 1982, 225–228.

12.

Jarrett

Ganiere

J.P.

Comparative studies of the efficacy of a recombinant feline leukaemia virus vaccine, Veterinary Record 138, 1996, 7–11.

13.

Jarrett

Detection of FeLV antigen, Veterinary Record 137, 1995, 127.

14.

Levy

Richards

Edwards

Elston

Hartmann

Rodan

Thayer

Tompkins

Wolf

Feline retrovirus testing and management, Compendium on Continuing Education for the Practicing Veterinarian 22, 2001, 652–657.

15.

Levy

J.K.

Scott

H.M.

Lachtara

J.L.

Crawford

P.C.

Seroprevalence of feline leukemia virus and feline immunodeficiency virus infection among cats in North America and risk factors for seropositivity, Journal of the American Veterinary Medical Association 228, 2006, 371–376.

16.

Polzin

D.J.

Lund

Walter

Klausner

From journal to medicine: evidence-based medicine. Bonagura

J.D.

Kirk's Current Veterinary Therapy XIII Small Animal Practice, 1999, WB Saunders: Philadelphia, 2–8.

17.

Reid

R.W.

Barr

M.C.

Scott

F.W.

Retrospective serological survey for the presence of feline immunodeficiency virus antibody: a comparison of ELISA and IFA techniques, Cornell Veterinarian 82, 1992, 359–369.

18.

Weijer

Van Herwijnen

Detection of FeLV antigen, Veterinary Record 137, 1995, 127.

19.

Wilson

E.B.

Probable inference, the law of succession, and statistical interference, Journal of the American Statistical Association 22, 1927, 209–212.