Abstract
Serum antibody titers are a useful measurement of protection against infection (feline panleukopenia virus [FPV]) or clinical disease (feline herpesvirus-1 [FHV] and feline calicivirus [FCV]), and their determination has been recommended as part of disease outbreak management in animal shelters. The objective of this study was to determine the sensitivity, specificity, and inter-observer and inter-assay agreement of two semi-quantitative point-of-care assays for the detection of protective antibody titers (PAT) against FPV, FHV and FCV in shelter cats. Low sensitivity for FPV antibodies (28%) rendered a canine point-of-care assay inappropriate for use in cats. The feline point-of-care assay also had low sensitivity (49%) and low negative predictive value (74%) for FPV PAT detection, but was highly accurate in the assessment of FHV and FCV PAT. Improvements in accuracy and repeatability of FPV PAT determination could make this tool a valuable component of a disease outbreak response in animal shelters.
In the owned pet cat population, illness caused by feline panleukopenia virus (FPV) is relatively uncommon and feline herpesvirus-1 (FHV) and feline calicivirus (FCV) infections usually result in mild illness. In contrast, these diseases are a constant and serious threat to the health and welfare of cats residing in animal shelters. Outbreaks of FPV are frequently managed by depopulation of both clinically diseased and healthy exposed cats.1–3 Upper respiratory disease caused by FHV and FCV is associated with increases in length of shelter stay, animal care costs, and euthanasia.4,5 Because of these threats, rapid recognition of cats at increased risk for infection, particularly FPV, is critical in the high-risk environment of an animal shelter.
Serum antibody titers have been shown to be a useful measurement of protection against infection (FPV) or clinical disease (FHV and FCV), and their determination has been recommended as part of disease outbreak management in animal shelters.6–8 However, titer determination at a reference laboratory is technical, time-consuming, and often cost-prohibitive for animal shelters. The availability of a rapid semi-quantitative point-of-care assay for protective antibody titers (PAT) would be a valuable tool for disease outbreak management. A commercially available point-of-care enzyme-linked immunosorbent assay (ELISA: TiterCHEK CDV/CPV, Synbiotics, San Diego, CA) previously found to be an accurate method of identifying PAT against canine parvovirus (CPV) 9 has been previously recommended for evaluation of FPV PAT in cats, 10 even though it is only labeled and approved for use in dogs. A feline-specific point-of-care ELISA (ImmunoComb, Feline VacciCheck, Biogal Galed Laboratories, Kibbutz Galed, Israel) has been developed for the determination of PAT against FPV, FHV and FCV. No information has been published about the accuracy of these two assays in cats compared to gold standard laboratory analysis.
The objective of this study was to determine the sensitivity, specificity, and inter-observer and interassay agreement of these two semi-quantitative point-of-care assays for the detection of PAT against FPV, FHV and FCV and to determine if the assays would be useful in the management of an FPV out-break in a shelter.
Materials and methods
Samples
A total of 356 serum samples from cats and kittens entering three shelters in north Florida between May 7, 2010 and August 15, 2010 were collected for analysis. A total of 56 archived serum samples collected from feral cats undergoing elective ovariohysterectomy or orchiectomy through a trap—neuter—return program (Operation Catnip, Gainesville, FL) were also used. Blood (3–5 ml) was collected via jugular, cephalic or femoral venipuncture into serum separator tubes within 1 day of admission to the shelter. Serum was separated by centrifugation, aliquoted into duplicate cryovials and stored at −20°C pending analysis. The study protocol was approved by the University of Florida Institutional Animal Care and Use Committee.
Serological testing
Serum antibody titers were measured by use of a hemagglutination inhibition assay (HI) for FPV and by a serum neutralization assay (SN) for FHV and FCV at a university-affiliated diagnostic laboratory (Animal Health Diagnostic Center, College of Veterinary Medicine, Cornell University, Ithaca, NY). The laboratory recommended assessment for booster vaccination when titers <40, <8, and <32 are reported for FPV, FHV, and FCV, respectively.
The canine assay utilizes color-coded plastic wells coated with purified CPV antigen. One microliter of serum or plasma is placed into a well and incubated with polyclonal rabbit anti-dog IgG conjugated to horseradish peroxidase. A chromogenic substrate is added and the subsequent color reaction is compared visually to positive and negative control wells on the plate. Samples with color reactions of equal or greater intensity than the positive control are considered to have PAT against CPV (equivalent to HI titers ≥80 in dogs). Those with reactions of less intensity than the positive control well are considered negative for PAT. The assay also includes canine distemper virus (CDV) antigencoated wells and can simultaneously determine PAT against CDV when used with canine samples.
The feline assay utilizes a plastic, comb-shaped card with FPV, FHV, and FCV antigen-coated test spots (in addition to a positive reference spot) attached to each of 12 teeth and a reagent-filled, six-row, multicompartment developing plate. A serum, plasma (5 μl) or whole blood (10 μl) sample is deposited in the first row of the developing plate using capillary tubes and a piston (provided in the kit) or a calibrated pipette. The comb is then inserted into the first row allowing for binding of antibodies present in the sample to the antigen spots on the comb device. At timed intervals, the comb is transferred to the remaining wells for washing, binding of labeled secondary antibodies (anti-cat IgG), and color development, resulting in the production of a gray color tone. The intensity of the gray tone is visually compared to the positive reference spot and to a gray scale provided with the kit to determine whether PAT is present. The kit instructions define PAT as HI titers ≥40 for FPV and SN titers ≥16 and ≥32 for FHV and FCV, respectively. Test spots equal to or darker than the positive reference spot are considered indicative of PAT. Those lighter than the positive reference are considered negative for PAT (Fig 1). Alternatively, a software program developed for the assay can be used to measure the relative density of spots on scanned images and to calculate a quantitative titer equivalent (see ImmunoComb, Feline VacciCheck product insert). Once developed, test spots are permanent allowing for archiving and review of results.
A feline point-of-care ELISA kit for the determination of PAT against FPV, FHV, and FCV. A 12-toothed comb with antigen test spots is moved through wells of the developing plate at timed intervals (A). The results are displayed as gray color tones and are compared to a Comb Scale for interpretation (B).
A set of feral (n = 56) and shelter (n = 20) cat serum samples were thawed and analyzed to evaluate both point-of-care assays according to manufacturers' instructions during pilot testing. Due to poor sensitivity of the canine assay for FPV antibodies in the pilot study, only the feline assay was subsequently used in the definitive study. In the definitive study, 356 serum samples from shelter cats were thawed and tested for FPV, FHV, and FCV PAT using the feline point-of-care assay according to manufacturer's instructions.
Results derived from the first run in the software program were used for calculation of sensitivity and specificity. Test combs were also evaluated visually by observers blinded to the software results. To assess inter-observer variability, assay results were evaluated by visual assessment of the color intensity using the scale provided in the kit by three different observers blinded to the previous results. The numeric scale score was translated into a ‘positive’ or ‘negative’ result according to manufacturer's instructions and compared to those reported by the assay software. To assess inter-assay variability, a subset of 60 serum samples was re-tested on a different day. To assess repeatability of scanned results, 476 samples (416 unique samples from pilot and definitive testing plus 60 samples from inter-assay testing) were scanned and analyzed by the software program a total of three times.
Statistical analysis
Sensitivity for each assay was calculated as the number of cats with positive results for PAT on the gold standard assay (true-positives) divided by the total number of cats with positive results on the gold standard assay (true-positives) plus the total number of cats with positive results on the gold standard assay but negative results on the test assay (false-negatives). Specificity was calculated as the number of cats with negative results on the gold standard assay (true-negatives) divided by the total number of cats with negative results on the gold standard assay (true-negatives) plus the total number of cats with negative results on the gold standard assay but positive results on the test assay (false-positives). PAT prevalence was defined as the number of cats with PAT as determined in the gold standard assays divided by the number of cats tested. Positive predictive value (PPV) was calculated as the number of true-positives divided by the total number of positive results. Negative predictive value (NPV) was calculated as the number of true-negatives divided by the total number of negative results. Overall accuracy was calculated as the number of true-positives plus true-negatives divided by the total number of results. Because PPV and NPV vary depending on actual prevalence in the specific population being tested, PPV and NPV were also calculated for theoretical PAT population prevalences of 25%, 50%, and 75% to mimic conditions found in various cat populations. Inter-observer and inter-assay agreement were assessed by calculating the kappa coefficient. (GraphPad Software, Quantify agreement with kappa. http://www.graphpad.com/quickcalcs/kappa1.cfm (accessed Oct 25, 2010)). Observations with kappa values <0.6 were considered to have poor agreement, those ≥0.6 and ≤0.8 to have good agreement, and those >0.8 to have very good agreement. 11
Results
Pilot testing of canine and feline assays
PAT prevalence for FPV in the 76 samples tested was 93% as determined by HI. The sensitivity of the canine assay for the study population was 28%, resulting in an NPV of 9% and overall accuracy of 33% (Table 1). The specificity and PPV of the assay were each 100%. The assay never reported a positive PAT result for samples with titers <640 (PAT is ≥40). The feline assay reported valid results for 73/76 samples. The three invalid results reported by the software program did not appear grossly abnormal and could be scored visually. The sensitivity was 72%, NPV was 21%, and overall accuracy was 74%. The specificity and PPV of the assay were each 100%. Because of the poor sensitivity of the canine assay for feline samples, no further testing was performed.
Pilot study results of two semi-quantitative point-of-care assays for the detection of PAT when used to detect FPV antibody titers in a population of 76 shelter and feral cats. Point-of-care assay results reported as positive or negative (indicating the presence or absence) of PAT were compared to the gold standard HI assay. HI titers ≥40 are considered positive for PAT. Canine assay titers ≥80 are considered positive for PAT against CPV
Definitive testing of feline assay
Of the 356 samples tested, 145 (41%) were identified by HI to have PAT for FPV. As identified by SN, 35 (10%) and 127 (36%) had PAT for FHV and FCV, respectively. Performance of the feline ELISA is reported in Table 2. The software program reported invalid results for a total of 12 samples. One of these 12 samples was also not able to be evaluated visually due to lack of development of any of the test spots on that tooth of the comb, including the positive control. The remaining invalid results did not appear grossly abnormal and could be read visually. Ten of these 11 invalid results occurred on the second tooth of the comb, suggesting a possible manufacturing or software defect for this tooth. A large proportion of erroneous results was due to false-negative results reported for samples with FPV HI titers of 40 (58/89 [65%] discordant FPV results) but false-negative results were reported for titers as high as 2560. False-negative results for FHV and FCV represented a variety of SN titers (FHV range 8–24; FCV range 32–768).
Performance of a feline point-of-care assay for the detection of PAT against FPV, FHV, and FCV in 344 shelter cats
The predictive value of a positive or negative result was calculated using the actual prevalence of PAT in the samples from the study population (Table 2). Because predictive values change when prevalence changes, PPV and NPV were also calculated for theoretical PAT prevalences of 25%, 50%, and 75% (Table 3).
Calculated positive and NPVs of a feline point-of-care ELISA for three different prevalence levels of FPV, FHV, and FCV
The strength of agreement between the software program and visual interpretation by three observers is reported in Table 4. In general, agreement was good for FPV (κ = 0.74–0.76) and FHV (κ = 0.78–0.80), and very good for FCV (κ = 0.87–0.90). A total of 60 samples was assayed a second time to evaluate interassay variability. The inter-assay agreement was very good for FPV (κ = 0.84, confidence interval [CI] = 0.67–1.02), good for FHV (κ = 0.69, CI = 0.39–0.98), and very good for FCV (κ = 0.93, CI = 0.83–1.03).
Percent of concordant results and strength of agreement between scanned images interpreted by a software program and visual interpretation by three observers when using a point-of-care assay for the detection of PAT against FPV, FHV, and FCV in 356 shelter cats
Among the 476 samples scanned three times on different days using the software program, valid results were available for 450 samples. Invalid results were reported for 19 samples during the first scan, 13 samples during the second scan, and 15 samples during the third scan, representing a total of 25 different serum samples. Invalid results were consistently reported across all three scans for eight samples. Excluding samples with invalid results during any scan, there were 57 samples (12%) with variable results between the three scans for FPV, 44 (9%) for FHV, and 10 (2%) for FCV. Four samples tested for FPV resulted in a different titer value reported for each scan; the remainder of the discordant results for FCV, FHV, and FPV resulted in two like titers and one different titer among the three scans. Four of the 57 (7%) variable FPV analyses resulted in discordant PAT results (ie, present vs absent). Two of the 44 (5%) variable FHV analyses and one of the 10 (10%) FCV analyses resulted in discordant PAT results. These errors would have resulted in an error in clinical diagnosis.
Discussion
Low sensitivity for FPV antibodies rendered the canine point-of-care assay inappropriate for use in cats. This is not surprising given that the assay relies on the detection of anti-dog IgG and the manufacturer makes no claims about its use in other species. The feline point-of-care assay also exhibited a high number of false-negative results for FPV PAT, resulting in low sensitivity and low NPV, but was highly accurate in the assessment of FHV and FCV PAT. When samples were analyzed a second time, and when developed combs were scanned multiple times visually or by computer, discordant results for PAT occurred in 5–10% of samples.
The assays evaluated in this study were designed to assess the need for cats and dogs to receive booster vaccines rather than reliance on a predetermined immunization schedule. These semiquantitative assays provide a positive result when serum antibody levels consistent with protection against infection are present. Although cats with PAT against FPV are considered to be immune to infection, those with PAT against FHV and FCV are protected only against severe clinical disease.7,8,12,13 It is not possible to use antibody titers to determine the converse, if cats are susceptible to infection, because the innate and cell-mediated arms of the immune system also contribute to host defense but are more difficult to measure.
While sensitivity and specificity are fixed characteristics of diagnostic tests, variations in actual PAT prevalence influence the predictive value of positive or negative results. That is, PPV decreases as prevalence decreases and NPV decreases as prevalence increases. Recent work by our laboratory indicates a PAT prevalence of 40% for FPV, 11% for FHV, and 37% for FCV in the population of shelter cats that provided samples for this study. 14 Another author has reported similar protective titers against FPV (48%) in shelter cats in Wisconsin. 15 Confirmatory testing of questionable results using gold standard assays may be indicated in some cases.
The first step in the management of a disease outbreak in an animal shelter is the identification and isolation of cats with active disease on the basis of clinical signs or diagnostic testing.15,16 Although there is no commercial antigen test designed specifically to detect FPV in cats, an assay for the detection of CPV antigen in dogs has been found to detect FPV in a majority of, but not all, infected cats. 17 Modified-livevirus vaccines should be administered to any cats not previously fully vaccinated and to any cat in which appropriate administration of previous vaccines cannot be assured. Next, measurement of antibody titers can be used to assign infection risk categories to exposed, asymptomatic cats. Exposed, asymptomatic cats with PAT are at lower risk for infection and may be placed for adoption. Those without PAT may be in a subclinical or incubation phase of infection and are kept in quarantine for the duration of the disease incubation period or otherwise removed from the population. This is an especially useful tool in the face of an FPV outbreak because the incubation period is short, there is no carrier state, and PAT is highly predictive of immunity. This contrasts with infection with FHV or FCV, which often result in chronic carrier states and the development of partial immunity, making titer interpretation more difficult.
Inaccurate antibody test results can have a substantial impact on management outcomes in any environment and their implications must be considered. During a shelter disease outbreak, failure to detect PAT may result in an exposed low-risk cat being kept unnecessarily in shelter quarantine or being euthanased. Failure to detect the absence of PAT may cause a high-risk exposed cat to be kept in the shelter population or to be adopted, potentially facilitating the spread of infection. A screening test with a high specificity for PAT, such as the feline point-of-care ELISA in this study, can help limit disease transmission, but the low sensitivity of this assay for FPV may result in inefficient management decisions such as increasing numbers of cats being held in quarantine and unnecessary euthanasia. However, given the high PPV of the feline ELISA, it may be a useful tool in the context of a shelter where large numbers of animals are routinely euthanased. Despite its lack of sensitivity, it will correctly identify a portion of animals at low risk for disease, ensuring that those selected to remain in the population are likely to be protected.
Cost, efficiency, and ease of use play an important role when choosing a diagnostic test in an animal shelter. The feline ELISA can be performed at the shelter on an as-needed basis using only 5 μl of serum or plasma or 10 μl of whole blood for all three titers. The assay takes approximately 30 min to perform (not including sample collection and preparation), but is a multi-step process requiring a proficient technician and can be time-consuming for shelter staff if there are many cats to evaluate. In this study, efficiency of sample handling was substantially improved through use of a calibrated pipette rather than the capillary tube and piston provided with the test kit, and ease of test spot interpretation was improved through use of the scanning software. In comparison, titer measurement at most reference laboratories requires 1 ml of serum or plasma and less shelter staff time because the samples are analyzed off-site. Availability of test results takes longer than with the ELISA, which may be problematic during disease outbreaks when prompt triage of exposed cats is essential.
The feline point-of-care screening ELISA, which had high diagnostic accuracy and can be performed in less than an hour, can be a useful tool for evaluation of PAT against FHV or FCV, but only identified about half of cats with FPV PAT. Improvements in accuracy and repeatability of FPV PAT determination could make this tool a valuable component of a disease outbreak response, assisting shelters in saving as many cats as possible while preventing the spread of infection to other cats, inadvertently adopting infected cats to the public, or resorting to depopulation in the event of an infectious disease outbreak.
Conflict of interest
Support for this study was provided by the Sweetbay Foundation and Maddie's Fund.
Footnotes
Acknowledgements
The authors would like to thank Biogal Galed Laboratories, Kibbutz Galed, Israel for supplying test kits used in this study as well as Michael Crandall and Patricia A Dingman for serving as test observers.