Developing an Automated Enzyme Immunoassay for High-Throughput Screening of Bovine Serum Antibodies to Neospora Caninum

Introduction

In veterinary science, especially in the field of population medicine, surveillance studies are important for the understanding of the disease dynamics in a targeted animal production group. The use of an automated high-throughput screening assay became necessary. Originally, HTS concentrated on acquiring the best robotic workstations and using them in performing more assays per day. This drive for higher throughput led to significant advances in miniaturization. In the future, in addition to the ability to test large numbers of specimens per day, providing flexibility for multiple tests scheduling and the ability to manipulate vast amounts of laboratory data accurately will play an important part in the automation of veterinary diagnostic test development.

The past decade saw the development of many automated enzyme immunoassays used to measure a variety of biological molecules.^1,2,3 During this time, robotic automation with microplates evolved to become the hardware basis to support high-throughput screening.^4,5 In order to ensure compatibility with the different commercial robotic liquid handling workstations, the Society of Biomolecular Screening (SBS) standard microplate footprint was used to develop the size and shape of these microplates.

The increased workload and demand to produce internationally recognized quality results from a veterinary laboratory led to the need for more cost-effective and reliable assays. Automated enzyme linked immunosorbent assay is the logical tool to be used for HTS tests in surveillance studies. Besides increasing the efficiency of the laboratory, an automated ELISA (a-ELISA) increases the bio-safety in the laboratory by minimizing risks due to human error. In the past five years, our laboratory has developed a number of cost-effective ELISA-based assays for the screening of antibodies and antigens that are of animal health and food safety concern. We have been employing a commercial, automated, liquid handling workstation and adopting the SBS standard 96-well microplate format in all our work. Two of these tests received accreditation from Standards Council of Canada by fulfilling the requirements of the ISO/IEC 17025 standard. ⁶

Neospora caninum is a protozoan parasite found in cattle, sheep, goats, deer, horses, and dogs worldwide. ⁷ In cattle, it causes abortion in cows and encephalomyelitis in congenitally infected calves, resulting in huge economic losses to cattle producers.^8,9,10 A number of manual enzyme linked immunosorbent assays used to screen for specific antibodies to the parasite in cattle are available.^11–15

The objectives of this project were to develop an automated ELISA using a robotic workstation for HTS of bovine serum for antibodies to N. caninum and to evaluate the applications of this test, based on its performance characteristics, that could fulfill the requirements of the ISO/IEC 17025 standard. ⁶

Experimental Design

The basic design consisted of three major stages. The three stages were test sample selection, automated ELISA development, and test validation.

TEST VALIDATION

The last stage was to validate the a-ELISA, a requirement of the ISO 17025 standard for test accreditation. ⁶ The test performance characteristics were examined. This included reproducibility, repeatability, sensitivity, specificity, positive (PV +) and negative (PV-) predictive values, and kappa (K) value evaluation.

The reproducibility studies were set up in the following manner. A blind protocol was used to compare the a-ELISA results to those of the manual ELISA obtained in our laboratory. The true identity information of these animals was kept from the technologists; therefore those who were performing the assays would not have any knowledge of the results of the other ELISA. The laboratory supervisor involved duplicated the serum samples to make up a test sample size of 400 (N = 400) sera and the duplicated samples were labelled in such a manner that the technologist would not be able to match up the duplicated samples. Again, this coding system was information privy only to the laboratory supervisor. Statistically, the sample size of 200 animals used was adequate to yield a 95% required precision for the assay. ¹⁷

A repeatability test was used to examine how a-ELISA quality assurance was being maintained. The IDEXX kits include both pooled positive and pooled negative controls. A total of 30 pairs of control sera from different test kits of the same lot were tested over a period of four weeks. The results were examined using a Levey-Jennings control chart. ¹⁸ The mean and standard deviation (SD) of O.D. values of the positive control sera from 30 individual measurements were plotted. The charts were constructed by scaling the x-axis to accommodate the data from 30 runs, and by scaling the y-axis to include a range from the (mean + 3 SD _mean) to the (mean − 3 SD _mean). Lines representing the (mean), (mean ±2 SD _mean), and (mean± 3 SD _mean) were drawn on the chart. All 30 measurements were plotted directly on the chart. The same procedure was followed for the negative control sera. Using the two charts, the agreement between replicates and the amount of between-run agreement for each control serum were analyzed.

OPEN IN VIEWER

The proficiency test results, sensitivity, specificity, positive and negative predictive values were assessed using a two by two table. ¹⁹ The identity scoring of the two tests was compared by using the kappa quotient (K) calculation.

Results

The screening for specific N. caninum antibodies in the duplicated serum samples (N = 400) was completed in one test run when using the a-ELISA, whereas the manual ELISA could only test 40 samples each run. The turn around time (TAT) was approximately two hours in each case. Both ELISAs showed identical results between duplicated serum samples. The results of the duplicated pair were either both positive or both negative. Based on the data comparison studies using the two by two table (Table 1), of the 400 tests compared, 392 test results matched exactly between the a-ELISA and the manual test. A total of eight results were different between the two assays. The eight (four pairs) different results were represented by three animals tested positive by manual ELISA that tested negative with the a-ELISA, and one animal tested negative by the manual ELISA that tested positive with the a-ELISA. The good matching results between the two assays of duplicated samples demonstrated that a-ELISA results were reproducible.

OPEN IN VIEWER

Table 1.

Two by two table for the comparison of the a-ELISA and the manual ELISA.

		Manual ELISA
		Positive	Negative
A-ELISA	Positive	168	2
	Negative	6	224

For the repeatability studies of the positive control and negative control sera of the a-ELISA, the two sets of results were plotted on Levey-Jennings control charts (Figure 2 and Figure 3), and the mean and SD were calculated. For the positive control serum, the mean was 0.387 and the SD was 0.053. For the negative control serum, the mean was 0.143 and the SD was 0.009. Examining the two charts, the O.D. of the 30 individual runs of either control were all within ± 2SD O.D. values. The ELISA was considered repeatable if the variation of each of the 30 positive and 30 negative control O.D. values was within ±2 SD of the mean of the individual runs. These charts showed that the a-ELISA developed was a precise test procedure, since the results were repeatable between tests performed on different dates.

OPEN IN VIEWER

Figure 2.

Levey-Jennings control chart of the positive control serum tested by a-ELISA.

By using the two by two table data, the two ELISAs were compared and the performance characteristics were assessed. ²⁰ Sensitivity (96.55%) and specificity (99.12%), PV+ (98.82%) and PV- (97.39%) were obtained. The K value was 0.9592.

The apparent prevalence in this infected herd was 0.42 using the manual ELISA, whereas with the a-ELISA it was 0.425, a difference of 1.19%. Using the sensitivity of 96.55% and the specificity of 99.12%, the true prevalence rate of the herd was calculated to be equal to 0.435. ²⁰

Discussion

When automating any laboratory test, several important attributes must be considered. These are automation flexibility, operation complexity, and system throughput. Of equal importance, we must examine the automated assay performance characteristics and its applications. Furthermore, in order for the results to be accepted by national and international clients, accreditation of the test by an internationally recognized organisation should be in place.

There was a high degree of automation flexibility with the B2K workstation. It allowed easy adjustment of the different liquid handling steps, incorporation of a variety of equipment and provided the user with an integrated software program. We were able to combine the different essential steps of the ELISA protocol into one combined program. An open design, reinforced by a modular approach for setting up the hardware during the planning and scheduling of the test, provided great flexibility.

OPEN IN VIEWER

The degree of operation complexity was kept to a minimum for this a-ELISA. The programming was straightforward and user friendly. One in-house program was written to run this a-ELISA and a second one for the handling of the data. All necessary liquid handling steps, including liquid transfer, microplate shaking, incubation, washing, and reading steps were automated. Each time, the end user had simply to use the “run” command to start the procedure. The workstation is maintained and certified for accuracy by the manufacturer annually. The technologist only has to perform routine weekly workstation alignment procedures.

Improving the throughput of the ELISA for screening antibodies was the goal of this project. Forced downtime due to human fatigue between steps of the assay was eliminated. In order to prevent forced downtime due to machine failure, the B2K is maintained in good working condition by the manufacturer and our routine maintenance measures. These measures were necessary quality assurance steps according to requirements of ISO 17025 standard. ⁶ The automated procedure enabled the technologist to increase walk-away time (WAT) during each screen. The time could be used to complete other major tasks of the assay, such as specimen preparation and data reporting. HTS was achieved by reducing the time required to complete a screen (i.e., TAT) and an increase of output capacity. One technologist applying the a-ELISA can run 10 times more specimens per test run than can be done by relying on a manual ELISA, a huge decrease of TAT. In any survey involving a large number of specimens, the inconsistency of results between different test runs is a source of uncertainty that can affect the interpretation reliability of the project. HTS can minimize this source of uncertainty by allowing smaller numbers of test runs per project. Furthermore, the ability of the automation station to perform correctly each and every liquid handling step of the assay eliminates one major source of mistake, human error.

The commercial manual N. caninum antibody ELISA is highly labor intensive. The a-ELISA, on the other hand, is not labor demanding. At the end of a test, the results should be communicated to the clients in an easy-to-read format. The “paper trail” of all the different steps of the ELISA should be traceable and devoid of any mistake. The a-ELISA accomplished both these tasks - all the relevant data of the different liquid handling steps and laboratory results were handled by computer hardware and software programs that recorded and processed these data and quality assured the accuracy and integrity of the results.

In order to assure reliability of the proficiency test in this project, sample size and age of animal considerations were required. Use of a small sample size can be considered as a confounding factor in any comparison study. A relevant sample size based on the prevalence of the disease can provide a better comparison test. In this project, it was limited by the actual herd size from which the sera were collected. For the purpose of a comparison study of two tests, the number used was sufficient to provide the 95% required precision. Since the prevalence rate was < 40%, if the herd size had been much bigger than 250 head, around 336 to 400 animals would have been required for sampling and testing. However, for a herd size of 250 animals, the required sample size was calculated to be around 150. ¹⁷ Mature cattle of at least two years old, which should have a proper immune response to this parasite, were used. In order to minimize sampling bias, the animals were chosen at random, irrespective of their clinical history. Furthermore, to eliminate bias during testing, a blind test was used in the comparison studies. The a-ELISA was able to ensure repeatability and reproducibility of the test results with an excellent kappa value of 0.9592. A K value close to 1 (or 100%) indicated a good match and high reproducibility of the a-ELISA. A-ELISA results were reproducible compared to results of identical sera tested manually.

In order for the a-ELISA to be accredited under the requirements of the ISO 17025 standard, the test had to be validated and the quality assurance tests maintained. This was done by repeatability and reproducibility studies. Repeatability and precision studies, referring to minimum dispersion, and reproducibility and accuracy studies, referring to minimum shift from the expected value, are necessary. ²¹ The repeatability studies of the positive and negative controls provided excellent results. This characteristic was extremely important in screening for the specific antibodies to N. caninum in a large number of sera. Run to run or microplate to microplate difference cannot be large for accurate comparison of data between tests. As previously mentioned, this a-ELISA was validated following the ISO 17025 standard for test accreditation. ⁶ This will make the use of the a-ELISA for N. caninum accessible to other laboratories, both government and private, that require tests with ISO standards. In fact, this test should be acceptable both nationally and internationally. Countries such as the United States and the countries comprising the European Union all require tests with ISO standards.

In surveillance projects, the need to know the true prevalence of the disease is important. Laboratory data can provide the apparent prevalence value. The apparent prevalence is a reasonable estimate of the true prevalence of the infection only if both the sensitivity and specificity values of the test used are high. The a-ELISA, having high values for both the sensitivity and specificity (96% or above), was useful in calculating the true prevalence of N. caninum in the infected herd. Furthermore, the possibility of false negative or false positive results was small. The high sensitivity was reflected in a high negative predictive value, and the high specificity resulted in a high positive predictive value. The predictive values, PV+ and PV-, of 98% would be very useful for veterinarians in assessing the conditions of the herd if they were relying on the a-ELISA results. The a-ELISA is therefore well suited as a screening method. However, if the apparent prevalence of infection is low, such as 5-10%, with the specificity and sensitivity of 99.12% and 96.55% respectively, the predictive values will change. The predictive value of a positive test result will be lower (between 83-92%), and the predictive value of a negative test result will not have any significant change. ²¹

The combination of a-ELISA and computerized data processing permits HTS for N. caninum in cattle. In our laboratory, this a-ELISA has been successfully used in a sero-prevalence study of N. caninum in cattle in Alberta, Canada. ²² These data were useful and gave us confidence to apply this method in future surveillance studies.

Finally, we have done an analysis of the costs involved in adopting an automated system. The equipment costs and the expenses involved in purchasing commercial, and developing in-house, software have been considered. In our case, the initial start-up costs were in the range of $65,000 (USD). This included the purchase costs of the Biomek 2000 workstation, the stacker carousel, the plate reader, the gripper tool, and the software - as well as the technologist time required to become familiar with the software package and develop and troubleshoot a viable program. Because this work unit is performing tests almost continuously - testing 15,000 to 20,000 samples a year - the automated system will pay for itself in two years. The development time required to automate an ELISA test depends on the test itself. Basic technical time ranges from two to four weeks. We have been using our automated system for the last five years to perform a variety of ELISA tests in our laboratory. We estimate that a fully functional robotic workstation provides us with the equivalent of two extra fulltime equivalent technical positions.

Acknowledgments

This work was supported totally by the Agri-Food Laboratories Branch, Food Safety Division, Alberta Agriculture Food and Rural Development, Alberta, Canada.

Mention of trade names or commercial products in this article is solely for the purpose of providing specific information and does not imply recommendation or endorsement by the Alberta Department of Agriculture Food and Rural Development.