Comparison of four high-throughput,automated immunoassays for the detection of SARS-CoV-2 antibodies

Introduction

The severe acute respiratory syndrome – coronavirus 2 (SARS-CoV-2) causing COVID-19 (Coronavirus infectious disease 2019) was first reported in China at the end of 2019. Following extensive human-to-human transmission of the virus, it became a major worldwide concern with the World Health Organization (WHO) declaring a pandemic on 11 March 2020. ¹ The diagnosis of current SARS-CoV-2 infection is carried out using real-time polymerase chain reaction (RT-PCR) by detecting viral RNA in respiratory samples.

Coronaviruses are enveloped RNA viruses, and the 3′ part of the genome encodes four major structural proteins: spike (S), membrane (M), envelope (E) and nucleocapsid (N). ¹ Antibody responses to SARS-CoV-2 have been shown to occur 8 to 10 days following infection with IgG and IgM rising either simultaneously or sequentially.^2,3 It is still unclear whether the presence of anti-SARS-CoV-2 antibodies in the serum confers that individual any immunity. ⁴ However, these assays are useful to determine previous exposure to SARS-CoV-2 infection and are being used as tools for determining seroprevalence in different populations.

The major assay manufacturers have rapidly developed serological assays to detect antibodies to SARS-CoV-2 (targeted to the nucleocapsid or spike protein) using high-throughput immunoassay analysers that are already available in routine clinical laboratories.^5,6 As the Roche Elecsys (total) and Abbott (IgG) assays were early to market, a number of studies have been published with these assays, generally using serum samples from patients who have been hospitalized with severe COVID-19.^7,8 However, there are limited published data and assay comparison data using the more recently released high-throughput assays (such as the Beckman Coulter IgG assay and the Siemens total antibody assay). It is therefore important to see how these assays compare against one another particularly, as they are being used routinely to determine previous exposure to SARS-CoV-2. In the UK, large-scale sero-surveillance studies using samples collected from blood donors are being performed by Public Health England using the EUROIMMUN anti-SARS-CoV-2 assay. Recent data from the North West England region (incorporating our Lancashire and South Cumbria network) show that the adjusted seroprevalence has decreased to 6.8% in August 2020 (from above 10% in May 2020). ⁹ There has been some recent debate about the suitability of the serological assays to detect mild infection with a suggestion that assays that target the spike protein may be most suitable for this purpose. ¹⁰

As in the rest of the UK, anti-SARS-CoV-2 analysis was introduced in our region in Summer 2020. This test has been available for both healthcare workers and also for patients attending general practice to determine previous exposure to SARS-CoV-2. Within Lancashire and South Cumbria, we have four different high-throughput immunoassay analysers within our Biochemistry and Virology departments (Abbott Architect, Beckman Coulter DxI 800, Roche Cobas e801 and the Siemens Advia Centaur XPT). Anti-SARS-CoV-2 assays are available on all the analysers and we have three of these assays in routine clinical use. There has been no published comparison data of the anti-SARS-CoV-2 assays on these four high-throughput immunoassay analysers.

We wanted to directly compare the performance of the assays to understand the impact of using different assays within our network in individuals with confirmed previous SARS-CoV-2 and within the general practice population. The first aim was to directly compare the assays in hospitalized patients and healthcare workers who had tested positive for SARS-CoV-2 by RT-PCR in serum samples collected at 14 days and at 28 days later. The second aim was to determine how the assays compared by randomly selecting a subset of serum samples received from general practice and determining the seroprevalence.

Materials and methods

Anti-SARS-CoV-2 analysis

All serum aliquots were sent to the Royal Preston Hospital and distributed from there to the other laboratories for further analysis. Anti-SARS-CoV-2 analysis was carried out using the Abbott Architect (Abbott Diagnostics, Maidenhead, UK) at the Blackpool Victoria Hospital, the Beckman Coulter DxI 800 (Beckman Coulter, High Wycombe, UK) at the Royal Lancaster Infirmary, the Roche Cobas e801 (Roche Diagnostics Ltd., Burgess Hill, UK) at the Royal Preston Hospital and the Siemens Advia Centaur XPT (Siemens Healthcare Diagnostics, Frimley, UK) at the Royal Blackburn Hospital. All assays were performed according to manufacturer’s instructions; the assay characteristics and cut-off index recommended for the interpretation of a positive result can be seen in Table 1. The Beckman Coulter assay also has an equivocal range which is shown in Table 1. The reportable range for the Siemens assay is 0.05–10.0; for data analysis, any results <0.05 were converted to 0.05 and any results >10.0 were converted to 10.0.

OPEN IN VIEWER

Table 1.

The antibody detected, the target antigen and the cut-off index for the four anti-SARS-CoV-2 immunoassays.

Manufacturer	Antibody detected	Target antigen	Cut-off index
Abbott	IgG	Nucleocapsid protein	≥1.4
Beckman Coulter	IgG	Spike protein S1 RBD	≥1.0 (Equivocal 0.81–0.99)
Roche	Total antibody	Nucleocapsid protein	≥1.0
Siemens	Total antibody	Spike protein S1 RBD	≥1.0

RBD: receptor-binding domain.

Results

Comparison of anti-SARS-CoV-2 assays in hospitalized patients with confirmed SARS-CoV-2 infection

Serum samples were obtained from 31 hospitalized patients who had tested positive for SARS-CoV-2 RNA by RT-PCR 14 days earlier (median = 14 days, range 12–18 days). The age of the patients ranged from 35 to 95 years with a median age of 81 years; 19 patients were male. Figure 1 shows the results using all four assays; 26/31 (83.9%) samples showed concordance across all four assays with 24 positive results and two negative results. The discordant results are also shown in Table 2. For two samples, the result was positive using one assay only (Abbott and Siemens). The other three discordant samples were positive using three of the assays and negative using the Roche assay or the Siemens assay or the Abbott assay in one sample. The sensitivities of the assays for samples collected at 14 days after the positive RT-PCR result were 87.1%, 87.1%, 83.9% and 83.9% for the Abbott, Beckman Coulter, Roche and Siemens anti-SARS CoV-2 assays, respectively.

OPEN IN VIEWER

Figure 1.

Anti-SARS-CoV-2 results using the Abbott Architect IgG, the Beckman Coulter IgG, the Roche total and the Siemens Centaur total assays (left to right) in hospitalized patients at 14 days after a confirmed positive SARS-CoV-2 RNA result. Filled diamonds show results where there was concordance between all four assays. The other symbols show the results from serum samples that were not concordant. The positive cut-off for the Beckman Coulter, Roche and Siemens assays is 1.0 (x-axis). The positive cut-off for the Abbott assay is 1.4 (shown by the dotted lines). The Beckman Coulter assay has an equivocal range between 0.81 and 0.99 (shown by a dashed line at 0.81).

OPEN IN VIEWER

Table 2.

The results using all assays are shown for all samples where the results were discordant.

Individual	No. of days after positive swab for SARS-CoV-2	Gender	Age	Abbott Architect IgG	Beckman Coulter DxI 800 IgG	Roche Cobas e801 Total Antibody	Siemens Advia Centaur XPT total antibody
Patient-hospital	15	F	85	4.55	48.26	0.64	2.69
Patient-hospital	13	M	83	2.75	0.27	0.98	0.12
Patient-hospital	13	F	76	0.21	0.33	0.72	3.78
Patient-hospital	14	M	89	5.04	7.06	8.31	0.68
Patient-hospital	14	M	90	1.39	4.52	3.05	>10
Healthcare worker B	14	M	31	1.44	5.86	0.47	2.65
Healthcare worker D	15	F	58	1.12	6.32	6.04	>10
Patient-hospital	30	F	78	2.47	5.03	4.96	0.18
Patient-hospital	26	F	83	5.20	18.81	12.10	0.92
Patient-hospital	29	M	62	4.70	26.67	0.61	>10
Healthcare worker D	28	F	58	1.04	4.45	7.30	>10
Healthcare worker F	30	F	32	1.34	0.89	1.43	0.31
Patient-general practice	Not known	F	23	0.05	1.68	0.09	0.05
Patient-general practice	Not known	M	86	0.05	1.38	0.11	0.05
Patient-general practice	Not known	F	58	0.43	0.23	9.38	>10
Patient-general practice	Not known	F	86	0.02	0.03	0.10	1.56
Patient-general practice	Not known	F	67	3.41	0.02	0.09	0.05
Patient-general practice	Not known	F	27	0.90	1.06	0.59	6.26
Patient-general practice	Not known	F	62	0.01	1.76	0.07	0.05
Patient-general practice	Not known	M	65	6.89	0.58	28.40	3.19
Patient-general practice	Not known	F	29	0.52	2.10	6.21	0.75
Patient-general practice	Not known	F	71	1.89	0.05	0.09	0.05
Patient-general practice	Not known	M	80	5.55	0.37	0.09	0.07

Note: Positive results are shown in the grey boxes and equivocal results (Beckman Coulter assay only) are highlighted in bold italic.

Serum samples were obtained from 26 hospitalized patients who had tested positive for SARS-CoV-2 RNA by RT-PCR 28 days earlier (median = 28 days, range 26–35 days). The age of the patients ranged from 42 to 88 years with a median age of 76 years; 18 patients were male. Figure 2 shows the results using all four assays; 23/26 (88.4%) samples showed concordance across all four assays with 23 positive results. The discordant results are also shown in Table 2; all three were positive using all assays except Roche for one sample and Siemens for two samples. The sensitivities of the assays for samples collected at 28 days after the positive RT-PCR result were 100.0%, 100.0%, 96.2% and 92.3% for the Abbott, Beckman Coulter, Roche and Siemens anti-SARS CoV-2 assays, respectively.

OPEN IN VIEWER

Figure 2.

Anti-SARS-CoV-2 results using the Abbott Architect IgG, the Beckman Coulter IgG, the Roche total and the Siemens Centaur total assays (left to right) in hospitalized patients at 28 days after a confirmed positive SARS-CoV-2 RNA result. Filled diamonds show results where there was concordance between all four assays. The other symbols show the results from serum samples that were not concordant. The positive cut-off for the Beckman Coulter, Roche and Siemens assays is 1.0 (x-axis). The positive cut-off for the Abbott assay is 1.4 (shown by the dotted lines). The Beckman Coulter assay has an equivocal range between 0.81 and 0.99 (shown by a dashed line at 0.81).

Comparison of anti-SARS-CoV-2 assays in healthcare workers with confirmed SARS-CoV-2 infection

Nine healthcare workers provided serum samples at 14–20 days (median = 14 days) and at 28–33 days (median = 29 days) after testing positive for SARS-CoV-2 RNA by RT-PCR. The age of the healthcare workers ranged from 31 to 58 years with a median age of 49 years; eight were female. None were hospitalized with COVID-19. Figure 3(a) and (b) shows the results using all four assays at 14 days and 29 days, respectively. Five healthcare workers showed positive or equivocal results using all four assays on both serum samples at 14 days and 28 days. One healthcare worker was negative using all four assays on both samples (Healthcare worker C in Figure 3). The concordance between the four assays for either a negative result or a positive/equivocal result was 66.7% and 77.8% for serum samples collected a median of 14 days and 29 days after a positive SARS-CoV-2 RNA result, respectively. The discordant results are shown in Table 2.

OPEN IN VIEWER

Figure 3.

SARS-CoV-2 antibody results using the Abbott Architect IgG (triangle), the Beckman Coulter IgG (cross), the Roche total (circle) and the Siemens Centaur total (square) assays in nine healthcare workers (A-I) at (a) 14 days after a confirmed positive SARS-CoV-2 RNA result and (b) 29 days after a confirmed positive SARS-CoV-2 RNA result. The positive cut-off for the Beckman Coulter, Roche and Siemens assays is 1.0 (x-axis). The positive cut-off for the Abbott assay is 1.4 (shown by the dotted lines). The Beckman Coulter assay has an equivocal range between 0.81 and 0.99 (shown by a dashed line at 0.81).

Comparison of anti-SARS-CoV-2 assays in general practice samples and seroprevalence

There were a total of 225 serum samples collected from patients who had attended general practice. All were anonymized and previous exposure to SARS-CoV-2 was not known. The age of the patients ranged from 13 to 94 years with a median age of 62 years; 93 were male (41%). Using all four assays, the results were either consistently negative or consistently positive for 206 samples and eight samples, respectively (95.1% concordance). The results are shown in Figure 4(a)). For 11 samples, there were differing results between the four assays (see Figure 4(b)) and Table 2. Overall, the seroprevalence using the four assays in the general practice samples was 5.3%, 5.8%, 4.9% and 5.3% using the Abbott, Beckman Coulter, Roche and Siemens assays, respectively.

OPEN IN VIEWER

Figure 4.

Anti-SARS-CoV-2 results using the Abbott Architect IgG, the Beckman Coulter IgG, the Roche total and the Siemens Centaur total assays (left to right) in general practice samples (n = 225). (a) The results that were concordant between the four assays (n = 215) and (b) the discordant results between the assays (n = 11). The positive cut-off for the Beckman Coulter, Roche and Siemens assays is 1.0 (x-axis). The positive cut-off for the Abbott assay is 1.4 (shown by the dotted lines). The Beckman Coulter assay has an equivocal range between 0.81 and 0.99 (shown by a dashed line at 0.81).

Discussion

This is the first study to compare all four high-throughput anti-SARS-CoV-2 immunoassays (including the Beckman Coulter assay) in both hospitalized patients and healthcare workers with confirmed SARS-CoV-2. In addition, we have determined how these assays differ when determining seroprevalence within the general practice population.

The four high-throughput anti-SARS-CoV-2 assays generally compared very well and showed good concordance, in particular for the patients hospitalized with COVID-19. However, the comparison indicated that it may be possible to obtain a negative result for any of the anti-SARS-CoV-2 assays in patients who have had a previous positive SARS-CoV-2 RNA result on RT-PCR. At 28 days, there was a higher proportion of positive results using all four assays in the hospitalized patients compared with the healthcare workers (88% vs. 67%), suggesting that these assays may not be optimized to detect antibodies in those with milder SARS-CoV-2 infection. Similar findings have been reported from other studies.¹¹ In this study, we have compared the assays at a time period after diagnostic confirmation of SARS-CoV infection by RT-PCR rather than post onset of COVID-19 symptoms. The preincubation period after infection is now established at approximately five days.¹² In our hospitalized patient group, there may have been a number of days between symptom onset and the confirmation of SARS-CoV-2 by RT-PCR on hospital admission. In our healthcare worker group, symptom onset and test date are likely to be similar. However, it is a limitation of our study that we are only able to characterize the antibody response post diagnostic confirmation by RT-PCR rather than post symptom onset.

In one of the healthcare workers, there was no antibody response using any of the assays at either 14 days or 28 days. It is possible that their positive SARS-CoV-2 RNA result was a false-positive result. There is no gold standard assay for determining previous viral exposure. It is possible for both false-positive results to occur (e.g. due to contamination) or false-negative results to occur (e.g. due to inadequate swabbing or length of time from infection).

In terms of the sensitivity of the assays to detect antibodies following SARS-CoV-2 infection, there were slightly increased sensitivities using both the IgG assays (Abbott and Beckman Coulter) at both 14 days and 28 days in the hospitalized patients. Interestingly, these assays have different antigen targets with the Abbott assay targeting the nucleocapsid and the Beckman Coulter assay targeting the receptor-binding domain of the spike protein. This may be due to the IgG assays being more suitable for a predominantly IgG response which may be expected at 28 days following SARS-CoV-2 exposure, compared with the total assays which measure both IgG and IgM. ⁴

A limitation of our study is that our numbers for calculating sensitivity are low; however, they clearly show differences between the different assays. In particular, our healthcare worker group contained only nine individuals. We have included the results from this small group, as they suggest a lower proportion of positive antibody results in non-hospitalized individuals with confirmed SARS-CoV-2 infection. The lower concordance between the four assays in this group compared with the hospitalized patients suggests that the choice of assay and the testing strategy may be more important for the general population. However, further investigation of a wider group of individuals with confirmed SARS-CoV-2 infection who were not admitted to hospital is required to confirm this.

There is no clear pattern in the discordant results based on whether the assay is an IgG/total antibody assay or the antigen target. This indicates that there may be individual-specific differences in the type of antibody response and the timing of the antibody response to SARS-CoV-2 infection. This was particularly evident in our healthcare worker group where the serum samples were collected from the same individuals at different time points. Similar findings have been concluded using different assays. ⁷ Given that these are immunoassays, we also know that there are other factors that may affect the results of specific assays such as medications, the presence of other analytes in the serum and heterophilic antibodies. Although rare, these factors should always be considered when there is an unexpected result using an immunoassay. The recommended practice is to confirm the results using an alternative immunoassay method.¹³

For all groups, the results show that the Roche anti-SARS-CoV-2 total antibody assay demonstrates the clearest distinction between a ‘positive’ result and a ‘negative’ result; for example, in Figure 4, the majority of results using the Roche assay are clustered around 0.1 or >10 (compared with a cut-off of 1.0). Only the Beckman Coulter assay has an equivocal range; interestingly, some of the discordant results were due to an assay having a result very close to but not above the cut-off for that assay. For example, one hospitalized patient had a result of 0.92 (<1.0) using the Siemens assay when all other assays showed results above the respective cut-off. Similarly, a hospitalized patient had a result of 1.34 (<1.4) using the Abbott assay when all other assays were positive. In both cases, the result would be reported as ‘negative’ using one of the assays; however, if a repeat sample had been collected a few days later, it is possible that the result would have been above the cut-off. For a population study or for a person wanting to know if they have had previous exposure to SARS-CoV-2, a result close to the cut-off may be more correctly interpreted as ‘equivocal’ with a repeat sample to be collected at a later time. Alternatively, a second anti-SARS-CoV-2 assay could be used on these borderline results. Further work is needed to determine the most appropriate testing strategy and a suitable equivocal range for the Abbott, Roche and Siemens assays.

In our general practice population, the seroprevalence using the four different assays ranged from 4.9 to 5.8%. This is lower than the 6.8% reported in the sero-surveillance studies for our region by Public Health England using blood donor samples (and a different assay) at a similar time. ⁷ This may be due to the small sample size or the preselection of our population of patients attending general practice. It is possible that some of these individuals may have been shielding at the height of the pandemic due to other underlying health conditions or it is possible that some of these individuals only had mild infection and the assays were not able to measure the antibody response. ¹⁰ The only way to get a true snapshot of seroprevalence would be to measure antibodies in a completely unselected population.

Despite good overall conformance between the four high-throughput assays studied here, our findings do indicate that the choice of assay will impact on the results of sero-surveillance studies and re-evaluation by the manufacturers may be required for those results close to the cut-off. Additionally, in patients who present with a clear documented history of SARS-CoV-2 infection but get a negative result, analysis using a different assay may be required. Furthermore, with the roll out of SARS-CoV-2 vaccinations, the use of the assays will be changing, so a clear understanding of their application and consideration of using a second assay for confirmation of equivocal results are required.