The end of the gold standard in syphilis diagnostics - A comparison of alternative screening test methods

Introduction

Infections with Treponema pallidum ssp. pallidum leading to syphilis cause a substantial disease burden, especially if infections remain undiagnosed and untreated. Beyond infections in adults, congenital syphilis represents an additional health threat. As syphilis incidence increased distinctly since more than a decade in many countries worldwide, and also congenital syphilis is on the rise in several countries, timely and accurate diagnosis as well as treatment is of utmost importance.^1,2 Also, in Germany the numbers of infections are rising, especially in men who have sex with men (MSM). ³

At present, serological tests are used for detection of syphilis as part of a stepwise diagnostic. Current European guidelines^4–6 recommend the following procedure: First, positive samples are identified using a polyvalent screening test covering IgG and IgM antibodies. Second, findings are confirmed using a treponema-specific IgG test. Treponema-specific IgM antibodies and non-treponema-specific lipoid antibodies are then assessed to determine disease activity. ⁴

Especially in the early, still IgM-dominated phase of a syphilis infection, the sensitivity of polyvalent immunoassays, which are mostly used as screening tests, is crucial. Due to its high sensitivity and specificity and ease of use, the TPPA had been employed in many laboratories throughout Europe as both a screening and confirmatory test. The production stop announced by the world’s only manufacturer of TPPA in 2022 created a major diagnostic gap for various issues, such as the sensitive initial sample screening or the determination of overall antibody kinetics to evaluate treatment success or detect re-infections.

Following treatment of syphilis according to recognized medical standards, a significant (4-fold) decrease in the titer of the activity parameters RPR and IgM is needed to classify infection as successfully treated.^2,4 The total antibody kinetics, which could be visualized by the TPPA, had been used by most laboratories and physicians, especially in Germany, as an additional tool to assess therapeutic success. In contrast, European and American guidelines rely mainly on the RPR titer for monitoring therapy and for detecting re-infections. The TPPA generally shows an average fourfold increase in titer in the event of re-infection. ⁷

A former study already showed that the sensitivity of the available TPHAs based on avian erythrocytes was insufficient to be used as an alternative to TPPA. ⁸ Polyvalent screening tests to detect treponema-specific antibodies are also approved for the initial testing of patient samples and could potentially serve as alternatives to TPPA.^9,10

The aim of our study was to investigate whether samples tested positive for syphilis by TPPA as gold standard can be reliably detected using alternative polyvalent screening tests. The second aim was to evaluate if a reduction of the manufacturer`s cutoff of the specific tests could enhance the sensitivity of these tests. In addition, the study assessed the potential of these screening tests to monitor antibody kinetics in a similar way as those of the TPPA.

Material and methods

Results

Cutoff reduction

Overall, the sensitivity of the screening tests according to the manufacturer’s specifications was significantly reduced compared to the TPPA (100 %) (Table 1). In particular, weakly reactive samples with low antibody concentrations could not be detected when using the manufacturer cutoffs, as shown for some examples in Table 2 in detail. To enhance sensitivity of established test systems, we explored reducing the manufacturer’s cutoffs.

OPEN IN VIEWER

Table 1.

Sensitivity [%] and Specificity [%] for cutoffs according to manufacturer specifications and reduced cutoffs of CMIA, ECLIA and CLIA tests compared to the TPPA.

Patient population total (n = 1768)	CMIA [Index]		EIA [RE/mL]	ECLIA [COI]		CLIA [Index]
	<1.0 manufacturer’s specification	<0.3 reduced cutoff	<22.0 manufacturer’s specification	<1.0 manufacturer’s specification	<0.5 reduced cutoff	<0.9 – 1.1 manufacturer’s specification	<0.4 reduced cutoff
Sensitivity	81.4	97.6	62.4	91.4	96.2	87.6	95.2
Specificity	99.7	84.3	98.9	99.0	98.5	99.2	97.8

OPEN IN VIEWER

Table 2.

Case reports of false negative results using the manufacture´s cutoffs from patients with different stages of infection.

	CMIA (Index)	TPPA (titer)	IgG-FTA-abs (titer)	19S-IgM-FTA-abs (titer)	RPR (titer)	EIA (RE/ml)	ECLIA (COI)	CLIA (Index)	Infection stage
Manufacturer´s cutoff	<1.0	<1:80	Negative	Negative	Negative	<16 - 22	<1,0	<1.1
Patient 1	0.75	1:80	1:10 (weakly positive)	1:40	n	11.08	0.350	1.4	Early latent
Patient 2	0.52	1:640	1:20 (positive)	1:80	n	9.73	0.583	2	Primary syphilis
Patient 3	0.42	1:80	1:5 (borderline)	1:160	n	7.85	1.760	0.7	Residual findings

As already shown in a prior study, lowering the CMIA threshold to 0.3 index improves sensitivity of the test without a significant loss of specificity. ¹²

Comparison of the diagnostic result-status with the EIA results, obtained in ascending order, revealed no specific numerical threshold distinguishing negative from positive samples. Therefore, it was not possible to reduce the cutoff without a significant loss of specificity for this test.

For the ECLIA, a numerical limit of ≥ 0.5 (COI) was identified as the lowest value at which at least one positive sample could be detected. Using the CLIA, positive samples were detected from a result of ≥ 0.4 (Index).

Table 1 shows the calculated sensitivity of the CMIA, EIA, ECLIA and CLIA tests across the different populations using both, the cutoffs according to the manufacturer’s specifications and the reduced cutoffs, if possible. Applying the reduced cutoffs resulted in a clear increase in sensitivity.

Sensitivity of the CMIA improved from 81.4 to 97.6 by reducing the cutoff value. The adjusted cutoff was also applied for data of 61,677 blood donors (Suppl.1): 74 samples underwent confirmatory testing after showing a reactive screening test index > 1 according to the manufacturer, which corresponds to a proportion of 0.12 %. Of these, 7 samples were identified as correctly positive. By reducing the cutoff to an index of ≤ 0.3, the proportion of samples requiring retesting increased to 0.77 %.

Table 2 shows a selection of samples that were not detected using the thresholds according to the manufacturer’s instructions. The table includes data from three patients subjected to confirmatory diagnostics based on the in-house threshold for the CMIA of 0.3-0.99. All three samples have a positive TPPA titer and contain treponema-specific antibodies with negative lipoid antibody findings. Based on the findings and available anamnestic information (time of infection and lack of symptoms), patient 1 is assignable to the early latent stage. Patient 2 exhibited a genital syphilitic lesion at the time of blood sampling, indicating an active and infectious stage. For patient 3 the anamnestic information indicated the results to be a residual finding following a previously treated treponema infection.

The CMIA would not have identified any of these samples as positive when applying the manufacturer’s cutoff. The EIA showed also negative results in all cases. The ECLIA was only able to detect the sample of patient 3 as positive, whereas the CLIA failed to detect the residual findings in patient 3.

Table 3 shows the distribution of samples that identified as positive with the newly defined threshold ranges. Early infection phases were defined based on any available clinical information, such as the presence of syphilitic lesions. Additionally, detection of T. pallidum using NAAT (Nucleic acid amplification technology) from smears and the pattern of serological test results were utilized to define the infection stage, whereby the RPR result was negative in all cases of early infections. The activity of the diseases was determined by the detection of treponema-specific IgM antibodies, as it is usual in the early phases of infection.^16,17

OPEN IN VIEWER

Table 3.

Overview of confirmed early phases, residual findings and unspecific reactions [%] in patient populations selected over the total data set (n = 1768) according to the reduced cutoffs in A: CMIA (n = 261), B: ECLIA (n = 18) and C: CLIA (n = 39) divided into five different cohorts.

Patient population	A: CMIA (n = 261)
	0.3-0.99 [Index]
	Confirmed early phases of infection [%]	Confirmed residual findings [%]	Unspecific reaction [%]
Samples from HIV counseling centers (10%)	24	56	20
Samples from the public health service (5%)	31	46	23
Maternity care (2%)	-	33	67
Blood donation service (82%)	-	-	100
Others (1%)	33	33	33
	B: ECLIA (n = 18)
	0.5 - 0.99 [COI]
Samples from HIV counseling centers (50%)	56	33	11
Samples from the public health service (0%)	-	-	-
Maternity care (0%)	-	-	-
Blood donation service (39%)	-	-	100
Others (11%)	100	-	-
	C: CLIA (n = 39)
	0.4 - 0.9 [Index]
Samples from HIV counseling centers (28%)	9	82	9
Samples from the public health service (15%)	-	67	33
Maternity care (8%)	-	33	67
Blood donation service (46%)	-	-	100
Others (3%)	-	100	-

Confirmed active early infection cases were primarily found in samples from HIV counseling centers, whereas such samples did not appear in tests from blood donation services or maternity care.

Comparison with TPPA

Comparisons between CMIA, EIA, ECLIA and CLIA results and TPPA titer levels mostly showed a continuous, but partly not proportional increase of the measured values with increasing TPPA titer (Table 4).

OPEN IN VIEWER

Table 4.

Medians of the CMIA, EIA, ECLIA and CLIA test results according to the TPPA titer levels and correlation of the test results to the TPPA titer.

TPPA titre ≤ 1:80	CMIA [Index] <1.0	EIA [RE/mL] <16-22	ECLIA [COI] <1.0	CLIA [Index] <0.9-1.1
1:80 (n = 25)	1.00	11.08	2.90	1.00
1:160 (n = 22)	0.99	7.72	2.99	1.20
1:320 (n = 19)	2.53	14.31	7.59	3.20
1:640 (n = 23)	5.60	17.01	19.02	6.19
1:1280 (n = 22)	9.92	51.14	47.10	14.30
1:2560 (n = 20)	12.77	94.31	71.90	38.50
1:5120 (n = 19)	15.82	164.92	136.00	60.80
1:10240 (n = 15)	15.47	163.83	158.00	71.80
1:20480 (n = 8)	19.32	200.00	158.50	49.85
>1:20480 (n = 37)	17.85	1258.00	231.00	265.00
Correlation to TPPA titer (p < 0.001)	0.82	0.85	0.84	0.83

The CMIA had a much smaller range of measured values compared to the EIA, ECLIA and CLIA, therefore a corresponding increase in value was limited, particularly at higher TPPA titer levels.

The EIA showed a continuous increase in values up to a TPPA titer level of 1:20,480. For results >1:20,480 a significant increase of around 6 times of the test result was detected.

In contrast, the ECLIA showed relative low results at TPPA levels of 1:20,480, whereas results at lower titer levels were comparable. A similar pattern was observed with the CLIA, which also yielded lower test results at a TPPA level of 1:20,480.

Antibody kinetics in follow-up controls

In treated patients, antibody titers have previously been tracked using TPPA, with re-infection indicated by an increase in TPPA titer by at least two titer levels. Based on the measured values of the CMIA, EIA, ECLIA and CLIA test systems shown as an example, it can be assumed that such kinetics can also be depicted with aid of the index values, COI or RE/ml (Table 5).

OPEN IN VIEWER

Table 5.

Results of TPPA, 19S-IgM-FTA-abs, RPR, CMIA, EIA, ECLIA and CLIA in follow-up controls after treatment (A) and in case of reinfection (B). n/a = not available due to lack of material, or because the measurement was not performed for this sample.

	Date	TPPA (titer)	IgG-FTA-abs (titer)	19S-IgM-FTA-abs (titer)	RPR (titer)	CMIA (Index)	EIA (RE/ml)	ECLIA (COI)	CLIA (Index)
A
1	05/07/2018	1:20,480	n/a	1:80	1:16	19.69	1123	246	187
	11/13/2018	1:20,480	n/a	Negativ	1:4	17.44	452	133	94.6
	12/17/2018	1:20,480	n/a	Negativ	1:4	14.73	194	113	75.6
2	08/21/2018	1:10,240	>1:20	1:80	1:32	14.89	898	115	111
	12/27/2019	1:5120	n/a	1:40	1:1	13.82	101	93	27.7
B
3	01/14/2022	1:5120	n/a	1:10	n	15.98	135	27	83.46
	04/23/2022	1:5120	n/a	1:10	n	15.41	99	35	89.28
	09/14/2022	1:640,000	n/a	1:2560	1:256	22	169	504	3878.6
4	02/04/2022	1:20,480	n/a	n	n	20.03	169	121	384.55
	05/06/2022	1:20,480	n/a	n	n	19.43	140	140	330.95
	07/13/2022	1:160,000	n/a	n	n	19.43	165	149	336.25
	10/14/2022	1:320,000	n/a	1:2560	1:256	24.01	134	249	6847.2

No increase could be recorded in ECLIA for patient 4 at the time of re-infection. This is possibly due to a high-dose hook effect, which can yield falsely low results in the presence of very high antibody concentrations; however, dilution testing was not possible due to limited specimen material.

Discussion

Some of the evaluated screening tests already have a relatively high sensitivity if they are used according to the manufacturer’s instructions. However, for samples with a very low antibody response, such as those with a TPPA titer of 1:80 in the early stages of infection or in rare cases with residual findings, the sensitivity was significantly lower. Our comparisons with TPPA-based results as gold standard revealed sensitivities of 81.4 % for CMIA, for ECLIA 91.4 % and 87.6 % for CLIA using the manufacturer instructions. The EIA demonstrated the lowest sensitivity among these assays at 62.4 % compared to the TPPA.

The reduced sensitivities compared to the TPPA are likely attributable to differences in antigen composition among the test systems. While the TPPA works with a whole-cell lysate of the entire bacterium and thus offers the entire antigen spectrum of the pathogen, ¹⁸ the polyvalent screening tests contain recombinant individual antigens in different combinations.^11,13–15

The data shown in Table 2 demonstrate that samples with low antibody titers may remain undetected if the limit values specified by the manufacturers are applied. This poses a risk of missing early and highly contagious treponema infections. Only the TPPA, which is no longer available, was able to reliably identify these samples as positive in all cases. ¹⁸

A closer examination of these samples using the newly defined reduced cutoffs, indicates that these are particularly clinically relevant for HIV counseling centers and public health services (Tab.3). This patient population is often characterized by a higher number of sexual partners, lower condom use and a higher baseline prevalence of STI such as syphilis. Hence, a higher proportion of infections in the early phase of infection is expected for this population, which would have been classified as negative and thus overlooked without the reduction of the threshold. This means that a reduction of the threshold is particularly recommended for samples from patients of high-risk groups. In contrast in low-prevalence groups, such as pregnant women or blood donors, no early, active infections are likely to be found.

According to the manufacturer specifications, the tests which were included in our study did not show a comparable performance to the TPPA. Preliminary investigations had already suggested that a considerable increase in sensitivity can be achieved by lowering the cutoff specified by the manufacturer used to trigger follow-up diagnostics. ¹² Our study results confirm the hypothesis with a larger number of samples and additional test systems. Lowering the cutoffs increased sensitivity for CMIA, ECLIA and CLIA tests. Only for the EIA it was not possible to determine a cutoff that allowed an increase in sensitivity, without a major loss of specificity. The reduction in specificity is not a significant problem from a diagnostic point of view, because any false positive samples are excluded by follow-up diagnostics. Nevertheless, it must be weighed up whether the additional expense of retesting weakly reactive samples is worthwhile. This should always be done in consideration of the patient clientele to be tested.

Our study data indicate a cutoff of ≤ 0.3 (Index) is useful for the CMIA. For the ECLIA, our data suggest a cutoff value of ≤ 0.5 (COI) and for the CLIA of ≤ 0.4 (Index). With these cutoff adjustments, a comparable sensitivity in syphilis diagnostics can be guaranteed using other test systems than TPPA. An identical cutoff for all test systems would lead to a considerable loss of specificity, or even an overlook of reactive samples.

With exception of the EIA, all polyvalent screening tests are only approved for qualitative testing. In addition to RPR and IgM kinetics the TPPA titer based on a semi-quantitative test has traditionally been employed to monitor overall antibody kinetics during the course of treponema infections after therapy, but also in cases of suspected reinfection. In order to identify an alternative for this issue as well, TPPA titer levels were correlated with measured values of the screening tests. This application is not intended for three of the four manufacturers and is therefore outside the stated purpose. Nevertheless, the measured values could be taken into account when interpreting the findings. Our study data show a certain correlation between the TPPA titer levels and the measured values (Tab. 2). Only few, exemplary patient histories could be analyzed as part of our study. EIA and CLIA results are most comparable to TPPA values for assessing total antibody kinetics. It could be helpful for the application of the tests to compare samples of the same patient in one run or on the same day in order to avoid variations caused by the calibrators.

Although it is not provided for in international guidelines, at least in Germany the overall kinetics of treponema-specific antibodies after therapy and especially in cases of suspected re-infection are considered in many laboratories. Of course, the kinetics of the activity parameters (lipoid and IgM antibodies) are always used in parallel as an assessment criterion for the treatment success.^4,5,19

Differences in the detection of immunoglobulin types M and G can cause variations in test performance. Therefore, very recent infections could be not detected by the evaluated tests, compared to the more valid results of the TPPA.

One possible explanation may be that polyvalent screening tests have reduced sensitivity for detecting IgM antibodies compared to the TPPA and therefore only achieve a positive result at a later stage of the infection, when the IgG antibody response has developed.

The detection of treponema-specific IgM antibodies is particularly indicative of an early infection phase. The clinical data (Tab. 2) show that low antibody titers and IgM antibodies can be found in all stages of a treponema infection, i.e. both with an active infection (patient 2) and with residual findings (patient 3).

The results of the alternative test systems can contribute to the overall assessment of the laboratory findings. In the long term, it would be valuable for manufacturers to consider semi-quantitative application across all test systems. All test systems examined are suitable for the assessment of antibody kinetics.

Overall, three of four polyvalent screening tests demonstrated a good performance for screening analysis for diagnosis of syphilis by the application of a reduced cutoff. These tests can be considered as substitutes for TPPA, considering the limitations previously mentioned.