Abstract
Summary:
The burden of disease and associated healthcare costs of syphilis are significant despite widespread screening and treatment. Our objective was to conduct an economic evaluation using a simulation model when comparing enzyme immunoassay (EIA) initial testing and Inno-Lia (IL) confirmatory testing (EIA + IL) with rapid plasma reagin (RPR) initial testing and Treponema pallidum particle agglutination assay (TPPA) and fluorescent treponemal antibody absorption assay (FTA-ABS) confirmatory testing (RPR + TPPA/FTA). Estimates of prevalence, test costs and utilization of services for 2006 were derived from Alberta databases. Estimates of test characteristics were derived from the available literature. The incremental cost-effectiveness ratio was Canadian $461 per additional correct diagnosis (less costly and more effective). EIA + IL is cost-effective when compared with RPR + TPPA/FTA for screening and diagnosis of syphilis.
Introduction
Syphilis is a sexually transmitted infection (STI) caused by the bacterium Treponema pallidum. As the organism can be detected in the exudates from early lesions by microscopy, the vast majority of syphilis tests are serological and diagnosis of infection is therefore dependent on laboratory testing.
Initial syphilis testing is often conducted with non-treponemal assays such as the rapid plasma reagin (RPR) followed by treponemal assays such as T. pallidum particle agglutination (TPPA) and fluorescent treponemal antibody absorption (FTA-ABS) confirmatory tests (RPR + TPPA/FTA). However, this testing protocol has three major limitations leading to incorrect diagnosis and unnecessary costs. First, RPR is a non-treponemal assay that may not identify individuals early in primary syphilis or late in its progression (late latent and tertiary stages). Secondly, RPR results are confounded by various conditions including tuberculosis, mononucleosis, pregnancy and autoimmune disease, which lead to false-positive results. Thirdly, confirmatory testing with TPPA and FTA-ABS is subjective requiring evaluation by experienced technologists, particularly when antibody levels are low. Consequently, confirmatory testing with TPPA and FTA-ABS may lead to both false-positive and false-negative results if the particle agglutination or fluorescence intensity is misinterpreted. 1
In contrast to RPR, enzyme immunoassay (EIA) is a treponemal assay able to detect untreated and treated syphilis at all stages and has been shown to have greater diagnostic accuracy,2,3 although at a higher cost per test. Additional advantages of EIA include high throughput automation and the electronic generation and dissemination of results. 4 In contrast to TPPA and FTA-ABS, Inno-Lia (IL) is a multiparameter line immunoassay that uses recombinant polypeptide antigens derived from T. pallidum. 1 IL employs simple criteria which reduce subjectivity when interpreting results and is reported to be the most accurate single assay for confirmatory testing of syphilis.1,5
It has been proposed that EIA can be used as the initial test2,3 followed by confirmatory testing with IL (EIA + IL).1,5 However, there have been no studies that have evaluated the economic implications (costs and benefits) of this testing process. Information regarding the cost effectiveness of EIA + IL is necessary for clinicians and policy-makers to rationally guide syphilis testing and management recommendations. The aim of this study is to conduct an economic evaluation of syphilis testing using EIA + IL compared with the current RPR + TPPA/FTA process.
Materials and Methods
Health context
The study was conducted in Alberta, Canada. The Alberta Health Ministry provides a comprehensive syphilis programme that coordinates diagnosis, treatment, partner notification, prevention, surveillance, research and education. Patient-care services were provided by regional health authorities within Alberta, while the health ministry coordinates case management and partner notification.
Current testing protocols: RPR + TPPA/FTA
In the prenatal population, all pregnant women accessing antenatal care were screened for syphilis. Pregnant women were identified after a blood test during their first visit to the clinician (e.g. family physician, obstetrician or midwife). In the non-prenatal population, testing for syphilis was conducted for individuals who either self-refer themselves to a clinician because of the presence of symptoms or for routine sexually transmitted disease (STD) testing or because they have been identified as a sexual partner from a diagnosed case. For the prenatal population, blood is drawn at a local lab and forwarded to the Alberta Provincial Laboratory for Public Health (ProvLab) for analysis by RPR. Specimens with non-reactive results (using RPR) were censored from further testing, while reactive results were titrated and receive confirmatory testing with TPPA. Non-reactive TPPA results were censored from further testing, while reactive results receive final confirmatory testing with FTA-ABS. As antibody against T. pallidum persists for decades and reactive results may not indicate an acute infection, all results were forwarded to the physician for a final clinical diagnosis. For non-prenatal populations, RPR testing was performed by several other laboratories but all reactive specimens were forwarded to the ProvLab for confirmatory testing and RPR titration. The ProvLab performs approximately 75% of all RPR testing for Alberta. As the RPR screening test is known to miss early seroconverting cases, all serum samples from STD clinics were also tested by TPPA, even when the RPR test was negative.
When a positive serum was identified, provincial records were reviewed to determine whether the specimen was a follow-up sample on a known (and treated) case, or whether it is from a newly identified case. For newly identified cases, the Infectious Diseases Medical Consultant (IDMC) in the Alberta Health Ministry was contacted and treatment was coordinated. All cases were interviewed for a list of sexual partners who were contacted and scheduled for syphilis testing. Follow-up testing with RPR titration was performed on all new cases depending on the stage of syphilis. All follow-up results were reviewed by the IDMC.
For discrepant cases (i.e. individuals with discrepant results on TPPA and FTA-ABS), follow-up testing with RPR, TPPA and FTA-ABS (simultaneous testing) was conducted within two to four weeks. All results were reviewed by the IDMC who provides a final diagnosis.
It is important to recognize that the objective of laboratory testing was to identify seropositive individuals (i.e. presence of antibodies against T. pallidum) in the testing population to assist in clinical evaluation (i.e. positive confirmatory test results do not necessarily identify a new case because antibodies can be present in both new and previously treated cases).
Alternative testing protocol - EIA + IL
In the alternative protocol for syphilis testing, EIA is performed first. For positive results, an RPR titration was performed to guide response to therapy. All new EIA-positive cases were confirmed by IL. Given the high reported sensitivity of EIA, negative serum samples were not retested by confirmatory assays.
Model design
A cohort simulation model was constructed to determine from a health systems perspective, the cost effectiveness of EIA + IL versus RPR + TPPA/FTA. Using 2006 Alberta testing utilization levels for prenatal and non-prenatal populations, the model comparatively evaluates the costs and diagnostic outcomes of EIA + IL versus RPR + TPPA/FTA. The model simulated the cohort of individuals from prenatal and non-prenatal testing populations (separately) according to each protocol (EIA + IL or RPR + TPPA/FTA) to generate costs and outcomes.
Prenatal and non-prenatal populations were analysed separately because prevalence in the testing populations will affect the predictive values of initial and confirmatory tests and will therefore have a significant impact on both costs and diagnostic outcomes. All analyses were conducted using TreeAge Pro Suite (TREEAGE Software Inc., Williamstown, MA, USA).
Model inputs
All modelling assumptions and their sources are shown in Tables 1-3. Inputs were derived primarily from Alberta data. Inputs for which Alberta data was unavailable were supplemented with published research. Cost factors for which there was limited information available were estimated through consultation with experts and from available data.
Inputs – population and diagnostic tests
Refers to the mathematical distribution assigned to incorporate the likelihood of possible values (i.e. inherent variance) of the input during model simulation. NONE means that input does not vary during model simulation. Distributions are fitted based on primary data. In general, parameters estimated from larger sample sizes generate smaller ranges of possible values (consistent with statistical theory). Therefore inputs with very small standard errors indicate they were fitted from large sample sizes
Based on local evaluation of Enzygnost 9 Siemens, Deerfield IL, USA), Architect (Abbott, Mississauga ON, Canada) & Trepsure kits (Phoenix Biotech, Mississauga ON, Canada)
Provincial Lab Internal Accounting
Inputs – treatment and routine follow-ups
Note: Typical cases are patients who are easily contacted and located and who are also cooperative. A-typical cases are patients who are difficult to contact and locate and who are also uncooperative
Standard errors are estimated
Refers to the mathematical distribution assigned to incorporate the likelihood of possible values (i.e. inherent variance) of the input during model simulation. NONE means that input does not vary during model simulation
Search is conducted for contacting patient for scheduling treatment, treatment follow up visit, and follow up visit for indeterminate diagnosis
Personal communication from IDMC
Provincial Lab Internal Accounting
Inputs – outcomes for false-negatives and costs
Standard errors are estimated
Refers to the mathematical distribution assigned to incorporate the likelihood of possible values (i.e. inherent variance) of the input during model simulation. NONE means that input does not vary during model simulation
Provincial Lab Internal Accounting
Neurosyphilis is an irreversible condition. Cost reflects the management of the condition including eventual nursing home care. Cost is adjusted to Can$ 2006 using the Canadian consumer price index. Hospital services for caring for newborn infants with syphilis. This cost is the incremental cost when compared with healthy infants (i.e. cost attributable to congenital syphilis). Cost is adjusted to Can$ 2006 using the Canadian consumer price index
Personal communication from IDMC
Population characteristics
The prevalence and utilization levels of syphilis testing services (Table 1) were based on data collected by the ProvLab and Alberta Health and Wellness in 2006. In 2006, serum samples were received at ProvLab for syphilis testing from 89,647 individuals. Of this total sample, 51,523 were tested for prenatal screening and 38,124 were tested for diagnostic purposes in the non-prenatal population (e.g. STI clinics and physician offices).
Test characteristics
The sensitivity and specificity of EIA (Table 1) was derived from the performance of the top three EIA kits in the provincial evaluation: 6 Enzygnost (Siemens, Deerfield, IL, USA), Architect (Abbot, Mississauga, ON, Canada) and Trep-Sure (Phoenix Biotech, Mississauga, ON, Canada) (distributed by Phoenix Biotech). The specificity of TPPA and the sensitivity and specificity of FTA-ABS and IL were derived from the available literature and provincial STI databases. 7
Published sensitivity data were derived from performance of the test on known seropositive samples and because antibodies were not yet present in about 30% of early primary syphilis cases, 8 these values overestimate performance in routine use. In order to calculate the performance of the assays against all cases, the sensitivity values of the tests were reduced by 5%. This adjustment better reflects the test performance as used for all patients, as opposed to selected seropositive cases.
Costs and outcomes
A cost driver in the delivery of syphilis testing services is the labour resource required for searching and locating individuals, administering treatment, contacting sexual partners and reviewing charts. Information relating to service delivery was obtained from several Alberta sources.9,10 Estimates of labour time associated with specific tasks were obtained from STI directors, managers and nurses. Costs of tests and labour were obtained from Alberta sources (Table 2). 10 12
Cost of testing services included test reagents and labour, resource costs associated with false-negatives (e.g. congenital and neurosyphilis), treatment, treatment follow-up, contact-tracing, follow up of indeterminate cases, chart reviews from IDMC and community search and patient contacting for treatment and follow up (Table 3). Costs of testing services were obtained from the ProvLab accounting records.
Severe adverse health outcomes associated with false-negatives include neurosyphilis and congenital syphilis. Information relating to the probability of these outcomes and the resource costs associated with the outcome was derived from the published literature. 13 16 Costs of neurosyphilis reflect the management of the condition including eventual nursing home care. Symptoms of neurosyphilis may not occur for up to 30 years after initial infection. To convert future costs to 2006 values, costs associated with neurosyphilis were discounted at a rate of 5% per year and were assumed to occur in 30 years. This estimate is conservative because neurosyphilis may occur earlier than 30 years.
Costs of congenital syphilis reflect the incremental hospital cost for caring for the newborn syphilitic infant, when compared with healthy infants, reflecting the cost attributable to congenital syphilis. Costs of neurosyphilis and congenital syphilis were based on American data and were adjusted to Can$ 2006 using the Canadian Consumer Price Index.
The objective of laboratory testing is to identify seropositive individuals (i.e. presence of antibodies against T. pallidum) in the testing population in order to inform the clinical decision of diagnosis by the clinician (i.e. identify new cases of syphilis from both previously treated individuals who have antibody and individuals who are true-negatives; Table 2). Therefore, effectiveness was defined as the number of correctly identified new cases (CNC) and the number of correctly identified true-negatives (CTN), which included previous cases that have antibody but are not a new case.
Criteria for cost effectiveness
The analysis will identify additional costs (or savings) and additional effectiveness between EIA + IL and RPR + TPPA/FTA. If costs are less and effectiveness more, the decision should be to adopt the more effective and less costly alternative. However, if costs are more and effectiveness more, decisionmakers must decide whether the additional benefits are worth the additional costs by using the incremental cost effectiveness ratio (ICER).
The ICER represents the cost per additional correct diagnosis and EIA + IL is judged to be cost-effective if its ICER is below society's willingness to pay (WTP) for an additional benefit. Currently, there are no guidelines informing the appropriate societal but society should be WTP at least the opportunity cost of misdiagnosis (i.e. the impact of false-positives and false-negatives on health services). In the prenatal population, the cost of a false-negative is Canadian (Can) $17,445 (congenital syphilis + neurosyphilis) and the cost of a false-positive is Can$2962 (represents unnecessary resource use). In the non-prenatal population, the cost of a false-negative is Can$1428 and the cost of a false-positive is Can$3113. Therefore, an ICER below Can$20,000 per additional correct diagnosis and an ICER below Can$5000 were used as benchmarks to indicate whether EIA + IL is cost-effective in the prenatal and non-prenatal populations, respectively.
Model simulations and uncertainty
Expected values of costs and effectiveness were calculated from 100,000 Monte Carlo simulations using the distributions and standard errors listed in Tables 1-3. 17 During each simulation, for each input with a fitted distribution, a value is randomly sampled from the distribution and the costs and effectiveness were calculated for the simulation.
Based on the 100,000 sample sets, we generated an ‘acceptability curve’ 18 which depicts the proportion of simulations that had ICERs (when compared with RPR + TPPA/FTA) below a range of WTP thresholds for an additional correct diagnosis. Therefore, the curve represents the probability that EIA + IL is cost-effective at the cost per correct diagnosis thresholds.
Model assumptions
The treatment regimen for a case of syphilis is dependent on the stage of infection, but these data were unavailable. Therefore, our analysis applied the treatment regimen for primary syphilis and assumes no allergy to penicillin. Cases identified from contact-tracing generate further testing and contact-tracing. The costs and outcomes were calculated to include the costs associated with contact-tracing. Costs and outcomes were not calculated for second-order individuals (i.e. sexual partners identified from the initial list of sexual partners).
Results
Costs
Results are expressed in Can$ 2006. In the prenatal population, cost of the current protocol was Can$1,904,935 while cost of EIA + IL was Can$1,914,439. In the non-prenatal population, cost of the current protocol was Can$2,320,967 while cost of EIA + IL was Can$2,234,914. Thus, the total cost of current protocols (prenatal and non-prenatal) was Can$4,225,902 and the total cost of EIA + IL (prenatal and non-prenatal) was Can$4,149,353.
Effectiveness
In the prenatal population, the current protocol generated 51,510 (13 CNC and 51,497 CTN) correct diagnoses. EIA + IL generated 51,517 (14 CNC and 51,503 CTN) correct diagnoses producing seven additional correct diagnoses. In the non-prenatal population, the current protocol generated 37,876 (224 CNC and 37,652 CTN) correct diagnoses. EIA + IL generated 38,035 (247 CNC and 37,787 CTN) correct diagnoses producing 159 additional correct diagnoses.
Incremental cost-effectiveness ratio
In the prenatal population, the ICER of EIA + IL (when compared with the current protocol) is Can$1358 per additional correct diagnosis (i.e. EIA + IL will cost Can$1358 to produce one additional correct diagnosis). In the non-prenatal population, the ICER of EIA + IL is -Can$541 per additional correct diagnosis (i.e. EIA + IL will save Can$541 to produce one additional correct diagnosis). Overall (prenatal and non-prenatal), the ICER of EIA + IL is -Can$461 per additional correct diagnosis (i.e. EIA + IL will save Can$461 to produce one additional correct diagnosis).
Uncertainty
Figure 1 shows the distribution of incremental costs and effectiveness (difference in costs and effectiveness between EIA + IL and RPR + TPPA/FTA) associated with EIA + IL in the prenatal and non-prenatal populations. In the prenatal population, EIA + IL can range from being more costly and less effective to being less costly and more effective. However, EIA + IL is most likely to be more costly and more effective. In the non-prenatal population, EIA + IL is always more effective but can be either more costly or less costly than the current protocol. However, EIA + IL is most likely to be more effective and less costly.
Distribution of incremental costs and effectiveness of enzyme immunoassay initial test and Inno-Lia confirmatory test (EIA + IL). Constructed from 100,000 sample sets generated from a Monte Carlo simulation. Using rapid plasma reagin initial test and Treponema pallidum particle agglutination assay and fluorescent treponemal antibody absorbed assay confirmatory test as the comparator, the figure depicts the distribution of incremental costs and effectiveness of EIA + IL
Figure 2 shows the acceptability curve of EIA + IL. In the prenatal population, at a WTP of $0 per additional correct diagnosis (i.e. if society was WTP nothing to have more correct diagnoses in syphilis), EIA + IL is 45% cost-effective when compared with the current protocol (this is because of the proportion of simulations that are less costly and more effective). At our benchmark of Can$20,000, the probability that EIA + IL is cost-effective is greater than 90%. In the non-prenatal population, the probability of EIA + IL being cost-effective is 80% at a WTP of Can$0 per additional correct diagnosis. At our set benchmark of Can$5000, the probability that EIA + IL is cost-effective is 100%. Overall (prenatal and non-prenatal) results are similar to the results of the non-prenatal population.
Acceptability curve for enzyme immunoassay initial test and Inno-Lia confirmatory test (EIA + IL). Constructed from 100,000 sample sets generated from a Monte Carlo simulation. Using rapid plasma reagin initial test and Treponema pallidum particle agglutination assay and fluorescent treponemal antibody absorbed assay confirmatory test (RPR + TPPA/FTA) as the comparator, the curve depicts the proportion of incremental cost effectiveness ratio of EIA + IL that were below a range of cost per additional correct diagnosis thresholds. Therefore, the curve represents the probability that EIA + IL is cost-effective (when compared with RPR + TPPA/FTA) at particular cost per correct diagnosis thresholds. $ = Canadian dollars
Discussion
Control of syphilis is based on early identification and treatment of cases. Therefore, accurate and cost-effective testing and clinical diagnosis is critical. We evaluated the economic impact of EIA + IL separately in the prenatal and non-prenatal population because of the difference in prevalence of syphilis between the populations. Our results suggest that when compared with RPR + TPPA/FTA, EIA + IL is cost-effective in both prenatal and non-prenatal populations. EIA + IL will not only generate more correct diagnoses, but will also generate less heath-service cost. Our study has several strengths that increase the validity of our findings. First, model inputs for tests and population characteristics were derived from Alberta data, which provides greater confidence that the inputs for test characteristics represent effectiveness (performance under real-world conditions) rather than efficacy (performance under ideal conditions). Cost inputs were derived from actual accounting data and therefore reflect real-world costs. Secondly, our analysis was a multivariate probabilistic model that incorporates the uncertainty associated within each input (i.e. accounts for variation in inputs) and the potential interactions between inputs, and therefore provides greater confidence in the reliability and validity of the results.
However, there are limitations to our analysis. It is uncertain how EIA + IL will be implemented and how clinicians will ultimately use the test. While the analysis indicates that EIA + IL is cost-effective, it is based on the assumption that persons testing negative on EIA and IL do not receive further confirmatory testing or follow up. Although in actual conditions there will be variation in how EIA + IL is used depending on clinical presentation, protocols for EIA + IL outlined in this report should be adhered to (at least in general) in order to achieve the cost-effectiveness outcomes described. Thus, clear diagnostic protocols and education for clinicians ordering syphilis tests will be important for the implementation of the EIA + IL protocol.
We assumed that neurological involvement with syphilis would occur in 30 years. This was a conservative estimate because symptomatic neurosyphilis can occur within the first year of infection. Our results are therefore a conservative estimate of the cost savings associated with EIA + IL.
Our analysis was conducted using a health-service perspective and therefore, does not capture employment effects, lost productivity or effects on health-related quality of life.
Finally, the analysis does not include the testing volumes from testing service providers other than the ProvLab. Therefore, the cost of testing services in the analysis does not reflect the total volume of testing services conducted in Alberta; although it does cover approximately 75% of RPR testing and all confirmatory testing.
To our knowledge, this is the first economic evaluation of EIA + IL. At current prevalence levels, EIA + IL would generate cost savings to the healthcare system while also generating more correct diagnoses, when compared with RPR + TPPA/FTA. Therefore, EIA + IL should replace RPR + TPPA/FTA for the testing and diagnosis of syphilis provided that clear diagnostic guidelines are developed and continuing education is provided for clinicians on how to best use EIA and IL.
Footnotes
Acknowledgements
Supported by a financial contribution from Alberta Health and Wellness through its Alberta Health Technologies Decision Process. The authors are indebted to the staff of Alberta Health and Wellness STD services and staff of the regional health authorities for coordination and provision of data.