Surveillance of an infectious syphilis outbreak in south London,UK: a comparison of reporting systems

INTRODUCTION

In the UK, syphilis diagnoses reached a nadir in the 1990s alongside decreased rates of other sexually transmitted infections (STIs) following educational campaigns and behavioural changes in the face of a then untreatable AIDS epidemic. ¹ The resurgence of syphilis in the UK was first seen in men who have sex with men (MSM) in the late-1990s; then described in heterosexuals and subsequently in new born children. ^2,3 All genitourinary (GU) medicine clinics in the UK are statutorily required to file quarterly reports on the number of sexually transmitted disease (STD) diagnoses and clinical activity. STD diagnoses are reported using Korner Codes (KC60). However, KC60 data are deficient in demographic and risk factor information that are required to describe epidemics in sufficient detail. Other additional drawbacks of KC60 returns are their inaccuracy and delay in reporting. ⁴ In 2000, the Health Protection Agency (HPA) started a voluntary National Enhanced Syphilis Surveillance (NESS) programme to provide greater detail and further information about risk factors and sexual contacts for the evolving syphilis epidemic. ⁵ This passive reporting system was developed initially to report epidemics in MSM and has subsequently been applied to the heterosexual population.

The Health Protection Units (HPUs) in south London were informed by GU medicine staff in 2002 of a local rise in heterosexual syphilis potentially linked with sex work. In order to measure, monitor and respond to this epidemic, a Local Enhanced Syphilis Surveillance (LESS) was developed. The information sought was designed to directly inform local prevention initiatives and included specific risk factor data such as sexual behaviours, local venues frequented, engagement in and locations of any commercial sex work. Five of the ten south London clinics were asked to participate in LESS and outreach programmes were initiated to access and test commercial sex workers. This surveillance data were useful initially and helped health professionals target local prevention and control activities. The data subsequently seemed to indicate that the epidemic was declining towards the end of 2004 and this pattern was also seen in NESS. However, the fall in cases reported to the enhanced surveillance schemes was not supported by KC60 reports. In view of the importance of accurate surveillance in the management of epidemics, the three surveillance data sources were therefore evaluated.

MATERIALS AND METHODS

Three data sources for the year 2004 at three clinics in south London reporting cases of heterosexual syphilis were compared. These three clinics were chosen as they participated in both the NESS and LESS and could be visited for patient-note review. Information was obtained from the HPA for the first two datasets (KC60 and NESS). KC60 data included the aggregate quarterly returns from each clinic broken down by gender and homosexual acquisition or not. Heterosexual syphilis cases were identified by the codes A1–A3, indicating infectious syphilis, for patients not coded as homosexually acquired. This classification matches the inclusion criteria for both NESS and LESS. For analysis of NESS, database filters were applied to select women, and men not classified as homosexual. The LESS database was originally designed to collect information on cases of heterosexual infectious syphilis and these data were obtained from the southeast London HPU.

In order to disaggregate the KC60 reports so that the three databases could be compared, a series of clinical visits were arranged with lead clinicians. Prior to these visits, clinic managers compiled a list of all the cases of heterosexual infectious syphilis that had been diagnosed in 2004 and arranged for notes to be pulled by clinical staff for review. The clinic visits, lasting for four hours on average, consisted of a review of notes by one of the authors (ASM-J) within the clinic. For each patient, all visits to the clinic in 2004 were reviewed noting diagnoses made and the sexual histories to determine the sexual orientation. These were compared with NESS documentation. If the patient was not present in NESS, then a new NESS form was completed for the clinic and the HPA. Conversely, if a patient was listed in NESS but not identified by the clinic then these clinical notes were requested on the day of the clinic visit and subsequently checked. Confidential patient information was treated in accordance with Caldicott and HPA guidelines and only unique clinic identifiers were collected when the three databases were compared in a password-protected secondary Excel database. The three databases were compared using the syphilis cases identified during the clinic visit as the reference for comparison. Over- and under-reporting of the enhanced databases were described by percentage changes from this reference. Clinic visits were made in January and February 2006. Ethical committee approval was not sought as this was undertaken as part of an outbreak investigation review.

RESULTS

In 2004, only six of the 10 south London clinics participated in NESS despite all clinics reporting A1–A3 cases via their KC60 returns. The three clinics visited in this study were all participants of NESS. Overall, 200 cases of heterosexual infectious syphilis were reported from KC60 data returns from the three clinics in 2004; in contrast, only 73 and 62 reports were made for the same clinics by the NESS and LESS systems, respectively (Table 1).

One hundred and sixty-three case-notes (82% of the anticipated caseload based on KC60 returns) were identified by the clinics using the relevant KC60 codes. On review of these notes, 79 (48%) were confirmed as heterosexuals with infectious syphilis. Of the remaining 84 cases, 58 were MSM and 26 were incorrect diagnoses: alternative STDs (gonorrhoea, chlamydia, herpes and candida), an incorrect year (2003 and 2005) or non-infectious forms of syphilis (codes A4–A9).

KC60 over-reporting was suggested by the fact that 18.5% (37 of 200) of cases reported to the HPA could not be re-identified by clinic searches under the KC60 code A1–A3 in 2004. However, using the list of patients reported via NESS, an additional 12 patients were identified bringing the total number of confirmed cases of heterosexual syphilis to 91. Ten of these 12 case-notes had A1–A3 coding but none were identified by the clinics from their KC60 databases. A further nine cases reported to the enhanced surveillance schemes (NESS or LESS) could not be confirmed during clinic visits because the medical notes could not be located. Figure 1 shows the overlap between the three databases. Only 60 cases of heterosexuals with infectious syphilis were present in all the three databases.

OPEN IN VIEWER

Figure 1

This figure shows all the cases of heterosexual infectious syphilis in three south London clinics in 2004 and their location in the three databases. Patients identified during the clinic visit (Clinic) were compared with the National Enhanced Syphilis Surveillance (NESS) and the Local Enhanced Syphilis Surveillance (LESS) reports. Sixty patients were present in all three databases

Table 1 uses cases confirmed by clinic-note review as the ‘true’ denominator. Overall, KC60 data overestimated male heterosexual cases by 316%, while female cases were more likely to be confirmed. The NESS system under-reported by 20% while the LESS system under-reported the number of heterosexual syphilis cases by 32%. These results varied by clinic.

DISCUSSION

The inaccuracy of KC60 coding is clear from this study. The main source of error was in men where patients were misclassified as heterosexual. Coding errors can occur due to distraction with local codes ⁶ or time pressures leading to omission. The default answer to the question ‘homosexually acquired’ is negative; therefore any omission to code will report the male patient as heterosexual. The over-reporting of heterosexual cases is a major concern since KC60 data are used to monitor national and regional trends. Despite the inaccuracy of KC60 coding, this system made it possible to identify cases of heterosexuals with infectious syphilis that were missed by enhanced surveillance.

Surveillance is never perfect. In previous analysis of surveillance passive reporting was shown to have only 64% completeness. ⁷ In this review of clinical notes, NESS and LESS were confirmed to have captured 80% (73 of 91) and 68% (62 of 91) of cases, respectively. In this case a mandatory but inferior KC60 reporting system captured more cases, but overestimated the contribution of heterosexually transmitted infection due to incorrect coding of sexual orientation. Incentives are necessary to motivate staff but it may be more effective to align appropriate technology with normal clinic workflow, so that diagnostic information can be readily extracted and reported. An active surveillance of syphilis by cross comparisons between KC60, NESS, serology and parental penicillin prescriptions would likely identify a greater number of patients and be superior to the current passive reporting system. Independent sources of data could also be used for capture-recapture analysis in order to estimate the true size of heterosexual syphilis.

The purpose of LESS was to describe the epidemic and associated local risk factors among heterosexual patients in order to manage this local outbreak more effectively. LESS used equivalent questions to NESS except that it recorded the address of those venues where transmission was thought to occur. The additional benefit of LESS was that it was under local HPU control thereby providing more timely information. Traditionally, NESS returns are submitted centrally to the HPA in Colindale and there is a delay in reporting that makes local epidemic management less effective. In this review, LESS captured fewer cases of heterosexuals with infectious syphilis than both NESS and KC60. LESS required additional staff work to complete and added some additional locally relevant information over and above that obtained from NESS. The added value of that local perspective was thrown into doubt once the main characteristics of cases affected in the early stage of the outbreak were understood. It then appeared that this local initiative was no longer an effective use of resources.

The lack of completeness of both enhanced surveillance systems in these three clinics is of concern for a number of reasons. First, enhanced surveillance indicated that the heterosexual epidemic was waning and yet both databases under-reported cases of heterosexual infectious syphilis. Secondly, the epidemic is not being well-described in terms of demographics and risk factors making it difficult to ensure that local control measures are still appropriate and balanced correctly. This incomplete description of the heterosexual syphilis epidemic is compounded by non-participation in NESS. Thirdly, inaccuracies were found and some KC60 reported cases could not be identified and validated at clinical level.

This study of syphilis surveillance was limited by the nature of the surveillance tools themselves, since there is no one system that can be assumed to be complete. The enhanced systems are limited by the need to motivate and sustain the extra effort associated with participation in active reporting. On the other hand aggregation of data in the KC60 system makes it less useful in an outbreak or epidemic situation. A further problem is the regional variation in the laboratory diagnosis of infectious syphilis, ⁸ such that laboratory case reports are not standard for an active surveillance system.

There are a number of caveats specific to this study. First, the clinical notes were reviewed by only one person (ASM-J). Secondly, this work was all done with the aim of causing as little clinic disruption as possible; consequently, clinic visits were brief and it was not possible to review 9% (9 of 100) of the clinical notes. Thirdly, each clinic used a different computer and filing system as well as coding rules (local codes and methods to identify MSM) making each clinic visit unique. Only KC60 codes and the NESS forms were standard across the three clinics. During clinic visits 12 additional patients were identified and clinical notes reviewed because their clinic numbers were present in the NESS database. These inconsistencies highlight the need to invest in effective informatics systems to facilitate audit, pay for performance and surveillance.

Web-based reporting is already being used in a number of countries for a wide range of disease surveillance ^9–11 and it appears that UK sexual health services would benefit from this strategy. When a patient is given a KC60 code A1–A3, a NESS form should be generated automatically, completed and then submitted electronically. Web-based reporting is more timely, complete and accessible making it an attractive mechanism for local and national evaluation of epidemics. Since the heterosexual syphilis epidemic does not appear to be resolving, significant efforts should be made to improve surveillance, so that this and other epidemics can be appropriately managed.