Abstract
Abstract
Background:
The incidence of surgical site infections (SSI) is considered increasingly to be an indicator of quality of care. We conducted a study in which daily inspection of the surgical incision was performed by an independent, trained team to monitor the incidence of SSI using U.S. Centers for Disease Control and Prevention (CDC) definitions, as a gold-standard measure of care. In the department of surgery, two registration systems for SSI were used routinely by the surgeon: An electronic and a plenary tracking system. The results of the independent team were compared with the outcomes provided by two registration systems for SSI, so as to evaluate the reliability of these systems as a possible alternative for indicating quality of care.
Methods:
The study was an incidence study conducted from May 2007 to January 2009 that included 1,000 adult patients scheduled to undergo open abdominal surgery in an academic teaching hospital. Surgical incisions were inspected daily to check for SSI according to definitions of health care-associated infections established by the CDC. Follow-up after discharge was done at the outpatient clinic of the hospital by telephone or letter in combination with patient diaries and reviews of patient charts, discharge letters, electronic files, and reported complications. Univariate and multivariable analyses were done to identify putative risk factors for missing registrations.
Results:
Of the 1,000 patients in the study, 33 were not evaluated. Surgical site infections were diagnosed in 26.8% of the 967 remaining patients, of which 18.0% were superficial incisional infections, 5.4% were deep incisional infections, and 3.4% were organ/space infections. More than 60% of SSIs were unreported in either of the department's two tracking systems for such infections. For these two systems, independent major risk factors for missing registrations were (1) the lack of occurrence of an SSI, (2) transplantation surgery, and (3) admission to non-surgical departments.
Conclusions:
Most SSIs were not tracked with the department's two systems. These systems proved poor alternatives to the gold-standard method of quantifying the incidence of Surgical Site Infection SSI and, therefore, the quality of care. Both protocolized wound assessment and on-site documentation are mandatory for realistic quantification of the incidence of SSI.
Methods
The study was designed as a prospective observational cohort study and was conducted at a 1,200-bed academic teaching hospital. Approval for the study was obtained from the local ethics committee. Inclusion criteria were a minimum age of 18 years and open abdominal surgery or a laparoscopic procedure that was converted intraoperatively to an open abdominal procedure. Exclusion criteria were laparoscopic surgery, inguinal/umbilical hernia repair, and day surgery. The inclusion/exclusion criteria were established for the purpose of providing a patient group that could be subjected to repetitious, daily in-hospital surveillance, with a high estimated risk of developing any of the three degrees of a superficial incisional, deep incisional, or organ/space SSI. Informed consent was obtained from all study participants. In addition to demographic data (age, gender, department of admission), the following data were documented: Body mass index, comorbidity (chronic obstructive pulmonary disease, diabetes mellitus), systemic corticosteroid use (oral, but not inhalation or dermal corticosteroid use), smoking, American Society of Anaesthesiologists score, duration of surgery, type of surgery, emergency surgery, transplantation surgery, National Research Council wound contamination class, and National Nosocomial Infection Risk Index. Data were also collected on duration of hospital stay, reoperation within 30 days after primary surgery, and in-hospital mortality.
Patients included in the study were subjected to the tracking method of daily surveillance, the primary outcome of which was the incidence of SSI according to the definitions of the U.S. Centers for Disease Control and Prevention (CDC) [10]. Other endpoints included abdominal incision dehiscence and pain. However, these results will not be discussed in this paper. For each patient, data reported in the department's two routine tracking systems were reviewed for the occurrences of SSIs in that particular system. In The Netherlands, national nosocomial infection tracking systems include the Dutch National Nosocomial Infection Surveillance System (Preventie Ziekenhuisinfecties door Surveillance [PREZIES), which is used mainly for prevalence studies. The definition of SSI used in PREZIES is based on the definition used by the CDC, with the additional obligatory presence of clinical symptoms; diagnosis of SSI by the surgeon alone was not sufficient for the determination of an SSI. Our study differed from PREZIES in that it did not require the presence of clinical symptoms in addition to the CDC definitions.
Daily surveillance tracking method (“gold-standard” method)
Abdominal incisions were inspected and photographed daily by two research fellows from postoperative day two onward (including weekends and holidays) to observe for the presence of SSIs. The research fellows were medical students in the fourth to sixth year of training who participated in the study for a minimum of five months, and were supervised daily by the first author (G.H.vR.). All participants in the inspection team tracked infections independently of the surgeon involved in the operation. On at least one occasion per week, inspection rounds were performed in conjunction with the supervisor. After 21 days of clinical observation or at discharge if it was earlier than 21 days, patients were given diaries in which to record problems with their incisions until postoperative day 30. This period was chosen because most SSIs have been reported to present within 21 days postoperatively [1]. Follow-up was done at the outpatient clinic on postoperative day 30, or alternatively by telephone or letter. Patient charts, discharge letters, wound photographs, and culture results were reviewed by the first author (G.H.vR.) after a minimum period of three months following discharge to verify the incidence of SSI. The surveillance team was impartial, and did not itself promote compliance with the electronic and plenary tracking systems, so as to preserve the validity of the sensitivities of the systems described below. Data collected by the surveillance team were not submitted to any national surveillance system.
Electronic tracking system
The electronic ward system was introduced at our institution in January 2007 and required residents to track in-hospital complications on a daily basis. Electronic record sheets allowed documentation of the absence or presence of complications and their severity. No application was included in the electronic system for alerting physicians to whether this sheet was missing at discharge. These data were not submitted to any national surveillance system.
Plenary tracking system
For several years, the plenary tracking system in our department consisted of a daily review of all patients discharged from surgical wards, which was part of the plenary morning report led by the head of the department. A few days after discharge, the occurrence of complications, including a short description and the severity of the most serious complication, were scored on hard copy discharge lists during the plenary report and filed for all patients. These data were not submitted to any national surveillance system. In addition to the scoring and filing of discharged patients' most serious complications, physicians were required to issue discharge letters for all patients, including those who died in-hospital.
Statistical analysis
A total of 1,000 patients were to be recruited for the study, based on the hypothesis that 10% of included patients would develop SSIs, thereby allowing group sizes sufficient for the comparison of patients with and without SSIs. Putative risk factors for registrations missing from the tracking systems were evaluated with univariate analysis, using the χ2 test and Mann–Whitney U test for categorical and continuous data, respectively. Subsequently, variables that were significant in univariate analysis were entered in multivariable stepwise logistic regression analyses with backward elimination to identify major independent predictors of missing registrations. Cox regression analysis, with the occurrence of SSI as a time-dependent factor, was done to investigate the association between SSI and 30-day survival. The SPSS software system version 15 (SPSS, Chicago, IL) was used for all analyses. Two-sided values of p<0.05 were considered significant in all analyses.
Results
Between May 2007 and January 2009, 1,000 of 1,459 eligible patients were included in the study, with 459 patients not giving informed consent for participation. Thirty-three patients were not evaluated because of cancelled surgery or because they met the exclusion criteria for the study, leaving 967 patients whose data were available for analysis.
Surgical procedures done on these patients included kidney transplantation (19.5%), and liver (13.8%), colorectal (13.0%), esophageal (11.5%), stomach–small intestinal (9.4%), pancreatic (8.3%), vascular (6.2%), and other (18.3%) procedures. Length of stay (10th–90th percentile) was 5–31 days for the study group as a whole, with a median hospital stay of 11 days (5–25 days) for patients without SSI and 16 days (8–51 days) for patients with SSI (p<0.001). Forty-five patients with SSI underwent reoperation (17.4%; of whom 27 had a diagnosis of SSI before or during reoperation and 18 patients had this diagnosis after reoperation), in contrast to 26 patients without SSI who underwent reoperation (p<0.001). In total, 41 patients (4.2%) died within 30 days after surgery (36 in the hospital, of whom 11 died after developing a SSI and 25 died without developing a SSI). Thirteen patients with a SSI died within 30 days after surgery versus 28 patients without a SSI who died within this period (5.0% vs. 2.9%, respectively). A survival analysis showed the hazard ratio for risk of death associated with SSI in the period of 30 days after surgery to be 2.7 (95% CI 1.3–5.5, p=0.006). Additional population characteristics of the study population are shown in Table 1.
BMI=body mass index; COPD=chronic obstructive pulmonary disease; NRC class=National Research Council (NRC) wound contamination class.
Values represent mean±standard deviation and range (10th–90th percentile) or numbers of patients (percentages).
†NNIS risk index: National Nosocomial Infection (NNIS) risk index was calculated by awarding 1 point for each of the following: (1) a patient with ASA class 3, 4, or 5; (2) an operation classified as contaminated or dirty-infected; and (3) an operation lasting more than T hours, where T depends upon the operative procedure being performed [11].
Daily surveillance tracking method (“gold-standard” method)
Thirty-day follow-up was completed for 85.4% of the 967 patients whose data were available for analysis. Of these patients 643 (77.9%) had their follow-up completed at the outpatient clinic, 170 (20.6%) by telephone, and 14 (1.7%) by letter or e-mail. Data from the plenary and electronic tracking systems were reviewed for all 967 patients. The charts of 946 of the 967 patients (97.8%) were examined; 21 of the patients' charts (2.2%) were lost. Using the daily surveillance tracking method, SSIs were diagnosed in 259 of the 967 patients in the study (26.8%), with 174 of these being superficial incisional (18.0%), 52 being deep incisional (5.4%), and 33 being organ/space infections (3.4%) (Fig. 1).

Surgical site infections (SSI) according to gold standard were tracked as SSI, not tracked as SSI (non-SSI) or registrations were missing (missing). Bars demonstrate proportionate distribution by tracking method (plenary, electronic, and discharge letters) and timing of diagnosis.
The median time of diagnosis of SSI was at postoperative day 9 (interquartile range 6–13 d). The majority of infections (81%) were diagnosed in the hospital. The median hospital stay increased with the severity of SSI, and was 14, 25, and 27 days for superficial incisional, deep incisional, and organ/space infections, respectively (p<0.001). Patients with a SSI were readmitted more than twice as often as patients without a SSI (15.1% vs. 7.4%, p=0.009). The mean number of ambulatory care visits within the three months after surgery was 1.9 for patients with a SSI versus 1.1 for patients without a SSI (p<0.001).
Electronic tracking system
Registrations of SSIs were entered in the electronic tracking system for 458 of the 967 patients whose data were analyzed (47.4%). Registrations were missing for 509 patients (52.6%), and no registrations of any complications were entered for these patients. SSIs were tracked according to the daily surveillance tracking method for 64 of 259 patients with SSIs (24.7%). In compliance with the electronic tracking method, infections diagnosed after discharge were tracked significantly less often than infections diagnosed while patients were in the hospital (6/49 vs. 58/210 patients; p<0.001). Deep incisional and organ/space infections were not reported more often than superficial incisional SSIs (p=0.488).
Plenary tracking system
Registrations in the plenary tracking system were available for 709 of the 967 patients (73.3%) whose data were analyzed, and were missing for the remaining 258 of 967 patients (26.7%). Of all patients with SSIs tracked with the daily surveillance tracking method (n=259), 79/259 patients (30.5%) were tracked with the plenary tracking system. In compliance with the plenary tracking method, infections diagnosed after discharge were tracked significantly less often than infections diagnosed while patients were in the hospital (4/49 vs. 75/210 patients; p=0.042). Deep incisional and organ/space infections were not reported more often than superficial incisional SSIs (p=0.097).
Discharge letters were available for 946 patients (97.8%), with letters missing for 21 patients (2.2%). Surgical site infections were reported in 40% of patients with superficial incisional infections (68/171), 70% of patients with deep incisional infections (37/53), and 61% of patients with organ/space infections (20/33). Deep incisional and organ/space infections were reported significantly more often than superficial incisional infections (p<0.001).
Missing registrations
In total, 40.2% of patients with SSIs were reported in the plenary or electronic tracking systems. For the total study population (n=967), registrations of SSIs were missing for 52.6% of patients in the electronic system and for 26.7% of patients in the plenary morning report. Patients with infections of which the records were missing from the plenary or electronic tracking systems (n=218) required readmission within 30 days in 14% of cases (n=30), reoperation following a diagnosis of SSI in 15% of cases (n=32), and use of antibiotics in 32% of cases (n=46); in the remaining 8% of cases (n=18) these unrecorded SSIs resulted in death. Table 2 shows the frequency of various putative risk factors for missing registrations of SSIs and the results of univariate analyses of these putative risk factors. Risk factors for missing SSI registrations in the plenary and electronic systems by univariate analysis were the lack of occurrence of an SSI, transplantation surgery, emergency surgery, and admission to a nonsurgical department. In-hospital mortality and length of hospital stay were not risk factors for missing SSI registrations. By multivariable analysis, no occurrence of SSI, transplantation surgery, or admission to a nonsurgical department proved to be a significant independent risk factor for missing SSI registration (Table 3).
Values represent median and range (10th–90th percentile).
CI=confidence interval; OR=odds ratio; SSI=surgical site infection.
Table 4 shows infection rates calculated for different combinations of the tracking systems investigated in the study. The infection rate for data from the plenary and electronic systems combined was 10.3% (100/967). After the addition of discharge-letter data, the infection rate was 16.9%, and sensitivity with the daily surveillance tracking method as a reference increased from 38.6% (100/259) to 61.8% (160/259). This rate remained significantly lower than that with the daily surveillance tracking method, for which the rate of SSI was 26.8% (p<0.001). A number of patients with opened incisions but with negative cultures and without other findings needed to fulfill the CDC criteria for SSI were diagnosed wrongly as having infected incisions in the electronic (n=2) and plenary systems (n=7). Specificities of the electronic and plenary system were 99.4% and 98.6%, respectively.
Infection rate calculated as number of patients with tracked surgical site infections (SSIs) divided by number of patients for whom data were analyzed (n=946 for “Discharge letters only” and “Plenary and electronic systems and discharge letters combined,” and n=967 for all others).
Discussion
The incidence of SSI in this study was 26.8%; SSIs were associated with substantial morbidity and with increased 30-day mortality. The majority of SSIs (61%) were not reported in any of the tracking systems used by surgeons. The substantial morbidity associated with missed SSIs suggests that a large proportion of these missed infections were relevant clinically. Methods of tracking complications of surgery and classification systems for such complications vary substantially in reports in the literature [12–20]. Tracking systems used by surgeons proved unreliable for monitoring the incidence of SSI, and this method of self-reporting therefore constitutes a poor indicator of quality of care. A self-reporting bias for surgeons and residents may partly explain the low sensitivity of the tracking systems for SSI, whereas such a bias was unlikely to exist for members of the team engaged in the tracking of SSI through the daily surveillance method. The sensitivity of the daily surveillance tracking method and the high rate of follow-up may also explain the high infection rate identified through this method, which was comparable to rates of SSI reported by other authors [21,22]. Furthermore, tracking by a specialized team cannot be exchanged for surgeons' tracking systems without a substantial loss of sensitivity. Comparison of these data with national data was possible because of a validation study of the PREZIES data that was conducted from 1999 to 2004. This latter study consisted of systematic retrospective chart review by the team that validated the PREZIES data and interviews with local infection-control professionals. In that study, a positive predictive value of 0.97 and negative predictive value of 0.99 were found, both of which were comparable with the findings in our study [23].
Independent risk factors for missing registrations in our surgery department's systems included the lack of occurrence of SSI, transplantation surgery, and admission to a nonsurgical department. These risk factors most likely indicate non-compliance with registration on the part of surgeons and surgical residents in charge of patients admitted to the transplantation department and to non-surgical departments, which may not be generally true for other institutions.
Our routine electronic tracking system relied on the individual responsibility of the residents and supervising surgeons involved in performing surgery and detecting SSIs. A heavy workload may have hindered the reliability of medical records or electronic tracking by creating forgetfulness on the part of physicians or causing them to neglect the documentation of complications. Continuity of care can be endangered by frequent changes in staff shifts and compensatory leave, allowing a superficial incisional SSI to remain untracked. It might be useful to involve (trained) nurses or physician assistants in the tracking of SSIs, because nurses generally inspect incisions more often than do physicians. One study reported good results of routine electronic (computer) tracking on a daily basis by nurses of deviations from the normal postoperative course of patients undergoing cardiac surgery, and specifically of tracking before discharge from the postoperative ward of a Swedish hospital, after which supervising physicians decided whether or not complications had occurred [24]. An additional weakness of our electronic system was its lack of alarm signals (e.g., flagging of patients) and supervision, which allowed missing registrations to remain unnoticed without consequences. However, we doubt whether patient safety would have been guaranteed through alarm signals and whether this would thereby have covered 100% of all patients, as the sensitivity of the self-reporting system was far lower than that of the tracking method used by the specialized team.
The plenary tracking system used during the morning report depended on the continuity of care in the surgery department. It was problematic that patients who had undergone surgery and were admitted to non-surgical wards in our local hospital and who were not registered primarily as surgical patients (e.g., through registration in urology, internal medicine, and other departments) were not featured on discharge lists. As a result, complications associated with various operations remained untracked. Furthermore, data kept in the tracking systems were not combined routinely or evaluated. Evaluation of any single self-reporting system revealed the reporting of a lower number of infections than were reported when all of the tracking systems were combined. Routine evaluation of the tracking systems or other types of exogenous inducements to complete missing SSI data might have helped to achieve higher compliance.
The results of this study have raised awareness of the underestimation of rates of SSI in the departments of general surgery and infection control at our hospital. Embedding the gold-standard surveillance approach (including patients' diaries and postdischarge follow-up) into routine practice has thus far proven challenging on financial and practical grounds, because this method is associated with an increased workload, and post-discharge surveillance of surgical patients has remained voluntary in the national surveillance system used in The Netherlands for nosocomial infections. Another way in which cost-effective surveillance can be conducted is through an electronic or automatic selection of patients for nosocomial infections. This system of automated selection will be validated against the time-consuming but very reliable system of direct surveillance. The automated selection of high-risk patients, which deserves increasing attention and will be a reality in the future, will provide information about rates of infection while at the same time leaving personnel free to provide the interventional care needed for treating infections.
The discharge letters issued for all patients who underwent surgery at our hospital were expected to be fairly complete in the description of complications such as SSI, and it was hoped that the efficacy of the electronic and plenary complication tracking systems could be easily determined from these data. However, our data illustrated that the review of discharge letters for the follow-up of patients with SSIs was of limited value because of its mediocre sensitivity, especially for superficial incisional SSIs. Considering that 19% of the SSIs in our patients were detected after discharge, and as many as 84% were found in previous studies to have been detected after discharge, depending on the patient population, post-discharge data ought to be included in audit meetings [1]. Infection rates reported in the literature, possibly as distinct from the rates reported in well-performed randomized controlled trials, will generally represent only a fraction of the true number of affected patients. It is notoriously difficult to achieve complete follow-up and documentation of SSIs even when charts are reviewed [13,19]. In many studies, follow-up after discharge is omitted or performed only through questionnaires administered by the surgeon. The relative percentage of SSIs detected after discharge depends strongly on the intensity of post-discharge surveillance, and can differ significantly in diffent countries even if comparable protocols and definitions are used in surveillance [25]. According to a report by the European Centre for Disease Prevention and Control, The Netherlands were, in 2007, among the countries with the highest percentages of infections diagnosed after discharge, at 58% [26]. Furthermore, the type of surgery proved relevant for the percentage of SSIs detected after discharge, with a relatively low percentage of SSIs reported for colon surgery (13%) in 2009, as compared with an overall rate of SSI of 48% in that same year [27]. Although follow-up of patients with implanted materials of non-human origin at one year after surgery is recommended by the CDC, it is often not done in studies with general-surgery patients. Implanted materials of non-human origin, such as mesh, are not used frequently in this patient group. The period of follow-up in our study was limited to 30 days.
Use of a self-reported incidence of infection as an indicator of quality of care is not advisable because of the low rate at which it can be documented (depending on the method used), and favors hospitals with the smallest rate of self-reported infection. Punishment of hospitals with high infection rates will promote a low documentation of infections and the alternative interpretation or use of definitions of infection, which does not contribute either to patient safety or to the development of a self-critical atmosphere for the documentation of hospital-acquired infections. Structural shortcomings in care may then be identified less easily. A better system for measuring patient safety is not comparing outcome data (e.g., SSI, depending on the case mix), but comparing process indicators for the prevention of infections, such as timely antibiotic prophylaxis. To truly improve quality of care, a constructive approach to the problem of SSI is needed on the part of politicians and health insurers. In order to be able to compare hospital infection rates, uniform definitions must be used and surveillance for and prevention of SSI should be integrated into resident training programs. Lastly, comparing rates of infection requires accurate correction for case mix and large numbers of patients for adequate statistical power. The reliability of incidence rates of SSI depends on the quality of documentation of SSIs in both the inpatient and outpatient departments, which should involve the training of nurses and doctors in the protocol-governed, regularly supervised assessment of fresh surgical incisions as part of a continuous validation process.
Footnotes
Author Disclosure Statement
None of the authors has competing interests. Preliminary results of this study were presented at the Third Combined Meeting of the Surgical Infection Societies of North America and Europe (May 6–9, 2009, Chicago, IL) and received the European residents award for the best oral presentation at this meeting.
