Abstract
Abstract
Background:
Organ/space surgical site infections (SSIs) are serious complications after cesarean delivery. However, no scoring tool to predict these complications has yet been developed. This study sought to develop and validate a prognostic scoring tool for cesarean organ/space SSIs.
Methods:
Data for case and non-case of cesarean organ/space SSI between January 1, 2007 and December 31, 2012 from a tertiary care hospital in Thailand were analyzed. Stepwise multivariable logistic regression was used to select the best predictor combination and their coefficients were transformed to a risk scoring tool. The likelihood ratio of positive for each risk category and the area under receiver operating characteristic (AUROC) curves were analyzed on total scores. Internal validation using bootstrap re-sampling was tested for reproducibility.
Results:
The predictors of 243 organ/space SSIs from 4,988 eligible cesarean delivery cases comprised the presence of foul-smelling amniotic fluid (four points), vaginal examination five or more times before incision (two points), wound class III or greater (two points), being referred from local setting (two points), hemoglobin less than 11 g/dL (one point), and ethnic minorities (one point). The likelihood ratio of cesarean organ/space SSIs with 95% confidence interval among low (total score of 0–1 point), medium (total score of 2–5 points), and high risk (total score of ≥6 points) categories were 0.11 (0.07–0.19), 1.03 (0.89–1.18), and 13.25 (10.87–16.14), respectively. Both AUROCs of the derivation and validation data were comparable (87.57% versus 86.08%; p = 0.418).
Conclusions:
This scoring tool showed a high predictive ability regarding cesarean organ/space SSIs on the derivation data and reproducibility was demonstrated on internal validation. It could assist practitioners prioritize patient care and management depending on risk category and decrease SSI rates in cesarean deliveries.
C
Although standard clinical practice guidelines for antibiotic prophylaxis in cesarean delivery [3–5] have been utilized in our setting, only ampicillin or ceftriaxone were prescribed for all patients undergoing cesarean delivery in the study site according to surgeon's preference. These antibiotics provide reasonable coverage against potential pathogens for patients undergoing cesarean delivery and are referred to as the antibiogram in our facility. The incidence of cesarean SSIs in our setting remains high at 5.9% and cesarean organ/space SSIs were responsible for 82.9% of total cesarean SSIs [6].
Predicting case outcome may be valuable for planning appropriate surveillance and follow-up. Tools to predict the risk of associated conditions can be constructed from a set of specific and optimal risk factors. Each risk factor's effect is combined statistically for the most appropriate predictive ability and the included risk factors become predictors of the model [7,8]. It may be integrated in the SSI surveillance system to direct health care workers to observe more closely and detect SSI cases among high-risk groups. This may also be useful for obstetricians to assist in decision making regarding antibiotic use as treatment or prophylaxis for such patients.
Although procedure-specific risk models have been studied [9,10], a specific prognostic scoring system for cesarean organ/space SSIs has not yet been developed and validated. Because of the major impact of cesarean organ/space SSIs, we conducted this study to develop a scoring tool to predict cesarean organ/space SSIs. This study was in addition to our previous study [6]. The prognostic scoring tool from this study could be applied easily in clinical practice after validity and reliability have been proven.
Patients and Methods
Research design, setting, and data collection
This prognostic predictive research study was conducted in a tertiary hospital in northern Thailand. Prospective data were collected from inpatient records of pregnant women undergoing cesarean delivery and the infection control database of the hospital between January 1, 2007 and December 31, 2012.
Relevant clinical data included age, ethnicity, hemoglobin concentration, gravida, gestational age, source of referral, American Society of Anesthesiology (ASA) score, wound class, duration of operation, emergency status, number of vaginal examinations before cesarean delivery, characteristics of amniotic fluid, blood loss volume, skin and uterine incision pattern, surgeon, performing concurrent operations, and time duration from the rupture of membranes to incision.
Case definition
All patients were followed up for 30 d of cesarean delivery for SSI surveillance. The cesarean organ/space SSI cases were defined formerly according to the U.S. Centers for Disease Control and Prevention (CDC) [11], which has been changed to the National Healthcare Safety Network (NHSN) criteria since 2008 [12].
Sample size
Sample size was estimated using the rule-of-thumb method requiring 10 or more per event per covariate in the logistic regression analysis [13]. Because the 11 candidate risk factors were to be the predictors in the full model and the incidence of cesarean organ/space SSIs in our center was approximately 4.87% [6], the study size required a minimum of 2,258 patients.
Missing data were explored for their distribution. No imputation was performed when missing data resulted in less than 5% of total predictor values in complete case analysis.
Statistical analysis
Baseline characteristics were analyzed and presented as frequency, percentage, mean, and standard deviation. Continuous data were compared among groups using Student t test, or Wilcoxon signed-rank test. Fisher exact test was used to compare discrete data.
Some continuous clinical data were stratified clinically and meaningfully to categorical data. The optimal and clinically meaningful cutoff point for each parameter was determined. Univariable logistic regression and non-parametric receiver operating characteristics (ROC) analysis were then performed to obtain a crude odds ratio, a 95% confidence interval (CI), and area under the receiver operating characteristics (AUROC) curves for each parameter.
A test on the collinearity of the predictors in the full model on multivariable logistic regression of cesarean organ/space SSIs was performed. All predictors of cesarean organ/space SSI on univariable analysis were included in the full model for backward-selection stepwise logistic regression to determine predictors in the final model. The significance level for term removal from the model to obtain the final model was set at >0.0005.
Using the least coefficient, assigned item scores were obtained by rounding off the figures of the result of the division of all coefficients from logistic regression. The sum of the given total score for each patient was calculated. The logistic regression of cesarean organ/space SSIs on the total score and post-estimation prediction were performed. The distribution of cesarean organ/space SSIs cases versus non-cases among the total score was summarized and split in three risk categories by which cutoff points were determined from inflection points of the total score-to-risk of the cesarean organ/space SSIs prediction curve. The likelihood ratio of positive (LR+) for cesarean organ/space SSIs, 95% CI, discrimination, and goodness of fit were calculated for each risk category.
Data for internal validation were obtained using bootstrap re-sampling from the derivation data with the same size, namely 4,988 observations. Their distribution of cesarean organ/space SSIs cases versus non-cases among the total score and risk categories were also summarized. The likelihood ratio of positive (LR+) for cesarean organ/space SSIs with 95% CI were calculated for each risk category. A non-parametric test on the equality of the AUROCs of both derivation and validation groups was conducted. The study was approved by the ethics committees of the Faculty of Medicine at Chiang Mai University and the research conducting hospital. Data analysis were conducting using STATA® version 11.2 (StataCorp, College Station, TX).
Results
Of a total of 4,988 eligible cesarean delivery cases, 243 cases were diagnosed as cesarean organ/space SSIs. All patients with cesarean organ/space infection cases were routinely treated empirically by ceftriaxone plus metronidazole, but 9.5% of cases (23/243) needed re-operation and three cases (1.2%) were fatal after re-operation.
Among these 243 cases, 77 patients (31.7%) were tested to determine the pathogens. A total of 83 isolates were identified from these patients (one isolate in 73 cases, two isolates in two cases, and three isolates in two cases). Of these isolates, 47% showed no pathogenic growth. Escherichia coli was the dominant organism (24.1%). The organisms that were resistant to the antibiotic used were isolated in 22.9% (Table 1). No statistical association between the presence of resistant organism and subpopulation with any of predictors in the model was detected.
ESBLs = extended-spectrum β-lactamases.
Compared with the non-case group, patients developing cesarean organ/space SSIs were observed to have a higher proportion of ethnic minorities, hemoglobin concentration less than 11 g/dL, primigravida, gestational age less than 37 weeks, being referred by community hospitals/refugee camps, having a wound class greater than II, operative time more than 55 min, emergency operation, five or more vaginal examinations before cesarean delivery, foul-smelling amniotic fluid, and 12-h or more duration of ruptured membranes before incision. Their crude odds ratios were greater than 1.0 and AUROCs were greater than 0.5 (Table 2).
Shown are the confidence interval calculated by univariable logistic regression.
Shown are area under receiver operating characteristic curves estimated by non-parametric univariable ROC analysis.
ASA = American Society of Anesthesiology; AUROC = area under receiver operating characteristic curves; CI = confidence intervals; IQR = interquartile range; OR = odds ratios; ROM = rupture of membranes; SD = standard deviation; SSIs = surgical site infections.
All of the abovementioned parameters were included in the full model for backward stepwise multivariable logistic regression. Six clinical predictors remained in the final model after the regression. No collinearity was found among predictors in the model because the variance inflated factor of all predictors both in the full model and the final model were less than 2.5.
The coefficients and assigned scores of all predictors are shown in Table 3 including ethnic minorities, hemoglobin concentration less than 11 g/dL, vaginal examination five or more times before cesarean delivery, wound class more than II, referred from community hospitals/refugee camps, and foul-smelling amniotic fluid.
AOR = adjusted odds ratio; β = logistic regression beta coefficient; CI = confidence interval.
Shown are the confidence intervals calculated by multivariable logistic regression.
The distribution of total scores ranged from 0 to 12 between cases and non-cases showing substantial difference and implying its discriminative ability (Fig. 1). Discriminative performance of the full and reduced models among derivation data, represented by their AUROCs, were 0.8867 and 0.8722, respectively. The proximity of observed risks to the total score-to-risk of cesarean organ/space SSIs prediction curve with the p value from the Hosmer-Lemeshow goodness-of-fit test are presented in Figure 2.

Percentage distribution of clinical risk scores of cesarean organ/space surgical site infections (SSIs) cases (n = 243) versus cesarean organ/space SSIs non-cases (n = 4,745).

Observed risk (circle) versus score predicted risk (solid line) of cesarean organ/space surgical site infections (SSIs) among derivation data; size of circle represents the frequency of women in each score. The p value was obtained from the Hosmer-Lemeshow goodness-of-fit test.
For clinical implications, we divided the total score in three zones using the inflection points according to the total score-to-risk of cesarean organ/space SSIs predicted curve in Figure 2. The total score was 0–1 of 12 for low risk, 2–5 of 12 for moderate risk, and more than 5 of 12 for high risk. The likelihood ratio of cesarean organ/space SSIs with 95% CI for each risk category of derivation data were analyzed and are presented in Table 4. Total score among cases versus non-cases of derivation data are also summarized in Table 4. The validation data were analyzed in the same fashion and are presented in Table 4 for comparison. Both derivation and validation data had comparable AUROCs on the non-parametric test of equality as illustrated in Figure 3. Assigned scores were re-drawn in Table 5 for better understanding.

Area under receiver operating characteristic (AUROC) curves of clinical risk score on prediction of cesarean organ/space surgical site infections (SSIs) comparing bootstrap validation data with derivation data. The AUROCs with 95% confidence intervals (95% CIs) and p value were obtained from non-parametric analysis of the equality of the AUROCs.
CI = confidence interval; IQR = interquartile range; LR+ = likelihood ratio of positive; SSIs = surgical site infections.
Discussion
Cesarean organ/space SSIs have profound impact [14]. The affected patients have to undergo antibiotic treatment, re-operation for drainage, hysterectomy, and may even die. These factors can be complicated by delays in recognizing, diagnosing, and treating.
In our study, only one third of cases were tested to identify pathogens. This may be because all patients who developed cesarean/organ space SSIs in our facility were routinely treated empirically by ceftriaxone plus metronidazole. They were tested to determined pathogens when no clinical improvement was observed within 24–48 h after starting antibiotic treatment for cesarean organ/space SSIs or when intra-abdominal collections were questionable. These included abdominal/trans-vaginal ultrasonography, re-operation, and pus or body fluid culture either collected during re-operation or ultrasound-guided needle aspiration.
In addition, no pathogenic growth was observed in nearly half of isolates in this study. This may be because specimens were collected after empirical antibiotic therapy. Antibiotics may inhibit the growth of the bacteria causing cesarean organ/space SSIs, which may also be caused by improper collection techniques. Additionally, foul-smelling organ/space SSIs may be caused by anaerobic bacteria for which there is no test in our facility.
Although antibiotic therapy of cesarean/organ space SSIs changed potentially based on the susceptibility after pathogen identification and antibiogram determination, approximately one fifth of pathogens are resistant to the antibiotics used. Compliance with updated clinical practice guidelines for antimicrobial prophylaxis in cesarean delivery [3] may help lower multi-drug–resistant organisms in our setting.
A scoring tool to predict cesarean organ/space SSIs is clinically essential, serving as a procedure-specific scoring tool for cesarean delivery. It will allow meaningful improvement of the procedure performance. We believe that a prognostic scoring tool for cesarean organ/space SSIs can be used as to categorize, predict chance of SSIs, and provide tailor-made surveillance planning for patients with cesarean deliveries. In addition, the obtained prediction model will be valuable, simple to use, and feasibly applicable clinically.
The risk index developed by the NHSN has been used to predict SSIs universally. It comprises three predictors: ASA score more than two, wound class more than II, and operation time longer than 75th percentile, with equal score of one point for each item [15]. However, this is not a procedure-specific tool and lacks patient-level predictors to account for intrinsic risks of SSIs. Recently, a more procedure-specific tool to predict and monitor SSIs including more patient-related predictors has been developed [9]. However, no obvious score weighting method was proposed, although its c-index for cesarean delivery is up to 75% and classified as acceptable for discrimination ability [16].
The procedure-specific prediction tool for cesarean organ/space SSIs should comprise patient-related predictors collected from assessing bedside clinical parameters and procedure-related predictors [17]. The predictors should also be properly weighted scores depending on their magnitude of association in multivariable risk model regression and ensure internal and external orderly validation [8].
Six influential risk factors constitute the working predictors in this study. These include ethnic minority status, anemia [18], multiple vaginal examinations before cesarean delivery [6,19,20], foul-smelling amniotic fluid [21], wound class III or greater [22], and being referred from refugee camps or community hospitals [6]. Among these, only a wound class III or greater was set to be the same predictor as in a previous study [9].
Being an ethnic minority was a predictor by its own factors, i.e., geographically remote area, socio-economic and political backgrounds and creating a major hindrance to accessing the health care system. Anemia has been demonstrated to be a predictor for cesarean organ/space SSI on basis of its association with nutrient deficiencies [23], resulting in immune dysfunction and incision healing delay.
In this study, the number of vaginal examinations was five or more times before cesarean delivery. This increased the likelihood of infection by the process of inoculating bacterial flora from the lower part of the genital tract to the amniotic cavity [14]. It may also be similar to vaginal delivery trial-predictor in one previous study [9].
Wound class III or more was a predictor in our model on the basis of its general risk to post-operative SSIs and cesarean SSIs [22,24]. Foul-smelling amniotic fluid resembled the wound class predictor. It increased the chance of cesarean SSIs [25] and was evidence of chorioamnionitis [21] that may be a predictor. However, foul-smelling amniotic fluid was not exactly the same as the wound class III or more predictor. No collinearity was found between them in our study. In general, the fact that not all patients with a wound class III or greater had foul-smelling amniotic fluid may account for this finding.
Patients referred from local settings including refugee camps and community hospitals were under a time-consuming process for referral after vaginal delivery assistance failure. These may be accompanied by prolonged duration of ruptured membranes and multiple trans-vaginal procedures for birth assistance. These were the reasons it remained one meaningful predictor in the model after regression.
In comparison, patients with ruptured membranes occurring in the hospital will receive prompt intervention following criteria and based on the World Health Organization partograph. Hence, the duration of ruptured membranes was a partially important risk factor of cesarean organ/space SSIs and was not a predictor in the model.
Other abovementioned potential risk factors were removed from the model of regression because of their magnitudes and because they lacked significance to be included in the model. The emergency operation was demonstrated as a risk factor for cesarean incision SSIs but not for organ/space SSIs [6] and was removed from the model.
The strength of this study is because it is the first clinical prediction tool for cesarean organ/space SSIs constructed from a large study population. The cost of predictors in the model was well worth the outlay because they were performed routinely. The AUROCs of both the derivation data and internal validation data were greater than 0.8 demonstrating their discriminative performance [16]. Moreover, they were similar in AUROCs that the model was implicitly reproducible [8] and remained stable on internal validation, although missing data occurred.
In the derivation and internal validation, patients were approximately 10 times at risk of cesarean organ/space SSIs for each increment of risk categorization. This may provide useful information for obstetricians in decision making of antibiotic use regarding risk-specific groups.
This predicting tool can be used to screen high-risk cases of organ/space SSIs. The predictors in this tool can be also used to convince policy makers, obstetricians, midwives, and infection control professionals to develop international multi-disciplinary infection prevention and control programs between Thailand and Myanmar to reduce organ/space SSIs and many of the burdens of SSIs such as antibiotic use, re-operation, and mortality. The programs should involve prenatal care, labor care, cesarean delivery, and post-cesarean care. These programs should aim to support pregnant women's health, i.e., having normal concentration of hemoglobin, early detection and prompt referral for cesarean delivery, and improved quality of care in all phases. The practice guidelines to prevent organ/space SSIs in this special setting should be developed. Screening and referral systems should be included in the guidelines.
In addition, the tool could be applied equally to the post-discharge surveillance system regarding risk-specific groups and hence to reduced workload. It could also help monitor and provide periodic feedback to the local network of partnering medical facilities with high prevalence of cesarean organ/space SSIs to improve quality of care.
This prognostic scoring tool may have some limitations for some predictors when applied to other settings, including being referred and an ethnic minority. However, many pregnant women are being referred for better deliver or life saving. Additionally, several countries have ethnic minorities. Therefore, this prognostic scoring tool is of theoretical benefit for these settings. Last, globalization may potentially result in a change in population structure [26]. Thus, the tool should be temporally and geographically validated and updated periodically.
Conclusions
The developed prognostic risk score for cesarean organ/space SSIs may provide useful information for obstetricians in decision making or prioritizing management for their patients according to the risk category. In addition, it may help in post-discharge surveillance and to reduce SSI rates among cesarean deliveries.
Footnotes
Acknowledgments
The authors wish to thank Professor Dr. Jayanton Patumanond at the Clinical Research Center, Faculty of Medicine, Thammasat University for statistical analysis assistance.
An abstract of this manuscript was accepted as a poster presentation at IDWeek 2016. It has been published as an online supplement to Open Forum Infectious Diseases (OFID), the new Open Access Journal from IDSA.
Author Disclosure Statement
All named authors have no conflicting financial interests to disclose.
