External Validation of a Decision Tool To Guide Post-Operative Management of Patients with Secondary Peritonitis

Abstract

Background:

Timely identification of patients in need of an intervention for abdominal sepsis after initial surgical management of secondary peritonitis is vital but complex. The aim of this study was to validate a decision tool for this purpose and to evaluate its potential to guide post-operative management.

Methods:

A prospective cohort study was conducted on consecutive adult patients undergoing surgery for secondary peritonitis in a single hospital. Assessments using the decision tool, based on one intra-operative and five post-operative variables, were performed on the second and third post-operative days and when the patients' clinical status deteriorated. Scores were compared with the clinical reference standard of persistent sepsis based on the clinical course or findings at imaging or surgery. Additionally, the potential of the decision tool to guide management in terms of diagnostic imaging in three previously defined score categories (low, intermediate, and high) was evaluated.

Results:

A total of 161 assessments were performed in 69 patients. The majority of cases of secondary peritonitis (68%) were caused by perforation of the gastrointestinal tract. Post-operative persistent sepsis occurred in 28 patients. The discriminative capacity of the decision tool score was fair (area under the curve of the receiver operating characteristic = 0.79). The incidence rate differed significantly between the three score categories (p < 0.001). The negative predictive value of a decision tool score categorized as low probability was 89% (95% confidence interval [CI] 82–94) and 65% (95% CI 47–79) for an intermediate score. Diagnostic imaging was performed more frequently when there was an intermediate score than when the score was categorized as low (46% vs. 24%; p < 0.001).

Conclusion:

In patients operated on for secondary peritonitis, the decision tool score predicts with fair accuracy whether persistent sepsis is present.

Secondary peritonitis is caused by a breach in the integrity of the gastrointestinal tract, either by perforation, infection, inflammation, or ischemia. Despite the diversity in specific underlying diseases, management is based on several general principles [1]. Achieving adequate source control is one of the most important initial aspects, and for most patients, this entails surgical intervention [2,3]. After the initial surgical procedure, persisting or new-onset abdominal sepsis may necessitate re-intervention in as many as 34% of patients [3,4]. It has been well recognized that timely identification contributes substantially to a better survival rate. Planned re-operations therefore were performed historically to avoid delay in detection. However, during recent decades, a more conservative “on-demand” approach, with re-operations performed only in some patients with clinical deterioration, has been shown to be superior in terms of patient-related outcomes and direct medical costs [4].

Essential to the on-demand strategy is vigilant observation during the post-operative course with round-the-clock monitoring and decision-making [5]. The decision to perform a re-intervention is not straightforward. No well-defined and objective criteria exist for something that we call “clinical assessment” in daily practice. Available widely used intensive care unit (ICU) scoring systems have limited value for the identification of patients with continuing infection after the initial emergency laparotomy [6]. Although computed tomography (CT) is an important and useful diagnostic for abdominal sepsis, several drawbacks warrant selective usage. First, CT necessitates transfer of a potentially unstable patient from the ICU. Second, the administration of contrast agents may induce or aggravate renal injury, and third, CT entails concerns regarding cost-effectiveness in cases of unclear selection criteria.

In an attempt to improve post-operative management of patients operated on for secondary peritonitis, our study group developed and internally validated a predictive model [7]. The model included one intra-operative variable (extent of the contamination during initial surgery) and five post-operative variables (heart rate, hemoglobin concentration, body temperature, defecation, and the need for administration of inotropic agents) and predicted the probability of persistent sepsis from an infectious abdominal focus. The prediction model was subsequently translated into a decision tool to guide the use of diagnostic imaging for three score-based categories. The aim of the current study was to validate this decision tool externally in a new cohort of patients with secondary peritonitis and to evaluate its potential to guide and improve post-operative management.

Patients and Methods

This was an external validation of a decision tool previously developed by our study group [7]. Validation was done using a new cohort of patients cared for in a single hospital, which was one of the hospitals that contributed patients to the development of the tool.

The manuscript was written according to the items for external validation of a prediction model as proposed in the “Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD)” statement [8]. The ethics committee of the Academic Medical Centre approved the study protocol and waived informed consent.

Study design and data collection

Consecutive adult patients (≥18 years of age) undergoing emergency surgery (laparotomy or laparoscopy) for secondary peritonitis between January 2014 and March 2015 were included prospectively. Secondary peritonitis was defined as an intra-abdominal infection, verified during surgery, caused by perforation or infection of a visceral organ, ischemia, or necrosis of part of the gastrointestinal tract or a post-operative peritoneal infection secondary to anastomotic leakage [4].

Data on patient demographics, etiology of peritonitis (perforation, ischemia/necrosis or post-operative anastomotic leakage), severity of peritonitis (pre-operative Acute Physiology and Chronic Health Evaluation [APACHE] II score and extent of peritonitis during the index operation) and surgical details (approach, whether source control was achieved, and the employment of a planned re-laparotomy strategy) were collected. Furthermore, data on the post-operative course were collected, including the use of diagnostic imaging, number of radiologic and surgical interventions, length of post-operative hospital stay, and in-hospital mortality rate.

Decision tool assessments

The development of the decision tool has been described elsewhere [7]. In short, the tool is composed of six items: Heart rate (≤90/>90 bpm), hemoglobin concentration (≤5.0/>5.0 mmol/L), body temperature (≤35.5°C or >39°C/>35.5 and ≤39.0°C), defecation (yes/no), extent of contamination found at the index operation (diffuse/local), and the need for administration of inotropic agents (yes/no). With use of a nomogram [7], depicted in Figure 1, a probability score can be calculated. For each item, points can be obtained with a maximum possible score of 60, with a higher score being associated with a greater probability of persisting sepsis. Three separate categories were formulated during development: Low probability (0–19 points), intermediate probability (20–40 points), and high probability (>40 points). Given the design of the decision tool, patients can be evaluated more than once during a single post-operative period.

FIG. 1.

Nomogram depicting decision tool use to predict persistent abdominal sepsis with advice regarding monitoring and performing imaging studies.

With use of the decision tool, a score was calculated for each patient (except those who died soon after the index operation) on the second and third post-operative days (PODs). Furthermore, a score was calculated on every POD within a 10-d period after surgery when the status of the patient deteriorated or led the treating physicians to consider performing diagnostic imaging or re-interventions.

Outcomes

The diagnostic accuracy of the decision tool was calculated by comparing the calculated score with the clinical reference outcome; i.e., persisting sepsis from an abdominal focus (within 24 h after the decision tool assessment). Positive results were patients who had positive findings of an infected abdominal focus at re-laparotomy, re-laparoscopy, or diagnostic imaging. Patients who died from presumed sepsis within 24 h of the assessment were also considered positives. Negative results were patients with negative findings at re-operation or imaging and patients with an uncomplicated post-operative course during the first 24 h after the decision tool assessment.

Secondary outcome was the potential of the decision rule to improve patient management in terms of selection for diagnostic imaging on suspicion of an abdominal infected focus by comparing the number of imaging studies as advised by the decision tool with the number of imaging studies actually performed.

Statistical analysis

If a patient had an episode of abdominal sepsis, all subsequent decision tool assessments were censored from the analyses of diagnostic accuracy and impact on management. For every assessment using the decision tool, a predicted probability of persistent sepsis was calculated using the full original model [7]. Subsequently, the full model was updated by re-calibrating the intercept [9]. The discriminative capacity of the updated model was evaluated by calculating the area under the receiving operator characteristic curve (AUC ROC) with corresponding 95% confidence intervals (CI). In a ROC curve, the diagnostic accuracy of a test is depicted by plotting the sensitivity against 1 – specificity. A perfect diagnostic test with 100% sensitivity and 100% specificity will have its coordinates in the upper left corner, resulting in an AUC of 1. Calibration of the updated model was assessed with the Hosmer-Lemeshow (H-L) goodness-of-fit test. With this test, assessments were grouped by decile of predicted probability of continuing sepsis, and the expected number of cases (persisting sepsis) was compared with the observed number in the 10 groups with a χ² test. A p value <0.05 indicates good performance.

The scores calculated using the decision tool were categorized into one of the three original categories (low, intermediate, and high). For the low and intermediate categories, the negative predictive value was evaluated by calculating the probability of not having persisting sepsis associated with a score classified as low or intermediate, respectively. For the scores in the high category, the probability of persisting sepsis was calculated (i.e., positive predictive value).

Normally distributed data were expressed as mean (standard deviation [SD]) and non-normally distributed data as median (interquartile range [IQR]). All statistical analyses were performed with SPSS® software version 20.0 (IBM, Armonk, NY). P values of <0.05 were considered to indicate statistical significance.

Results

A total of 72 adult patients with intra-operatively proved secondary peritonitis underwent surgical intervention during the study period. Three patients died within 24 h after the index operation and were excluded from analysis. The median age of the 69 remaining patients was 56 years (IQR 36–65), 49% were men, and the median APACHE II score was 11 (IQR 5–13) (Table 1). The most common cause of peritonitis was perforation of a visceral organ (68%) followed by ischemia or necrosis of part of the gastrointestinal tract (17%). A laparoscopic procedure was performed in more than half of all patients (36; 52%). Ten of the 69 patients died post-operatively, resulting in an in-hospital mortality rate of 15%, 18% when including the early deaths (13/72). The median length of the post-operative hospital stay of the surviving patients was 13 d (IQR 6–22 d).

Table 1.

Patient Characteristics and Clinical Outcomes (n = 69)

Mean age (y) (IQR)	56 (36–65)
No. male (%)	34 (49)
Median APACHE II score (IQR)	11 ( 5–13)
Etiology of peritonitis (%)
Perforation	47 (68)
Ischemia	12 (17)
Anastomotic leakage	9 (13)
Other	1 ( 1)
Type of surgical approach (%)
Laparotomy	33 (48)
Laparoscopy	36 (52)
Extent of peritonitis at index operation (%)
Local	50 (73)
Diffuse	19 (28)
No. of post-operative imaging studies (range for a patient)	53 ( 0– 4)
No. of percutaneous drainage procedures (range for a patient)	21 ( 0– 2)
No. of re-operations (range for a patient)	11 ( 0– 2)
Median post-operative hospital stay (d) (IQR)	13 ( 6–22)
In-hospital deaths (%)^a	10 (14)

Excluding early (within 24 h of admission) post-operative deaths.

APACHE = Acute Physiology and Chronic Health Evaluation; IQR = interquartile range.

Outcomes

An abdominal focus for the sepsis was diagnosed 32 times in 28 of 69 patients, with four patients having two separate episodes of persisting abdominal sepsis. With respect to the reference standard, the majority of these episodes were diagnosed by imaging studies (21/32; 66%), four episodes were diagnosed by imaging studies with subsequent positive findings at re-operation (13%), three episodes were confirmed intra-operatively without previous imaging (39%), and four patients died within a 10-d period after surgery from presumed abdominal sepsis (13%).

A total of 53 diagnostic imaging studies were performed post-operatively in 37 patients (47 CT scans, 5 ultrasound scans, and 1 magnetic resonance study), with a median number of one imaging study per patient (range 0–4). Positive findings on imaging suggestive of an abdominal focus of sepsis were found in 25 of the 53 studies (47%) and in 22 of 69 patients. Eighteen patients underwent 21 percutaneous interventions. Eleven re-operations were performed in ten patients. Four of these reoperations concerned planned re-laparotomies; all were performed as second-look operations for intestinal ischemia, with positive findings in one. The seven other re-operations were performed in six patients and were indicated by the suspicion of persisting abdominal sepsis with a previous CT scan in four patients, all 7 re-laparotomies having positive findings.

Diagnostic accuracy of the decision tool

A total of 161 assessments were performed using the decision tool, with a median number of 2 (IQR 1–2) per patient (Table 2). Of all assessments, 57 (35%) were indicated by the clinical status of the patient. Using the original model [7], the probability of continuing sepsis for every decision tool assessment was calculated in the present cohort, and the model was updated by re-calibrating the intercept (Appendix 1) [9]. The discriminative capacity of the updated model, expressed as AUC ROC, was 0.79 (95% CI 0.70–0.87), suggesting fair discriminatory capacity (Fig. 2). This AUC was comparable to the discriminative capacity found in the original development cohort (AUC 0.80; 95% CI 0.69–0.82) [7]. Assessment of calibration with use of the H-L test showed adequate goodness-of-fit (p = 0.20).

FIG. 2.

Receiver operating characteristic (ROC) curve showing the sensitivity plotted against 1 – specificity of the decision tool score in present cohort of patients operated on for secondary peritonitis. The area under this curve (AUC) was 0.79 (95% confidence interval 0.70–0.87).

Table 2.

Decision Tool Assessments

Total number of assessments	161
Median number of assessments/patient (IQR)	2 ( 1– 2)
Median decision tool score (IQR)	10 ( 6–19)
Decision tool score by category (%)
Low (0–19 points)	121 (75)
Intermediate (20–40 points)	37 (23)
High (41–60 points)	3 ( 2)

IQR = interquartile range.

A total of 32 assessments were performed on POD 2 with five cases of persistent sepsis. The AUC of these assessments was 0.93 (95% CI 0.84–1.00). Forty-one assessments performed on POD 3, with only one case of continuing sepsis, resulted in an AUC of 0.99 (95% CI 0.95–1.00). The remaining 88 assessments were performed on POD 4–POD 10, with 22 cases of persistent abdominal sepsis. The AUC of these assessments was 0.73 (95% CI 0.62–0.85).

The median calculated score was 10 (IQR 6–19) of a maximum of 60 points. The majority of the decision tool scores were categorized as representing a low probability (0–19 points) of persistent sepsis (121/161; 75%). A score in the intermediate category (20–40 points) was found in 23% of the assessments (37/161), whereas only three assessments (2%) had a score of 41 points or more (high probability of persistent sepsis). The actual incidence of abdominal sepsis per probability score category is shown in Table 3. The incidence rate differed significantly among the three categories (p < 0.001). The negative predictive value of a decision tool score categorized as low probability was 89% (95% CI 82–94), and 65% (95% CI 47–79) for an intermediate score. The positive predictive value of a score in the high category was 67% (95% CI 12–95), as two of the three patients with a high score had persistent abdominal sepsis.

Table 3.

Incidence of Persistent Sepsis, Diagnostic Imaging, and Re-Ooperations Categorized by Decision Tool Score Category

		Actual abdominal sepsis (%)		Diagnostic imaging performed (%)			Re-operation ^b
Score	N	No	Yes	Total	Positive	Negative	Total	Positive	Negative
Low (0–19 points)	121	108 (89)	13 (11)	29	13 (45)	16 (55)	0	–	–
Intermediate (20–40 points)	37	24 (65)	13 (35)	17	7 (41)	10 (59)	5	5 (100)	0
High (41–60 points)	3	1 (33)	2 (67)	3	2 (67)	1 (33)	1	1 (100)	0
Total^a	161	133 (83)	28 (17)	49	22 (45)	27 (55)	6	6 (100)	0

Numbers of diagnostic imaging studies and re-operations differ from those in Table 1, as patients were censored from the analyses after an episode of abdominal sepsis was confirmed.

Excluding planned reoperations.

For the 10 patients who died during the post-operative course (excluding the three early deaths), a total of 19 assessments were done resulting in a median score of 30 (IQR 20–34), compared with an overall median score of 10 (IQR 6–19). The assessments in these patients included all three high scores and eight of the 37 intermediate scores (22%).

Of all 161 assessments, 92 were repeat procedures. Seventy-three of these 92 repeated assessments resulted in a low score. The negative predictive value of low scores after a previous low score (10 positives of 57 assessments) was similar to the low scores following an intermediate score (three positive results of 13 assessments); 83% (95% CI 67–91) vs. 81% (95% CI 54–95) (p = 0.911). For an intermediate score, the positive predictive value (probability of persistent sepsis) was greater if the assessment succeeded a low score (i.e., an increasing score) compared with an intermediate score (50%; 95% CI 20–80 vs. 22%; 95% CI 4–60), although the difference was not statistically significant (p = 0.210).

Potential impact of decision tool on post-operative management

Management decisions with respect to post-operative diagnostic imaging (in the majority followed by percutaneous drainage) and surgical interventions categorized according to the decision tool score are displayed in Table 3. Following 121 decision tool scores in the low category, diagnostic imaging still was performed in 29 instances, which diagnosed an abdominal focus in 13 instances (108/121 or 89% true negative results of management decisions by the decision tool). If the advice of the decision tool for the low category had been adhered to (“no indication for diagnostic imaging”), 16 “unnecessary” imaging studies would have been avoided at the cost of 13 wrongfully omitted studies (missed cases of continuing abdominal sepsis), while reducing the overall number of imaging studies in the total cohort by 59% (29 of 49 studies).

The advice for a patient with an intermediate score was to consider imaging. Following 37 intermediate scores, 17 imaging studies were performed (46%), which were positive for an abdominal infections in seven instances (20% true positives of management decisions by the decision tool). Diagnostic imaging was performed more frequently after an intermediate score than after scores categorized as low (46% vs. 24%; p < 0.001). This is in line with the advice of the decision tool corresponding to the intermediate category (“consider imaging”). Although more imaging studies were performed after an intermediate score than after a low score, the proportion of appropriate imaging studies (i.e., imaging studies with positive findings) was similar (45 vs. 41%; p = 0.806).

All scores in the high category were followed by imaging, in line with the advice of the decision tool, and two of the three imaging studies were positive for an abdominal infection focus.

Discussion

In this study, a decision tool developed to predict the probability of persistent abdominal sepsis in patients operated on for secondary peritonitis was externally validated. The scores showed good calibration and fair discriminative capacity in the new cohort. The performance was similar to that found during internal validation, supporting the external validity of the decision tool. Furthermore, the tool had a good negative predictive value of a score categorized as low (i.e., the probability of not having sepsis) of 89%, whereas the positive predictive value (i.e., the probability of having sepsis) of a score categorized as high was moderate (65%). Given the design of the decision tool, assessments can be done multiple times during the post-operative course. The documented results showed that, irrespective of previous assessments, low scores preserved their negative predictive value. Furthermore, although the difference was not statistically significant, an intermediate score succeeding a low score indicated a greater probability of persistent sepsis than a constant intermediate score, illustrating that the scores reflected a change in clinical condition adequately. All and all, the negative predictive value of a low decision tool score was superior to its positive predictive value at an intermediate or high score, although still not perfect.

The potential of the decision model to guide post-operative decision making to perform diagnostic imaging was evaluated as well. Adherence to the advice of the decision tool for scores categorized as low (omitting diagnostic imaging) would have reduced the number of imaging studies in the total cohort by 59%, at the cost 11% missed cases of persistent sepsis (i.e., false negatives). The advice to perform imaging after finding a high score did seem accurate, although only a few assessments resulted in a score categorized as high.

Despite progress in the initial surgical management of secondary peritonitis, persisting or new-onset abdominal sepsis by an infectious focus during the post-operative course continues to be a problem. Timely and adequate re-intervention is important, but repeated and unnecessary surgical interventions involve additional risks and do not improve outcome [4]. Because of this dilemma, clinical predictors enabling early and correct diagnosis of sepsis are of great interests. There is no consensus among surgeons which variables are most import during post-operative decision making in patients with peritonitis [10]. Multiple widely used tests such as the APACHE II score and the Sepsis-related Organ Failure Assessment (SOFA) score predict death in critically ill patients [11,12[. However, when evaluating their ability to predict the necessity of surgical re-intervention in patients with secondary peritonitis, they perform significantly worse [6]. Long ago, a prediction model specifically designed to identify patients in need of surgical re-intervention was described (Abdominal Reoperation Predictive Index [ARPI]), but the ARPI score has several limitations that have been addressed previously [7]. Multiple clinical variables are described as predictors of the necessity of a re-laparotomy as well, including patient and peritonitis characteristics, surgical variables, and clinical and laboratory variables [13]. Several other markers and variables have been related to recurrent or persistent sepsis in patients with peritonitis, such as the serum procalcitonin ratio, increased intra-abdominal pressure, and assessments done using intra-peritoneal microdialysis [14 –16]. However, these markers lack evaluation of their diagnostic performance and external validation, and not all are widely available. More commonly used diagnostic markers of infectious processes are white blood cell (WBC) count and C-reactive protein (CRP) concentration. During the development of the current decision tool, these markers failed to show adequate association and were dropped. As these inflammatory markers are markedly increased in patients with secondary peritonitis in need of surgery, they are less likely to be informative during the post-operative course.

Although the decision tool showed reasonable external validity with fair discriminative capacity, its diagnostic performance was not good enough to abstain completely from clinical judgment. It is not likely that any single marker or decision tool can replace clinical decision-making by the multidisciplinary team taking care of the critically ill patient. The number of missed cases of persistent sepsis by the decision tool (11% false negatives) may not be low enough for clinical practice and is in need of improvement, preferably by integrating the tool into the team's clinical judgment. No direct comparison of the decision tool's rate of false negative results with that of sole clinical decision-making could be made, as this would require diagnostic imaging in every patient, even when it is not deemed indicated clinically. The ability of the tool to reduce the number of imaging studies was considerable. However, this may not outbalance the number of missed cases of peritonitis. Importantly, liberal use of imaging likely will result in more false-positive cases, as restricted use is associated with more missed cases; these two aspects need to be weighed against each other, and cut-off choices may differ between individual clinicians.

The potential of the decision tool to influence the number of “unnecessary” re-operations (i.e., negative findings) could not be assessed in our study, because only a few patients underwent a re-operation. This low number of re-operations reflects current clinical practice with on-demand re-laparotomy and increased use of percutaneous drainage. The on-demand strategy has become standard for most patients, and progress in imaging techniques, in particular CT, and the introduction of image-guided percutaneous interventions has substituted for some re-operations. Furthermore, as CT imaging is now widely used and almost always precedes re-intervention, a decision tool should be aimed at triage of diagnostic imaging rather than the decision to perform a surgical intervention.

Some limitations of our study need to be addressed. First, the number of positive cases (continuing abdominal sepsis) was small. A larger number of positive cases would have enabled more accurate analyses of the positive predictive value. Therefore, no firm conclusion could be drawn regarding this category. The majority of the decision tool assessments resulted in scores categorized as low. As only a few intermediate scores were found, and even fewer high scores, for these categories, the analyses are less accurate. Another limitation is the risk of verification bias, given our study design. A positive reference standard test was defined as positive findings at imaging or re-operation. Not all patients received the same reference test, as not all patients underwent post-operative imaging or a re-operation. Theoretically, patients with an abdominal infection focus for persistent peritonitis who did not undergo imaging or a re-operation could have been missed as “positives.” However, in clinical practice, such patients probably would deteriorate, with imaging likely being performed.

Vital to the current standard re-operation on-demand approach in patients operated on for secondary peritonitis is vigilant monitoring, with round-the-clock decision-making. To aid this complex process, a decision tool has been developed, which was externally validated in the current study. In patients operated on for secondary peritonitis, the decision tool score predicts with fair accuracy whether persistent sepsis is present. A low score on the decision tool has a reasonable negative predictive value but still is not perfect, and the decision tool cannot substitute completely for clinical decision-making. As imaging has become standard and almost always precedes re-interventions, the step-up-approach of clinical suspicion–imaging–re-intervention with efficient use of diagnostic imaging without impeding timely diagnosis could be facilitated by the decision tool. Future studies should assess its impact on decision-making and subsequent patient outcomes.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Appendix 1.

Full Model on Which the Decision Tool is Based Enabling Calculation of the Probability of Persistent Sepsis

Variable	β
Contamination (diffuse)	0.76695
Defecation (no)	1.46956
Inotropic medication (Yes)	1.40856
Temperature (≤35.5°C or >39°C)	0.91645
Heart rate (>90 bpm)	0.67818
Hemoglobin (≤5.0 mmol/L)	1.09552
Intercept (original)	−4.412803
Intercept (updated)	−3.135282

The predicted probability of persistent sepsis was calculated using the following formula:

P = 1/(1+EXP(-( - 3.135282 + 0.76695 (if diffuse contamination was present during initial surgery) +1.46956 (if no defecation) +1.40856 (if inotropic medication is needed) +0.91645 (if temperature is ≤35.5°C or >39°C) +0.67818 (if heart rate >90bpm) +1.09552 (if hemoglobin is ≤5.0 mmol/L)

References

Solomkin

, Mazuski

, Bradley

, et al. Diagnosis and management of complicated intra-abdominal infection in adults and children: Guidelines by the Surgical Infection Society and the Infectious Diseases Society of America. Surg Infect, 2010; 11:79–109.

Dellinger

, Levy

, Rhodes

, et al. Surviving Sepsis Campaign: International guidelines for management of severe sepsis and septic shock: 2012. Crit Care Med, 2013; 41:580–637.

Sawyer

, Claridge

, Nathens

, et al. Trial of short-course antimicrobial therapy for intraabdominal infection. N Engl J Med, 2015; 372:1996–2005.

van Ruler

, Mahler

, Boer

, et al. Comparison of on-demand vs planned relaparotomy strategy in patients with severe peritonitis: A randomized trial. JAMA, 2007; 298:865–872.

Boermeester

. Surgical approaches to peritonitis. Br J Surg, 2007; 94:1317–1318.

van Ruler

, Kiewiet

, Boer

, et al. Failure of available scoring systems to predict ongoing infection in patients with abdominal sepsis after their initial emergency laparotomy. BMC Surg, 2011; 11:38.

Kiewiet

, van Ruler

, Boermeester

, Reitsma

. A decision rule to aid selection of patients with abdominal sepsis requiring a relaparotomy. BMC Surg, 2013; 13:28.

Collins

, Reitsma

, Altman

, Moons

. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): The TRIPOD statement. BMJ, 2015; 350:g7594.

Janssen

, Moons

, Kalkman

, et al. Updating methods improved the performance of a clinical prediction model in new patients. J Clin Epidemiol, 2008; 61:76–86.

10.

van Ruler

, Lamme

, de Vos

, et al. Decision making for relaparotomy in secondary peritonitis. Dig Surg, 2008; 25:339–346.

11.

Knaus

, Draper

, Wagner

, Zimmerman

. APACHE II: A severity of disease classification system. Crit Care Med, 1985; 13:818–829.

12.

Vincent

, Moreno

, Takala

, et al. The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med, 1996; 22:707–710.

13.

Lamme

, Mahler

, van Ruler

, et al. Clinical predictors of ongoing infection in secondary peritonitis: Systematic review. World J Surg, 2006; 30:2170–2181.

14.

Novotny

, Emmanuel

, Hueser

, et al. Procalcitonin ratio indicates successful surgical treatment of abdominal sepsis. Surgery, 2009; 145:20–26.

15.

Basu

, Pai

. Early elevation of intra-abdominal pressure after laparotomy for secondary peritonitis: A predictor of relaparotomy?. World J Surg, 2008; 32:1851–1856.

16.

Verdant

, Chierego

, De Vos

, et al. Prediction of postoperative complications after urgent laparotomy by intraperitoneal microdialysis: A pilot study. Ann Surg, 2006; 244:994–1002.