Abstract
Abstract
Background:
Necrotizing soft tissue infections (NSTIs) are associated with a high mortality rate; however, there is no uniform way to categorize the severity of this disease early in its course. The goal of this study was to develop a clinical score based on data available at the time of initial assessment to aid in stratifying patients according to their risk of death.
Methods:
A cohort of all 350 patients admitted with NSTI to two institutions over a nine-year period was examined retrospectively. Using random split sampling, two datasets were created: Prediction (PD) and validation (VD). Multivariable stepwise regression analysis of the PD identified independent predictors of death using data available at the time of admission. Model performance was evaluated for accuracy, discrimination, and calibration. A clinical score to predict death was created, and using the Trauma and Injury Severity Score (TRISS) methodology, the score was validated on the VD (z-statistic).
Results:
Six admission parameters independently predicted death: Age > 50 years, heart rate > 110 beats/min, temperature <36°C, white blood cell count > 40,000/mcL, serum creatinine concentration > 1.5 mg/dL, and hematocrit > 50%. The accuracy of this model was 86.8%; the area under the receiver-operating characteristic curve was 0.81, and the Hosmer-Lemeshow statistic was 11.8. Additionally, the score had excellent performance in evaluation on the VD (z-score/statistic 0.23 to − 0.83).
Conclusion:
A clinical score that categorizes patients with NSTI according to the risk of death was created. It uses simple variables, all available at the time of first assessment. It stratifies patients according to disease severity and can guide the use of aggressive or novel therapeutic strategies and selection of patients for clinical trials.
Over the last decade, there has been an interest in understanding NSTI better. Some investigators have focused on methods that aid in early diagnosis so that surgical debridement can be accomplished promptly [20,21], whereas other researchers have focused on identifying patients at higher risk of death [4,6,7,10–13]. Although several predictors of death have been identified, differences in patients across series limit their broad applicability.
The primary objective of this analysis was to create a simple clinical score to aid in the prediction of death in patients with NSTI at the time of first assessment. In contrast to prior works, we expanded the scope of our cohort beyond that of a single center to incorporate a wider range of etiologies and clinical practice strategies, thus allowing the development of a prediction rule that will have broader utility.
Patients and Methods
Subjects
Patients with NSTIs who were admitted to Harborview Medical Center (HMC) in Seattle, WA, or to Lyndon Baines Johnson General Hospital (LBJGH) in Houston, TX, over a nine-year period (1995–2003) were included in the study. Both HMC and LBJGH are county hospitals and tertiary referral centers serving patients from a wide area, including Washington, Wyoming, Alaska, Montana, and Idaho (HMC); and Texas. In both centers, patients with NSTI were treated using a standardized approach that included early and scheduled repeated debridements in concert with aggressive physiologic and nutritional support in the intensive care unit. Empiric broad-spectrum antibiotics were initiated on all patients, with de-escalation once the predominant organism(s) was identified. Hyperbaric oxygen therapy was not used routinely.
Subjects were identified by first screening administrative discharge data in each institution. Discharges (or deaths) with any one of several ICD-9 discharge codes (040.0, 728.86, 785.4, 608.83, 682.9, 686.0) underwent formal medical record review to establish eligibility. Patients were deemed to have an NSTI if there was necrotic tissue at the time of surgery (as documented by the operating surgeon) or necrotic tissue (fascia or muscle) in pathology specimens. If the patient was eligible, the medical record was abstracted using a standardized case report form. This study was approved by the Institutional Review Board at each hospital.
Patient and disease-specific parameters
A detailed chart review was carried out by trained abstractors to capture information on baseline patient characteristics, including co-morbidities, the degree of physiologic derangement, and the primary site and cause of the infection. The physiologic parameters retrieved were systolic blood pressure (SBP), heart rate (HR), and temperature (°C) at the time of initial presentation to the emergency department. Laboratory values included admission white blood cell (WBC) count; hematocrit (or hemoglobin); serum bicarbonate, sodium, glucose, and creatinine concentrations; and the anion gap. The site of infection was classified as head and neck, chest, abdomen, perineum, buttocks, or extremities, whereas the cause of infection was classified as chronic wound/ulcer, injection during intravenous drug use, boil/furuncle, bite, idiopathic, trauma, postoperative incisional infections, perirectal abscess, or other.
The primary endpoint of interest was mortality status at the time of hospital discharge. Subjects were not followed beyond discharge.
Data analysis
Data from the two participating centers were pooled into a single dataset, and one-half of the 350 subjects were selected randomly to form the prediction dataset (PD), whereas the remaining half comprised the validation dataset (VD). We specifically elected not to perform model development on all data from one institution and validation on data from the other. When comparing the baseline characteristics of the patients at the different institutions, the main difference was a higher proportion of patients with polymicrobial and clostridial infections at HMC, as well as a higher proportion of staphylococcal infections at LBJGH. Baseline and physiological variables were evenly distributed among the two groups, and on initial analysis (results not shown), there was no hospital effect on the mortality rate. These institutions were selected because each has a significant volume of disease, yet the distribution of etiologies and microbiology differ considerably. Thus, the combined cohort provides a more representative sample of NSTI, enhancing the ability to use the clinical prediction rule in other centers.
Prediction model development
Using the PD, we first evaluated the relation between several admission variables and death using the χ2 test for categorical variables and the t-test for continuous variables. All variables with α < 0.2 on univariable analyses and those found important in prior analyses were submitted to multivariable analyses. A backward, stepwise logistic regression model identified the variables most strongly associated with death. Model parameters for entry and exit into the model were α = 0.1 and α = 0.2, respectively. To identify discrete thresholds for many of the continuous parameters (age, SBP, HR, temperature, WBC count, hematocrit, bicarbonate, sodium, glucose, creatinine, and anion gap), we allowed a series of thresholds to compete in the model. For example, binary variables representing each age (>30, >40, >50, >60 years) were assessed for the strength of their association, and only the variable meeting the parameter selection criteria specified above was retained. The performance of the model was then evaluated by calculating its discrimination and calibration. Discrimination is the ability to distinguish between those who survive and those who die and was estimated using the concordance index, also known as the c-statistic or the area under the receiver-operating characteristic (ROC) curve. Calibration was measured by the Hosmer-Lemeshow statistic (goodness-of-fit test) and represents the model's ability to classify outcomes across all risk strata accurately.
Development and validation of the score
An ordinal scale was created by converting the exponential regression coefficients of the model parameters into integers and then taking their sum. For simplicity and easy bedside use, the score was categorized into three strata, each associated with a progressive increase in the mortality rate.
Despite the good performance of the model from the dataset in which the score was derived, we wanted to evaluate the performance (validity) of the actual score. To validate the score, we tested it on the VD using a previously described approach comparing observed and expected outcomes using the z-statistic (22), defined as follows:
In essence, the z-statistic allows one to assign a probability that the difference in observed and expected results is attributable to chance alone. If the z-statistic is in the range of −1.96 to 1.96, this probability is greater than p > 0.05, permitting the inference that the observed and expected results are no different [22]. All statistical analyses were performed using Stata version 8 (Stata, College Station, TX).
Results
The prediction dataset (PD) consisted of 175 patients with a hospital mortality rate of 16.6% (29 patients). Of these, 141 patients (80.1%) were treated at HMC. Tables 1 and 2 describe the baseline clinical and physiologic characteristics of the cohort and the relation of these characteristics to death. Table 3 lists the microbiological findings in patients for whom there was information regarding cultures.
Some patients had more than one comorbidity.
Evaluation of the distribution of either site or cause of infection and death.
Results are given as mean (standard deviation).
Prediction model development
On the basis of the results of the univariable analyses and the approach described in Patients and Methods, we evaluated the association of age, chronic obstructive pulmonary disease (COPD), heart disease, site of infection, HR, temperature, WBC count, hematocrit, serum bicarbonate and creatinine concentrations, and anion gap with death in a backward stepwise logistic regression model. Six parameters were associated independently with death: age > 50 years, HR > 110 bpm, temperature < 36°C, WBC >40,000/mcL, serum creatinine concentration > 1.5 mg/dL, and hematocrit >50%. The strength of the association between each parameter and death is represented as an odds ratio (OR) with 95% confidence intervals (CI) in Table 4. The Hosmer-Lemeshow statistic was 11.8, and the area under the ROC curve was 0.81, suggesting good calibration and discrimination, respectively. Table 5 illustrates the number of patients in each dataset presenting with these independent predictors of death.
Development of the clinical prediction rule
The score was then derived by converting into integers the coefficients of each variable from the regression model. Three minor and three major predictors were identified (Table 6). The presence of any of the minor predictors (HR > 110 bpm, temperature <36°C, and creatinine concentration >1.5 mg/dL) added one point to the total score, whereas the presence of any major predictor (age > 50 years, WBC count > 40,000/mcL, and hematocrit > 50%) added three points to the total score, yielding a possible range of 0 to 12. To simplify the score for clinical use, three strata were created, each associated with a given risk of death.
Model validation
As expected after random split sampling, the mortality rate in the validation dataset was similar to that in the prediction dataset (PD 16.6%, VD 18.8%; p = 0.58). To validate our score, we compared the numbers of predicted and observed deaths using the validation dataset. As demonstrated in Table 7, the number of deaths observed in each risk stratum was not different statistically from that predicted.
A z-score between 1.96 and −1.96 assumes no statistical difference between the two datasets when assigning death, which is interpreted as adequate performance after internal validation.
Discussion
This is the first clinical score designed to predict death in patients with NTSI. It is derived from a large cohort of patients from two tertiary referral institutions. It uses simple variables, all available at the time of admission, and is able to stratify patients accurately according to disease severity. The score incorporates six parameters: HR > 110 bpm, temperature < 36°C, serum creatinine concentration > 1.5 mg/dL, age > 50 years, WBC count > 40,000/mcL, and hematocrit > 50%. Three risk strata were identified on the basis of mortality risk and validated using a split sample technique.
Many reports have described predictors of poor outcome in patients with NSTI. These include parameters related to baseline characteristics, extent of physiologic derangement, culture results, site of involvement, and treatment strategies (time to and number of procedures, body surface area of debridement, and use of hyperbaric oxygen) [2–19,23]. However, many of these factors are not predictive in all series, and some of them are not available until the course of the disease has already been established. For example, in our prior work, we demonstrated the adverse prognostic importance of clostridial infections [13]. However, in many cases, the organism may not be isolated, or if it is isolated, the information is unavailable for the first 24–48 h after admission, limiting the potential impact on early decision making. These observations and the value of being able to identify high-risk patients early in the course were the primary motivation for the development of our clinical prediction rule.
This score allows early classification of patients according to disease severity and uses simple, readily available parameters. The advantages of the use of this prediction rule are several. First, it provides an opportunity to identify patients at high risk of death who, when evaluated at a center with limited resources, might best be transferred to a center where adequate critical care support is available. Such is the case for patients classified as having stage II or III NSTI, although these recommendations may differ according to the hospital's volume and the clinician's experience. These data also can be used to identify a subset of high-risk patients in whom surgical decision making may present the challenge of competing options such as primary amputation for definitive source control or attempts at limb salvage with repeat operative debridements. Identification of this subset of patients enables better-focused research efforts that can be designed to evaluate this question specifically. With the availability of novel interventions (e.g., hyperbaric oxygen), a prognostic score will allow the physician to weigh the relative risks and benefits of an intervention better, particularly when considering an additional invasive procedure or transporting the patient to another institution where these novel interventions are available. Finally, and equally important, the score can be used for comparisons across institutions and over time, providing a more formal means to assess the impact of interventions.
Mortality rates across series differ markedly [2–13], an effect likely attributable to the heterogeneity of the patient populations studied. For example, the pooled mortality rate from a large series of patients with perineal infections (Fournier gangrene) was 16% in a recently published literature review [24], which is lower than that of a large published series of patients with predominantly extra-perineal NSTIs [6,11]. Another example is the difference in the mortality rate in series with a high percentage of patients using intravenous drugs or populations with a high incidence of diabetes mellitus (28% vs. 12%, respectively) [7,25]. The generalizability of our score is improved significantly by including a large cohort of patients from different institutions. This causes the demographic and clinical characteristics (co-morbidities, site and cause of infection) to be distributed more evenly, so that our results become more applicable to patients cared for at other centers. It should be noted, however, that the generalizability of these results also depends on management by the same standards herein described. Specifically, early debridement (within 24 h of admission) was achieved in many of our patients (73%), and when surgical treatment was not performed until later, this was related to delay in diagnosis and time to surgical consultation by other admitting services.
The score derived from this study fulfills many of the standard criteria required for adequate scoring systems. Not only is it relevant and generalizable, as stated above, but it is simple and practical because it uses straightforward variables, all of which are readily available at the time of first admission in most centers. This finding compares favorably with prognostic scores that, although widely validated, are derived from multiple variables, some of which are not obtained routinely at first assessment, making their use cumbersome outside clinical trials. Most importantly, the model has good performance and is able to classify patients accurately when tested using a split sample validation technique [26–28].
This study has several limitations. Because of its retrospective nature, it is difficult to distinguish results from NSTI and those from other complicated skin and soft tissue lesions such as diabetic foot infections. Additionally, our ability to identify patients with an NSTI using ICD-9 codes derived from administrative discharge data might provide a biased sample of subjects. However, our selection of ICD-9 codes was overly inclusive, with patients not meeting the criteria being excluded by chart review. Also, there is a strong correlation between our data and the diagnostic criteria proposed by Wall et al. [20], providing further evidence that we are capturing a population highly representative of those with NSTI. This analysis also suffers from the limitations associated with internal validation of the prognostic model. Clearly, this model requires external validation in other series, ideally through multicenter experience, and we welcome future efforts in this area. Further, many of the prognostic variables are relatively extreme, and it is possible that these data add little to an experienced physician's impression of prognosis. However, the strength of this tool derives from the interactions between the presence of more than one variable (some more common than others) rather the presence of one isolated and maybe uncommon characteristic. Additionally, many of these patients might first be assessed in environments where no such expertise exists; thus, this prognostic score provides an objective means to assess prognosis and might affect the decision to transfer a patient.
In summary, we present a simple score to help stratify risk in patients with NSTI. As always, the use of prediction models should supplement, rather than replace, clinical expertise. Nevertheless, this simple scheme identified patients at risk for an adverse outcome who might benefit from transfer to an institution with adequate resources; more aggressive surgical debridement; and the use of alternate strategies where weighting of the risks and benefits is complex.
Footnotes
Author Disclosure Statement
We certify that we all have participated sufficiently in the work to take public responsibility for the appropriateness of the experimental design and method and the collection, analysis, and interpretation of the data. We have no financial conflicts to disclose.
Presented at the 25th Annual Meeting of the Surgical Infection Society and the International Sepsis Forum, Miami, Florida, 2005. Recipient of the Resident=Fellow Presentation Award of the Surgical Infection Society.