Abstract
Sepsis is an inflammatory response triggered by infection, with risk of in-hospital mortality fueled by disease progression. Early recognition and intervention by multidisciplinary sepsis programs may reverse the inflammatory response among at-risk patient populations, potentially improving outcomes. This retrospective study of a sepsis program enabled by a 2-stage sepsis Clinical Decision Support (CDS) system sought to evaluate the program’s impact, identify early indicators that may influence outcomes, and uncover opportunities for quality improvement. Data encompassed 16 527 adult hospitalizations from 2014 and 2015. Of 2108 non–intensive care unit patients screened-in by sepsis CDS, 97% patients were stratified by 177 providers. Risk of adverse outcome improved 30% from baseline to year end, with gains materializing and stabilizing at month 7 after sepsis program go-live. Early indicators likely to influence outcomes include patient age, recent hospitalization, electrolyte abnormalities, hypovolemic shock, hypoxemia, patient location when sepsis CDS activated, and specific alert patterns.
Keywords
Sepsis is an uncontrolled inflammatory response to an infection and can strike anyone at any age. 1 A spectrum of sepsis exists and mortality increases as severity of sepsis increases.2,3 Thus, early recognition of sepsis and early intervention are paramount in improving outcomes. 4 Despite risks of hospital-acquired sepsis, most patients with sepsis initially present to the hospital’s emergency department and possibly have an underlying infectious process. 5
Results of initial routine diagnostics and vital signs offer insights into complexities that affect patients with sepsis, and challenge providers working in multiple-task clinical environments. Recent studies have reported that patients with severe sepsis are likely to have indications of tachypnea and tachycardia with prevalence rates near or above 80%, 6 and may be confounded by electrolyte abnormalities and metabolic disturbance,7-10 hypovolemic shock,11-13 and hypoxemia.14-16 Survivors are likely to experience a much lower physical quality of life17,18; have increased demands for a broad array of concrete, rehabilitative, and therapeutic services 19 ; and pose elevated risk of readmission to hospital.20,21
Given the adverse consequences of sepsis among hospitalized patients, hospital-based sepsis programs enabled by Clinical Decision Support (CDS) systems are being implemented for electronic surveillance of patients, 22 and may be coupled with early warning systems and rapid response teams to achieve earlier intervention. 23 Despite mixed reviews of early generation sepsis CDS, 24 provider alerts have been shown to increase early goal-directed therapy. 25 An ideal sepsis CDS is available in real time and at the point of care, integrating cloud-based CDS with the host electronic health record (EHR) system in a patient-centric clinical workflow. 26 These systems facilitate awareness and transparency of sepsis across the hospital’s patient care units, and have been shown to appropriately increase and accelerate diagnostics and interventions if clinimetric performance is acceptable to providers,27-29 as measured by sensitivity and positive predictive value.
The objective of this study was 3-fold: first, to determine clinimetric performance of a 2-stage sepsis CDS for early recognition of sepsis in the hospital; second, to demonstrate the effectiveness of a new sepsis program; and third, to identify factors (ie, early indicators) that likely influence outcomes and, therefore, become candidate variables for future quality improvement initiatives.
Methods
Patients and Data Collection
This was an observational study of prospectively screened patients admitted to a 284-bed urban, nonprofit community hospital in the United States, with more than 16 000 annual admissions. The hospital had an enterprise EHR system (Millennium: Cerner Corporation, Kansas City, Missouri) and a new sepsis CDS system for early recognition of sepsis (St John Sepsis Rescue Agent: Cerner Corporation, Kansas City, Missouri). The hospital’s sepsis program was designed to improve early recognition and intervention of sepsis enabled by the sepsis CDS. The sepsis protocol was guided by the Surviving Sepsis Campaign resuscitation and management bundles 30 and was communicated broadly by education emphasizing the “magic four” concept, which highlights clinical events and processes key to saving lives (ie, STAT lactic acid, obtain cultures prior to antibiotics, early administration of antibiotics, early administration of fluids in patients with severe sepsis or septic shock). A new 2-stage sepsis CDS was implemented to enable providers to detect and manage patients with sepsis. The alert system applied a binary alarm system paradigm with 2 alerts: (1) indications of systematic inflammatory response syndrome (SIRS; proxy for sepsis) and (2) indications of sepsis (proxy for severe sepsis). 31 Each alert included specific clinical criteria responsible for activation. The cloud-based system, running continuously to monitor patient diagnostics from arrival until discharge, was integrated into the EHR and clinical workflow with the following process: alert notifications with clinical indications were delivered to a designated nurse who became responsible for contacting a provider within 5 minutes of receiving the alert. The provider then conducted a sepsis screening and stratification of patients with sepsis at bedside, and if indicated, submitted orders for the suggested sepsis plan of care, including initiating the initial resuscitation bundle as delineated in the Surviving Sepsis guidelines. A provider–nurse relationship was then established to ensure completion of the resuscitation bundle. 27
The study included adult (≥18 years old) patients hospitalized over 12 months during 2014 and 2015; this observation period encompassed the first year following implementation of the hospital’s sepsis program. The clinical process applied a day-in-the-life of a patient paradigm from arrival to hospital discharge. Patients were candidates if they had a sepsis CDS alert activated, then sepsis rule-in if a provider suspected infection. Patients were excluded from the study if their sepsis CDS first alert activated while they were in an intensive care unit (ICU). Sepsis rule-in patients were grouped into 13 admission cohorts, each encompassing 28 days of contiguous dates of admission. The first admission cohort “0” was considered the baseline because it was the first cohort after implementation of the sepsis program. Source data included EHR registration, vital signs, laboratory, pharmacy, and clinical orders. The US Department of Health and Human Services’ Office for Human Research Protections clarified that quality improvement activities, described herein, often qualify for institutional review board exemption and do not require individual informed consent. 32
Definitions
The primary outcome was defined as a dichotomous variable, where adverse outcome = expired or discharged to hospice, and positive outcome = survive. Sepsis and severe sepsis were defined per the American College of Chest Physicians/Society of Critical Care Medicine Consensus Conference.33,34 Sepsis was defined as suspected or confirmed infection with clinical evidence of SIRS; while severe sepsis additionally required evidence of organ system dysfunction. “Suspected infection” gold standard required at least one microbiology culture be obtained (eg, blood, urine, sputum, oxacillin-resistant Staphylococcus aureus, methicillin-resistant Staphylococcus aureus, or soft tissue) and administration of at least one intravenous or oral anti-infective antibiotics (eg, antibacterial/fungal medication). Thresholds for SIRS were established when ≥3 of the following 5 criteria were satisfied: (1) temperature >38.3°C or <35°C; (2) heart rate >95 beats/min; (3) respiratory rate >22 breaths/min; (4) white blood cell count >12 000 cells/mm3, or <4000 cells/mm3, or >10% immature (band) forms; or (5) glucose 141 < 200 mg/dL. (Note: the threshold for temperature was lowered to <35°C because the surveillance alerting system was unable to distinguish core from noncore temperature.) Threshold for sepsis was established when ≥2 SIRS criteria were present, and ≥1 of the following 4 organ system dysfunction criteria were satisfied: (1) cardiovascular system: systolic blood pressure <90 mm Hg and/or mean arterial pressure <65 mm Hg; (2) tissue perfusion: serum lactate >2.0 mmol/L; (3) hepatic system: total bilirubin: ≥2.0 mg/dL and <10.0 mg/dL; and (4) renal system: serum creatinine: Δ↑0.5 mg/dL from baseline. A look-back period consisted of 12 hours for serum lactate, 30 hours for the other criteria, and 72 hours for Δ↑ serum creatinine. Alert notifications for patients in an ICU location were not delivered to providers.
Elevated Shock Index (heart rate divided by systolic blood pressure, normalized by age and gender) was defined when a patient’s initial Shock Index was ≥95th percentile. 12 The calculation for a corrected apparent strong ion difference (SIDa) was derived from first lab results. Because lactate is an independent determinant of mortality in critically ill patients, the SIDa was “partitioned” into inorganic ion difference [(Na+ + K+ + Mg2+ + Ca2+) − Cl−] and lactate plasma level.35,36 Lactate plasma level was removed from the calculation because it was typically unavailable with initial laboratory results. Because Ca2+ and Mg2+ were not always resulted with the initial laboratory order sets, a summative value of 1.85 mmol/L was incorporated into the SIDa model, based on reported results of ionized calcium and magnesium among critically ill patients: Ca2+ = 1.11 mmol/L 37 and Mg2+ = 0.78 mmol/L. 38 Thus, the calculation for the corrected SIDa = [(Na+ + K+ + 1.85) − Cl−] was established on patient arrival (ie, the first set of tests resulted). Electrolyte abnormality and metabolic disturbance were based on corrected SIDa ≤ 37.0 or ≥40.1 mmol/L. Importantly, the baseline laboratory results used to compute the corrected SIDa preceded orders for blood gas analysis. Hypoxemia was defined as PaO2/FiO2 < 300. 14 Recent discharge was defined as a prior discharge from the same hospital within 30 days of the current arrival date. Patient location when the sepsis CDS first alert activated includes emergency department; medicine (general medicine, oncology, and pulmonary); critical care units, excluding ICU (general medicine, intensive nursing care unit, medical cardiac, and medical stroke and cardiac); surgery (general surgery, joint replacement, surgical neurosciences, and neurology and rehabilitation center).
Statistical Analysis
Data were analyzed retrospectively. A confusion matrix was applied to report sepsis prevalence, sensitivity, and positive predictive values (PPV) for the sepsis program. Unadjusted bivariate analyses applied Fisher exact and χ2 (2-tail, P value) for dichotomous variables in 2 × 2 and 2 × n contingency tables, respectively. Multivariable logistic regression (MLR) was used to identify predictors of the primary outcome. The odds were performed unadjusted, then controlled for demographics and clinical risk factors. Mann–Whitney U test was applied to estimate differences in medians and distributions in MLR residuals, which established a deterministic basis for stratifying the study population into 2 subcohorts, defined by months 1 to 6 and months 7 to 12. Separate MLR was then used to identify predictors of the primary outcome for each subcohort. Metrics were established for observed adverse outcome per 1000 patient days, expected adverse outcome per 1000 patient days, and ratio of observed to expected (O/E) adverse outcomes per 1000 patient days. One-sample Z-test for proportions was used to analyze rates pertaining to observed and expected outcomes over time. All analyses were conducted using SPSS v21 (IBM, Inc., Armonk, New York).
Results
Of 16 527 hospitalizations encompassing 103 013 patient days examined, the cloud-based sepsis CDS screened-in 2108 (13%) patients corresponding to 21 patients per 1000 patient days [(2108/103 013) × 1000]. Of patients with an activated CDS alert, 177 different providers screened and stratified patients with sepsis. Provider compliance with screening patients with a CDS alert was 97% (n = 2035 of 2108 patients); with 3 of 4 (n = 1480 of 2035, 73%) patients screened by providers being suspected of infection, and more than 80% (n = 61 of 73, 84%) of patients bypassed from screening also suspected of infection (P = .043). Thus, sepsis prevalence was 9.3% (n = 1541/16 527) or 15.0 patients per 1000 patient days, where [(n = 1541/103 013 patient days) × 1000]. Figure 1 illustrates the clinical process to rule in/out patients with sepsis. Clinimetric performance of the sepsis CDS was established by applying the 2108 patient-level results in a confusion matrix to derive accuracy metrics of 83% sensitivity (n = 1280 of 1541 patients) and 87% PPV (n = 1280 of 1466 patients).
Patient selection schematic.
Characteristics of the 1541 patients with an activated CDS alert and suspected of infection by providers are shown in Table 1. Nearly all patients arrived to the emergency department. About 1 in 6 patients had been recently (ie, <30 days) discharged from the hospital and were now returning. Initial diagnostics indicated approximately 3 in 4 patients had abnormal corrected SIDa, more than half of patients had an elevated Shock Index, and 1 in 5 patients had hypoxemia. A majority (n = 937, 61%) of patients’ first activated alert was SIRS versus severe SIRS; a small fraction (n = 20 of 604, 3%) of patients with severe SIRS first alert included multiple organ dysfunction. The dominant SIRS alert profile included heart rate and respiratory rate (SIRS: HR & RR) paired with a third SIRS criterion and the next most prevalent SIRS alert profile contained temperature and white blood cell count (SIRS: Temp & WBC). The dominant severe SIRS alert profile included perfusion (ie, lactic acid ≥2) organ dysfunction followed by cardiovascular organ dysfunction. Two thirds of patients were alerted while in the emergency department. Total patients days were 15 279 days among 1541 patients with sepsis rule-in; median 7.0, 95% confidence interval (CI; 3.9 to 12.0) days.
Patient Characteristics by Outcome. a
Abbreviations: CDS, clinical decision support system; CV, cardiovascular; HR, heart rate; IQR, interquartile range; RR, respiratory rate; SIDa, corrected apparent strong ion difference; SIRS, systemic inflammatory response syndrome; Temp, temperature; WBC, white blood cell count.
Recent Discharge: discharged from hospital within 30 days of the current hospitalization. Clinical Results: first diagnostics resulted. Patient Location first activated alert includes emergency department; medicine (general medicine, oncology, and pulmonary); critical care units, excluding the intensive care unit (general medicine, intensive nursing care unit, medical cardiac, and medical stroke and cardiac); surgery (general surgery, joint replacement, surgical neurosciences, and neurology and rehabilitation center).
Table 1 reports bivariate association of patient characteristics with adverse outcome of “expired or discharged to hospice.” Results showed patients were more likely to be older, hospitalized recently, have an abnormal corrected SIDa, elevated Shock Index, and hypoxemia. These patients were more likely to have either sepsis CDS first alert SIRS: HR & RR paired with a third SIRS criterion, or Severe SIRS indicating cardiovascular or renal organ dysfunction. Patients whose first alert activated while in medicine or critical care non-ICU patient care units were more likely to experience an adverse outcome.
Table 2 illustrates the effects of risk factors on primary outcome. The MLR model demonstrated high discriminatory characteristics (C-statistic = .81, 95% CI = .78 to .84) for predicting outcomes. The model illustrated predictive qualities of specific SIRS patterns generated by the sepsis CDS for early recognition, particularly because alerts activated before organ dysfunction. The likelihood of adverse outcome was diametrically opposite among patients with first alert SIRS: HR & RR (odds ratio [OR] = 1.56, 95% CI = 1.01 to 2.42) and patients with SIRS: Temp & WBC (OR = 0.42, 95% CI = 0.21 to 0.86) after controlling for other factors.
Factors of Adverse Outcomes. a
Abbreviations: CDS, clinical decision support; HR, heart rate; RR, respiratory rate; SIDa, corrected apparent strong ion difference; SIRS, systemic inflammatory response syndrome; Temp, temperature; WBC, white blood cell count.
Admission cohort months 0 to 12 (N = 1541). Multivariable Logistic Regression model constant = −6.528. Model performance: Nagelkerke R2 = 0.27; Hosmer and Lemeshow test χ2 = 6.90, df = 8, P = .55; C-statistic = 0.81, 95% confidence interval (0.78 to 0.84). Age: one unit increase. Recent Discharge: patient discharged from hospital within 30 days of current hospitalization. Clinical Results: first diagnostics resulted at presentation include SIDa, Shock Index, and Hypoxemia. Patient Location at the patient’s first activated alert includes: Medicine (general medicine, oncology and pulmonary) and Critical Care units (general medicine, intensive nursing care unit, medical cardiac, and medical stroke and cardiac); ICU excluded.
Clinimetric performance was examined further by analyzing metrics for observed and expected adverse outcome per 1000 patient days across admission cohorts (Figure 2). Estimates from the MLR model indicate the predicted rate of adverse outcomes declined 17.3%, from 18.5 to 15.3 patients per 1000 patients days (P < .001), while the observed rate of adverse outcomes per 1000 patient days declined by 36.3%, from 24.0 to 15.3 (P < .001). The metric O/E adverse outcomes per 1000 patient days in the month 0 baseline had an O/E = 1.30; then O/E fluctuated during months 1 to 6 before materializing and stabilizing O/E ≤1 from months 7 to 12. Based on these findings, 2 patient cohorts were established: months 1 to 6 (cohort H1) and months 7 to 12 (cohort H2). Analysis of MLR residuals grouped by cohort H1 and cohort H2 indicated a significantly tighter distribution of residuals among patients in H2 compared to H1 (P < .01), thereby affirming the estimated difference in outcomes between these cohorts.
Observed and expected outcomes by admission cohort.
Two separate MLR models were used to predict effects of risk factors on outcomes for cohort H1 and cohort H2 (Table 3). Both models demonstrated high discriminatory characteristics on predicting outcomes after controlling for other risk factors, as reported by the C-statistic. The key differences between models were 2-fold. First, risk factor of corrected SIDa: cohort H1 (OR = 0.57, 95% CI = .36 to .88) compared to cohort H2 (OR = 1.88, 95% CI = 1.10 to 3.23) illustrates a reversal in odds of corrected SIDa on adverse outcome. Second, sepsis CDS alerts with specific criteria: whereas none of the 5 alerts was significant for cohort H1, 4 of 5 first alerts with specific criteria were significant for cohort H2 (P < .05); these latter alerts were defined as SIRS: HR & RR, SIRS: Temp & WBC, Sepsis: cardiovascular organ dysfunction, and Sepsis: renal organ dysfunction.
Factors of Adverse Outcomes by Admission Cohort. a
Abbreviations: CDS, clinical decision support; HR, heart rate; RR, respiratory rate; SIDa, corrected apparent strong ion difference; SIRS, systemic inflammatory response syndrome; Temp, temperature; WBC, white blood cell count.
Multivariable Logistic Regression (MLR) model for each patient cohort; MLR model cohort months 7 to 12 constant = −7.987, model performance: Nagelkerke R2 = 0.31; Hosmer and Lemeshow test χ2 = 8.61, df = 8, P = .38; C-statistic = 0.83, 95% confidence interval (0.80 to 0.87). Age: one unit increase. Recent Discharge: patient discharged from hospital within 30 days of current hospitalization. Clinical Results: first diagnostics resulted at presentation include SIDa, Shock Index, and Hypoxemia. Patient Location at the patient’s first activated alert includes: Medicine (general medicine, oncology and pulmonary) and Critical Care units (general medicine, intensive nursing care unit, medical cardiac, and medical stroke and cardiac); ICU excluded.
Discussion
This multidisciplinary hospital-wide sepsis program led to a significant improvement in patient outcomes. Despite substantial variation in outcomes during the initial 6 months after launch of the sepsis program, gains materialized, and were sustained during the second half of the year. Change in the observed adverse outcome rate was twice the magnitude than expected (ie, Δ↓ 36% observed versus Δ↓ 17% expected patients per 1000 patient days) at baseline and compared to the study average. This positive impact was driven by the sepsis program’s purpose of risk reduction through early recognition and treatment, and corresponds to a 30% improvement in sepsis O/E outcomes, similar to other sepsis quality improvement initiatives. 39
The 2-stage sepsis CDS is an effective approach toward detecting and facilitating the management of patients with sepsis in a non-ICU setting by achieving broad adoption among providers, a finding shared by others who have implemented similar systems. 40 Definitions of sepsis CDS SIRS and severe SIRS alerts are robust,30,33,34 as reported by clinimetric performance results in this study and previous publications.27,31 Fidelity of the sepsis CDS was high, as indicated by the fact that 97% patients detected by the cloud-based sepsis CDS surveillance system were screened and stratified by providers; a small percentage of patients “bypassed” from the secondary screening were mostly suspected of infection by providers. This high level of system utilization was likely predicated on provider acceptance of the sepsis program, the sepsis CDS alert robust definition and high predictive characteristics, and the system being integrated into clinical workflow coupled with institutional requirements for providers to immediately examine identified patients using a standardized, evidence-based sepsis screening and stratification protocol.
More than 6 in 10 patients’ first sepsis CDS alert indicated SIRS. A substantial finding suggests a specific SIRS alert definition that combines heart rate and respiratory rate is dominant and responsible for first alert activation among many patients in non-ICU settings; this specific SIRS alert definition is also predictive of adverse outcomes when controlling for other factors. Therefore, early recognition systems should include a SIRS alert containing heart rate and respiratory rate as being clinically actionable because this pattern is also highly prevalent among patients with severe sepsis in the ICU. 6
Other patient factors likely to influence outcomes are identifiable at presentation, such as recent hospitalization and diagnostics indicating electrolyte abnormalities, hypovolemic shock, and hypoxemia, thereby corroborating findings of other studies.13,14,20 These key predictive factors can facilitate clinical assessment and management of patients with sepsis located on various hospital wards. Sepsis programs with enabling CDS technology are positioned to further improve outcomes by using predictive clinical events to trigger a critical care consult, especially among patients at risk of deterioration.
There are some limitations to this study to consider. First, the setting was a single-center 284-bed urban nonprofit community hospital within a regional health system in the United States; this setting may not be generalizable to other clinical settings. Second, although the CDS was built on current clinical evidence and good best practices for a binary alarm system with cross-check and validation functionality, the application was developed to promote broad adoption across the hospital’s multidisciplinary provider and nursing groups; other health systems may have different circumstances. Third, the sepsis program’s impact on outcomes may not be fully known because the study began immediately after the program’s go-live date; however, the longitudinal study design permitted a case-control definition of 13 distinct admission cohorts over a one-year time frame, with a notable reduction in risk and observed adverse outcomes among patients during the second half of the program year compared to earlier observation periods.
An area of future research includes a focus on improving outcomes of patients with sepsis by incorporating key risk factors with historical odds of adverse outcomes into the patient’s alert notifications delivered to clinicians, thereby documenting actionable information to inform medical decision support at the point of care. Risk factors targeted for inclusion into the next quality improvement phase include: first, patients who were recently hospitalized and now presenting to the emergency department, and have an sepsis CDS alert activated; second, patients whose first alert indicates SIRS: HR & RR combination; and third, patients with an activated alert and experiencing complications of respiratory distress, metabolic disturbance, and shock. A 2-stage sepsis CDS is applicable and facilitates future study.
Conclusion
Early recognition of sepsis reduces risk of adverse outcomes. A multidisciplinary sepsis program enabled by a 2-stage sepsis CDS expedites accurate detection, stratification of patients with sepsis, and intervention. Achieving substantive outcomes requires patience for program effects to materialize and stabilize, with implications for translational research and quality improvement initiatives.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors received no financial support for the research, authorship, and/or publication of this article.