Validation of the Intensive Care Unit Early Warning Dashboard: Quality Improvement Utilizing a Retrospective Case

Abstract

Introduction: Risk stratification with the Modified Early Warning System (MEWS) or electronic cardiac arrest trigger (eCART) has been utilized with ward patients to preemptively identify high-risk patients who might benefit from enhanced monitoring, including early intensive care unit (ICU) transfer. In-hospital mortality from cardiac arrest is ∼80%, making preventative interventions an important focus area. ICUs have lower patient to nurse ratios than wards, resulting in less emphasis on the development of ICU early warning systems. Materials and Methods: Our institution developed an early warning dashboard (EWD) identifying patients who may benefit from earlier interventions. Using the adverse outcomes of cardiac arrest, ICU mortality, and ICU readmissions, a retrospective case–control study was performed using three demographic items (age, diabetes, and morbid obesity) and 24 EWD measured items, including vital signs, laboratory values, ventilator information, and other clinical information, to validate the EWD. Results: Ten statistically significant areas were identified for cardiac arrest and 13 for ICU death. Identified items included heart rate, dialysis, leukocytosis, and lactate. The ICU readmission outcome was compared to controls from both ICU patients and ward patients, and statistical significance was identified for respiratory rate >30. Discussion: With several statistically significant data elements, the EWD parameters have been incorporated into advanced clinical decision algorithms to identify at-risk ICU patients. Conclusion: Earlier identification and treatment of organ failure in the ICU improve outcomes and the EWD can serve as a safety measure for both at-risk in-house patients and also extend critical care expertise through telemedicine to smaller hospitals.

Introduction

Intensive care unit (ICU) beds are at a premium due to limited availability, high cost, and a shortage of trained intensive care physicians and nurses. Based on a 2007 census, there were 67,357 ICU beds¹ in the United States with average occupancy of 68% in 2005.² ICU capacity, however, is often limited by the number of qualified staff as opposed to merely the actual bed space. The daily cost varies, but the total cost of ICU care in the United States exceeded $55 billion dollars in 2005, and has continued to rise over the last decade.² With the increased average age of the U.S. population increasing the need for ICU beds, coupled with an intensivist shortage, the necessity of telemedicine and telecritical care has increased to meet the national demand.

Technology has significantly improved clinical outcomes in the field of patient monitoring over the last 15 years. Electronic early warning algorithms, including the Modified Early Warning System (MEWS), have been utilized since 1999 as a result of the Audit Commission report on variability between hospitals.³ Before MEWS, subjective clinical judgment alone determined who received ICU resources.⁴ Furthermore, accurate predictors of adverse outcomes have been sought with varying success. These include simple laboratory values (such as lactate), vital sign measurements, the effect of changes in parameters (including central venous pressure cardiac output, lactate, brain natriuretic peptide [BNP], and blood urea nitrogen), as well as standardized illness severity scoring systems (such as the Acute Physiology and Chronic Health Evaluation [APACHE] score). Thus far, however, there is no universal predictor of adverse outcome; therefore, the hope is that combining different parameters may help to define an at-risk physiological state, in which intervention may improve outcomes.⁵

MEWS provides an objective manner of triaging high-risk ward patients who require earlier ICU transfer and early intervention. MEWS utilizes respiratory rate, heart rate, systolic blood pressure, urine output, temperature, and neurological status.³ Gardner et al., evaluated MEWS in 334 surgical patients and 57 (17%) met their predefined ICU transfer criteria of a MEWS score ≥4.³ In this patient review, only 16 (5%) were transferred to the ICU. This article reported MEWS score (sensitivities and specificities) as a predictor of meeting the hospital's ICU admission criteria, but there was no control group of patients who were not monitored by MEWS for comparison. There were also no reported patient-centered outcomes such as mortality. Subbe et al., provided a similar evaluation of MEWS for medical admissions, evaluating 709 of them and utilizing a MEWS ≥5 as their ICU transfer trigger. In this retrospective review, the author reported that it was unclear if early warning affected outcomes.⁶

Later studies, however, have shown that automated early warning systems are a benefit. Moon et al. 2011, performed an 8-year audit of patients transferred to the ICU following cardiopulmonary resuscitation and found that MEWS scores did correlate with acuity and ultimately mortality.⁷ They compared a 4-year pre-MEWS period (2002–2005) to a 4-year MEWS capable period (2006–2009) and reported a reduction of in-house cardiac arrest from 0.4% to 0.2% (p = 0.004) and decreased mortality from 52% to 42% (p = 0.05).⁷ Huh et al., defined “calling criteria” as clinical judgment used by nurses and physicians to determine the acuity necessary to transfer a patient to the ICU. They compared “calling criteria” to an electronic medical record (EMR) system with criteria, including mean blood pressure, respiratory rate, pulse rate, PaCO₂ and glucose.⁸

Utilizing the EMR resulted in a lower 28-day mortality: 10.5% versus 26.7% p = 0.022.⁸ Lee et al., also showed a mortality benefit from early intervention⁹ by evaluating timing of medical emergency team (MET) interventions through a retrospective analysis of 525 critically ill cancer patients over a 2-year period. They defined early intervention as <1.5 h from “physiological derangement” and late intervention as >1.5 h and found that the early intervention group had a lower 30-day mortality (29% vs. 55.3%; p < 0.001).⁹ Delayed MET activation and intervention in the form of Rapid Response Teams have been associated with higher 30-day mortality.¹⁰ A similar warning program is the electronic cardiac arrest trigger (eCART), which expanded over MEWS by utilizing laboratory values, vitals, and demographics compared to vital-based MEWS.⁴ This system was developed on the premise that even a modest delay of a few hours in ICU transfer resulted in significant morbidity and mortality. For predicting cardiac arrest, MEWS >2 has a 67% sensitivity and 71% specificity, MEWS >3 is 39% sensitive and 90% specific, compared to eCART >6, which is 89% sensitive and 52% specific.⁴ eCART also detected cardiac arrest earlier than MEWS (22 h vs. 37 h).

The concept that early recognition and intervention results in better outcomes are intuitive, but logistically creating a model that determines which patient is at high risk is the challenge. The aforementioned studies have provided very interesting data regarding the effectiveness of various electronic identification algorithms designed to augment clinical judgment. Until now, however, there has not been significant study involving early warning systems in the critically ill patient population. This study is a retrospective analysis to validate the algorithms used in the early warning dashboard (EWD). It evaluates 23 online individual data elements composing our algorithms and four test data element items for future algorithms, against the adverse outcomes of ICU death, cardiac arrest in the ICU, and ICU readmission. To our knowledge, this is the first study evaluating an electronic early warning system for ICU patients.

Materials and Methods

Purpose

Research data were derived from an approved Naval Medical Center San Diego IRB protocol. The study aim is to validate the individual data elements that comprise the algorithms used in the Intensive Care Unit Early Warning Dashboard (EWD) and test four new elements that could be incorporated in future algorithms. The EWD uses clinical decision-making algorithms based on data obtained from the EMR to identify patients at risk for adverse outcomes.

EWD Development

Working with CliniComp^®, the algorithms were coded to create a computer-based dashboard linked to the Navy Medicine inpatient electronic medical record (Essentris). The algorithms were arranged into the following categories: Sepsis-related Organ Failure Assessment score, electrolytes, bleeding, glucose, sepsis, acute respiratory distress syndrome, ventilator weaning, cardiac markers, respiratory depression, and MEWS (Fig. 1). Algorithms are composed of several different variables to include vital signs, laboratory values, and other assessments. The EWD also allows for changes in parameters (or a specified rate of change) to act as a variable. Scores for each algorithm are automatically calculated and updated on the EWD (Table 1). If a preset value is exceeded, the EWD can then flag either a yellow or red warning for that particular category. Initial criteria used were based on prior algorithms developed by CliniComp and the Veteran's Administration, a literature search for risk factors associated with adverse outcomes, and consensus of opinion from providers in critical care, emergency medicine, and cardiology. The EWD was then initiated in test mode monitoring patients currently in our ICU without driving patient interventions.

Fig. 1.

Early Warning Dashboard Patient names were blacked out in accordance with Health Insurance Portability and Accountability Act (HIPAA).

Table 1.

Components of Algorithms

ALGORITHMS	ALGORITHMS (TIME OF STUDY)	EXPANDED ALGORITHMS (CURRENT OPERATIONAL)	DATA ELEMENTS
Bleed	X	x	Heart rate, INR, hematocrit drop, platelet drop
Glucose	X	x	Blood sugar (>180, <70)
Electrolytes	X	x	Potassium and sodium
Cardiac markers	X	x	Troponin, CKMB, BNP, heart rate
Vent wean	X	x	Static compliance, PaO₂/FiO₂, respiratory rate, minute ventilation
ARDS	X	x	Tidal volume, plateau pressure, pH, respiratory rate (observed), PaO₂/FiO₂, tidal volume
SEPSIS	X	x	WBC, procalcitonin, bands, temperature
SOFA	X	x	PaO₂/FiO₂, SaO₂/FIO₂, platelets, bilirubin, hypotension, vasopressors, Glasgow Coma Scale, creatinine, urine output
Resp	X	x	PaO₂/FiO₂, SaO₂/FIO₂
Flu (influenza)		x	Influenza swab, CPK, temperature >100.5, BMI
Coag		x	Platelets
Cardio		x	Hypotension, vasopressor
Liver		x	Bilirubin
CNS		x	Glasgow Coma Scale
Renal		x	Creatinine, urine output
MEWS	X		Respiratory rate, heart rate, systolic blood pressure, urine output, temperature, neurological status

See Supplementary Appendix for Precise Scoring Systems for each algorithm.

ARDS, acute respiratory distress syndrome; BMI, body mass index; BNP, brain natriuretic peptide; CKMB, creatine kinase muscle and brain; CNS, central nervous system; CPK, creatine phosphokinase; INR, International Normalized Ratio; MEWS, Modified Early Warning System; SOFA, Sepsis-related Organ Failure Assessment score.

Study Design

This study was a retrospective case–control using ICU mega data over time period of September 1, 2013–October 1, 2014. The adverse outcomes of ICU death, cardiac arrest (CODE), and ICU readmissions were selected. Utilizing the Naval Medical Center San Diego Intensive Care Unit admission data, we identified all patients during the time period of September 1, 2013–October 1, 2014. In addition, a control group of ICU patients who did not experience the adverse outcomes was also obtained from the same admission data and time period. Three demographic data points, 23 data elements from current algorithm composition, and four elements for consideration in future algorithms were evaluated for all patients who experienced an adverse outcome and all controls (Table 2). The four proposed test data elements included an extrapolation of the current EWD temperature of >100.5 F to the proposed data element of 101 F, as well as new dialysis, ectopy, and atrial fibrillation, with rapid ventricular response. For a full list of the specific scoring systems involved in each of the algorithms, see Supplementary Appendix 1; Supplementary Data are available online at www.liebertpub.com/tmj

Table 2.

Data Elements in the Study

Demographic elements

Age >65

Diabetes

BMI >30

Current EWD elements

Heart rate >120

Respiratory rate >30

Temperature >100.5

Vasopressors at event

Anuria

Oliguria

HCO₃ ≤15

Decrease HCO₃ 4 pt

Creatinine >2

Creatinine change >0.5

INR >3

Platelets <20

HCT drop >6/24 h

wbc >12

Troponin >0.5

BNP ≥300

Potassium <2.5 or >7

Sodium <120

Blood culture positive

PaO₂/FiO₂ < 300

PaO₂/FiO₂ < 200

Lactate >2.5

Plateau pressure >30

Proposed future data elements

Temperature >101

Dialysis

Ectopy

Afib with RVR

Afib, atrial fibrillation; EWD, early warning dashboard; HCT, hermatocrit; RVR, rapid ventricular response; wbc, white blood cell.

Method of Evaluation

The individual demographic items and data elements were reviewed for all identified patients through the history and physical examination (H&P) or from the data sections of the EMR. The following clarifications apply: oliguria is defined as <500 mL in 24 h; thus, an anuric patient was positive for both anuria and oliguria by our system of counting. Blood culture positivity was defined as a positive blood culture from a sample drawn before the adverse event, but during the same admission. Coagulase-negative Staphylococcus was excluded due to its high likelihood of false positivity. New dialysis was also considered positive if it occurred during that specific admission. For all other criteria, positive was defined as within the previous 24 h of the adverse event. For controls, a 24-h period was selected for comparison. In addition, the plateau pressure was occasionally not documented by the respiratory therapist, but if the peak pressure was documented as <30, then the value was accepted. For ICU death, comfort care death versus unexpected mortality was not factored into the final patient number. Finally, the manner of counting data elements that were not documented was to count them as neither positive nor negative; rather, the denominator was decreased by one for that element.

Statistical Analysis

Odds ratios (ORs) and one tailed p-values were calculated using mid-P exact values from the OpenEpi computer program. For situations where no patients in the control were positive for a data element, 0.5 was used for calculations since the equation for an OR would make it undefined with a zero in the denominator. An example of this adjustment occurred with “New Dialysis” in cardiac arrest where 5/15 patients who arrested had the risk factor of new dialysis. None (zero) of the 33 control patients were on dialysis. This results in an indeterminate statistical result despite a likely very important clinical result. Thus, the zero was replaced with 0.5, which modestly decreases the proposed impact of data element, but allows for statistical evaluation. The proposed definition of significance was to achieve an 80% power and p ≤ 0.05.

Results

The analysis of the data displayed 13 statistically significant data elements predictive of ICU death (Table 3). The risk factor for BNP >300 was present in all 24 patients; thus, an OR becomes undefined by a zero in the denominator. With the substitution of 0.5, an OR of 22.15 was obtained (p = 0.005). Similarly, the data element “new dialysis” was not present in any member of the control group of survivors; therefore, the same substitution of 0.05 was made with resultant ORs of 9.4 (p = 0.05). Cardiac arrest (Code) resulted in 10 statistically significant data elements (Table 4). New dialysis (OR 33 p = 0.002) required the same modification of 0.05 since no control patient had this risk factor. In addition to the 10 statistically significant data elements, one demographic item, diabetes (OR 2.63 p = 0.07), and two data elements, troponin >0.5 (OR 4 p = 0.08) and ectopy (OR 2.6 p = 0.07), trended toward statistical significance. For ICU readmission, there were comparisons to both the ICU control, as well as a ward control. For the comparison to the ICU control, only respiratory rate >30 (OR 3.1 p = 0.059) and the demographic item diabetes (OR 2.8 p = 0.71) were significant or trending toward significance (Table 5). Compared to the ward control, only respiratory rate >30 (OR 21.38 p = 0.0000284) was significant, with diabetes (OR 2.03 p = 0.065) and white blood cells (wbcs) >12 (OR 3.033 p = 0.0845) trending toward significance (Table 6). In addition, the raw data evaluating ICU death, cardiac arrest, and ICU readmissions and controls can be seen in Supplementary Appendices 2, 3, 4, 5a, and 5b.

Table 3.

Intensive Care Unit Deaths

	ICU DEATH AVERAGE AGE	73.1	CONTROL GROUP AVERAGE AGE	67.1
	YES	NO	YES	NO	ODDS RATIO	p (1 TAILED)
Age >65	23	10	19	14	1.695	0.32
Diabetes	14	19	12	21	1.29	0.626
BMI >30	13	20	14	19	0.882	0.809
hr >120	19	14	9	24	3.62	0.0149
RR >30	19	14	14	19	1.82	0.232
Temp >100.5	7	26	7	26	1	>0.999
Temp >101	5	28	4	29	1.295	0.738
Pressors at event	23	10	6	27	10.35	0.0000273
Anuria	2	30	0	33	^*Undef (4.4)	0.207
Oliguria	8	24	2	31	5.18	0.042
Dialysis	4	28	0	33	^*Undef (9.4)	0.05
HCO₃ ≤15	14	19	2	31	11.4	0.00059
Decrease HCOS 4 pt	13	18	6	27	3.25	0.044
Creatinine >2	15	18	4	29	6.04	0.0032
Creatinine change >0.5	11	21	4	29	3.8	0.039
INR >3	7	24	1	28	8.17	0.036
Platelets <20	2	31	0	33	^*Undef (4.28)	0.213
HCT drop >6/24 h	2	30	5	28	0.373	0.283
wbc >12	19	14	17	16	1.28	0.632
Trop >0.5	5	22	2	22	2.5	0.331
BNP ≥300	24	0	13	6	^*Undef (22.15)	0.005
K < 2.5 or >7	0	33	0	33	n/a	n/a
Na <120	0	33	0	33	n/a	n/a
BCX positive	9	20	4	19	2.14	0.283
PaO₂/FiO₂ < 300	23	9	3	30	25.56	0.00000016
PaO₂/FiO₂ < 200	14	18	2	31	12.06	0.000444
Lactate >2.5	20	11	8	18	4.09	0.0134
Plateau pressure >30	2	17	0	0	^*Undefined	^*Undefined
Ectopy	12	20	10	23	1.38	0.554
Afib with RVR	9	23	5	28	2.19	0.223

Undef refers to Undefined for the odds ratio when utilizing the raw data as 0 in the denominator of a number, which is mathematically undefined. The number in parentheses is obtained by modifying the 0 to 0.5 and performing the odds ratio calculation. Plateau pressures could not be evaluated (undefined) as the control group did not have reported plateau pressures, and thus, no extrapolation was performed. Potassium and sodium both had zeros for exposures and, thus, were neither positive risk factors for either the experimental group or control. If modified to 0.5 for both values, then odds ratio would be 1, but reported in this study as n/a.

BCX, blood cultures; hr, heart rate; ICU, intensive care unit; RR, respiratory rate; Temp, temperature; Trop, troponin.

Table 4.

Cardiac Arrest (Code)

	CODE GROUP AVERAGE AGE	73.1	CONTROL GROUP AVERAGE AGE	67.1
	YES	NO	YES	NO	ODDS RATIO	p (1 TAILED)
Age >65	11	4	19	14	2.03	0.16
Diabetes	9	6	12	21	2.63	0.07
BMI >30	5	10	14	19	0.68	0.29
hr >120	8	7	9	24	3.05	0.049
RR >30	7	8	14	19	1.18	0.396
Temp >100.5	4	9	7	26	1.65	0.26
Temp >101	2	13	4	29	1.15	0.44
Pressors at event	10	5	6	27	9	0.00089
Anuria	1	14	0	33	^*Undef (4.7)	0.235
Oliguria	5	10	2	31	7.75	0.013
Dialysis	5	10	0	33	^*Undef (33)	0.002
HCO₃ ≤15	6	9	2	31	10.3	0.004
Decrease HCOS 4 pt	5	10	6	27	2.25	0.14
Creatinine >2	8	7	4	29	8.3	0.0023
Creatinine change >0.5	3	12	4	29	1.8	0.25
INR >3	4	11	1	28	10.2	0.02
Platelets <20	0	15	0	33	n/a	n/a
HCT drop >6/24 h	1	14	5	28	0.4	0.24
wbc >12	12	3	17	16	3.8	0.035
Trop >0.5	4	11	2	22	4	0.08
BNP ≥300	11	4	13	6	1.3	0.39
K < 2.5 or >7	0	15	0	33	n/a	n/a
Na <120	0	15	0	33	n/a	n/a
BCX positive	2	13	4	19	0.73	0.38
PaO₂/FiO₂ < 300	10	5	3	30	20	0.000046
PaO₂/FiO₂ < 200	6	9	2	31	10.3	0.0041
Lactate >2.5	8	7	8	18	2.6	0.089
Plateau pressure >30	3	12	0	0	Undef
Ectopy	8	7	10	23	2.6	0.07
Afib with RVR	1	14	5	28	0.4	0.24

Undef refers to Undefined for the odds ratio when utilizing the raw data as 0 in the denominator of a number, which is mathematically undefined. The number in parentheses is obtained by modifying the 0 to 0.5 and performing the odds ratio calculation. Plateau pressures could not be evaluated (undefined) as the control group did not have reported plateau pressures, and thus, no extrapolation was performed. Platelets, potassium, and sodium all had zeros for exposures and, thus, were neither positive risk factors for either the experimental group or control. We reported them statistically as n/a.

Table 5.

Intensive Care Unit Readmission (ICU Control)

	READMISSION AVERAGE AGE	73.1	CONTROL GROUP AVERAGE AGE	67.1
	YES	NO	YES	NO	ODDS RATIO	p (1 TAILED)
Age >65	6	7	19	14	0.62	0.253
Diabetes	8	5	12	21	2.8	0.071
BMI >30	4	9	14	19	0.6	0.253
hr >120	2	11	9	24	0.48	0.22
RR >30	9	4	14	19	3.1	0.059
Temp >100.5	1	12	7	26	0.31	0.16
Temp >101	0	13	4	29	^*Undef (0.28)	0.40
Pressors at event	1	12	6	27	0.375	0.21
Anuria	0	13	0	33	n/a	n/a
Oliguria	1	12	2	31	1.28	0.41
Dialysis	1	12	0	33	^*Undef (5.5)	0.212
HCO₃ ≤15	1	12	2	31	1.28	0.41
Decrease HCOS 4 pt	3	10	6	27	1.35	0.35
Creatinine >2	2	11	4	29	1.32	0.38
Creatinine change >0.5	2	11	4	29	1.32	0.38
INR >3	0	13	1	28	^*Undef (1.1)	0.24
Platelets <20	0	13	0	33	n/a	n/a
HCT drop >6/24 h	1	12	5	28	0.467	0.283
wbc >12	7	6	17	16	1.098	0.447
Trop >0.5	1	12	2	22	0.92	0.49
BNP ≥300	6	7	13	6	0.396	0.186
K < 2.5 or >7	0	13	0	33	n/a	n/a
Na <120	0	13	0	33	n/a	n/a
BCX positive	0	13	4	19	^*Undef (0.18)	0.28
PaO₂/FiO₂ < 300	1	12	3	30	0.833	0.468
PaO₂/FiO₂ < 200	1	12	2	31	1.28	0.41
Lactate >2.5	3	10	8	18	0.675	0.235
Plateau pressure >30	0	13	0	0	^*Undefined	^*Undefined
Ectopy	2	11	10	23	0.418	0.167
Afib with RVR	1	12	5	28	0.467	0.283

Undef refers to Undefined for the odds ratio when utilizing the raw data as 0 in the denominator of a number, which is mathematically undefined. The number in parentheses is obtained by modifying the 0 to 0.5 and performing the odds ratio calculation. Plateau pressures could not be evaluated (undefined) as the control group did not have reported plateau pressures, and thus, no extrapolation was performed. Anuria, platelets, potassium, and sodium all had zeros for exposures and, thus, were neither positive risk factors for either the experimental group or control. We reported them statistically as n/a.

Table 6.

Intensive Care Unit Readmission (Ward Control)

	READMISSION AVERAGE AGE	73.1	CONTROL GROUP AVERAGE AGE	67.1
	YES	NO	YES	NO	ODDS RATIO	p (1 TAILED)
Age >65	6	7	12	9	0.643	0.279
Diabetes	8	5	7	14	2.03	0.065
BMI >30	4	9	8	13	0.722	0.345
hr >120	2	11	1	20	3.64	0.185
RR >30	9	4	2	19	21.38	0.000284
Temp >100.5	1	12	2	19	0.792	0.45
Temp >101	0	13	2	19	^*Undef (0.37)	0.48
Pressors at event	1	12	0	21	Undef	0.191
Anuria	0	13	1	19	^*Undef (0.73)	0.31
Oliguria	1	12	1	19	1.58	0.394
Dialysis	1	12	0	21	^*Undef (3.5)	0.288
HCO₃ ≤15	1	12	0	21	^*Undef (3.5)	0.288
Decrease HCOS 4 pt	3	10	4	17	1.275	0.393
Creatinine >2	2	11	2	19	1.725	0.322
Creatinine change >0.5	2	11	2	19	1.725	0.322
INR >3	0	13	2	11	0	0.12
Platelets <20	0	13	0	21	n/a	n/a
HCT drop >6/24 h	1	12	1	20	1.667	0.382
wbc >12	7	6	5	16	3.033	0.0845
Trop >0.5	1	12	0	10	^*Undef (1.7)	0.43
BNP ≥300	6	7	2	5	2.14	0.249
K < 2.5 or >7	0	13	0	21	n/a	n/a
Na <120	0	13	0	21	n/a	n/a
BCX positive	0	13	0	8	n/a	n/a
PaO₂/FiO₂ < 300	1	12	0	3	^*Undef 0.5	0.34
PaO₂/FiO₂ < 200	1	12	0	3	^*Undef 0.5	0.34
Lactate >2.5	3	10	0	8	^*Undef (4.8)	0.19
Plateau pressure >30	0	13	0	0	^*Undef	Undefined
Ectopy	2	11	1	0	^*Undef (0.09)	0.04
Afib with RVR	1	12	1	0	^*Undef (0.04)	0.08

Undef refers to Undefined for the odds ratio when utilizing the raw data as 0 in the denominator of a number, which is mathematically undefined. The number in parentheses is obtained by modifying the 0 to 0.5 and performing the odds ratio calculation. Plateau pressures could not be evaluated (undefined) as the control group did not have reported plateau pressures, and thus, no extrapolation was performed. Anuria, platelets, potassium, and sodium all had zeros for exposures and, thus, were neither positive risk factors for either the experimental group or control. We reported them statistically as n/a.

Discussion

With the success of the MEWS as a ward tool to improve patient safety, the military has developed a prototype for an EWD to proactively monitor ICU patients. In particular, recent evaluation of EMR criteria versus “call criteria” by Lee et al. displayed mortality benefit for a protocol-based approach.⁹ In an ICU setting, however, the nurse to patient ratio will generally be lower than it is on the floor, resulting in less of a drive for an electronic monitoring system. However, the acuity of many critically ill patients and the need for time consuming procedures can result in patient data elements being missed or seen at a later time period, while the nurse or provider is engaged in another activity. Thus, the EWD provides an excellent adjunct monitoring system for critically ill patients. In the military, there is a shortage of trained intensivists to cover all the military hospitals worldwide. Thus, we have traditionally staffed these smaller hospitals with internists with protocols for early transfers to larger military medical centers or nearby civilian hospitals with more advanced ICU capability. In medium-sized hospitals, we often have only one or two intensivists who provide 7-day 24-h ICU coverage. In response, the Navy is now exploring telecritical care monitoring for some of the smaller hospitals. The EWD is a tool that can help augment this capability by remotely identifying high-risk patients who require closer monitoring and more immediate evaluation. With the validation of the data elements composing the algorithms, we see great potential for the EWD's use as a patient safety tool for in-house augmentation and to enhance and enable telecritical care support. Based on variability of intensivist staffing at several community hospitals, telemedicine and telecritical care have been utilized, and their role is expected to significantly expand in the coming years. The EWD is a proactive EMR-linked tool to assist with earlier identification of high-risk patients. This study has validated the individual data elements of the EWD algorithms against patient-centered outcomes. With the EWD operation, we next plan to prospectively evaluate its effectiveness on patient outcomes. Electronic algorithms have been shown to improve patient safety in the ward function; and we see monitoring systems, such as the EWD, to be a very effective option for many hospital systems and telecritical care providers.

Footnotes

Acknowledgments

Source of Work: Retrospective Case–Control utilizing electronic medical record at the Naval Medical Center San Diego. Kavanaugh et al. are military service members (or employees of the U.S. Government). This work was prepared as part of official duties. Title 17 U.S.C. 105 provides that “Copyright protection under this title is not available for any work of the United States Government.” Title 17 U.S.C. 101 defines a United States Government work as a work prepared by a military service member or employee of the United States Government as part of that person's official duties. The study protocol was approved by the Naval Medical Center San Diego Institutional Review Board in compliance with all applicable Federal regulations governing the protection of human subjects.

Disclosure Statement

No competing financial interests exist.

References

Carr

, Addyson

, Kahn

. Variation in critical care beds per capita in the United States: Implications for pandemic and disaster planning. JAMA, 2010; 303:1371–1372.

Halpern

, Pastores

. Critical care medicine in the United States 2000–2005: An analysis of bed numbers, occupancy rates, payer mix and costs. Crit Care Med, 2010; 38:65–71.

Gardner-Thorpe

, Love

, Wrightson

, Walsh

, Keeling

. The value of Modified Early Warning Score (MEWS) in surgical in-patients: A prospective observational study. Ann R Coll Surg Engl, 2006; 88:571–575.

Churpek

, Yuen

, Winslow

, Robicsek

, Meltzer

, Gibbons

, Edelson

. Multicenter development and validation of a risk stratification tool for ward patients. Am J Respir Crit Care Med, 2014; 190:649–655.

Pinsky

, Dubrawski

. Gleaning knowledge from data in the intensive care unit. Am J Respir Crit Care Med, 2014; 190:606–610.

Subbe

, Kruger

, Rutherford

, Gemmel

. Validation of a modified Early Warning Score in medical admissions. Q J Med, 2001; 94:521–526.

Moon

, Cosgrove

, Lea

, Fairs

, Cressey

. An eight year audit before and after the introduction of modified early warning score (MEWS) charts, of patients admitted to a tertiary referral intensive care unit after CPR. Resuscitation, 2011; 82:150–154.

Huh

, Lim

, Koh

, Lee

, Jung

, Seo

, Hong

. Activation of a medical emergency team using an electronic medical recording-based screening system. Crit Care Med, 2013; 42:1–8.

Lee

, Suh

, Ryu

, Chung

, Yang

, Park

, Jeon

. Effect of early intervention on long term outcomes of critically III cancer patients admitted to ICUs. Crit Care Med, 2015; 43:1439–1448.

10.

Boniatti

, Azzolini

, Viana

, Ribeiro

BSP

, Coelho

, Castilho

, Guimarães

, Zorzi

, Schulz

, Rodrigues Filho

. Delayed medical emergency team calls and associated outcomes. Crit Care Med, 2013; 42:26–30.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.29 MB

0.34 MB

0.18 MB

0.31 MB

0.22 MB

0.00 MB

Validation of the Intensive Care Unit Early Warning Dashboard: Quality Improvement Utilizing a Retrospective Case–Control Evaluation