Current Pneumonia Surveillance Methodology: Similar Underestimation in Trauma and Surgical Patients in the Intensive Care Unit

Abstract

Background:

In 2013, the Centers for Disease Control and Prevention (CDC) developed new surveillance definitions for ventilator-associated events (VAE), leading to concerns that hospitals may be underreporting the true incidence of ventilator-associated pneumonias (VAPs). We sought to compare rates of clinically diagnosed VAP with CDC defined possible VAPs (PVAPs) in patients with a VAE in the surgical/trauma intensive care unit (STICU). Hypothesis: Significant difference exists between rates of clinical VAP and PVAP in patients with at least one VAE.

Patients and Methods:

All STICU patients with ≥1 VAE, between 1/1/2013 and 10/31/2015 were identified. Age, length of stay (LOS), ICU and ventilator days were collected.

Results:

There were 134 patients who had ≥1 VAE. Mean age was 54.3 (±17.1) years. Mean LOS, median ICU, and median ventilator days were 26.3 (±14.1), 21.0 (17.0–33.0), and 17.0 (12.8–24.0) days, respectively. There were 68 cases of clinically diagnosed VAP, but only 37% met PVAP criteria. We compared 43 cases of clinical VAP, not meeting PVAP criteria, with the 25 PVAPs. Both groups had similar outcomes. The PVAPs were more likely to have an abnormal temperature (48.0% vs. 14.0%, p = 0.004), abnormal white blood cell count (84.0% vs. 18.6%, p < 0.001), or new antibiotic agent initiated (100% vs. 18.6%, p < 0.001) as VAE triggers. Comparison of the 93 trauma and 41 surgical patients demonstrated trauma patients were younger (51.2 vs. 61.5 y, p = 0.001), but had similar outcomes and rates of clinical VAP (48.4% and 43.9%, p = NS). Only 20.4% of trauma and 14.6% of surgical patients, however, had a PVAP reported. For patients with at least one VAE, the sensitivity and specificity for PVAP detecting VAP was 36.8% and 96.0%, respectively.

Conclusion:

The new CDC definition for PVAP grossly underestimates the clinical diagnosis of VAP and reports less than a third of the patients treated for VAP. Reporting differences were similar for trauma and surgical patients.

Ventilator-associated pneumonia (VAP) is one of the most common nosocomial infections [1]. Effective preventive strategies have been reported, but how we benchmark our success at combating this problem has been challenging because of the ever-changing definition of VAP. This evolving definition has made comparisons difficult, and looming financial penalties raise the question of how accurately we are reporting the true incidence of this disease. In 2013, the Centers for Disease Control and Prevention (CDC) developed new surveillance definitions for ventilator-associated events (VAE) in attempts to more objectively identify ventilator-related complications.

This new surveillance system uses a three-tiered algorithm for defining VAEs [2]. The patient must first have two days of sustained increase in respiratory support (increase in F_IO₂ ≥0.20 or increase in positive end-expiratory pressure ≥3 cm H₂O) after at least two days of stability to meet criteria for a ventilator-associated condition (VAC). If the VAC is associated with an abnormal temperature or white blood cell count, plus administration of a new antibiotic agent for at least four days, the VAE is classified as an infection-related ventilator-associated complication (IVAC). If after meeting criteria for an IVAC, there is evidence of pulmonary infection based on positive respiratory cultures or other diagnostic tests, the VAE is considered a possible VAP (PVAP).

There are limited data regarding the utility of the new surveillance algorithm in identifying clinically managed VAP in surgical and trauma intensive care units (ICU). There is emerging literature, however, suggesting significant underestimation of clinical ventilator-associated pneumonias captured by this new system [3 –6]. This finding has led to concerns that hospitals may be underreporting the true incidence of VAPs, which may lead to the conclusion that pneumonia should be a “never event” and thus not reimbursed.

We sought to compare our rates of clinically diagnosed and treated pneumonia with the rates of PVAPs as defined by the CDC, in STICU patients who had at least one VAE. Further, we wanted to determine whether there was a difference in these rates between trauma and surgical patients.

Patients and Methods

This was a retrospective study of all adult patients admitted to either the surgical or trauma ICU at a level I trauma center with at least one VAE between January 2013 and November 2015. Patients were identified from infection control records of VAE reported to the CDC's National Healthcare Safety Network. Demographic data including gender, age, reason for admission, admitting service, and admitting location were collected. The primary outcome was clinical diagnosis of VAP, which was determined by documentation and treatment of pneumonia by the attending intensivist.

Our standard method of diagnosis has been described in previous work from our group [7], where we use the previous CDC definitions from 2008 [8] as an indication to obtain a bronchoalveolar lavage (BAL), if possible, and base diagnosis on the results of that culture. However, we use a higher temperature cutoff (>38.4°C) to define fever. Our definition of a positive respiratory culture was modified to include growth of any abnormal respiratory pathogen from a tracheal aspirate or BAL culture and was further classified as meeting quantitative thresholds if there were ≥10,000 colony-forming units/mL. Other clinical outcomes including duration of mechanical ventilation, hospital length of stay (LOS), ICU days, as well as rates of bronchoscopy and tracheostomy were recorded. Patients were excluded if admitted to a medical service but were boarding in either the surgical or trauma ICU.

Statistical analysis was performed using IBM SPSS^© version 22 (IBM, Armonk, NY). Continuous variables were compared using the Student t-test or Mann–Whitney U test. Categoric data were analyzed using either the chi-square test or Fisher exact test as appropriate. A p value ≤0.05 was deemed statistically significant. Means in this study are reported as mean ± standard deviation, and medians are reported as median (interquartile range [IQR]). Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated by total cases of VAP identified or not identified in this population of patients with at least one VAE. This study was approved by the MetroHealth Medical Center Institution Review Board.

Results

Study population

From January 2013 to November 2015, there were 134 patients admitted to the STICU who had at least one VAE. Demographics are demonstrated in Table 1. The mean age was 54.3 (±17.1) years. The mean hospital LOS was 26.3 (±14.1) days, median ICU days and median ventilator days were 21.0 (17.0–33.0) and 17.0 (12.8–24.0) days, respectively. Tracheostomy was performed in 59.5% of patients during their hospitalization. There were three patients who had an existing tracheostomy or tracheostomy performed on the day of admission. Overall death was 33.6% (45 patients).

Table 1.

Patient Demographics and Outcomes

		Subgroup comparison
	All patients n = 134	Trauma n = 93	Surgical n = 41	p ^*
Mean age, y ± SD	54.3 ± 17.1	51.2 ± 18.0	61.5 ± 12.3	0.001
Male	107 (79.9%)	75 (80.6%)	32 (78.0%)	0.816
Mean hospital LOS, days ± SD	26.3 ± 14.1	26.3 ± 13.8	26.4 ± 14.7	0.972
Median ICU days (IQR)	21.0 (17.0–33.0)	22.0 (16.0–30.5)	19.0 (15.0–24.5)	0.175
Median ventilator days (IQR)	17.0 (12.8–24.0)	17.0 (12.5–25.5)	15.0 (12.5–22.0)	0.241
Clinical VAP	63 (47.0%)	45 (48.4%)	18 (43.9%)	0.709
PVAP	25 (18.7%)	19 (20.4%)	6 (14.6%)	0.481
Bronchoscopy	76 (56.7%)	60 (64.5%)	16 (39.0%)	0.008
Tracheostomy	78/131 (59.5%)	58/91 (63.7%)	20/40 (50.0%)	0.177
Mortality	45 (33.6%)	28 (30.1%)	17 (41.5%)	0.235

Cases of VAP	All n = 68	Trauma n = 49	Surgical n = 19	p ^*
PVAP	25 (36.8%)	18 (36.7%)	7 (36.8%)	1.00
Positive culture	61 (89.7%)	47 (95.9%)	14 (94.7%)	1.00
VAP diagnosis before VAE	17 (25.0%)	15 (30.6%)	2 (10.5%)	0.121

SD = standard deviation; LOS = length of stay; IQR = interquartile range; ICU = intensive care unit; VAP = ventilator-associated pneumonia; PVAP = possible ventilator-associated pneumonia; VAE = ventilator-associated event.

Comparison of trauma patients with surgical patients with a VAE.

Evaluation of pneumonias and VAEs

In the 134 patients, there were a total of 139 VAEs. There were four patients with multiple VAEs on the trauma service. Of the 139 VAEs, there were 90 (64.7%) VACs, 23 (16.5%) IVACs, and 26 (18.7%) PVAPs (Fig. 1). Median onset of VAE was 5.0 (IQR 3.0–11.0) days after admission. There were 83 clinically diagnosed pneumonias (Fig. 2). Specifically, there were 68 VAPs, nine hospital-acquired pneumonias (HAP), five community-acquired pneumonias (CAP), and one health-care–associated pneumonia (HCAP) diagnosed. Of the 68 clinical VAPs, only 25 (36.8%) were determined to have a PVAP. Twenty-seven (19.4%) VAEs occurred after a patient had received a clinical diagnosis of pneumonia, on average 5.78 ± 6.25 days later.

FIG. 1.

Ventilator-associated events (VAE). VAC = ventilator-associated condition; IVAC = infection-related ventilator-associated complication; PVAP = possible ventilator-associated pneumonia.

FIG. 2.

Cases of clinically diagnosed pneumonia. CAP = community-acquired pneumonia; HAP = hospital-acquired pneumonia; HCAP = healthcare-associated pneumonia; VAP = ventilator-associated pneumonia; PVAP = possible ventilator-associated pneumonia.

Comparison of clinical VAP with and without PVAP

There were 68 cases of clinically diagnosed VAP. We compared the 43 cases of clinically diagnosed VAP that did not meet PVAP criteria with the 25 cases of VAP that did meet PVAP criteria (Table 2). Both groups had similar age, gender, hospital LOS, ICU days, ventilator days, and days to onset of VAE. The PVAPs were more likely to have abnormal temperature (48.0% vs. 14.0%, p = 0.004), abnormal white blood cell count (84.0% vs. 18.6%, p < 0.001), or new antibiotic agent started within two days from VAE and continued for at least 4 days (100% vs. 18.6%, p = < 0.001) as criteria for their VAE. Comparison of the most common pathogens that met quantitative thresholds are listed in Table 3. The most common organism, Staphylococcus aureus, was twice as high in the PVAP group; however, there were no statistically significant differences in the rates of any specific pathogen between the two groups.

Table 2.

Comparison of Clinical Ventilator-Associated Pneumonia Episodes without Possible Ventilator-Associated Pneumonia with Possible Ventilator-Associated Pneumonia

Analysis by cases of VAP = 68	VAP without PVAP n = 43	PVAP n = 25	p
Male	34 (79.1%)	22 (88.0%)	0.513
Trauma	31 (72.1%)	18 (72.0%)	1.0
Mean age, years ± SD	52.4 ± 18.7	49.0 ± 16.3	0.445
Median hospital LOS (IQR)	30.0 (22.0–35.0)	28.0 (20.0–34.5)	0.571
Median ICU days (IQR)	25.0 (20.0–34.0)	24.0 (17.0–30.0)	0.468
Median ventilator days (IQR)	22.0 (17.0–31.0)	21.0 (15.0–26.0)	0.397
Median days to VAE (IQR)	5.0 (3.0–10.0)	5.0 (4.0–11.0)	0.292
Median days to VAP diagnosis (IQR)	10.0 (6.0–14.0)	6.0 (4.0–11.5)	0.074
VAE after pneumonia diagnosis	9 (20.9%)	8 (32.0%)	0.387
Positive respiratory culture	40 (93.0%)	25 (100.0%)	0.248
Tracheostomy	35/41 (85.4%)	18/24 (75.0%)	0.335
Mortality	9 (20.9%)	5 (20.0%)	1.0
Triggers for VAE
Increased F_iO₂	10 (23.3%)	8 (32.0%)	0.570
Increased PEEP	40 (93.0%)	22 (88.0%)	0.662
Abnormal temperature	6 (14.0%)	12 (48.0%)	0.004
Abnormal WBC	8 (18.6%)	21 (84.0%)	<0.001
New antibiotic agent initiated	8 (18.6%)	25 (100%)	<0.001

VAP = ventilator-associated pneumonia; PVAP = possible ventilator-associated pneumonia; SD = standard deviation; LOS = length of stay; IQR = interquartile range; ICU = intensive care unit; VAE = ventilator-associated event; F_iO₂ = fraction of inspired oxygen; PEEP = positive end-expiratory pressure; WBC = white blood cell count.

Table 3.

Comparison of the Most Common Pathogens that Met Quantitative Thresholds for Diagnosing Pneumonia in Ventilator-Associated Pneumonia without Possible Ventilator-Associated Pneumonia versus Possible Ventilator-Associated Pneumonia

Pathogens	VAP w/o PVAP n = 43	PVAP n = 25	p
Staphylococcus aureus	10 (23.3%)	12 (48.0%)	0.059
Haemophilus species	10 (23.3%)	5 (20.0%)	1.0
Enterobacter species	8 (18.6%)	2 (8.0%)	0.304
Pseudomonas aeruginosa	6 (14.0%)	1 (4.0%)	0.248
Streptococcus pneumoniae	2 (4.7%)	2 (8.0%)	0.621
Klebsiella species	2 (4.7%)	2 (8.0%)	0.621
Moraxella catarrhalis	2 (4.7%)	2 (8.0%)	0.621
Actinetobacter baumannii	2 (4.7%)	1 (4.0%)	1.0

VAP = ventilator-associated pneumonia; PVAP = possible ventilator-associated pneumonia.

Comparison of trauma and surgical patients

We then compared the 93 trauma patients with the 41 surgical patients with a VAE (Table 1). Trauma patients were younger (51.19 ± 18.04 vs. 61.46 ± 12.25 y, p = 0.001), but both groups had similar hospital LOS, ICU days, and ventilator days. VAP was clinically diagnosed in 48.4% of trauma patients and 43.9% of surgical patients with a VAE. Three trauma patients who had a PVAP reported did not have a clinical diagnosis of VAP, because of two having a HAP and one patient having tracheobronchitis. Trauma patients were more likely to have undergone bronchoscopy (64.5% vs. 39.0%, p = 0.008). Both groups had similar rates of any positive respiratory cultures associated with their diagnosis of VAP (95.5% vs. 94.4%, p = 1.0), but four patients with VAP in the surgical group had subthreshold growth. Still, approximately only one third of clinically diagnosed VAP met PVAP criteria in both groups.

Sensitivity, specificity, PPV, and NPV

The sensitivity of PVAP in identifying cases of clinical VAP in patients with at least one VAE was 36.8%. Specificity and PPV was high at 96.0% and 92.6%, respectively, and NPV was 62.3%. These values were similar for both trauma and surgical patients (Table 4).

Table 4.

Sensitivity, Specificity, Positive Predictive Value, and Negative Predictive Value for Possible Ventilator-Associated Pneumonia Detecting Clinical Ventilator-Associated Pneumonia Cases in Patients with at Least One Ventilator-Associated Event

	All patients n = 142	Trauma n = 100	Surgical n = 42
Sensitivity	36.8%	36.7%	36.8%
Specificity	96.0%	94.1%	100%
PPV	92.6%	81.8%	100%
NPV	62.3%	60.8%	65.7%

PPV = positive predictive value; NPV = negative predictive value.

Discussion

This study adds to the growing body of literature regarding the applicability of the CDC's new surveillance algorithm for identifying ventilator-associated complications in the surgical and trauma populations. The VAEs capture a broader array of respiratory complications and may be better at identifying more severe respiratory dysfunction. Several studies have noted worse outcomes in patients with a VAE than those with VAP alone [6,9,10]. However, VAEs fall short in identifying the true incidence of VAP. Previous studies have reported up to 80% of clinical VAP cases are missed by the VAE algorithm [4,6,9,10]. Conversely, there are many VAE episodes that are not linked to a VAP. Studies including mixed ICUs have reported VAP represents 15%–28% of VAC cases [10,11], and 28%–40% of IVACs [9,11], but up to 58.1% of IVACs in a trauma population [5]. Less than half (48.2%) of VAEs in our study were associated with a clinical VAP. These discrepancies lead to the question of what we are actually measuring by tracking VAE rates and how we focus preventive measures to combat them.

Another major concern is determining the usefulness of the PVAP definition that was meant to more closely represent VAP diagnostic criteria used at the bedside [12]. In this high-risk group of patients with at least one VAE, sensitivity of PVAP in identifying clinical VAP was only 36.8%, although the specificity was 96.0%. While PVAP serves as good surveillance marker because of its high specificity and low likelihood of false negatives, it falls short in reporting the true incidence of clinical VAP, which is paramount when considering it as an internal quality measure. Others have reported 88%–95% of clinically diagnosed VAPs failed to meet PVAP criteria [3,4].

At our institution we make every effort to obtain an adequate lower respiratory sample when the suspicion of pneumonia arises. Therefore, we would expect the PVAP rate to more accurately capture our rates of clinical VAP, when in fact it did not. In our study population, which only included patients with at least one VAE, 63.2% clinical VAPs did not meet PVAP criteria, despite 93.0% of those patients having a positive respiratory culture that would be consistent with the laboratory requirements for a PVAP. Both clinical VAP and PVAP patients, however, had similar hospital LOS, ICU days, ventilator days, and death, making PVAP a less useful marker of poor outcomes in our population. This discrepancy in meeting PVAP criteria may be because of the temporal differences of when the first signs of worsening oxygenation occur, triggering the VAE, and when signs of infection develop, leading the clinician to obtain respiratory samples and initiate antibiotic agents. In our study, patients who had a VAP but did not meet PVAP criteria were more likely to have pneumonia diagnosed an average of four days after triggering a VAE versus one day later in the PVAP group.

There were several limitations to our study. This was a retrospective study at a single institution, and we did not have data available on clinically diagnosed VAP that did not trigger a VAE. Also, we recognize that there is no gold standard for diagnosing pneumonia. Despite consistencies in how our group defines VAP, there is likely an element of bias given the diagnosis was made by the treating intensivist. We used our definition for clinical pneumonia as the standard for determining sensitivity and specificity, because we wanted to get a real sense of the difference between the actual patients with VAP we are treating and the mark by which the CDC is measuring our performance. One could argue that we are overdiagnosing pneumonia and treating too many patients with antibiotic agents. We pride ourselves on our antibiotic stewardship and rigorous diagnosis of VAP, however. Regardless, there is significant discrepancy in clinical VAP and PVAPs, which could ultimately lead to us focusing too may resources on monitoring and tracking PVAP and not having appropriate resources to follow clinical VAP.

Conclusions

The new CDC definition for PVAP grossly underestimates the clinical diagnosis of VAP and reports less than one third of patients whom we actually treat for VAP in the STICU. There were similar reporting differences between trauma and surgical ICU patients. Further studies are needed to assess the true validity of the new VAE surveillance paradigm in identifying preventable complications of mechanical ventilation in this population. Further, we are concerned that focusing on improving surveillance data may, in fact, cause us to lose sight of managing the patient's disease—VAP. We must continue to focus our efforts on appropriate diagnosis, treatment, and antibiotic stewardship.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

References

Dudeck

, Weiner

, Allen-Bridson

, et al. National Healthcare Safety Network (NHSN) report, data summary for 2012, Device-associated module. Am J Infect Control, 2013; 41:1148–1166.

Centers for Disease Control and Prevention. Ventilator-associated event (VAE) protocol. www.cdc.gov/nhsn/pdfs/pscmanual/10-vae_final.pdf. 2016. (Last accessed May 1, 2016).

Klein Klouwenberg

, van Mourik

, Ong

, et al. Electronic implementation of a novel surveillance paradigm for ventilator-associated events. Feasibility and validation. Am J Respir Crit Care Med, 2014; 189:947–955.

Lilly

, Landry

, Sood

, et al. Prevalence and test characteristics of national health safety network ventilator-associated events. Crit Care Med, 2014; 42:2019–2028.

Piriyapatsom

, Lin

, Pirrone

, et al. Evaluation of the infection-related ventilator-associated events algorithm for ventilator-associated pneumonia surveillance in a trauma population. Respir Care, 2016; 61:269–276.

Stoeppel

, Eriksson

, Hawkins

, et al. Applicability of the National Healthcare Safety Network's surveillance definition of ventilator-associated events in the surgical intensive care unit: A 1-year review. J Trauma Acute Care Surg, 2014; 77:934–937.

Claridge

, Golob

Jr. , Leukhardt

, et al. The “fever workup” and respiratory culture practice in critically ill trauma patients. J Crit Care, 2010; 25:493–500.

Horan

, Andrus

, Dudeck

. CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting. Am J Infect Control, 2008; 36:309–332.

Muscedere

, Sinuff

, Heyland

, et al. The clinical impact and preventability of ventilator-associated conditions in critically ill patients who are mechanically ventilated. Chest, 2013; 144:1453–1460.

10.

Klompas

, Khan

, Kleinman

, et al. Multicenter evaluation of a novel surveillance paradigm for complications of mechanical ventilation. PLoS One. 2011; 6:e18062.

11.

Bouadma

, Sonneville

, Garrouste-Orgeas

, et al. Ventilator-associated events: Prevalence, outcome, and relationship with ventilator-associated pneumonia. Crit Care Med, 2015; 43:1798–1806.

12.

Magill

, Klompas

, Balk

, et al. Executive summary: Developing a new, national approach to surveillance for ventilator-associated events. Ann Am Thorac Soc, 2013; 10:S220–S223.