Abstract
Abstract
Background:
No consensus exists regarding the definition of ventilator-associated pneumonia (VAP). Even within a single institution, inconsistent diagnostic criteria result in conflicting rates of VAP. As a Level 1 trauma center participating in the Trauma Quality Improvement Project (TQIP) and the National Healthcare Safety Network (NHSN), our institution showed inconsistencies in VAP rates depending on which criteria was applied. The purpose of this study was to compare VAP definitions, defined by culture-based criteria, National Trauma Data Bank (NTDB) and NHSN, using incidence in trauma patients.
Methods:
A retrospective chart review of consecutive trauma patients who were diagnosed with VAP and met pre-determined inclusion and exclusion criteria admitted to our rural, 861-bed, Level 1 trauma and tertiary care center between January 2008 and December 2011 was performed. These patients were identified from the National Trauma Registry of the American College of Surgeons (NTRACS) database and an in-house infection control database. Ventilator-associated pneumonia diagnosis criteria defined by the U.S. Center for Disease Control and Prevention (used by the NHSN), the NTDB, and our institutional, culture-based criteria gold standard were compared among patients.
Results:
Two hundred seventy-nine patients were diagnosed with VAP (25.4% met NHSN criteria, 88.2% met NTDB, and 76.3% met culture-based criteria). Only 58 (20.1%) patients met all three criteria. When NHSN criteria were compared with culture-based criteria, NHSN showed a high specificity (92.5%) and low sensitivity (28.2%). The positive predictive value (PPV) was 84.5%, but the negative predictive value (NPV) was 47.1%. The agreement between the NHSN and the culture-based criteria was poor (κ = 0.18). Conversely, the NTDB showed a lower specificity (57.8%), but greater sensitivity (86.4%) compared with culture-based criteria. The PPV and NPV were both 74% and the two criteria showed fair agreement (κ = 0.41).
Conclusions:
The lack of standard diagnostic criteria for VAP resulted in variable reporting to different agencies. Emphasis on establishing a consensus VAP definition should be undertaken.
V
As a Level I trauma center participating in both the Trauma Quality Improvement Project (TQIP) and the National Healthcare Safety Network (NHSN), our institution has observed inconsistencies in the rates of VAP based on variable diagnostic criteria (i.e., deep lung bronchoalveolar lavage [BAL]) compared with the National Trauma Data Bank (NTDB)/National Trauma Registry of the American College of Surgeons (NTRACS)/TQIP and U.S. Centers for Disease Control and Prevention National Healthcare Safety Network (CDC/NHSN) definitions. There has been much effort over the past decade to establish a definition of VAP that would be uniform and easily reproducible among hospitalized patients. Currently, multiple definitions of VAP exist that include a combination of clinical, radiologic, and microbiologic data [5,20,21]. The purpose of this study was to compare VAP definitions (culture-based versus NTDB and NHSN criteria) using incidence, not to follow outcomes of these patients.
Patients and Methods
Setting
The setting was a 24-bed surgical intensive care unit (SICU) in a rural, 861-bed, Level 1 trauma center and tertiary care academic teaching hospital.
Definitions
Three methods are used commonly in the diagnosis of VAP: the NHSN criteria, the NTDB definition based on NTRACS criteria, and microbiologic criteria based on lower respiratory tract sampling [5]. Although the CDC has three definitions of pneumonia (i.e., PNU 1, PNU 2, and PNU 3), clinically defined pneumonia (PNU 1) and pneumonia with specific laboratory findings (PNU 2) best fit our trauma population (PNU 3 is used to define VAP in immunocompromised patients and so was not used for the purposes of this study) [20]. The CDC criterion for diagnosing VAP uses radiography, clinical signs and symptoms, and laboratory data (Tables 1 and 2). The NTDB criterion utilizes a different combination of radiography, clinical, and laboratory data (Table 3). At our institution, clinical suspicion for VAP is based on two or more of the following: fever, leukocytosis, purulent sputum, chest radiograph findings, and/or deteriorating respiratory function (e.g., decreasing oxygenation). Diagnosis is then based on deep respiratory culture obtained via bronchoscopy. Our institution performs bronchoscopies in a standard fashion by inserting the bronchoscope through the endotracheal tube or tracheostomy tube into the airway without use of suctioning to avoid contamination. We use a 100-mL aliquot that is divided into 20-mL aliquot samples and the culture is determined to be positive if containing greater than 104 colony-forming units (CFU) of speciated bacteria (Table 4).
Source: Horan et al. [20].
WBC = white blood cell count.
Source: Horan et al. [20].
BAL = bronchoalveolar lavage; PMN = polymorphic neutrophil.
Source: National Trauma Data Standard [21].
IgM = immunoglobulin M; IgG = immunoglobulin G.
If >104 CFU of a speciated organism was obtained from culture then patient would be diagnosed with VAP.
CFU = colony-forming unit; VAP = ventilator-associated pneumonia.
Patient and data selection
After obtaining approval from the local Institutional Review Board, we performed a retrospective review of consecutive adult (age ≥18 y) trauma patients admitted to our institution between January 2008 and December 2011 who developed VAP (mechanical ventilation for >48 h prior to VAP diagnosis). Patients who sustained burn injuries or were transplant recipients were excluded. The NTRACS database (local trauma registry) was merged with our in-house infection control database to identify all trauma patients who met inclusion criteria for VAP. Each patient was evaluated for inclusion in the study by meeting the NHSN, NTRACS, or microbiology culture-based criterion for VAP. We used >104 CFU/mL criteria for a positive quantitative culture to diagnose VAP per CDC guidelines. Individual chart review was performed to ensure only trauma patients who met our inclusion criteria (e.g., mechanical ventilation for >48 h) were included.
Statistical analysis
Sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were calculated comparing our gold standard of culture-based diagnosis and the NHSN and NTDB criteria. The Cohen κ statistic, which measures the inter-observer variation, was also calculated to evaluate the degree of agreement by chance between the culture-based criteria and the NHSN and NTDB criteria. The κ statistic is used commonly in the medical literature to show the difference between “observed” agreement, or the degree of agreement that is present, and the “expected” agreement, or the degree of agreement present by chance alone [22]. The κ value is calculated as
Results
After the databases were queried, a total of 279 unique patients met VAP diagnosis utilizing one or more of the criteria: NHSN (n = 71; 25%), NTDB (n = 246; 88%), and culture-based (n = 213; 76%). Patients were then separated based on any overlap between criteria (Table 5). When comparing the NHSN criteria with our culture-based criteria, a specificity of 92.5%, sensitivity of 28.2%, PPV of 84.5%, and NPV of 47.1% was observed for the NHSN criteria. The κ statistic showed poor agreement between the NHSN and culture-based criteria (κ = 0.18). When comparing the NTDB criteria with our culture-based criteria, a specificity of 57.8%, sensitivity of 86.4%, PPV of 74%, and NPV of 74% was observed for the NTDB criteria. The κ statistic showed fair agreement between the NTDB and culture-based criteria (κ = 0.47; Table 6).
NHSN = National Healthcare Safety Network; NTDB = National Trauma Data Bank.
NHSN = National Healthcare Safety Network; NTDB = National Trauma Data Bank; NTRACS = National Trauma Registry of the American College of Surgeons; NPV = negative predictive value; PPV = positive predictive value.
Discussion
Severely injured trauma patients who require mechanical ventilation are at a greater risk of VAP than other intensive care unit (ICU) populations, leading to proportional increases in morbidity, mortality, and cost [1,3–5,13,14,18,25–27]. Although wide implementation of ventilator bundles along with increased awareness have been shown in the literature to decrease VAP rates [10,27–30], bundles may not be as effective in trauma patients [15,16], likely related to unique risk factors such as injury severity, emergent placement of an airway, and chest trauma. Compounding the incidence and preventative challenges of VAP after trauma, a financial disincentive may evolve. In October 2008, the Centers for Medicare and Medicaid Services began to discontinue payments for certain hospital-acquired infections as financial penalties to encourage hospitals to improve quality of care [31]. There has been interest in including VAP as a benchmark of quality of care [17,32]. One concern regarding this practice as mentioned by Wahl et al. [33] is that current “regulating bodies do not distinguish ICU populations or patient types when implementing best practices.” Furthermore, it has been suggested that VAP may not be a hospital-acquired infection at all, but rather a sequelae of pre-hospital or emergency intubations, which tend to be higher in the trauma population.
These challenges in benchmarking are further complicated by the myriad of definitions and diagnostic criteria for VAP. Therefore, we sought to evaluate potential concordance between a bronchoscopic BAL culture-based definition and those utilized in reporting by a Level 1 trauma center, the NHSN, and NTDB criteria.
Three key points within our data should be emphasized. First, only 20.1% of patients defined as having a VAP met all three diagnostic criteria. Second, there is little overlap between the culture definition and the NHSN criteria. Third, a large portion of patients who were diagnosed with VAP by NTDB criteria also had positive cultures. Although the NHSN criteria showed some overlap with the culture-based definition and was very specific, it failed to identify a substantial population of those patients with VAP thus lowering its sensitivity. This correlated with its low NPV. Comparatively the NTDB criteria had a fair overlap with the culture criteria, but there remained a fair percentage in the NTDB group that was not included in the culture-based definition. Even though its false-positive rate lowered its specificity, the NTDB group was more inclusive, thereby giving it a higher sensitivity and relatively good PPV and NPV. When compared with our culture criteria, both NHSN (κ = 0.18) and NTRACS (κ = 0.47) showed high variability. A recent retrospective review performed by Klompas [34] observed similar variability and likely is attributable to an absent gold standard in VAP diagnosis.
There have been at least three studies to date that have evaluated culture criteria versus the NHSN criteria, previously known as the National Nosocomial Infection Survey (NNIS). These studies demonstrated varying degrees of sensitivity, specificity, NPV, and PPV between the two criteria [19,35,36]. The degree of variability between the studies likely depends on the sensitivity of the chest radiography and quantification used for positive culture of BAL samples. Our results are similar to those of Thomas et al. [36], who compared culture criteria with NHSN criteria and observed a lower sensitivity and greater specificity of identifying VAP: 28.2% versus 29% and 92.5% versus 94.8%, respectively. Although comparative studies have been performed, our study is the first to compare all three VAP criteria.
The CDC has recently made changes in their surveillance definition of VAP, as their previous definition used clinical data that lacked specificity. As of January 2013, the algorithm has changed to use objective, clinical data and will now be used to detect ventilator-associated events (VAE) that can then be defined further as either “possible” or “probable” VAP depending on fulfillment of certain additional criteria [37]. These changes were made to aid in a surveillance capacity and not as a guide in clinical care of patients with VAP [37,38]. As the new criterion include non-infectious as well as infectious events and have disregarded radiologic data, future surveillance figures will be unable to be compared with prior CDC surveillance data [38,39]. Given the recent implementation of these new criteria, improved outcomes, development of prevention strategies, and utility for comparisons among hospitals has yet to be determined [39].
There are a few limitations to our data that should be acknowledged. First, this study is limited by all shortcomings inherent of a retrospective review. Second, this was a single-center study and thus was limited by the small sample size and external validity. Third, all assumptions used when comparing NTDB and NHSN criteria with our institution's culture-based criteria were predicated on the fact that a deep respiratory sample with positive culture who met clinical criteria defined a pneumonia. Although our clinical criterion does not have a numerical score, it does follow that of the CPIS, and thus contains parameters that may not be used accurately in trauma patients and be a source of bias. Finally, this study defined a positive culture as >104 CFU/mL of speciated bacteria but it is also been observed in prior studies that >105 CFU/mL maybe more appropriate in the trauma population [40,41]. Although we are currently limited in our laboratory's ability to re-analyze the data using different microbiologic thresholds, we hope to evaluate VAP rates prospectively utilizing 105 CFU/mL to define a positive quantitative culture in the near future.
Conclusions
Within our institution, there is a wide range of VAP reported depending on which set of criteria is used. Our research findings will add to the literature a comparison of the three major different VAP criteria. Ventilator-associated pneumonia is likely underreported to the NHSN because this criterion has low sensitivity thus missing a fair number of patients. This correlated with our data, which showed that only approximately one in four of all VAP identified met NHSN criteria. Alternatively, given the lower specificity of the NTDB criteria and that it included a fair amount of false-positives, VAP may be overreported to TQIP, as 88.2% met the NTDB criteria. The agreement between the NHSN and our culture criteria was poor, whereas the agreement between the NTDB and our culture criteria was fair. Given the variability of VAP rates, which are dependent on the definition used, and the importance of reporting VAP accurately as a measure of hospital quality, possibly influencing financial reimbursement, further investigation should be performed to establish a consensus VAP definition.
Footnotes
Acknowledgment
Presented at the Society of Critical Care Medicine 43rd Critical Care Congress, San Francisco, California, January 2013.
Author Disclosure Statement
None of the contributing authors have any conflict of interests to report.
