Abstract
Background:
Pneumonia is the most common intensive care unit (ICU)-acquired infection and source of potential sepsis in ICU populations but can be difficult to diagnose in real-time. Despite limited data, rapid initiation of antibiotic agents is endorsed by society guidelines. We hypothesized that a post hoc analysis of a recent randomized pilot study would show no difference between two antibiotic initiation strategies.
Patients and Methods:
The recent Trial of Antibiotic Restraint in Presumed Pneumonia (TARPP) was a pragmatic cluster-randomized pilot of antibiotic initiation strategies for patients with suspected ICU-acquired pneumonia. Participating ICUs were cluster-randomized to either an immediate initiation protocol or a specimen-initiated protocol where a gram stain was required for initiation of antibiotics. Patients in the study were divided into one of seven mutually exclusive outcome rankings (desirability of outcome ranking; DOOR): (1) Survival, No Pneumonia, No adverse events; (2) Survival, Pneumonia, No adverse events; (3) Survival, No Pneumonia, ventilator-free-alive days ≤14; (4) Survival, Pneumonia, ventilator-free-alive days ≤14; (5) Survival, No Pneumonia, Subsequent episode of suspected pneumonia; (6) Survival, Pneumonia, Subsequent episode of suspected pneumonia; and (7) Death. These rankings were further refined using the duration of antibiotics prescribed for pneumonia (response adjusted for antibiotic risk; RADAR).
Results:
There were 186 patients enrolled in the study. After applying the DOOR analysis, a randomly selected patient was equally likely to have a better outcome in specimen-initiated arm as in the immediate initiation arm (DOOR probability: 50.8%; 95% confidence interval [CI], 42.7%–58.9%). Outcome probabilities were similar after applying the RADAR analysis (52.5%; 95% CI, 44.2%–60.6%; p = 0.31).
Conclusions:
We found that patients for whom antibiotic agents were withheld until there was objective evidence (specimen-initiated group) had similar outcome rankings to patients for whom antibiotic agents were started immediately. This supports the findings of the TARPP pilot trial and provides further evidence for equipoise between these two treatment strategies.
Pneumonia is the most common infection and potential source of sepsis in the intensive care unit (ICU).1,2 Despite its prevalence, ICU-acquired pneumonia can be a challenge to diagnose. This diagnostic challenge is amplified in surgical and trauma ICU patients in whom many non-infectious factors can masquerade as infection.3–5 Because it can take substantial time to obtain results from respiratory cultures, there is a period of diagnostic uncertainty during which intensivists must choose between starting antibiotic agents and risking potential overuse versus waiting for additional evidence and risking clinical deterioration. The Trial of Antibiotic Restraint in Presumed Pneumonia: A Surgical Infection Society Multicenter Pilot (TARPP) was a multicenter, cluster-randomized, crossover trial of antibiotic initiation strategies in intubated surgical and trauma patients with suspected but not yet proven ICU-acquired pneumonia. 6 This trial randomized 186 patients (93 in each group) to either a protocol of immediate initiation after airway culture regardless of clinical status, or a specimen-initiated protocol of withholding antibiotic agents until additional objective evidence of infection was identified. Patients with shock were started on antibiotic agents immediately in both arms.
The desirability of outcome ranking (DOOR) and response adjusted for antibiotic risk (RADAR) statistical methodology provides a potential option for improving the power from this small pilot. 7 The purpose of this study is to analyze retrospectively the TARPP pilot data using a DOOR/RADAR analytic method. We hypothesized that this post hoc analysis would similarly find no differences between the two treatment arms.
Patients and Methods
The TARPP pilot (clinicaltrials.gov: NCT04438187) received individual Institutional Review Board approval from each participating institution. This study was conducted using the final fully de-identified database and so specific IRB approval for this study was not required. Ethics approval was not required.
Patients in the TARPP study were stratified retrospectively based on the following adverse events: ventilator-free alive days ≤14, recurrent episode of suspected pneumonia, and death. These events were chosen by the primary study authors before beginning the DOOR analysis. For the purposes of this study, the authors believed that a subsequent episode of suspected pneumonia was a worse outcome than remaining intubated for a longer period without a suspected infection. Because this was a study of suspected but not yet proven infection, each category was further stratified based on the presence or absence of pneumonia based on the final culture results. This strategy yielded the following seven mutually exclusive outcome rankings:
Survival, No Pneumonia, No adverse events, ventilator-free-alive days >14. Survival, Pneumonia, No adverse events, ventilator-free alive days >14. Survival, No Pneumonia, ventilator-free alive days ≤14 (no subsequent episode of suspected pneumonia). Survival, Pneumonia, ventilator-free alive days ≤14 (no subsequent episode of suspected pneumonia). Survival, No Pneumonia, Subsequent episode of suspected pneumonia (any number of ventilator-free alive days). Survival, Pneumonia, Subsequent episode of suspected pneumonia (any number of ventilator-free alive days). Death.
For all patients, the probability that a randomly selected patient would have done better in the specimen-initiated arm compared with the immediate arm was calculated using 95% confidence intervals.
Next, RADAR analysis was applied to the initial DOOR rankings. The duration of antibiotic agents prescribed for the episode of suspected pneumonia that was studied rather than the total days of antibiotic agents for the hospitalization was used in the RADAR analysis. In the TARPP study, only the initial episode of suspected pneumonia was included. Probabilities that a randomly selected patient would do better in the specimen-initiated arm were similarly re-calculated following the addition of the RADAR analysis. Continuous variables were compared using Wilcoxon rank sum whereas categorical variables were compared using χ 2 or Fisher exact tests as appropriate. Statistics were performed using SAS software, version 9.4 (SAS Institute, Cary, NC), R Studio (version 4.2.1) and as well as a web-based DOOR/RADAR research tool from George Washington University Biostatistics Center. 8
Results
All 186 patients in the original TARPP study were included with 93 patients (50%) assigned to each treatment arm. The overall pneumonia rate was 65.6%, and 94.6% of patients were admitted after trauma. When comparing the Immediate versus the Specimen-initiated arm there were no differences in protocol compliance (74.2% vs. 82.8%; p = 0.15), median total days of antibiotic agents (14 vs. 13; p = 0.45), median days of antibiotic agents prescribed for pneumonia (7 vs. 7; p = 0.05), rates of overall resistance (24.2% vs. 26.8%; p = 0.75), or overall mortality (18.3% vs. 18.3%; p = 1.0). There was a median 9.3-hour difference in time to antibiotic agents between the two groups. Antibiotic agents were avoided completely in 19.4% of patients in the specimen-initiated arm. 6
Among patients with confirmed pneumonia, the rate of appropriate initial antibiotic therapy (AIAT) was 86.9% and was not statistically different when analyzed by treatment allocation (90.9% vs. 82.1%; p = 0.15). For trauma patients with confirmed pneumonia, 14.5% had early pneumonia (defined as occurring within 3 days of admission). The rate of early pneumonia did not differ according to treatment allocation (16.1% vs. 12.7%; p = 0.6). The duration of antibiotics for pneumonia was also not statistically different based on early versus late pneumonia.
The breakdown of patient outcomes by adverse event, mutually exclusive outcome rankings, and antibiotic exposure are listed in Table 1. Using the DOOR portion only, the likelihood that a randomly selected patient would have a higher outcome ranking in the specimen-initiated arm rather than the immediate arm was 50.8% (95% confidence interval [CI], 42.7%–58.9%; p = 0.84). Similarly, after applying the RADAR analysis, a randomly selected patient was no more likely to have a better outcome in the specimen-initiated arm when compared with the immediate arm (52.5% likelihood; 95% CI, 45.3–59.7%; p = 0.31).
Patient Factors Used to Define DOOR/RADAR Analysis
DOOR = desirability of outcome ranking; RADAR = response adjusted for antibiotic risk; SD = standard deviation; IQR = interquartile range;
For first instance of suspected pneumonia, not total hospitalization.
Discussion
Current Infectious Diseases Society of America (IDSA) and American Thoracic Society (ATS) guidelines recommend empiric treatment for patients with ICU-acquired pneumonia but provide no guidance on the timing of that intervention. 9 The Surviving Sepsis Campaign guidelines as well as the Centers for Medicare and Medicaid Services Severe Sepsis and Septic Shock Early Management bundle (CMS SEP-1) recommend rapid initiation of antibiotic agents for patients with suspected sepsis. 10 However, recent conflicting evidence for timing of antibiotic agents suggests that for patients without shock, the timing of antibiotic agents may not be as important as once thought.6,11–13
The TARPP pilot is a first of its kind study that found no difference between an immediate empiric antibiotic initiation protocol compared with a specimen-initiated protocol. Specifically, the TARPP pilot found no differences in compliance, mortality, or ventilator-free alive days despite a 9.3-hour median difference in time to antibiotic initiation. 6 In the specimen-initiated arm antibiotic agents were avoided completely in 19.4% of patients. However, as a pilot trial the study is inherently limited. The DOOR/RADAR methodology offers an informative supplement to a randomized trial and can in some cases increase power to detect differences between groups allowing for smaller sample sizes. 7 These potential benefits as well as inherently incorporating antibiotic risk make DOOR/RADAR an ideal methodology to apply to small trials of antibiotic stewardship such as TARPP. This methodology has been used previously to suggest superiority for short-course antibiotic agents using data from the STOP-IT trial. 14 In our case, DOOR/RADAR further highlights the outcome similarities between our two treatment groups. Because our analysis did not demonstrate superiority of one group over the other, this finding bolsters our argument for equipoise and further highlights the need for larger clinical trials of antibiotic initiation in the surgical ICU.
Our study is strengthened by the application of DOOR/RADAR methodology to this smaller trial design, but we also have several limitations. One notable limitation is that multiplex polymerase chain reaction (PCR) was not used in this study. 15 Given our sample size, we are underpowered to detect smaller differences between treatment groups. For example, 19.4% of patients avoided antibiotic agents completely in the specimen-initiated arm, which may lead one to expect superiority for a specimen-initiated protocol in a RADAR analysis that adjusts for antibiotic risk. However, those patients had similar outcomes compared with the immediate arm, suggesting equivalence. Given the small size of the pilot trial and relatively large number of DOOR levels, it is possible that our observed equivalence reflects an underpowered analysis to detect true superiority and reduced statistical benefits of DOOR/RADAR in this case. Additionally, this methodology is sensitive to the way the initial ordinal DOOR levels are defined. Although our DOOR levels were created after completion of the original study, with potential risk of unconscious bias, the DOOR levels were defined before beginning this analysis and based on clinical risk criteria generalizable to other similar patient populations.
Conclusions
In this retrospective DOOR/RADAR analysis, we found that a randomly selected patient from the TARPP study had the same outcome probability regardless of treatment allocation. This outcome supports the initial results of the TARPP study, providing further evidence for equipoise between immediate and specimen-initiated empiric treatment for patients with suspected ICU-acquired pneumonia and supporting the execution of a larger, multicenter trial.
Footnotes
Acknowledgments
The abstract from this manuscript was presented as a poster presentation at the IDWeek meeting on October 14, 2023 in Boston, Massachusetts. A version of the abstract for this study has been previously published in Open Forum Infectious Diseases: Guidry CA et al. 2581. Desirability of Outcome Ranking and Response Adjusted for Antibiotic Risk (DOOR/RADAR) Post-Hoc Analysis Supports Equipoise for Antibiotic Initiation Strategies in ICU-Acquired Pneumonia. Open Forum Infectious Diseases, Volume 10, Issue Supplement_2, December 2023, ofad500.2196; 
Authors' Contributions
Conceptualization: Guidry. Methodology: Guidry, Chollet-Hinton, Baker. Formal analysis: Chollet-Hinton, Baker. Investigation: O'Dell, Beyenne, Watson, Sawyer, Chollet-Hinton, Atchison, Derickson, Cooper, Pennington, VandenBerg, Halimeh, Guidry. Writing: Guidry. Editing and review: All authors
Funding acquisition: Guidry.
Funding Information
Dr. Guidry received a grant from the Surgical Infection Society Foundation for this study. For the remaining authors, no conflicts of interest were declared.
Author Disclosure Statement
The authors have no relevant financial interests to disclose.