Preoperative Verification of Timely Antimicrobial Prophylaxis Does Not Improve Compliance with Guidelines

Patients and Methods

The Institutional Review Board of the University of Minnesota approved the study. The change in preoperative procedure to include verification of timely antibiotic administration was implemented in March 2006. We conducted a retrospective review to determine whether the proportion of patients receiving timely prophylaxis increased after this intervention.

After one month of education of the anesthesia, nursing, and surgical staffs, the inclusion of the question “Has the antibiotic been administered within the correct time frame?” began officially on March 13, 2006. Previously, this “time out” included a verification only of patient identity, operative procedure, and surgical site.

We reviewed the medical records of all patients undergoing an operation at our hospital during two five-day periods in 2006, one pre-intervention (January 9–13) and one post-intervention (March 13–17). The data extracted from these records were operative procedure, time of antibiotic administration, time of incision, and the patient's preoperative inpatient or outpatient status. Excluded from analysis were patients whose operation by current surgical guidelines did not require antibiotics, patients who were inpatients prior to their procedure and may already have been receiving antibiotics, and those with incomplete records. Decisions regarding the exclusion of a case because it did not require antibiotic prophylaxis were made by reviewing recent literature in collaboration with practicing surgeons.

Data were entered in an SPSS database (SPSS, Inc., Chicago, IL) where the time between antibiotic administration and surgical incision was calculated. We defined compliance with the standard of care as antibiotic infusion occurring within 60 min prior to incision. Administration of antimicrobial prophylaxis was considered timely, early (>60 min prior to incision), late (beginning after incision), or not given (no antibiotics administered). Those cases where the start of antibiotic infusion occurred at the time of incision were considered to be timely. For operations requiring vancomycin, the analysis was modified to include administration up to 120 min prior to incision as timely.

The proportions of timely administration of antimicrobial prophylaxis in the pre-intervention and post-intervention groups were compared using the χ² statistic for non-continuous variables with significance noted as p ≤ 0.05. Statistical analyses were performed using SPSS for Windows v. 12.0.

Results

A total of 715 cases were reviewed, including 332 cases that took place at our hospital during the pre-intervention week and 383 during the post-intervention week. A total of 425 cases were excluded: 315 that did not require antibiotics, 88 inpatient procedures, and 22 cases with incomplete records. Similar percentages of cases were excluded in the two groups for inpatient status and incomplete records, but a larger portion of the pre-intervention patients were excluded because they did not require antibiotics, 56% vs. 33% (Table 1).

In the pre-intervention group, 87 of 97 patients (90%) received timely antibiotic administration. In the post-intervention group, antibiotic administration was timely in 163 of 193 cases (85%). Although pre-intervention compliance was slightly better than post-intervention, this difference was not significant (p = 0.223). Antibiotics were administered early in three patients (3%) in the pre-intervention group and 7 (4%) in the post-intervention group. Antibiotic administration was late in two (2%) of the pre-intervention cases and in 12 (6%) of the post-intervention group. Antibiotics were not given in five (5%) of pre-intervention cases or in 11 (6%) of post-intervention cases (Table 2 and Fig. 1).

OPEN IN VIEWER

FIG. 1.

Timeliness of antibiotic administration.

Discussion

The results of this study demonstrate that in the setting of good, but not perfect, compliance with antimicrobial prophylaxis guidelines, the addition of preoperative verification does not improve timely antibiotic administration significantly. In our post-intervention group, 4% of patients received antibiotics too early, 6% received them too late, and 6% did not receive them at all, demonstrating no significant improvements from the pre-intervention group. The possible explanations for this result are: (1) Preoperative verification was not done in some cases; 2) verification was done incorrectly; or 3) there was an inappropriate response or a lack of response to the verification.

In recent years, as it has become clear that we are falling short of our goals in the prevention of SSIs, many institutions have made changes in preoperative procedures in an attempt to increase compliance with the NSIPP guidelines. One of the first modifications shown to increase compliance with the guidelines was the transfer of responsibility for antibiotic administration from the nursing staff in the preoperative area to the anesthesia care provider in the operating room [12–15]. The study published by Matuschka et al. in 1997 showed an improvement from 32% to 88% on-time administration at the Louisville Veterans Affairs Medical Center after responsibility was shifted from ward or holding room nurses to the anesthesiologists [12]. Webb et al. also showed dramatic improvement, from 51% to 98% on-time administration, when responsibility was shifted to the anesthetist and administration occurred in the operating room rather than on the floor or in a preoperative holding area [14]. Antibiotic administration in the operating room by the anesthesia provider has become standard practice at many institutions, including our own.

Yet even in the context of these changes, many hospitals have demonstrated persistent inadequacy in this area and have continued to investigate ways to increase their compliance with the standard of care. One such effort, studied by O'Reilly et al. at the University of Michigan, not only transferred responsibility to the anesthesia provider, but used a computer system with checklist reminders and physician-specific feedback to improve the timely administration of antibiotics. Their results showed improvement from 69% to 92% on-time administration within one year [15]. Wax et al. evaluated the effect of an interactive visual reminder system integrated into the anesthesia provider's computer charting system. Their baseline compliance rate of 82.4% was much higher than that in the studies cited above and similar to our own. With the implementation of the visual reminder system, they demonstrated modest but statistically significant improvement to 89.1% [16]. Consistent in each of these efforts is the responsibility of timely antibiotic administration lying solely with the anesthesia provider. In some cases, these efforts also rely on the development and implementation of new computer systems in addition to practice changes.

In the complex perioperative setting, sustained behavior change may best be realized by the integration of new practices into reliable existing routines and by the inclusion of all present surgical staff in the new practice. A case study in brief published in 2006 in the Joint Commission Journal on Quality and Patient Safety studied a change, similar to our own, in the preoperative surgical time-out, which was the addition of the question of whether the antibiotics had been given. This study showed improvement from 86% to 98% in three months [17]. Our own inconsistency with this result may best be understood with consideration of the fact that this group had in the previous year implemented other quality improvement techniques, including signage, education, and informal feedback/encouragement. These efforts may have made the operating team prepared for responding effectively to formal preoperative procedure change when it occurred. Additionally, data were collected for three months after the intervention, allowing the observation of sustained improvement with time.

Other investigators have used an even broader strategy to address deficiencies in guideline compliance. Parker et al. evaluated in 2007 the effectiveness of implementing Six Sigma methodology to improve adherence to the antibiotic prophylaxis guidelines [18]. The approach, which is used widely in business and manufacturing, is now familiar to many in the healthcare industry. Its goal is to improve quality and efficiency by reducing process variability through rigorous data analysis, team training, process mapping, and multiple targeted interventions. In study by Parker et al., the pre-intervention audit of 615 cases revealed only 38% compliance with the guidelines for timing of antibiotic prophylaxis. Six Sigma methodology was designed and implemented to cover seven process inputs potentially affecting the timely preoperative administration of antibiotics. Examples included reinforcement of the use of standardized order forms and restriction of antibiotic administration to the operating room. The intervention also included an intensive one-week educational effort for all relevant staff. Data were collected prospectively, and the results showed significant improvement. Yet it is notable that of the 1,716 post-intervention cases analyzed, only 86% received antibiotics within 60 min of incision [18].

Although the surgical time-out has become an essential tool in our efforts to decrease injury to patients, it is vulnerable to human error. In a 2007 study, Clarke et al. reported 427 “near-misses” and surgical interventions actually started involving the wrong patient, procedure, side, or site over 30 months in Pennsylvania, despite the routine use of time-outs [19]. The authors found that in 34% of the cases where an incorrect surgical intervention was started, there was a documented failure of the time-out (e.g., done before positioning, started at or after the time of incision). Yet the authors also noted that “14 complete wrong-site procedures were done despite time-outs described without shortcomings.” Although the purpose of the surgical time-out is to prevent human error from precipitating injury to the patient, the time-out itself is vulnerable to our own mistakes. These mistakes theoretically include complete omissions, late time-outs, inaccurate verification, and inadequate responses to preoperative deficiencies revealed by the time-out.

Our own study demonstrates that preoperative oral verification is not fail-safe. It is interesting to consider further how failures of the time-out might occur. The possibilities specific to our study include complete omission, exclusion of the antibiotic verification, time-out occurring at the time of incision, which would not allow correction of inadequate antibiotic prophylaxis, or an incorrect or incomplete response to the verification of antibiotic administration. A consideration of these possibilities reveals shortcomings of our study, the foremost being that we did not collect our data from direct observation but rather through a review of operative documentation.

Partly on the basis of these data, our institution performed human factors analysis and modified the time-out process similar to the steps outlined by the Safe Surgery Saves Lives Study Group [20]. Initiation of the new process included a more robust, system-wide education effort, proctoring for appropriate performance, on-going monitoring for compliance, and possible consequences for noncompliance. However, even with successful implementation of checklists, timely preoperative antibiotic prophylaxis differs widely (50.6%–96.9%) and is followed in only 88% of patients [20], similar to our results.

In addition to being vulnerable to recorder error, the documentation used for our data collection did not include a record of whether the time-out was done correctly or even if it was done at all. It did not say whether the time-out revealed a preoperative deficiency or whether there was an appropriate response. We thus cannot confirm that the surgical time-out was delivered in its correct augmented form at the start of every operation in the post-intervention group, nor can we deny that there were any informal antibiotic verifications in the pre-intervention group. Anecdotal reports from surgeons endorse the consistency of time-outs and specifically that the augmented form was used routinely in the post-intervention period. Still, we do not have data that can reveal the specific ways in which the preoperative verification failed in some cases. Thus, our analysis appears to be incomplete.

Our study has major limitations in addition to our inability to verify the consistency and quality of the time-outs. Our post-intervention group included only those operations occurring in the five-day period following the change in preoperative procedure. This did not allow observation of potential improvement over time as the new practice became accepted as routine and as strategies to correct cases of non-compliance became habitual. Another important limitation is that although the total numbers of cases performed in the pre-intervention and post-intervention groups were similar, a much higher proportion of cases in the pre-intervention group did not require antibiotics and thus were excluded from analysis. Finally, longer interval followup, to include an audit of the new practice, would be useful in determining the value of our intervention.

Conclusions

We describe a single-center effort at improving preoperative prophylactic antimicrobial compliance by adding a reminder to the preoperative verification process. The addition of this step did not improve compliance significantly. It remains unclear whether improvement to a goal of 100% compliance with prophylactic antibiotic guidelines in the setting of good pre-intervention compliance requires more than the addition of a preoperative verbal verification, although our own experience as well as that of others suggests that it does.