Abstract
Abstract
Background:
High rates of inappropriate use of prophylactic antibiotics in surgery continue to be reported in the literature, with many institutions designing interventions aimed at improving prescription. This study evaluates the surgical arm of a clinician-focused educational antimicrobial stewardship program implemented in February 2014 at Blacktown Hospital, Australia.
Methods:
A before-after analysis of the surgical antibiotic prophylaxis intervention was conducted at Blacktown Hospital, New South Wales, Australia. Two hundred abdominal general surgical patients were selected via simple random sampling and categorized into pre-intervention (n = 100) and post-intervention (n = 100) groups. Antibiotic prophylaxis regimens were compared with the Australian guideline, Therapeutic Guidelines: Antibiotic (v14) with respect to drug choice, dosage, timing of administration, and duration of administration.
Results:
Overall adherence rates in the pre- and post-intervention periods were 18% and 15% respectively, demonstrating no substantial change (p = 0.568). No patients in either group were administered antibiotics without an appropriate indication. There were no substantial decreases in error rates across any category, including drug choice, dosage, timing of administration, duration of administration, or re-dosing. The apparent decrease in the rate of inappropriate broad-spectrum cephalosporin usage was not statistically significant (29.3% vs. 18.8%; p = 0.16).
Conclusions:
The educational intervention studied demonstrated no substantial change to overall adherence. Given the frequent failure of such interventions, stronger and more directly mandated adoption of prescribing guidelines is recommended for surgical services. Future consideration should be given to focused computer-based solutions, integrated with electronic medical records where possible.
S
As a natural progression of such research, many healthcare institutions have implemented programs in an attempt to improve appropriate antibiotic prescribing. Although some of these interventions are found to have no impact, a large number result in more than 100% improvement in adherence to prophylactic guidelines [3]. Further, studies concurrently reporting rates of surgical site infections often find a substantial decrease in these nosocomial infections, implying a tangible improvement in patient outcomes [4–6].
When designing an intervention, there is a suggestion that broad educational interventions are beneficial [3,7]. A systematic review reported that benefits from such interventions are independent of specific components [3]. Knowledge and awareness could therefore be responsible for improvement, rather than systemic and procedural changes [3]. Further, educational interventions have been associated with sustained results and a positive cost-benefit relation when compared with interventions with specific objectives [7].
However, identifying and explicitly addressing specific deficiencies have been frequently found to be more effective when large adherence problems exist. These interventions are tailored to specific institutions and are varied in scope, with the literature describing a number of novel measures. Some particularly effective examples have included the restriction of operating theatre antibiotic stocks to address issues with choice of antibiotics, and the transfer of responsibility for cessation of prophylaxis to pharmacy staff to address problems with extended duration of administration [4,8].
This paper describes the current surgical arm of an antimicrobial stewardship program implemented on February 14, 2014, at Blacktown Hospital, New South Wales (NSW), Australia. With limited adherence data and minimal detail on specific deficiencies, this program involved education and general awareness, rather than a unique intervention.
Patients and Methods
Setting and intervention
Designed as a before and after study, this project aimed to evaluate the surgical arm of the newly implemented Antimicrobial Stewardship Program at Blacktown Hospital, NSW, Australia. The 400-bed university teaching hospital provides both elective and emergency surgical services across a wide range of surgical disciplines. More than 9,300 surgical procedures are performed annually, servicing a population of more than 260,000 within Western Sydney.
In response to poor internal audit data, Blacktown Hospital implemented a network-wide stewardship program on February 14, 2014. This multi-faceted intervention focused on a computer-based antibiotic prescribing system, whereas multiple smaller arms addressed specific clinical areas. One of these was surgical prophylaxis. The program mandated the display of prophylactic guidelines in prominent surgical areas, notably throughout the operating suite. These displays included antibiotic prophylaxis guidelines for surgical procedures across the spectrum of major surgical procedures provided at the hospital. Recommended drug choices, dosages, and duration of administration were detailed for both regular cases as well as those involving penicillin hypersensitivity. Separate guidance was provided on appropriate timing of administration for both standard antibiotics and vancomycin infusions.
In addition, the intervention included substantial advertising throughout the hospital site aimed to generally raise awareness of appropriate prescription of antibiotics in all clinical areas. The intervention was one of education and increasing awareness, with no explicit changes to clinical practice.
Data collection
The study was a subanalysis of data collected as part of a larger project investigating overall adherence to surgical antibiotic prophylaxis guidelines, with Blacktown Hospital being one of the sample sites. Two sets of data were used, grouped by time: Pre-intervention (July 1 to December 31, 2013), and post-intervention (February 14 to June 30, 2014). Data were collected according to the protocol for the original study. All abdominal general surgical procedures performed at Blacktown Hospital within the stated time periods were eligible for inclusion. 756 pre-intervention cases and 582 post-intervention cases fulfilled these criteria and were added to a database. A total of 100 study cases were selected from each group via simple random sampling without replacement, utilizing random number generation.
Data were retrospectively sourced from the surgical, anesthetic, and medication records, including electronic and paper documentation. The data recorded included patient demographics (date of birth, gender), surgical characteristics (operation performed, date and time of procedure, duration, urgency classification) and antibiotic details (drugs and dosages administered, timing, duration of administration, allergies if applicable).
Data evaluation
Data were evaluated as per the protocol of the original study. Prophylactic antibiotic regimens were compared with a widely accepted Australian guideline, Therapeutic Guidelines: Antibiotic v14 [1]. This is also the source of the prescribing information provided by the intervention. Cases were considered for adherence to guidelines with respect to four key parameters: Drug choice, dosage, timing of administration, and duration of administration [9]. In addition, re-dosing was considered necessary when the duration of the operation exceeded two half-lives of the drug administered, as per the guideline [1]. Where no prescription was recorded, it was assumed that no prophylaxis had been given.
Non-adherence was determined by the presence of one or more errors in prophylaxis and this was reflected through the sums of error types.
Statistical analysis
SPSS version 21 (IBM Inc., Chicago, IL, USA) was used to perform all statistical calculations. The study was powered to individually meet the needs of both the original study and this subanalysis. Power calculations used standard parameters, with α = 0.05 and β = 0.2, indicating sample sizes of n = 100 per group were needed for the subanalysis reported here.
Frequencies and percentages were reported for categorical data, whereas means and standard deviations were reported for continuous data. Comparisons of categorical data were calculated using chi-square analysis or Fisher exact test where appropriate. Comparisons of continuous data were calculated using the Student t-test. Significance for all analyses was considered when p < 0.05.
Ethics
This study was subject to ethics review and governance under the Western Sydney Local Health District Human Research Ethics Committee.
Results
A total of 200 cases were collected, with pre- and post-intervention groups each containing 100 cases. No cases were excluded following randomized selection.
Comparison of demographic characteristics between cohorts is shown in Table 1. Groups were comparable in age, gender, approach (open/laparoscopic), urgency (elective/emergency), and staffing shift (standard/after hours). Surgical duration in the post-intervention group was substantially longer than in the pre-intervention group (p = 0.04). Distribution of procedures was similar, with laparoscopic cholecystectomy (pre =37%, post = 41%), laparoscopic appendicectomy (pre = 33%, post = 24%) and laparoscopic hernia repair (pre = 14%, post =11%) accounting for the three most common operations in both groups.
= mean ± standard deviation
Overall adherence rates were 18% prior to the intervention and 15% following the intervention, with no substantial change (p = 0.568). Antibiotics were inappropriately withheld in 5% of the pre-intervention group and 3% of the post-intervention group (p = 0.721). No patients in either group were administered antibiotic prophylaxis without an appropriate indication. None of the 10 different surgical procedures included in the study demonstrated a substantial improvement in adherence to antibiotic prophylaxis guidelines.
The frequencies of error categories are represented as a proportion of all cases with errors (pre[n = 82], post[n = 85]), as shown in Table 2. No specific area of adherence improved following the introduction of the intervention. There were non-substantial increases in errors involving timing of administration, duration, and re-dosing. Further analysis demonstrated an apparent reduction in cephalosporin substitution errors involving the inappropriate use of ceftriaxone in place of cephazolin, though this was not substantial (pre = 29.3%, post = 18.8%; p = 0.16).
Note: percentages represent data as a proportion of all cases with errors (pre = 82, post = 85).
Discussion
Appropriate use of prophylactic antibiotics in surgery is an effective measure in reducing the risk of surgical site infection, and guidelines exist to advise this clinical decision [10]. However, the large range of available agents and pressures within the operating environment may lead prescribers to stray from these recommendations. In order to improve the clinical outcomes for patients, as well as minimize the risk of antibiotic resistance, various institutions have introduced interventions aimed at improving adherence to guidelines.
This study investigated an educational intervention at Blacktown Hospital and demonstrated no substantial impact on adherence to antibiotic prophylactic guidelines in general surgery. Further analysis indicated that there was no substantial change in any aspect of prophylactic prescribing, though the increased number of errors involving antibiotic re-dosing approached significance.
Despite suggestions that intervention efficacy is independent of the specific components, this result is ultimately unsurprising [3,11,12]. A growing body of evidence continues to conclude that non-specific educational interventions are frequently ineffective [3,13–15]. A similarly designed study involving an intervention focused on the timing of administration also found a non-substantial increase in errors (p = 0.223) [13]. Further, a large randomized study involving 44 American hospitals concluded that educational quality improvement interventions offered no benefits over regular performance feedback [15].
The reasons for the failure of educational programs are unclear, although a few hypotheses have been suggested. Relatively high pre-intervention compliance has been postulated to indicate sufficient clinician understanding, with an alternate problem existing that prevents optimal prophylaxis [13]. Another suggestion involves cognitive dissonance, whereby generalized education fails to engage clinicians effectively, allowing them to ignore the evidence and continue their regular practice [3].
With respect to Blacktown Hospital, the particularly passive nature of the educational intervention makes the latter a likely contributing factor to the lack of improvement demonstrated. This could be effectively countered by employing evidence-based professional learning theories and heavily involving all clinicians in the development of protocols and the planning of educational sessions [16]. However, the poor pre-intervention adherence rates imply a serious systemic problem rather than an isolated lack of knowledge or understanding. Identification of the most frequent errors in prophylaxis, with subsequent material changes in practice to target these would be an effective and rapid solution.
This paradigm has led to some interesting and effective targeted interventions involving changes to clinical practice. Two studies identified choice of antibiotic agent as a problem and then examined the use of specialized prophylactic antibiotic prescription forms, both finding substantial decreases in errors [7,8]. Other approaches have included extended time-out sequences that include confirmation of choice of prophylaxis [7,13] as well as standard pharmacy-provided antibiotic kits appropriate for each operation [17,18]. Further, use of pharmacy-controlled automatic-stop protocols effectively reduced the extended duration of administration of antibiotics in one study [4].
However, as healthcare continues to adapt to the modern computer-based model, a whole new range of potential interventions are possible. Some specially designed computer-based solutions have already been designed and implemented to good effect. Real-time alert systems can identify incorrect drug choices and suggest alternatives, remind clinicians when the appropriate time for re-dosing is approaching and allow confirmation of the timing of administration [19–21]. Further, these systems allow for more accurate medication records and the automated generation and distribution of general and individual performance data [20].
However, the true potential of such a system is ultimately limited by the costs of a stand-alone solution. In September 2014 Blacktown Hospital adopted an electronic medical record system, pioneering the way for other NSW hospitals. It is the natural progression of this technology to integrate solutions that promote appropriate prophylactic prescribing, with the actual medication prescription modules themselves. This will allow real-time alerts for all aspects of the prescription to enhance and safeguard surgical prophylactic prescribing in just one step. Further, the digital records will allow easy auditing and facilitate simple database research in order to identify areas for further optimization in the prophylactic guidelines. It is reasonable to suggest that this proposed system could result in markedly increased rates of guideline adherence.
Yet, despite all of the evidence supporting these interventions as mechanisms to improve guideline compliance, the real world outcome of these efforts is unclear. Some studies have indicated that although rates of guideline adherence may improve, interventions often produce no substantial improvement in surgical site infection or patient outcomes [3,22–24]. Although it is clear that judicious prescription of prophylactic antibiotics is imperative to prevent antibiotic resistance, the direct clinical significance is uncertain. Further research into the impact on surgical site infections would be justified.
Recognition of the inherent limitations of this study is important when evaluating the results and conclusions. Being a subanalysis of a larger audit, the time frames of this study were limited. This study only examines the outcomes up to 4.5 mo post-intervention and the two groups have slightly unequal time periods. In addition, the retrospective design relies on the accuracy of the medical records used for data collection. Despite this, the study does provide a meaningful analysis of the intervention in its early stages and because of the intervention's static and non-interactive nature, it is unlikely that further improvements would have occurred.
In conclusion, this study adds to previous research that has suggested that educational interventions are ineffective in improving antibiotic prophylaxis prescribing in surgery. Given the consistent failure of these systems, stronger and more directly mandated adoption of prescribing guidelines is recommended for surgical services. The increased use of computer systems in healthcare implies that movement to computer-based interventions may be an effective direction to pursue. Further, research could be undertaken to correlate interventions with rates of surgical site infections, in order to determine the true clinical significance.
Footnotes
Author Disclosure Statement
The authors report no conflicts of interest, financial or otherwise.