Using simulation to improve pharmacy operators’ handling of cytotoxic spills

Introduction

Vial breakages or spills are common risky situations in cytotoxic production units. They lead to a high risk of contaminating pharmacy operators (POs) with these hazardous products and to a risk of drug dispersion in the environment if handling is done incorrectly. Considering that our pharmacy's chemotherapy unit employs 16 POs producing 25,000 cytotoxic preparations per year, they face a non-negligible risk of exposure.

As recommended by the International Society of Oncology Pharmacy Practitioners (ISOPP) guidelines, the handling of cytotoxic spills should be taught during POs’ initial training. ¹ This usually consists of a 1-hour presentation by the trainer responsible, who explains the standard operating procedures (SOPs) and shows the trainees what is included in the spill kit. The ISOPP guidelines also recommend annual refresher training in the form of a spill simulation exercise, which does not occur in our hospital.^1,2 During recent cytotoxic spills (fewer than five per year), POs were observed to be distraught, stressed and slow to react, with difficulties applying the SOPs. They admitted to being uncomfortable and lacking confidence about handling these spills.

Simulation is a training technique that can replace or amplify real-life activities through guided experiences, often immersive in nature, that evoke or replicate substantial aspects of the real world in a fully interactive manner. ³ The present study's objectives were to create an SBT program on handling cytotoxic spills and to evaluate its impact on a population of POs using four criteria: their satisfaction with the training program, improvements to their general knowledge of handling cytotoxic spills, their performance in real-life conditions, and the time they took to handle spills completely.

Methods

The study design is presented in Figure 1.

OPEN IN VIEWER

Figure 1.

Study design diagram.

Evaluation of the training

The SBT program's success was assessed according to Kirkpatrick's four-level scale for evaluating training programs. ⁵ The four levels are Reaction (measuring whether participants found the training relevant to their role, engaging, and useful), Learning (measuring whether participants acquired the knowledge that the training program focused on), Behavior (measuring post-training behavioral changes to see whether participants were applying what they learnt in their daily practice), and Results (measuring the training's impact at an operational level). We did not evaluate the fourth level, however.

Participants completed a mandatory, non-anonymous, 11-item satisfaction survey, on a 0-to-100% scale, to evaluate Reaction. Before and after their training, participants also completed a mandatory, non-anonymous, 20-question survey (1 point per question) to evaluate their Learning about various critical elements of cytotoxic exposure and handling cytotoxic spills. This was inspired by the Foch Hospital's questionnaire (Appendix 1). ⁶

PO Behavior was evaluated using real-life simulation tests 1 month before and 1 month after their SBT session. During these tests, operators had to handle a fake surprise cytotoxic vial breakage during their daily practice. The trainer-pharmacist broke a transparent water-filled glass vial with a fake chemotherapy label marked irinotecan on the floor of the cytotoxic storage area. Each time, the trainer discreetly called over a different operator, thus: “I think we have a problem. Could you please come and help?” The PO had to clean the area by applying the cytotoxic spill handling SOPs. If the PO was reluctant to handle the spill alone and wanted to get some help, the trainer-pharmacist offered that help. If the PO was afraid of being contaminated by the spill and wanted to leave the area, the trainer explained that it was a simulation and invited the PO to apply the SOP anyway. The trainer-pharmacist stood back to evaluate the operator according to a checklist prepared using the SOPs (Table 1). Each successfully completed step scored one point, with a maximum total of 20 points. A full 20 points was equivalent to a 100% score. The trainer also recorded the time needed to handle the spill, from its discovery to the PO leaving the area. The test was repeated one month after the training session using identical conditions.

OPEN IN VIEWER

Table 1.

SOPs.

Step	Objective	Points
Isolation	Evacuate people from the room (answer to the trainer's question: “Imagine there are other people in the room. What would you tell them?”)	1
	Inform a supervisor (pharmacist/pharmacy operator responsible for training) (answer to the trainer's question: “Would you warn someone about the incident?”)	1
	Take the spill kit	1
	Remove and prepare the equipment near the contaminated area	1
	Demarcate the area (warning tape + panel)	1
Total points for the isolation step		5
Protection	Put on the FFP3 mask	1
	Put on the overcoat	1
	Put on the overshoes	1
	Put on the protective glasses	1
	Put on a pair of nitrile gloves and a pair of chemical protection gloves	1 (0 if one type of glove is missing)
Total points for the protection step		5
Cleaning	Disperse the absorbent powder over all the liquid	1
	Pick up the pieces of glass and the powder using the dustpan and squeegee (if there are small pieces of glass, use the pliers)	1
	Clean the contaminated area using an absorbent wipe moistened with water and using spiral movements from the outside to the inside	1 (0 if the movement is not from the outside to the inside)
	Wash with water a total of 3 times	1 (0 if < 3, 1 if ≥ 3)
	Clean with disinfectant and dry the zone	1
Total points for the cleaning step		5
Elimination	Discard all soiled equipment as you go (dustpan, squeegee, absorbent wipe) and the warning tape and panel	1
	Discard the protective clothing (possibly retaining the mask)	1
	Close the kit's blue container or waste bag using the seal and dispose of it before leaving the room	1
	Do a handwash (answer to the trainer's question: “What would you do after handling the spill?”)	1
	Wait approximately 10 minutes before re-entering the cleanroom (answer to the trainer's question: “How long would you wait before re-entering the cleanroom if the incident occurred there?”)	1
Total points for the elimination step		5

Data were collected on a Microsoft Excel® spreadsheet (Microsoft Corporation, Redmond, USA). Differences between pre-training and post-training test scores for knowledge and performance were assessed using Student's t-tests for paired samples (with α = 0.05). The statistical analysis was done using BiostaTGV software (https://biostatgv.sentiweb.fr/), and boxplots were drawn using Datalab software (https://datatab.net/).

Results

Twelve POs participated in the present study, which was conducted over 4 months (from July to October 2023). Their mean post-training satisfaction score (the Reaction level of Kirkpatrick's pyramid) was 98.9% (Table 2). Several positive comments were given about the SBT itself, emphasizing the fact that it was fun and stimulating, necessary for maintaining POs’ professional skills, and updated knowledge on cytotoxic spill handling in a judgment-free environment that promoted collaboration. Other positive comments focused on the use of SBT itself, highlighting that simulation enabled better memorization and recall, that practice made it easier to integrate learning, and that the briefing was essential to bring the SOPs back to mind and put them in context. On the other hand, one participant underlined the fact that some of the questions in the knowledge survey were difficult to understand, and another regretted that it was not an electronic survey.

OPEN IN VIEWER

Table 2.

Results of the satisfaction survey (n = 12).

Question	Score (%)
This simulation is relevant to my professional practice	100
This experience will improve my professional practice	98.3
This simulation model is a relevant educational tool for operators’ continuing training	100
The simulation updated my knowledge of cytotoxic spill handling	98.2
The fun aspects added value for reviewing procedures compared to ‘traditional’ teaching	100
Total simulation training time was of the correct/adequate length	95
The simulation was well organized	96.7
The briefing and debriefing sessions were well facilitated	100
There was a good general atmosphere throughout the simulation	100
The simulation benefitted me	100
I will recommend this training program to colleagues	100
Mean	98.9

Knowledge scores (the Learning level of Kirkpatrick's pyramid) improved from 10.8/20 (SD = 2.0) before the SBT session to 14.5/20 (SD = 1.6) after it (p < 0.05) (Figure 2). Performance scores improved from 78.6% (SD = 7.4%) in the pre-SBT session test to 97.1% (SD = 5.2%) one month afterwards (p < 0.05) (Figure 3). Before the training, the main errors committed were forgetting a piece of personal protective equipment (PPE), such as the mask or the overshoes, and not following the recommendation of cleaning from the outside to the inside. ⁷ Time to handle the cytotoxic spill decreased from 17.3 minutes (SD = 3.6 minutes) before the training program to 11.9 minutes (SD = 1.5 minutes) after it (p < 0.05) (Figure 4). These two results represent the Behavior level of Kirkpatrick's pyramid.

OPEN IN VIEWER

Figure 2.

Improvement in knowledge questionnaire scores (in points), before and after the SBT session (n = 12).

OPEN IN VIEWER

Figure 3.

Improvement in the real-life performance test scores (in %), 1 month before and after the SBT session (n = 12).

OPEN IN VIEWER

Figure 4.

Improvement in time to handle the cytotoxic spill (in minutes), 1 month before and after the SBT session (n = 12).

Discussion

To the best of our knowledge, this was the first study to evaluate the impact of an SBT program to improve real-life cytotoxic spill handling by POs. Indeed, the Behavior level of Kirkpatrick's pyramid has not been previously attained on this subject. The Foch Hospital, in France, provides cytotoxic spill handling training similar to ours to POs from French-speaking countries, but although their knowledge is assessed before and after training, that training's impact on their real-life performance is not. ⁶

Other healthcare professionals could be interested in participating in this type of training. Indeed, simulated chemotherapy spills are widely used to increase the competencies and confidence of oncology nurses. For example, using three simulation scenarios involving leaking 5-fluorouracil, doxorubicin, and cyclophosphamide infusions, DeMacedo increased 29 nurses’ knowledge of the contents of a chemotherapy spill kit and their feelings of being prepared to deal with any future spills. ⁸ In Linnard-Palmer's study, pediatric oncology nurses had to use a spill kit to handle a pool of a simulated, red-colored chemotherapy spill representing doxorubicin. ⁹ The group's training rating scores demonstrated a high level of satisfaction, and they self-reported increased knowledge, confidence, and spill skills performance. ⁹ However, these results concentrated mostly on satisfaction with training or self-reported outcomes; they did not formally assess skills using a real-life evaluation. Once again, previously published studies did not manage to attain the Behavior level of Kirkpatrick's pyramid.

Our results for participants’ satisfaction with their training were similar to those found in the literature, and all our POs agreed that an SBT program was a relevant educational tool for their continuing professional training. As an appreciated and relevant means of training POs, SBT should be used to train other skills and improve knowledge on other topics.

Our POs improved their knowledge about cytotoxic spill handling (the Learning level) by almost 4/20 points, which means that, in theory, they should have a better understanding of how they should react. However, the present study's most valuable results were the SBT program's impact on their real-life performance, which improved by almost 20%, and on the time they needed to handle the spill, which decreased by an average of more than 5 minutes. Thus, after training, operators were acting in a safer way and spending less time in contact with cytotoxic drugs, which improved their personal safety.

Most cytotoxic drugs are transparent, making it complicated to verify whether all the necessary areas have been decontaminated. A solution for simulating environmental contamination by cytotoxic drugs is to use fluorescent markers. Semi-quantitative fluorescent methods have been widely used in environmental contamination evaluation studies involving nurses, ¹⁰ for verifying the effectiveness of closed-system drug transfer devices,^11–13 or for measuring the competency of POs handling cytotoxic drugs. ¹⁴ The concentration of fluorescein in water used is usually 10 g/L.^10,15 We did not use fluorescein, however, but considered that three cleaning operations were necessary, as per the European Society of Hospital Pharmaceutical Technologies video training. ¹⁵

Indeed, several videos exist presenting the procedures for handling cytotoxic spills.^15–17 One of them even explains how to make an “at-home” simulation by providing the list of necessary equipment. Video is an interesting medium with which to reach many people, but it is dependent on the will of the trainer to ensure the implementation of what is explained in the video. Video may also be complicated to update if SOPs change—removing out-of-date information from the internet is difficult. Based on the feedback from the POs who participated in our study, learning by doing is very powerful and watching a video probably does not have the same impact on knowledge and performance.

On the other hand, this type of SBT program is more resource-intensive than a classroom course or a simple demonstration. Firstly, it requires specific simulation material. The costs of the supplies needed for our simulation exercise were covered by our pharmacy's budget but, whenever possible, we used or re-used expired or unused open supplies to reduce waste and study expenditure. Secondly, it requires participant availability and, mostly, time on the trainer-pharmacist's schedule. A comparison of the relative impacts of SBT and a traditional classroom course or videos on POs’ performances in handling cytotoxic spills would be very interesting.

Conclusion

This fun SBT was highly appreciated by the POs who took part in it. It highlighted the added value of this kind of training compared to other, more traditional methods. POs improved their knowledge and their real-life performance when handling cytotoxic vial breakages. They also dealt with incidents faster, allowing production to start up again sooner. Our goal is to include this SBT program in our university hospital pharmacy's routine initial and continuing professional training for POs, as recommended by the ISOPP guidelines, and to provide it to POs in other French-speaking hospitals. There was also a request from our hospital's oncology nurses to adapt this training to them, especially using the scenario involving a leaking cytotoxic bag.