Development and Evaluation of an Interactive Simulation Module to Train the Use of an Electrosurgical Device

Abstract

Background:

The purpose of this study was to develop and evaluate an interactive Web-based training module for electrosurgery and use of an electrosurgical device.

Materials and Methods:

The training module consists of a theoretical part, a device tutorial, and an assessment. For evaluation, participants were recruited at the surgical departments from a university hospital and a non-university teaching hospital and were divided into a training group and a control group. All participants performed the same theoretical and practical tests. The training participants first completed the module before they performed the tests. The control participants immediately performed the tests. Results were compared between the training and control participants. To evaluate face validity, the training participants filled out a questionnaire on their opinion about the module.

Results:

In total, 39 participants were enrolled in the study: 20 in the training group and 19 in the control group. The training group answered significantly more theoretical questions correctly (15.7 versus 9.7; P<.001) and made significantly fewer errors in the practical test (2.2 versus 5.6; P=.007). The participants in the training group rated the usefulness and characteristics of the module with high marks. All of them indicated the module to be of additive value to surgical training programs.

Conclusions:

Training with an interactive Web-based module has a positive effect on both theoretical and practical competence regarding electrosurgery and use of an electrosurgical device. This module was rated positively by the participants and was indicated to be a useful addition to surgical training programs.

Introduction

The increasing use of surgical technical equipment in the operating room has introduced new challenges for medical personnel. Besides mastering operating skills, they also have to handle increasing amounts of equipment safely and prevent risks for patients and themselves. The need for such additional competencies has been illustrated by observational studies addressing incidents with laparoscopic operating equipment.^1–3 Electrosurgical devices are one of the multiple types of equipment and are already being used for more than a century in surgery.^4,5 Although these devices have become much safer than they were, laparoscopic surgery introduced a new form of application and with that new sorts of accidents.⁶ The literature has reported accidents occurring with electrosurgery in laparoscopic surgery varying from 2 to 5 per 1000 cases.^6,7 Small studies have shown deficiencies in knowledge on electrosurgery among residents and medical specialists.^8–11

Current education concerning equipment handling, including electrosurgery, is often organized in a traditional way, using lectures and demonstrations given on-site on an irregular basis.¹² Attending such training sessions is often not mandatory. Moreover, assessment as a warrant of competency is frequently lacking.¹² Outside of the hospital, skills courses exist for residents and consultants in which instructions for and training of equipment handling are expected to be part of the content. However, indications exist that such courses lack sufficient consideration of this subject and therefore do not have the intended effect on knowledge and skills.^13–15 In order to offer an easily accessible training alternative for equipment handling, which can be used independently and includes assessment, the purpose of this study was to develop an interactive Web-based training module on electrosurgery and the use of an electrosurgical device. The second purpose of this study was to evaluate this module's training effect and face validity.

Materials and Methods

Development of the module

First, a framework for the module and its training content was designed by the first author (P.D.v.H.). The framework and content were discussed with several experts, among whom were one of the other authors (L.P.S.S.) and a field expert from a company that produces and sells electrosurgical devices (ERBE Netherlands, Werkendam, The Netherlands). Then the actual Web-based module was built by a computer programmer and an interface designer from a company experienced in simulation for endoscopic surgical techniques (Simendo BV, Rotterdam, The Netherlands). The module consists of three parts: a theoretical part, a device tutorial, and an assessment. The electrosurgical device-producing company agreed on the use of one of their electrosurgical devices (ERBE VIO 300D) as a model for a virtual simulated device within the module.

Evaluation of training effect

To evaluate the training effect of the module under study, a prospective study was set up with a training group and a control group. Participants were recruited from the surgical departments of a university hospital and a non-university teaching hospital. Members of the surgical department of the non-university teaching hospital were invited to participate in the training group, and members of the surgical department of the university hospital were invited to participate in the control group. All participants volunteered and signed an informed consent upon participation.

The participants in the training group completed all three parts of the training module and after that performed a separate practical test on the real electrosurgical device (ERBE VIO 300D) to assess their practical competence. Their theoretical competence was extracted from answers to the theoretical questions in the assessment within the module. The participants in the control group did not have any previous training but started immediately with the theoretical and practical tests. The theoretical test for the control group was paper-based but contained exactly the same questions as the assessment within the module. These questions have been validated in a previous study.¹³ For the practical test the participants were given the following assignments: (1) to connect the generator to a power source; (2) to establish all connections to prepare the generator for monopolar diathermy; (3) to establish extra connections for bipolar diathermy; and (4) to adjust several settings for both monopolar and bipolar diathermy for given scenarios. Every assignment was divided into the required actions, and each action was scored as “completed correctly,” “completed incorrectly,” or “omission” as well as the number of attempts. The total time needed for the whole practical test was measured as well. The practical test was exactly the same for training and control participants. To determine the training effect of the module, the results for the theoretical and practical tests were compared between the training group and the control group.

Face validity

Face validity refers to a subjective evaluation of a training method's content and its resemblance to reality.^16,17 To evaluate face validity for the module in this study, all participants in the training group filled out a questionnaire after they had completed the module and the practical test. The questionnaire consisted of 27 questions addressing user-friendliness, level of difficulty, training capacity, level of realism, and target group for the module.

Statistical analysis

All data were analyzed using the statistical software package SPSS for Windows (version 16.0; SPSS Inc., Chicago, IL). For analysis of the training effect, the scores between training and control participants were compared using an independent-samples t test. A P value of <.05 was considered statistically significant. To verify the sample size a power analysis was performed. A difference in means of 3.4 errors for the practical test with a within-group standard deviation of 2.1 and a target significance level of .05 for 19 training participants and 19 control participants achieves a power of 99.7%.

Results

The module

The module consists of three parts: a theoretical part, a device tutorial, and an assessment. The theoretical part contains information about the physical background of electrosurgery, the operating principles, and the possible complications of the use of an electrosurgical generator and its alarms. The tutorial consists of a practical explanation of all connections, buttons, indicators, and menus of the device, including illustrative short videos. An important component of the tutorial is a virtual simulation of the real electrosurgical device in which all connections, buttons, and electrodes are featured (Fig. 1). In the tutorial this simulation can be used for free practice. In the assessment it is used for scenarios in which the user has to establish several connections and settings. These scenarios are alternated by theoretical (multiple choice) questions. At the end of the assessment the user receives a total score and subscores for “theory electrosurgery,” “theory electrosurgical generator,” “equipment handling,” and “trouble shooting.” Scores are given as a percentage of the maximum possible score.

FIG. 1.

Screenshots of the module showing the start screen with the three parts of the module and simulated electrosurgical device.

Evaluation

In total, 39 participants were voluntarily enrolled in the study: 20 participants in the training group and 19 participants in the control group. Table 1 shows that there were no important differences between the two groups, except for the fact that the control group contained more participants who were experienced with an electrosurgical device from ERBE than the training group.

Table 1.

Characteristics of Participants in the Different Groups

	Group
	Training (n=20)	Control (n=19)
Mean age (years)	27	28
Gender (M:F)	12:8	11:8
BLS completed	6	5
Experience as an attending surgeon
0 procedures	13	11
1–20 procedures	1	2
>20 procedures	6	6
Experience with the specific device used in the module in the last 2 years
Yes	0	5
No	11	7
Don't know	9	7

BLS, basic laparoscopic skills course; F, female; M, male.

Training effect

The participants in the training group outperformed those in the control group for both the theoretical test and the practical test (Table 2). There was a statistically significant difference for the mean amount of theoretical questions answered correctly in the theoretical test and for the mean amount of errors made during the practical test. The errors in the practical test mostly consisted of repetitions of the same actions.

Table 2.

Results for the Theoretical and Practical Tests

	Training group (n=20)	Control group (n=19)	P value^a
Theoretical test
Correctly answered theoretical questions (n)^b	15.7±1.8	9.7±2.4	<.001^c
Practical test
Time (seconds)	185±43	220±70	.063
Errors	2.2±2.1	5.6±4.7	.007^c
Repetitions	2.1±2.1	5.2±4.2	.006^c
Incorrect execution	0	0	—
Omissions	0.1±0.4	0.4±1.4	.344

Data are mean±standard deviation values.

By unpaired t test.

Maximum possible score was 18.

Significant difference.

Face validity

Of all 20 participants in the training group, 16 fully completed the questionnaire for face validity. One participant did not complete the questionnaire at all, and 3 other participants did not fill out part of the questions. Overall, the participants rated the user-friendliness, level of difficulty, training capacity, and especially level of realism with high scores (Table 3). Most participants indicated the module to be most appropriate for operating room nurses and junior residents, and all participants indicated the module to be of additive value for surgical training programs (Table 4).

Table 3.

Results for Face Validity, Presented as the Score on a 5-Point Likert Scale

Question	Score
How do you rate the user-friendliness of the … (from 1=very bad to 5=very good)
Module in total	4.0±0.7
Simulator	3.8±0.9
What do you think of the time needed to complete the … (from 1=too short to 3=exactly right to 5=too long)
Total module	3.3±0.8
Theory part	3.4±0.7
Tutorial	3.2±0.7
Assessment	3.5±0.8
How interesting or boring is the … (from 1=very boring to 5=very interesting)
Total module	3.4±0.9
Theory part	3.3±1.1
Tutorial	3.7±0.8
How do you rate the level of difficulty of the … (from 1=very easy to 5=very difficult)
Theory part	3.1±0.8
Tutorial	3.2±0.9
Assessment	2.8±0.7
The assessment tests the content of the theory part and tutorial (from 1=not at all to 5=completely)	4.0±1.0
How do you rate the training capacity of the … (from 1=very bad to 5=very good)
Total module	3.9±0.7
Theory part	3.7±0.7
Tutorial	3.7±0.8
How do you rate the module's training capacity for … (from 1=very bad to 5=very good)
Theoretical background concerning physiology and physics	3.8±0.8
Theoretical background concerning the device	3.8±1.0
Practical handling of the device	4.0±0.8
Troubleshooting for the device	3.4±0.9
How do you rate the realism of … (from 1=not realistic at all to 5=very realistic)
The looks of the simulated device	4.4±0.8
The sounds of the simulated device	4.2±0.8
Connecting the simulated device	4.5±0.8
Operating the simulated device	4.3±0.8
I considered any deficit in realism to be disturbing (from 1=very disturbing to 5=not disturbing at all)	4.3±1.1

Data are mean±standard deviation values.

Table 4.

Results for Face Validity, Presented as Number of Respondents (Total n=19)

Question	Number
I am now capable of operating the real device in a safe and responsible way:
I could already do this	0
I could already do this, but I have improved my skills	2
I was not capable of doing this, but now I am	13
I was not capable of doing this and I still am not	3
Don't know	1
This module is most appropriate for:
Operating room nurses	14
Interns	7
Junior residents	17
Senior residents	5
Consultants	4
This module is of additive value to surgical training programs:
Yes	17
No	0
No opinion	0

Discussion

This article discusses the development and evaluation of a newly designed Web-based module to train both theoretical and practical competence regarding electrosurgery and handling electrosurgical equipment. The evaluation of the module demonstrated a positive training effect as the training group outperformed the control group significantly for the theoretical test, but also for the practical test with the actual physical electrosurgical device. This confirms the hypothesis that it is possible to train practical equipment-related competence without the need of an actual device. This insight offers new opportunities for training and assessment of healthcare personnel. As it has been shown that incidents with technical equipment are not infrequent^1–3 and as requirements by healthcare inspectorates are sharpening, there is a constant need to keep personnel's skills and knowledge up-to-date. Not only is there a need for adequate training, but although few surgeons actually set up an electrosurgical generator themselves, they are legally responsible, and healthcare institutions are required to be able to show proof of the competence of their personnel. Web-based modules, like the one described and tested in this article, might offer a means to live up to such requirements and to offer personnel an easy-to-use alternative for independent training and assessment without the need of an instructor, an assessor, or the availability of equipment. Likewise, such modules can be applied as preparation for skills courses. In a previous study our group showed that laparoscopic skills courses only have a slightly positive effect on participants' knowledge of laparoscopic equipment.¹³ When course participants could use a Web-based module for home study in advance of a skills course, the eventual effect of a course, both theoretically and practically, may increase.

Similar modules are not on the market yet; however, the Society of American Gastrointestinal and Endoscopic Surgeons has developed the Fundamental Use of Surgical Energy (FUSE) program, which is currently undergoing beta testing (see www.fuseprogram.org/fuse-beta-exam/). As far as the authors can judge from the available literature, the FUSE program is mainly theoretical, and its assessment consists purely of multiple choice questions.^18,19 Although the theoretical part of the module described in this study does not seem to be as extensive as the theory in the FUSE program, our module contains the possibility of practicing the operating of an electrosurgical device, and this is also tested in the assessment of the module by the use of scenarios. To the authors' knowledge the FUSE program does not contain this feature.

Participants in the training group gave high scores for user-friendliness, training capacity, and realism. Moreover, all participants who filled out the questionnaire indicated the module to be of additive value to surgical training programs. The participants indicated they thought the module is most suitable for operating room nurses and junior residents. In its current form, the module was designed for residents and medical specialists and in the authors' opinions contains too much detailed theoretical information for operating room nurses. However, adaptations in its content can be made reasonably easily and will make it more appropriate for other groups and thereby expand its applicability.

There is a limitation to this study in the fact that participants were not randomly divided into groups, but that the two study groups were recruited at two different hospitals. This is a potential source of bias. Indeed, there was a difference in familiarity with the specific device used in the module. In the control group more participants had worked with this device in the last 2 years (Table 1). Despite this difference, the training group still outperformed the control group, which actually underlines the positive training effect of the module on practical competence even more. There were no other differences in prior training related to electrosurgery that could have interfered with this effect.

According to the results of this study, the training module can be used to train theoretical and practical competence regarding electrosurgery. However, as this was only a first evaluation of the effect of this module, some questions remain unanswered. For instance, it is not yet clear to which exact groups the module suits best. The assessment of the module was not validated in this study, so no conclusions can be drawn from its scores yet. Therefore, at this stage, the module should only be used for voluntary training, with no consequences related to the results of its assessment. Future research should aim at further distinct evaluation of the exact target group and validation of the assessment together with determination of cutoff scores for “pass” or “fail.”

Conclusions

An interactive Web-based training module was developed for both theoretical and practical competence regarding knowledge of electrosurgery and use of electrosurgical equipment. This study demonstrated a positive effect on theoretical and practical competence after training with the module. All participants who trained with the module rated it positively and indicated it to be a useful addition to surgical training programs.

Footnotes

Acknowledgments

The authors would like to thank all participants of the study for their voluntary contribution. This study was financially supported by a research grant (number P90200) provided by the Dutch national insurance company DSW (Schiedam, The Netherlands).

Disclosure Statement

No competing financial interests exist.

References

Verdaasdonk

, Stassen

, van der Elst

, Karsten

, Dankelman

. Problems with technical equipment during laparoscopic surgery. An observational study. Surg Endosc, 2007; 21:275–279.

Courdier

, Garbin

, Hummel

, Thoma

, Ball

, Favre

, et al. Equipment failure: Causes and consequences in endoscopic gynecologic surgery. J Minim Invasive Gynecol, 2009; 16:28–33.

Panesar

, Salvilla

, Patel

, Donaldson

. Laparoscopic cholecystectomy: Device-related errors revealed through a national database. Expert Rev Med Devices, 2011; 8:555–560.

Beer

. Removal of neoplasms of the urinary bladder. A new method employing high frequency (oudin) currents through a catheterizing cystoscope. Am J Med, 1910; 54:1768.

Cushing

. Electrosurgery as an aid to the removal of intracranial tumors. Surg Gynecol Obstet, 1928; 47:751–784.

Nduka

, Super

, Monson

, Darzi

. Cause and prevention of electrosurgical injuries in laparoscopy. J Am Coll Surg, 1994; 179:161–170.

, Ou

, Chen

, Yen

, Rowbotham

. Complications and recommended practices for electrosurgery in laparoscopy. Am J Surg, 2000; 179:67–73.

Pandey

, Killick

, Lindow

. Obstetrics and gynaecology trainees' knowledge of electrosurgical equipment. J Obstet Gynaecol, 2007; 27:721–722.

Assiotis

, Christofi

, Raptis

, Engledow

, Imber

, Huang

. Diathermy training and usage trends among surgical trainees—Will we get our fingers burnt?. Surgeon, 2009; 7:132–136.

10.

Sudhindra

, Joseph

, Hacking

, Haray

. Are surgeons aware of the dangers of diathermy?. Ann R Coll Surg Engl, 2000; 82:31–32.

11.

Tucker

. Laparoscopic electrosurgical injuries: Survey results and their implications. Surg Laparosc Endosc, 1995; 5:311–317.

12.

van Hove

, Verdaasdonk

, Dankelman

, Stassen

. [Use of checklists for laparoscopic equipment in Dutch hospitals]. Ned Tijdschr Geneeskd, 2012; 156:A4505.

13.

van Hove

, Verdaasdonk

, van der Harst

, Jansen

, Dankelman

, Stassen

. Effect of basic laparoscopic skills courses on essential knowledge of equipment. Surg Innov, 2012; 19:460–466.

14.

Menezes

, Birch

, Vizhul

, Shi

, Sherman

, Karmali

. A deficiency in knowledge of basic principles of laparoscopy among attendees of an advanced laparoscopic surgery course. J Surg Educ, 2011; 6:3–5.e2.

15.

Mayooran

, Pearce

, Tsaltas

, Rombauts

, Brown

, Lawrence

, et al. Ignorance of electrosurgery among obstetricians and gynaecologists. BJOG, 2004; 111:1413–1418.

16.

Gallagher

, Ritter

, Satava

. Fundamental principles of validation, and reliability: rigorous science for the assessment of surgical education and training. Surg Endosc, 2003; 17:1525–1529.

17.

Schijven

, Jakimowicz

. Validation of virtual reality simulators: Key to the successfull integration of a novel teaching technology into minimal access surgery. Minim Invasive Ther Allied Technol, 2005; 14:244–246.

18.

Fuchshuber

, Jones

, Feldman

, Schwaitzberg

, Rozner

. Ensuring safety in the operating room: The “fundamental use of surgical energy” (FUSE) program. Int Anesthesiol Clin, 2013; 51:65–80.

19.

Feldman

, Brunt

, Fuchshuber

, Jones

, Mischna

, et al. Rationale for the fundamental use of surgical energy (FUSE) curriculum assessment: Focus on safety. Surg Endosc, 2013; 27:4054–4059.