Independent Learning of Electrosurgery in Gynecology: A Randomized Controlled Trial

Abstract

Objective:

With limited time for gynecologist residents to learn essential topics, proponents of gynecology residency programs are increasingly advocating brief training programs. The effectiveness of two such programs—independent learning with conventional text review and viewing a web-based podcast—were compared for teaching gynecology residents electrosurgery principles. This research was the second randomized trial and one of few controlled studies to compare text review with web-based gynecologic electrosurgery learning.

Materials and Methods:

Of 38 obstetrics and gynecology residents in two large academic medical centers, 26 were available, consented to participate in the study, and were randomized according to baseline electrosurgery knowledge and residency levels to learn electrosurgery principles independently via chapter review or by viewing an online podcast. Learning of ten electrosurgery principles was assessed using 25 validated multiple-choice questions; 5 included simulated operating-room photographs. Participants were tested before and immediately after the intervention, and 4 weeks later, and completed a satisfaction survey. The primary outcome, test-score improvement, was assessed by Student's t-tests and learner satisfaction was determined using χ² tests.

Results:

Among the participants, 26 residents took the pre-test, 25 took the post-test, and 23 completed the fourth-week post-test. Pre-test scores were similar (p = 0.46). Immediately after the intervention, chapter participants answered 64.3% of the questions correctly and podcast participants answered 54.8% of the questions correctly (p = 0.08). Both groups' scores fell at 4 weeks (−6.5% chapter versus −1.3% podcast; p = 0.21). Most participants felt that they were more knowledgeable and that the training would influence their operating room work.

Conclusions:

The groups' score differences were not meaningful, and both groups' poor scores indicated that the kind of brief electrosurgery training currently promoted could have limited effect on educating residents. (J GYNECOL SURG 33:51)

Introduction

Electrosurgery is the passage of electrical current through a surgical instrument to a patient to produce cutting, coagulation, vessel sealing, and other tissue effects.¹ Electrosurgery may reduce blood loss, and decrease postoperative pain and operative time, but data regarding these outcomes is generally poor.^1–5 Electrosurgery has become an integral part of approaches used in gynecologic surgery, especially laparoscopy and minimally invasive procedures, and research suggests that electrosurgery use may affect clinical outcomes directly.^6,7

Complications from electrosurgery are rare and include the risk of inadvertent burns, poor wound healing, or development of scar tissue.^1–4,8 The International Society of Gynecological Endoscopy emerged in response to the tremendous rise in gynecologic laparoscopy and this society has promoted guidelines for the safe use of electrosurgery.⁹ Other surgical organizations have also published general recommendations on knowledge and use based on evidence of complications.⁸ While specific procedure-related suggestions exist, it is generally accepted that a thorough comprehension of the principles of electrosurgery, familiarity with electrosurgical instruments, and understanding of complex maneuvers and optimal surgical techniques, are essential to ensure the quality and safety of gynecologic use of electrosurgery.^8,10,11

In the few studies evaluating educational outcomes there is evidence that, even after formal training, obstetrics and gynecology (OB/GYN) residents lack knowledge that is essential to perform electrosurgery safely.^12,13 Conventional training in gynecology has included text reviews, didactic lectures, and hands-on training in operating rooms (ORs). However, there is increased knowledge to impart, limited time for education and mastery of many essential concepts, and a need to accommodate various learning styles. Thus, proponents of training programs in the United States and abroad are increasingly recommending brief simulation-based procedural training and novel independent computer-assisted learning, and relying less on conventional didactic sessions.^14–16 There is scant evidence demonstrating knowledge outcomes for any modality, but this is especially true regarding the potential benefits of existing webinars, podcasts, and other online learning sites for medical trainees.^17–20 Stimulation of multiple senses during web podcasts and other online learning resources where the student is able to control the media can improve a student's understanding of the material.²¹ To date, few controlled studies and only 1 randomized controlled trial (RCT) have been published comparing conventional text reading to online learning experiences.¹⁸

Materials and Methods

Design

To assess the efficiency of currently promoted ways to teach OB/GYN residents and augment the sparse evidence, this RCT utilized an opportunity to teach the principles of electrosurgery. The trial was designed to test the relative effectiveness of brief independent conventional chapter-based reading and independent online podcast learning. Assuming that baseline knowledge of the study participants was insufficient, the study's primary hypothesis was that, compared to independent conventional chapter reading, the podcast learning would be associated with a 40% higher mean (immediate) test-score change with a standard deviation (SD) equivalent to 15% of both groups' mean score, representing a meaningful improvement in knowledge of how to practice gynecologic electrosurgery.

Participants

The trial was conducted at two large academic medical centers, the Christiana Care Health System (CCHS) and Cooper University Hospital (Cooper), in the northeastern United States. All 38 trainees in the OB/GYN programs at two institutions were eligible. Of those residents, 26 were available on the day of the testing and training and all participants provided written voluntary informed consent. Those who did not participate had conflicting time constraints or were off-rotation. The 26 residents were randomly allocated (using the randomizer.com^* application) to one of two study groups to learn independently about electrosurgery in gynecology. To ensure the best internal validity (comparability between study groups), the randomization was stratified at each institution by baseline test scores and by postgraduate year (PGY). Every other consenting resident was assigned (in random rather than consecutive order) within each strata to one of the two study groups.

Procedure

Half of the participants were assigned to chapter review using the chapter entitled “Principles of Electrosurgery as Applied to Gynecology” in the 10th edition of Te Linde's Operative Gynecology,²² or to a podcast viewing of Electrosurgery and the Science of Energy, by Scott W. Biest, MD, an instructional 35-minute video created specifically for teaching and demonstrating principles of electrosurgery, which was available to all residents at Ethicon's ClinicalExpertise.com website www.clinicalexpertise.com/contents/video/1487, but is no longer available (online or in print). Both groups were allotted a protected 1-hour period for learning. Each group was gathered into a specified area and was supervised throughout the intervention by the investigators; performance or participation in the intervention was not enforced. The study was conducted in accordance with the latest version of the principles of the Declaration of Helsinki protocol²³ and was approved by the CCHS and Cooper University institutional review boards.

The primary learning objectives were assessed by tests before, immediately after, and at ∼4 weeks after the intervention. The last test was done to assess retention. The three tests contained 25 multiple-choice questions, including 5 simulated OR scenarios, compiled from previously validated electrosurgery questions and new questions validated for clarity and response between the coprincipal investigators (G.E.M. and K.D.S.). The questions were reviewed and selected by an OB/GYN generalist. They were aimed at testing ten main principles of electrosurgery (tissue vaporization, tissue fulguration/open circuit/activation, tissue desiccation, return electrode monitoring, bipolar energy, monopolar energy, cutting versus coagulation waveform, capacitive coupling, direct coupling, and current density). For each test (before, immediately, and at 4 weeks after the learning sessions), the same 25 questions and multiple-choice answers were posed in randomly allocated orders to minimize memory bias. To ensure that the order of the questions and the answers did not influence the total tests' scores across institutions, the order of the questions and their multiple-choice answers for each of the three tests was identical in both institutions. Between tests, study participants had access to both learning interventions, but they were not instructed to engage in or to avoid any further learning activity. No assessment was made of intervening self-initiated learning.

The study also assessed the trainees' satisfaction with the podcast and conventional chapter reading. A brief survey was administered to both groups at the post-test and at the 4-week delayed test to capture descriptions of learner satisfaction with each learning strategy. Satisfaction survey results for the immediate post-test were unavailable at one institution (Cooper), so this is reported here for a single institution alone (CCHS); satisfaction survey results for both institutions are reported here for the 4-week delayed test.

Statistics

Using a one-tailed test, an α error of 0.05 and 1-β = 80%, 11 participants were required in each study group (22 total) to test the primary hypothesis.

To enable longitudinal comparisons, names were obtained on the informed-consent forms and on all study tests. However, to protect confidentiality, anonymous study identification numbers, linked to the names and associated PGY levels and institutions, were assigned; all data entered into MS Excel files were de-identified. The MS Excel data files were converted to, merged and analyzed in SPSS for Windows version 21.

Student's t-tests were used to assess the change in mean test scores. Linear regression analysis with forced entry of study group, PGY, and pre-test scores were conducted to determine the effect of study group controlling for PGY (to represent professional maturity) on immediate and 4-week post-test scores. Answers to the 25 knowledge test questions were coded as correct or incorrect and the proportion of correct answers was summed. Questions answered correctly <30% of the time (therefore would be nearly randomly correct) were also individually analyzed for each of the three tests. Satisfaction with the training regimen was captured on a 3-point scale condensed from 5-point Likert scale questions and assessed by chi-square tests. Descriptive analyses of mean scores by PGY and institution presented here, although no causal inference is drawn from these stratified analyses.

Results

Of the 38 gynecology residents (22 CCHS, 16 Cooper), 26 were present and all consented to participate on the day of introduction (Table 1). All 26 took the pre-test, 25 took the post-test and 23 took the 4-week post-test. The average PGY was 2.5 ± 1.3 in the chapter group and 2.4 ± 1.0 in the podcast group (p = 0.74); PGY distribution between sites was similar. Participants randomly assigned to the chapter group had similar, 4.3% higher (51.7%), correct mean scores at pre-test than those in the podcast group (47.4%, Table 1, total sample p = 0.27) and scores were similar between the institutions.

Table 1.

Mean Scores by Study Group (Chapter and Podcast) and Institution (CCHS and Cooper University Hospital), 2014

	Total			CCHS			Cooper
Time frame	Mean	SD	n	Mean	SD	n	Mean	SD	n
Chapter
Pre	51.7	7.9	13	54.3	8.6	7	48.7	6.4	6
Post	64.3	8.9	12	65.3	10.6	6	63.3	7.8	6
4-week post	57.5	11.8	11	60.0	15.0	6	54.4	6.7	5
Web podcast
Pre	47.4	11.1	13	48.6	14.7	7	46.0	5.5	6
Post	54.8	4.4	13	54.9	6.0	7	54.7	2.1	6
4-week post	53.3	13.2	12	58.3	14.6	7	46.4	7.8	5

CCHS, Christiana Care Health System. SD, standard deviation

Immediately after the learning session, chapter group participants' scores increased by nearly 14 points (3.5 questions improvement), to 64.3% compared with the podcast group improvement of 7 points (1.75 questions), to 54.8% (Table 2; p = 0.08). Results for each institution were similar among both groups, with slightly less score improvement at CCHS and slightly more score improvement at Cooper. However, knowledge retention was worse in participants assigned to the chapter study group, with nearly half, 6.5 of the initial 13.7 points, improvement lost. Participants assigned to the podcast, who had less initial improvement, lost only 1.3 of the 7.4 points improvement in 4 weeks' time. Therefore, at 4 weeks after the learning session, there was only a 4.2% difference (roughly equivalent to 1 test question) in test scores between the chapter (57.5%) and podcast (53.3%; p = 0.84) group. Analyses excluding the three individuals who did not take the 4-week post-test were conducted to assess bias. The results were similar.

Table 2.

Chapter and Podcast Differences Between Pre, Post, and 4-Week Post Electrosurgery Test Scores by Institution (CCHS and Cooper University Hospital), 2014

	Total sample				CCHS				Cooper
Time frame	Mean difference	SD	n	p^a	Mean difference	SD	n	p^a	Mean difference	SD	n	p^a
Post vs. pre
Chapter	13.7	7.9	12	0.08	12.7	7.3	6	0.24	14.7	9.0	6	0.23
Web podcast	7.4	8.9	13		6.3	10.8	7		8.7	6.9	6
4 wks vs. pre
Chapter	5.8	6.3	11	0.97	7.3	8.2	6	0.67	4.0	2.8	5	0.41
Web P\podcast	5.7	11.1	12		9.7	11.0	7		0.0	9.4	5
4 wks vs. post
Chapter	−6.5	6.0	11	0.21	−5.3	7.0	7	0.15	−8.0	4.9	5	1.00
Web podcast	−1.3	12.5	12		−3.4	12.7	7		−8.0	9.4	5

Unequal variances assumed for p-values.

CCHS, Christiana Care Health System; wks, weeks; SD, standard deviation

Controlling for PGY and pre-test scores, the podcast group scored significantly (Table 3; p = 0.002) but not meaningfully (by <1 percentage point) lower than the chapter group on the immediate post-test. At the 4-week post-test, scores were virtually identical to the pre-test scores and showed no difference in scores by group. PGY was not associated with pre-test scores and, controlling for professional maturity as represented by PGY, it was found that higher PGY was not associated with higher post-test or 4-week delayed test scores. Higher PGY was marginally associated with less immediate improvement in score, but this was by <1 percentage point p = 0.08), which was not clinically meaningful. Assessing pre-test score as another measure of professional maturity showed that a higher pre-test score was associated with more immediate and delayed improvement (both p ≤ 0.005).

Table 3.

Linear Regression Analysis Assessing the Influence of Study Group ^a and PGY on Immediate and 4-Week Post-Test Scores Controlled for Pre-Test Scores, 2014

Immediate Post-Test (n = 24)
		95% CI
R² = 0.575	Beta	Lower	Upper	p
(Constant)		38.81	71.82	0.000
Study group	−8.35	−13.29	−3.41	0.002
PGY	−1.97	−4.20	0.27	0.081
Pre-test score	0.440	0.16	0.72	0.004

4-Week Post-Test (n = 22)
		95% CI
R² = 0.512	Beta	Lower	Upper	p
(Constant)		−17.34	38.94	0.432
Study group	0.017	−8.45	8.48	0.997
PGY	−1.74	−5.74	2.27	0.376
Pre-test score	0.98	0.50	1.46	0.000

(1 = Chapter, 2 = Web podcast)

PGY, postgraduate year; CI, confidence interval.

Analysis of questions answered correctly revealed that 9 pre-test questions, 3 post-test questions, and 4 delayed post-test questions had a 30% correct answer rate. Those questions varied across the test periods. Exclusion of these questions changed mean scores and the differences between test scores by <1%.

Survey results showed that, at CCHS, immediately after the learning intervention 50% of participants in the chapter and 42% of participants in the podcast group felt that they were generally knowledgeable (p = 0.43). After 4 weeks, when data were available from both institutions, 73% in the chapter and 67% in the podcast groups felt this way (p = 0.62; Fig. 1). Sixty-seven percent of CCHS chapter trainees felt somewhat or completely knowledgeable about energy coupling, return electrode monitoring, and cutting and coagulating waveforms immediately after training compared with 43% in the CCHS podcast group (Fig. 2; p = 0.32). This effect was 6% higher in the chapter and 24% higher in the podcast groups in both institutions after 4 weeks (p = 0.78). Immediately after training, 50% of the chapter and 43% of the podcast trainees at CCHS felt that the information was important for members of their profession, was presented in a style conducive to their learning, and that they were more knowledgeable about using electrosurgery. They also felt that the information would help them care for patients receiving electrosurgery better and would change how they managed these patients in the OR (p = 0.63), compared with 82% and 75% of chapter and podcast trainees from both institutions 4 weeks later (p = 0.62).

FIG. 1.

How prepared trainees felt about setting up and using a Bovie device safely in open surgery, and in monopolar and bipolar energy during laparoscopy after studying.

FIG. 2.

How knowledgeable trainees felt about energy coupling, return electrode monitoring, and cutting and coagulating waveforms.

Discussion

This trial showed that, while both independent chapter reading and podcast training in 10 basic principles of gynecologic electrosurgery improved knowledge immediately after training, it did not do so substantially and, after 4 weeks, there was no learning improvement in either of the study groups. Ultimately, both groups scored poorly overall after the training, with 40%–50% of questions answered incorrectly. As has been increasingly promoted for resident training, both learning interventions were conducted under the same circumstances within 1 hour of protected learning time. While podcast trainees might have been distracted by web availability and experienced less initial improvement, such distraction, if it occurred, had limited immediate effect and no influence on retention. Any initial concern that study participants could seek and find the learning intervention they were not assigned to and review it (crossover learning) was moot as indicated by the limited improvement, lack of retention, and generally low (< 70) scores.

Although this study did not assess residents' prior experience with electrosurgery, pre-test scores were not associated with PGY or institution, nor were there meaningful differences between groups. Therefore, there was little potential for bias based on this unknown experience. PGY was marginally associated with less immediate improvement in scores. Excluding PGY from the regression analyses minimally, by ∼5%, reduced the amount of variance accounted for by the model for immediate and 4-week post-test outcomes. This indicated that PGY alone might not have been a reasonable measure of ability to grasp and/or retain electrosurgery knowledge.

Contributing to medical education evidence is challenging.²⁴ One review of studies comparing face-to-face and computer-assisted learning found only 10 of 728 were controlled trials and only 1 of those was an RCT.¹⁸ The strengths of this multisite study included the tightly controlled study design with its highly stratified randomization, which achieved comparability and reduced potential confounding between the study groups. The study was purposefully powered a priori to test a meaningful knowledge improvement that would be consistent with an expectation of better practice. The available and attained sample exceeded the proposed 80% power. A larger sample would have allowed the study to test smaller differences than those observed. However, those differences were negligible and not considered to be clinically meaningful. The study also used validated questions. One trainee found that the format of some of the multiple-choice questions was confusing and might have affected responses negatively, but such question formats are standard in hospital compliance and knowledge tests. The trial also assessed change in knowledge at two points in time—immediately and 4 weeks after training—relative to baseline levels. Evaluating each participant's change in knowledge also reduced the primary outcome's variability, thus improving the measure's reliability.

While the study was conducted at two large institutions in the northeastern United States, the sample of residents was small and might or might not be generalizable. The findings of this study are consistent with a survey of OB/GYN trainees in the United Kingdom whose knowledge of electrosurgery was poor. That survey found that focused training in electrosurgery was ineffective.¹³ This might not mean that the chapter or podcast training mechanisms are ineffective, but rather that the 1-hour dedicated training period was insufficient. It is reasonable to question for which topics such time-limited intervention would be effective. It is possible that residents simply were not able to apply information conveyed in the podcast to answering the test questions, but poor design of computer-assisted learning tools has also been cited as a major reason for such tools not achieving their objectives.²⁵

While satisfaction questions often produce a positive response bias, the fact that most residents felt that the training had made them more informed and would influence their operating room performance—in the absence of knowledge improvement—is disconcerting given their poor scores.

Identifying the ideal educational tools is a constant challenge as medical knowledge and trainees continue to change. Some researchers have argued that conventional face-to-face lectures with minimal engagement can be overwhelming and ineffective. This concept has been part of the impetus to try new approaches.¹⁶ Only one of 10 controlled trials found face-to-face teaching superior to e-learning, although adding hands-on practice to face-to-face teaching produced the best results.¹⁸ Other researchers have suggested that the internet is a good mechanism for reaching wide audiences and providing supplemental educational material, although people with substantial time pressures tend to avoid reviewing supplemental material.²⁶ Reviewers have found no clear superiority of computer-assisted instruction over conventional training for OB/GYN, and the current study suggested that computer-assisted training was not preferable for these residents.^18,24

The extent of incorrect answers demonstrated in this study after both types of independent instruction was considerable and indicated that interactive, simulation, or additional electrosurgery instruction is necessary for trainees to master the complex principles involved. The current study and other studies have suggested that we need better assessments and evidence of our educational success before we assume that our residents have acquired new knowledge and/or skills. Moreover, such assessments are necessary to prevent trainees from poor performance and possibly poor patient outcomes as a result of perceived understanding of concepts.

Conclusions

Neither chapter review nor podcast viewing in a 1-hour structured learning environment affected the study residents' knowledge of electrosurgery meaningfully. More-effective means of teaching electrosurgery and/or more time for learning and standardized assessments of educational programs need to be developed and distributed for gynecology residents.

Footnotes

Acknowledgments

The authors thank Gunda Simpkins, MSN, MPH, manager of clinical research, in the department of obstetrics and gynecology at Cooper University Hospital for her contribution to the study's data management. The authors also thank the institutions' OB/GYN residents for their participation in the study.

Financial support for this study was provided by internal funding from both Christiana Care Health System and Cooper University Hospital.

Author Disclosure Statement

The authors report no conflicts of interest. No support was received from the National Institutes of Health, Wellcome Trust, the Howard Hughes Medical Institute, nor the Research Councils of the United Kingdom.

References

Charoenkwan

, Chotirosniramit

, Rerkasem

. Scalpel versus electrosurgery for abdominal incisions. Cochrane Database Syst Rev, 2012; 6:CD005987.

Ball

. Lasers: The Perioperative Challenge, 3rd ed. Denver, CO: AORN, Inc., 2004.

Ulmer

. The hazards of surgical smoke. AORN J, 2008; 87:721;quiz:735.

Barrett

, Garber

. Surgical smoke: A review of the literature. Is this just a lot of hot air?. Surg Endosc, 2003; 17:979.

, Mittal

, Windsor

. Systematic review and meta-analysis of cutting diathermy versus scalpel for skin incision. Br J Surg, 2012; 99:613.

Hur

, Donnellan

, Mansuria

, Barber

, Guido

, Lee

. Vaginal cuff dehiscence after different modes of hysterectomy. Obstet Gynecol, 2011; 118:794.

Levy

, Emery

. Randomized trial of suture versus electrosurgical bipolar vessel sealing in vaginal hysterectomy. Obstet Gynecol, 2003; 102:147.

, Ou

, Chen

, Yen

, Rowbotham

. Complications and recommended practices for electrosurgery in laparoscopy. Am J Surg, 2000; 179:67.

The National Institute of Occupational Safety and Health (NIOSH), Centers for Disease Control and Prevention. Control of Smoke from Laser/Electrical Surgical Procedures. [Department of Health and Human Services Publication 96–128]. 1996. Online document at: www.cdc.gov/niosh/docs/hazardcontrol/hc11.html Accessed October 11, 2016.

10.

Pandey

, Killick

, Lindow

. Obstetrics and gynaecology trainees' knowledge of electrosurgical equipment. J Obstet Gynaecol, 2007; 27:721.

11.

Makai

, Isaacson

. Complications of gynecologic laparoscopy. Clin Obstet Gynecol, 2009; 52:401.

12.

Fanning

, Kesterson

, Davies

, Green

, Penezic

, Vargas

, Harkins

. Effects of electrosurgery and vaginal closure technique on postoperative vaginal cuff dehiscence. JSLS, 2013; 17:414.

13.

Chen

, Korn

, Klingele

, Barber

, Paraiso

, Walters

, Jelovsek

. Objective assessment of vaginal surgical skills. Am J Obstet Gynecol, 2010; 203:79e1.

14.

Cohen

, Barsuk

, Moazed

, Caprio

, Didwania

, McGaghie

, Wayne

. Making July safer: Simulation-based mastery learning during intern boot camp. Acad Med, 2013; 88:233.

15.

Berkowitz

, Peyre

, Johnson

. Mobilizing faculty for simulation. Obstet Gynecol, 2011; 118:161.

16.

Jones

, Saunders

, Mires

. The e-learning revolution in obstetrics and gynaecology. Best Pract Res Clin Obstet Gynaecol, 2010; 24:731.

17.

Dauphinee

, Wood-Dauphinee

. The need for evidence in medical education: The development of best evidence medical education as an opportunity to inform, guide, and sustain medical education research. Acad Med, 2004; 79:925.

18.

Zhang

, Watson

, Banfield

. The efficacy of computer-assisted instruction versus face-to-face instruction in academic libraries: A systematic review. J Acad Libr, 2007; 33(4):478.

19.

Khasawneh

, Simonsen

, Snowden

, Higgins

, Beck

. The effectiveness of e-learning in pediatric medical student education. Med Educ Online, 2016; 21:29516.

20.

Kim

, Coolen

. A prospective cohort study using e-learning modules as a supplemental teaching resource for obstetrics and gynaecology clerkship students. J Obstet Gynaecol Can, 2015; 37:819.

21.

Mayer

. Applying the science of learning to medical education. Med Educ, 2010; 44:543.

22.

Sonderson

, Brill

. Principles of electrosurgery as applied to gynecology. In: Rock

, Jones

III . Te Linde's Operative Gynecology, 10th ed. Philadelphia: Wolters Kluwer/Lippincott Williams & Wilkins, 2008; 280.

23.

World Medical Association Declaration of Helsinki: Ethical principles for medical research involving human subjects. JAMA, 2013; 310:2191.

24.

Letterie

. Medical education as a science: The quality of evidence for computer-assisted instruction. Am J Obstet Gynecol, 2003; 188:849.

25.

Friedman

. Top ten reasons the world wide web may fail to change medical education. Acad Med, 1996; 71:979.

26.

Hammoud

, Gruppen

, Erickson

, Cox

, Espey

, Goepfert

, Katz

; Association of Professors of Gynecology and Obstetrics Undergraduate Medical Education Committee. To the point: Reviews in medical education online computer assisted instruction materials. Am J Obstet Gynecol, 2006; 194:1064.