Learning Laparoscopic Skills: Observation or Practice?

Abstract

Objective:

The aim of this study was to assess the respective roles of observation and direct practice in the retention of laparoscopic skills.

Materials and Methods:

Eighteen fifth-year medical students were included in a two-session laparoscopic learning course. During the first session, each participant was given four tasks to complete from the “Basic skills” and “Essential tasks” modules of the Simbionix LAP Mentor™, and another four tasks for observation only. During the second session, each participant completed all eight tasks. Performance evaluation was assessed using the objective structured assessment of technical skills (OSATS) global rating scale and LAP Mentor metrics.

Results:

The mean OSATS score during the first session (S1) was 16.7 ± 3.2. This increased by 34% during the second session (S2), reaching 21.8 ± 2.6 in the group of former observer students (S2O, P < .0001), and by 56% (25.1 ± 1.9) in the group of former practicing students (S2A, P < .0001). The analysis of LAP Mentor metrics showed that 14 of 28 parameters (50%) improved in the S2A group compared to S1, whereas only 25% of the parameters improved in the S2O group, the difference being significant (P = .048). In both groups, the more complex the task, the more the number of improved parameters decreased.

Conclusions:

Although simple observation of laparoscopic skills improved further performance, direct practice on the virtual reality trainer ensured more effective training. This work therefore advocates incorporating personal training on simulators into residents' surgical curricula.

Introduction

The development of laparoscopic surgery over the past 20 years, with its specific difficulties for surgeons, has challenged the traditional surgical apprenticeship in the operating room.^1,2 Various constraints (e.g., economic, medical, legal, ethical, and those linked to the restriction of working time for surgical residents) have led to the progressive shift of manual skills training to virtual reality simulator devices.³ Many studies have validated the use of low- and high-fidelity laparoscopic simulators (HFLS) for the training and/or assessment of laparoscopic skills,^4,5 but the question of the transfer of basic skills to procedural interventions, as well as from the simulation center environment to that of the operating room, remains controversial. Notably, most of the published studies compared trained participants to others who had not been exposed to learning modalities other than usual clinical practice,^6,7 prejudging the superiority of action compared to careful observation in the operating room. This postulate has recently been challenged by Ballouhey et al.⁸ in a study on urinary drainage apprenticeships, while a recent work by Lai et al.⁹ has shown that action is not more effective than observation, when learning crisis resource management. Consequently, the aim of this study was to explore the respective roles of active observation and direct practice in the process of learning laparoscopic skills.

Materials and Methods

This experiment is in accordance with current French laws and received the approval of the Commission Nationale Informatique et Liberte (CNIL). All participants gave their written informed consent before the start of the first training session. All sessions took place in the Simulation Centre of the University Hospital of Angers on an HFLS, the Simbionix LAP Mentor™ (Twin Medical, Angers, France). Participants were fifth-year medical students recruited on a voluntary basis. Exclusion criteria were previous training on a laparoscopic simulator and actual laparoscopy performed in the operating room. The minimum number of 11 participants per group was statistically assessed using the BiostaTGV online calculator (BiostaTGV, Jussieu, France), with an expected improvement of five points between the two sessions⁸ on the evaluation scale and statistical risks set at α = 0.05 and β = 0.80.

Two training sessions on laparoscopic skills, taking place 4 weeks apart, were offered to the participant in pairs. During the first session (S1), both participants were given initial training time on the high-fidelity simulator to adjust to its use. Subsequently, each participant completed a workshop of four tasks (either workshop 1 or workshop 2) selected from LAP Mentor's “Basic skills” and “Essential tasks” modules. Both workshops were designed to teach specific skills, such as camera manipulation, manipulation of objects, or cutting (Table 1), and were of equivalent difficulty. During the completion of a workshop by one participant, the other carefully observed his or her peer practicing (Fig. 1). Thereafter, they changed roles: the first participant observed the second, who performed the other workshop. A surgeon supervisor assisted the participants during the entire session, providing them with any required explanations about the tasks to be completed and advice as needed. As a result, at the end of the first session, both participants had access to the same pedagogical content; the only difference between them was the learning mode, that is, direct practice or peer observation.

FIG. 1.

Flowchart of the study. Black arrow: direct practice of a laparoscopic workshop of predefined tasks on an HFLS device; gray dotted arrow: observation of the intervention performed by peer. S1, first session; S2A, second session for participants completing the same workshop as during S1; S2O, second session for participants completing the workshop they observed during S1. HFLS, high-fidelity laparoscopic simulator.

Table 1.

Description of the Different Tasks of Laparoscopic Skills Workshops with Their Learning Goals

Workshop 1	Workshop 2	Main goals of procedural skills
Basic skills, Camera 30° manipulation	Basic skills, Camera 0° manipulation	Become familiarized with a 0°/angled camera
		Camera manipulation
		Camera orientation
Basic skills, clipping and grasping	Basic skills, electrocautery	Basic principles of safe clipping/safe electrocautery
		Clip applier/hook electrode manipulation
		Bimanual skills
		Eye–hand coordination
Essential tasks, peg transfer	Basic skills, translocation	Eye–hand coordination
		Bimanual skills
		Depth perception
		Laparoscopic instruments manipulation
		Space orientation and task planning
Essential tasks, pattern cutting (test)	Essential tasks, pattern cutting (training)	Eye–hand coordination
		Bimanual skills
		Tissue-handling skills
		Accuracy in cutting using scissors

During the second session (S2), each participant completed both workshops 1 and 2, and observed his or her counterpart perform the same, thus being, in turn, former actor (S2A) and former observer (S2O) for the workshop performed.

Laparoscopic skills were assessed using the objective structured assessment of technical skill (OSATS)¹⁰ global rating scale out of 35 points (item “Use of an assistant” not applicable) by the surgeon supervisor, and using self- and peer-evaluation. Some objective data were directly recorded by the HFLS, such as total time for completion of a task, average speed and total path length of the instruments, total number of movements, economy of motion, and accuracy rate.

Statistical analysis was performed using GraphPad Prism 5.04 for Windows (GraphPad Software, San Diego). Quantitative data were expressed as mean ± standard deviation or as median with extremes, and qualitative data as percentages. Normality was systematically assessed using the D'Agostino-Pearson omnibus normality test. Consequently, the comparison of quantitative OSATS score data between the first and the second training sessions was carried out using a one-tailed Student's t-test, with a two-tailed Student's t-test for intergroup comparisons in the second session and a two-tailed Wilcoxon matched-pairs signed-rank test for the HFLS data. Qualitative comparisons were made using Fisher's exact test, and correlation analysis was completed using Pearson's test. Linear regression was used to measure the coefficient of determination (R²). A P value of less than .05 was considered statistically significant.

Results

Description of the participants

In total, 18 fifth-year medical students volunteered for the study; all of them completed both training sessions. The mean age was 23 (22–25), the sex ratio was 1:1, and 3 (16.7%) students were left-handed. Ten (56%) wanted to become future surgeons, while 22% planned to join a medical practice, and the remaining 22% planned to become involved in forensic medicine or psychiatry. All of them had already observed between 2 and 40 laparoscopic interventions (median number: 10 per student). Their distribution according to the workshop carried out during the first session is summarized in Table 2; it did not differ except for the percentage of right-handed participants.

Table 2.

Baseline Characteristics of the Participants After Their Distribution According to the Workshop Performed During the First Training Session

	Workshop 1	Workshop 2	P
Mean age (years)	23.1 ± 0.9	22.6 ± 0.9	.26
Sex ratio (M/F)	5/5	5/5	1
Right handed (%)	100	66.6	.04
Expected surgical career (%)	44.4	66.7	.64
Median number of observed laparoscopic intervention	10 [2–40]	8 [2–30]	.77
Mean delay between the two training sessions (day)	25 ± 6	25 ± 6	1

Performance assessment using the OSATS global rating scale

During the first session, the mean OSATS score as assessed by the supervisor was 16.7 ± 3.2. It was higher in the case of self-grading (19.2 ± 3.0, P = .01) and peer-grading (23.2 ± 3.0, P < .0001). This breakdown remained the same during evaluations of the second session. Indeed, there was a positive correlation between the supervisor's assessment and self-grading (R² = 0.36, P < .0001) or peer-grading (R² = 0.42, P < .0001).

During the second laparoscopic session, the supervisor's assessment resulted in a 33.8% ± 21.3% improvement of the OSATS score after simple observation of the S1 workshops (group S2O-observers), with a mean OSATS score of 21.8 ± 2.6 (P < .0001). The OSATS score in the group of participants that had previously completed the workshop (group S2A-actors) was significantly higher than the one of the S2O group (P < .0001), with a mean score of 25.1 ± 1.9, resulting in an improvement of 56.3% ± 36.0% (P < .0001) compared to the first session (Fig. 2a). Self- and peer-grading OSATS scores confirmed that improvement was more significant in the S2A group than in the S2O group (Figs. 2b and 2c).

FIG. 2.

Evolution of the OSATS score between the first and second training session on the HFLS. The OSATS global rating scale was assessed by the supervisor (a) or using self- (b) or peer-grading (c). Moreover, a distinction between motor skills and procedural knowledge (d) has been made. S1, first session; S2A, second session for participants completing the same workshop as during S1; S2O, second session for participants completing the workshop they observed during S1. *P < .05. HFLS, high-fidelity laparoscopic simulator; OSATS, objective structured assessment of technical skills.

The OSATS global rating scale evaluates both manual skills (items “respect for tissue,” “time and motion,” and “instruments handling”) and procedural knowledge and planning (items “knowledge of instruments,” “flow of operation and forward planning,” and “knowledge of specific procedure”). A separate analysis showed that the improvement of skills focused on both motor skills and knowledge and it was more significant in the S2A group compared to the S2O group (P < .0001) (Fig. 2d).

Performance assessment using high-fidelity simulator metrics

Metrics from the HFLS were analyzed for each task as well as for each workshop as a whole. Global metrics were average speed of the dominant hand (cm.second⁻¹), total path length of the instruments (cm), and total time (second). Average speed did not differ between S1, S2A, and S2O (Fig. 3a), whereas total time decreased in both the S2A and S2O groups (431.8 ± 224.5 second and 441.9 ± 263.1, respectively, P ≤ .03) compared to S1 session (5408 ± 279 second), without significant difference between S2A and S2O results (Fig. 3c). On the contrary, total path length of the instrument (Fig. 3c) was found to improve by only 6% in the S2A group compared to the first session (1692 ± 1072 cm versus 1991 ± 1231, P = .003). A detailed metrics analysis for each task has been reported in Table 3 and shows a significant improvement for 14 out of the 28 recorded items in the group S2A (50%) compared to the first S1 session, whereas only 7/28 (25%) of the metrics improved in the S2O group (P = .048). Improved items were time to complete the tasks, path length of the instruments, and accuracy rates. The four tasks have been ranked in order of increasing difficulty, which showed that the more complex the task, the fewer the improved parameters.

FIG. 3.

Evolution of the global LAP Mentor™ metrics between the first (S1) and second training sessions, for actor (S2A) and observer (S2O) medical students. (a) Reports the average speed of the movements, (b) the total path length of the instruments, and (c) the total time of completion of the four tasks of the laparoscopic workshops.

Table 3.

Detail of the High-Fidelity Simulator's Metrics for Each of the Four Tasks of the Training Workshops

Task	Event description	S1	S2A	S2O
Camera manipulation	Accuracy rate—target hits (%)	76.8 ± 16.3	85.9 ± 11.0^*	83.8 ± 14.4
	Average speed—camera movements (cm.second⁻¹)	8.8 ± 0.9	9.7 ± 1.9^*	9.7 ± 1.4^*
	Total path length of camera (cm)	477.7 ± 161.2	364.6 ± 89.1^*	404.3 ± 177.1^*
	Total time (second)	133.6 ± 44.6	87.43 ± 21.1^*	97.21 ± 44.8^*
Clipping and grasping/electrocautery	Average speed of nondominant hand (cm.second⁻¹)	3.3 ± 1.0	3.4 ± 1.0	3.4 ± 1.0
	Average speed of dominant hand (cm.second⁻¹)	2.8 ± 0.8	2.9 ± 0.7	2.8 ± 0.6
	Total path length, nondominant hand (cm)	431.2 ± 191.5	311.6 ± 76.0^*	305.6 ± 89.7^*
	Total path length, dominant hand (cm)	461.7 ± 238.1	313 ± 98.8^*	337 ± 126.1^*
	Total path length (cm)	892.9 ± 406.2	624.6 ± 151.9^*	642.6 ± 188.5^*
	Accuracy rate (%)	88.3 ± 11.8	93.1 ± 9.5^*	89.36 ± 11.9
	Total time (second)	296.9 ± 170.8	195.6 ± 95.6^*	213.2 ± 111.0
Peg transfer/translocation	Average speed of nondominant hand (cm.second⁻¹)	3.2 ± 0.6	3.3 ± 0.6	3.1 ± 0.3
	Average speed of dominant hand (cm.second⁻¹)	3.4 ± 0.9	3.5 ± 0.8	3.2 ± 0.4
	Total path length, nondominant hand (cm)	1412 ± 1330	967.7 ± 711.6^*	981.5 ± 655.3
	Total path length, dominant hand (cm)	1604 ± 1617	1129 ± 892.4^*	1038 ± 668.1
	Total path length (cm)	3016 ± 2908	2097 ± 1566	2020 ± 1290
	Total time (second)	373 ± 223.1	251.4 ± 107.1^*	279.2 ± 111
Pattern cutting	Average speed of nondominant hand (cm.second⁻¹)	3.1 ± 0.7	3.4 ± 0.6^*	3.4 ± 0.7
	Average speed of dominant hand (cm.second⁻¹)	2.3 ± 0.6	2.3 ± 0.5	2.3 ± 0.6
	Number of movements, nondominant hand	202.6 ± 83.3	184.9 ± 80.1	172.8 ± 77.9
	Number of movements, dominant hand	448.6 ± 110.4	418.6 ± 12.5	403 ± 135.0
	Total path length, nondominant hand (cm)	307.4 ± 162.2	276.5 ± 136.4	256 ± 126.6
	Total path length, dominant hand (cm)	549.1 ± 220.7	506.4 ± 218.9	482.1 ± 226.4
	Total path length (cm)	856.5 ± 351.6	782.9 ± 303.5	738.1 ± 324.0
	Total time (second)	236.5 ± 36.2	225.6 ± 42.1	223.9 ± 54.2
	No accuracy error—outside of 5 mm area, n (%)	2/6 (33.3%)	9/14 (64.3%)	11/17 (64.7%)
	Area of accuracy error (mm²)	7	49.2 ± 58.4	40.4 ± 37.6
	Task complete within 300 second, n (%)	6 (33.3%)	14 (77.8%)^*	17 (94.4%)^*

S1, group of participants during the first training session; S2A, group of medical students who completed the tasks during the first session; S2O, group of medical students who observed the tasks being performed during the first session. The asterisk “^*” represents statistically significant differences between the first and the second sessions.

Discussion

The aim of this study was to compare the retention of basic laparoscopic skills after two different learning modes, namely personal practice and careful observation of a peer's performance. As a result, the assessment of the surgical performance using the OSATS rating scale and HFLS metrics showed that, following the second completion of a four-task workshop, there was an improvement in laparoscopic performance that was more significant after direct practice compared to simple observation. Consequently, participants who performed tasks during the first learning session showed an improvement of up to 50% in their OSATS score when completing a laparoscopy for the second time, well correlated to a significant improvement in 50% of the HFLS metrics. These results seem to coincide with previous reports on proficiency-based training.¹¹ Nevertheless, the group of participants who had first been mere observers achieved an improvement of 33% in the OSATS score and 25% of the metrics, which shows that active observation coupled with direct feedback from the supervisor can remain an effective learning modality.

Many studies have addressed the efficiency of simulation-based laparoscopic training to improve novices or surgical residents' skill retention and transferability to the operating room.^12–16 Almost all of them concluded that a proficiency-based simulation training on virtual reality simulators or video trainers improved skill retention¹⁷ and transferability to animal or human laparoscopic interventions.^7,18 In these studies, however, participants belonging to control groups were not exposed to any other learning modalities; at most, they received standard surgical practice during their residency, as did the participants belonging to the groups that had been subject to practical training. It is therefore possible to conclude that receiving any additional training, whatever its didactic modalities, improves further performance. With the current trend of limiting teaching staff and the development of various learning modalities,¹⁹ such as e-learning courses,²⁰ peer-assisted learning,^21,22 and observational²³ or mental practice,²⁴ surgical training may shift from the operating room to self-directed or student-led practice in simulation centers using computer-based video training. Although this study was not designed to assess the efficiency of such training modalities, it has shown that when novices were exposed to the same teaching modalities (i.e., being presented with the same tasks at the same time and receiving the same feedback from their supervisor, with one student practicing and the other observing), both improved their performance; direct practice, however, was more effective.

Our study is subject to several limitations, such as the small number of participants, which nevertheless remains within the range of comparable experiments.^{1,1,14,25–27} The small cohort size may also be counterbalanced by the homogeneity of both groups and the fact that each participant was his or her own control. The assessment of OSATS scores was not blinded, which may have induced some bias; however, results from the supervisor were consistent with those of the participants' evaluations and with LAP Mentor's metrics; during each session, only the OSATS scale was completed, without computing the final score to prevent assessors from having any knowledge of the final score or a participant's prior scores. Finally, we investigated a short laparoscopy training course that was not proficiency-based and had previously demonstrated no impact on the retention of surgical skills²⁸ or their transferability in the operating room.²⁷ Nevertheless, we demonstrated that a single training session improved laparoscopic performance. It remains to be determined, however, whether this effect persists with time. Further research is mandatory to assess whether the differences found between practice and observation as training modalities persist after longer curricula, and to define their “optimal” respective parts in a surgical apprenticeship.

Footnotes

Disclosure Statement

No competing financial interests exist.

References

Scott

, Bergen

, Rege

, Laycock

, Tesfay

, Valentine

, Euhus

, et al. Laparoscopic training on bench models: Better and more cost effective than operating room experience?. J Am Coll Surg, 2000; 191:272–283.

Ahlberg

, Enochsson

, Gallagher

, Hedman

, Hogman

, McClusky

3rd , Ramel

, et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg, 2007; 193:797–804.

Reznick

, MacRae

. Teaching surgical skills—Changes in the wind. N Engl J Med, 2006; 355:2664–2669.

Stunt

, Wulms

, Kerkhoffs

, Dankelman

, van Dijk

, Tuijthof

. How valid are commercially available medical simulators?. Adv Med Educ Pract, 2014; 5:385–395.

Faurie

, Khadra

. Technical competence in surgeons. ANZ J Surg, 2012; 82:682–690.

Al-Kadi

, Donnon

, Oddone Paolucci

, Mitchell

, Debru

, Church

. The effect of simulation in improving students' performance in laparoscopic surgery: A meta-analysis. Surg Endosc, 2012; 26:3215–3224.

Dawe

, Windsor

, Broeders

, Cregan

, Hewett

, Maddern

. A systematic review of surgical skills transfer after simulation-based training: laparoscopic cholecystectomy and endoscopy. Ann Surg, 2014; 259:236–248.

Ballouhey

, Cros

, Lescure

, Clermidi

, Romain

, Guigonis

, Bonnetblanc

, et al. Simulation training for urinary drainage improves skill retention. Prog Urol, 2015; 25:516–522.

Lai

, Haligua

, Dylan Bould

, Everett

, Gale

, Pigford

, Boet

. Learning crisis resource management: Practicing versus an observational role in simulation training—A randomized controlled trial. Anaesth Crit Care Pain Med, 2016; 35:275–281.

10.

Martin

, Regehr

, Reznick

, MacRae

, Murnaghan

, Hutchison

, Brown

. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg, 1997; 84:273–278.

11.

Stefanidis

, Korndorffer

Jr. , Sierra

, Touchard

, Dunne

, Scott

. Skill retention following proficiency-based laparoscopic simulator training. Surgery, 2005; 138:165–170.

12.

Andreatta

, Woodrum

, Birkmeyer

, Yellamanchilli

, Doherty

, Gauger

, Minter

. Laparoscopic skills are improved with LapMentor training: results of a randomized, double-blinded study. Ann Surg, 2006; 243:854–860; discussion 860–853.

13.

Grantcharov

, Kristiansen

, Bendix

, Bardram

, Rosenberg

, Funch-Jensen

. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg, 2004; 91:146–150.

14.

Schijven

, Jakimowicz

, Broeders

, Tseng

. The Eindhoven laparoscopic cholecystectomy training course—Improving operating room performance using virtual reality training: Results from the first E.A.E.S. accredited virtual reality trainings curriculum. Surg Endosc, 2005; 19:1220–1226.

15.

Larsen

, Soerensen

, Grantcharov

, Dalsgaard

, Schouenborg

, Ottosen

, Schroeder

, et al. Effect of virtual reality training on laparoscopic surgery: Randomised controlled trial. BMJ, 2009; 338:b1802.

16.

Lacreuse

I MG

, Becmeur

, Gomes Ferreira

, Moog

, et Kauffmann

. Evaluation clinique de l'efficacité de l'entraînement sur simulateur à la réalisation de noeuds intracorporels par les internes de chirurgie: un programme mis en place dans un service de chirurgie pédiatrique. Pédagogie Médicale, 2011; 12:213–221.

17.

Gurusamy

, Aggarwal

, Palanivelu

, Davidson

. Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. Br J Surg, 2008; 95:1088–1097.

18.

Buckley

, Kavanagh

, Traynor

, Neary

. Is the skillset obtained in surgical simulation transferable to the operating theatre?. Am J Surg, 2014; 207:146–157.

19.

Porte

, Xeroulis

, Reznick

, Dubrowski

. Verbal feedback from an expert is more effective than self-accessed feedback about motion efficiency in learning new surgical skills. Am J Surg, 2007; 193:105–110.

20.

Celentano

, Browning

, Hitchins

, Giglio

, Coleman

. Training value of laparoscopic colorectal videos on the World Wide Web: A pilot study on the educational quality of laparoscopic right hemicolectomy videos. Surg Endosc, 2017. DOI:10.1007/s00464-017-5504-2

21.

Saleh

, Sinha

, Weinberg

. Using peer-assisted learning to teach basic surgical skills: Medical students' experiences. Med Educ Online, 2013; 18:21065.

22.

Preece

, Dickinson

, Sherif

, Ibrahim

, Ninan

, Aildasani

, Ahmed

, et al. Peer-assisted teaching of basic surgical skills. Med Educ Online, 2015; 20:27579.

23.

Welsher

, Grierson

. Enhancing technical skill learning through interleaved mixed-model observational practice. Adv Health Sci Educ Theory Pract, 2017. DOI:10.1007/s10459-017-9759-0

24.

Yiasemidou

, Glassman

, Mushtaq

, Athanasiou

, Williams

, Jayne

, Miskovic

. Mental practice with interactive 3D visual aids enhances surgical performance. Surg Endosc, 2017. DOI: 10.1007/s00464-017-5459-3

25.

Sroka

, Feldman

, Vassiliou

, Kaneva

, Fayez

, Fried

. Fundamentals of laparoscopic surgery simulator training to proficiency improves laparoscopic performance in the operating room-a randomized controlled trial. Am J Surg, 2010; 199:115–120.

26.

Seymour

, Gallagher

, Roman

, O'Brien

, Bansal

, Andersen

, Satava

. Virtual reality training improves operating room performance: Results of a randomized, double-blinded study. Ann Surg, 2002; 236:458–463; discussion 463–454.

27.

Ahlberg

, Heikkinen

, Iselius

, Leijonmarck

, Rutqvist

, Arvidsson

. Does training in a virtual reality simulator improve surgical performance?. Surg Endosc, 2002; 16:126–129.

28.

Anastakis

, Wanzel

, Brown

, McIlroy

, Hamstra

, Ali

, Hutchison

, et al. Evaluating the effectiveness of a 2-year curriculum in a surgical skills center. Am J Surg, 2003; 185:378–385.