Learning Pattern of Two-Dimensional,Three-Dimensional,and Ultra-High-Definition Endovision System on Standardized Phantom Tasks: An Ex Vivo Randomized Study

Abstract

Introduction:

Three-dimensional (3D), high-definition (HD), and ultra-high-definition (4K HD) are recent additions over regular HD technology for laparoscopic surgery. The aim of this study was to evaluate the learning pattern of these systems on standardized phantom tasks.

Methodology:

Forty-five stereo-enabled resident doctors were randomly assigned into three groups. They performed three validated tasks, precision touch on flat surface, precision touch on uneven surface, surgical knot on rubber tube using either two-dimensional (2D) HD, 3D HD, or 4K HD Endovision systems. Each task was repeated 20 times. Data from four consecutive repetitions were pooled to make five blocks. Split group analysis by comparing the consecutive blocks in execution time and errors were made to see the learning pattern. A significant difference was accepted as continuous learning while no significant difference was accepted as learning stabilization.

Result:

Operating time was stabilized in two tasks after third block in 2D HD, one task after fourth block in 4K HD. There was continuous learning in all tasks with 3D HD. The 3D HD group was significantly faster than 2D HD and 4K HD in most of the tasks on fifth block. The error scores were similar between the consecutive blocks in 4K HD. It was stabilized after second block in 2D HD group and third block on 3D HD.

Conclusion:

The 3D HD Endovision system has more potential of faster execution of a task, but need more practice to reach similar safety profile. The 4K HD reached the safety plateau with minimal repetitions.

Introduction

Laparoscopic surgery is technology driven and technological advancement has played a crucial role in its rapid adaptation in clinical practice. Performing a three-dimensional (3D) surgical task by observing a two-dimensional (2D) image is one of the major challenges of laparoscopy. To overcome this limitation, 3D Endovision system was introduced three decades ago. The European Association of Endoscopic Surgery consensus statement suggests superiority of 3D Endovision system compared with 2D Endovision system in reducing operative time.¹ Other studies comparing 3D and 2D also support the same.^2,3

Recently, the ultra-high-definition (4K HD) Endovision system has emerged. Its ultra-high resolution (3840 × 2160) provides better image quality and indirect visual cues for mono-ocular depth perception. A few experimental trials comparing safety and efficacy of the systems showed superiority or equivalence of 4K over 3D,^4–6 whereas a few others suggested superiority of 3D over 4K.^7–9 It is likely that soon, these technologies will be used more widely and there may be an option to learn laparoscopic surgery using either of the systems.

There is limited comparative data on the learning process with 2D, 3D, and 4K Endovision systems. To the best of our knowledge, only one study has evaluated the difference between 3D and 4K in relation to the learning process. They found that the basic laparoscopic tasks can be learned quicker using 3D Endovision system. However, for tasks performed in narrow spaces, the learning speed with 4K HD Endovision system was comparable to 3D HD system.¹⁰ We designed this study to evaluate the learning pattern of standardized experimental tasks using 2D HD, 3D HD, and 4K HD Endovision systems. The primary outcome measures were improvement in execution time (speed) and error score (safety) on repeated practice.

Methods

This was a randomized experimental study done between January and June 2020. The study was approved by the Ethics Committee of the Institute (Ref. No. IECPG-261/5/2018). The participants were residents of the Department of Surgery and the Department of Gynecology who had not performed laparoscopic surgery. Residents with <3 months of exposure to laparoscopic surgery or who had undergone structured hands-on ex vivo training in laparoscopic surgery were excluded. The volunteered residents underwent Randot^® stereotest (Stereo Optical Inc., Chicago, IL) and those with stereoacuity of <100 seconds of arc were enrolled for the study.

All participants filled a predefined proforma elaborating their experience in open and laparoscopic surgeries. They were given a handout describing all the tasks and shown a narrated video of the tasks. They were informed regarding the different types of errors and that their performance in terms of execution time and error scores would be assessed. They signed a written consent for the study.

The enrolled residents were randomized into three groups, viz 2D HD, 3D HD, and 4K HD groups using a computer-generated block random sequence in 1:1:1 ratio (Sealed Envelope, London, UK). The allocation details were kept in sealed opaque envelopes, which was opened just before the experiment. Participants were given time to familiarize themselves with the laparoscopic instruments. In addition, participants of the 3D HD group performed an adaptation task twice to familiarize themselves with the 3D system. The adaptation task was a simple grasping and moving task, in which a ring had to be picked up with the left-hand instrument and passed to the right-hand instrument and put down again and were performed using the 3D HD system.¹¹

The participant then carried out three laparoscopic phantom tasks in a trainer box using the respective Endovision systems. Each task was repeated 20 times. Results were expressed as execution time in seconds and error scores of all the tasks.

After finishing tasks on the Endovision system, any adverse event, such as headache, blurred vision, dizziness, nausea, or vertigo, was recorded. The same surgical resident guided the camera in all the experiments. All procedures were recorded in 2D HD format.

Experimental setup

The phantom tasks were carried out in an indigenous laparoscopic box trainer. The box trainer was placed on an operating table and height of the table was adjusted to the mid-thigh level. The monitor was placed at a distance of 1.5 m from the participant, viewer's eye level was at the middle of the display. The monitor, box trainer, and the participating surgeon were positioned on a straight line. The phantom tasks set up were placed inside the box trainer and secured with magnetic attachment to maintain a uniform position.

The Endovision system used in 2D HD group was Stryker Laparoscopy System (Stryker Endoscopy, San Jose, CA) and 26-inch monitor (Vision pro 26-inch LED display, Model No. BPM150524511). In 3D HD group, Karl Storz system was used (Karl Storz Endoscopy, Tuttlingen, Germany). Participants wore a passive polarized glass and system used was TIPCAM^® telescope and monitor (LCD monitor, 32-inch Model No. EJ-NDA32EK2). In 4K HD group, Olympus system (Olympus Europa, Hamburg, Germany) was used with a 55-inch monitor (Sony-Olympus medical systems, LCD, 55-inch, Model No. IMD-X550S).

Tasks

Three phantom tasks were performed. The tasks were precision touch on flat surface (Task 1), precision touch on uneven surface (Task 2), and applying surgical knot on a rubber tube (Task 3). Tasks 1 and 2 were adapted from the experimental design by Storz et al.¹² Task 3 was adapted from fundamentals of laproscopic surgery¹³ task of intracorporeal suturing.

Task 1 (Precision touch on flat surface)

Seven circular target spots (Ø.6 mm in diameter) were made on a flat surface. The task was to touch these spots precisely without touching the surrounding area. A laparoscopic hook on dominant hand was used to touch these intended spots in a clockwise sequence (Fig. 1a).

FIG. 1.

The experimental task with its laparoscopic image. (a) Precision touch on flat surface (Task 1). (b) Precision touch on uneven surface (Task 2). (c) Surgical knot on rubber tube (Task 3). Color images are available online.

Task 2 (Precision touch on uneven surface)

It was similar to Task 1, however, the target spots were placed on an uneven surface (Fig. 1b).

Task 3 (Surgical knot on rubber tube)

A silk thread (15 cm length) was used to place a surgeon's knot (double throws followed by two single throws) on a predefined site (1 cm in length) on a rubber tube using two Maryland forceps. The task was to place a secure surgical knot over the predefined area (Fig. 1a).

Execution time

Time was measured in seconds from the recorded videos. For Tasks 1 and 2, it was measured from appearance of the laparoscopic hook on the monitor until the last target spot was touched. For Task 3, it was the duration from appearance of first instrument until the redundant ends of the silk thread were cut.

Error calculation

Tasks 1 and 2

Any deviation from touching the intended area was recorded as error. The error was counted using an electronic digital error counter to ensure objectivity and avoid variation in error calculation. The digital error counters recorded the number of times the wrong surface (unintended area, outside the target spot) was touched. The error counts were increased by one on each subsequent mistake. Maximum of error score was 7 and minimum score was 0 for each attempt.

Task 3

Accurate placement and adequate tightness of knot (safety) was assessed for error calculation. Any deviation in placement of the knot from the predefined area was considered as error and scored 1. Properly positioned knot and snugly placed nondisplaceable knot was given a score of 0. Nonsnugly displaceable knot was considered as an error and was given a score of 1.

Learning pattern

The data of four consecutive repetitions were pooled to make five blocks out of 20 repetitions of each task (1st to 4th repetition = Block 1, 5th to 8th repetition = Block 2, 9th to 12th repetition = Block 3, 13th to 16th repetition = Block 4, and 17th to 20th repetition = Block 5).

To see the learning pattern, split group analysis by comparing the consecutive blocks, that is, B1 versus B2, B2 versus B3, B3 versus B4, and B4 versus B5 in terms of execution time and error score were made. The presence of significant difference between the consecutive blocks was accepted as continuous learning. The absence of significant difference between the blocks was accepted as stabilization on learning.

Statistical analyses

Data were analyzed using statistical software SPSS Statistics 25.0 (IBM Inc.). The quantitative measures of Tasks 1 and 2 (time and error) were reported as mean (standard deviation). Errors in Task 3 (if the knot was snug or not and in position or not) was considered as the categorical variable and analyzed separately counting the specific combination of the errors (no error, error in any form, and dual error). The analyses were performed for all possible combination [stratified across all the three technology (2D HD, 3D HD, 4K HD) and also across the tasks (Tasks 1, 2, and 3)]. The within comparison was performed by Friedmann test and Wilcoxon signed-rank sum test. Likewise, comparison between the groups was performed by Mann–Whitney U test and Kruskal–Wallis test.

Task 3 errors were represented categorically with either the presence or absence of adequately snug knot and whether the position of knot was correct. The comparison between various systems was done for corresponding blocks using χ². A value of <0.05 was taken as significant. Comparison in the same technology between subsequent repetitions was not done due to the categorical nature of data.

Results

Sixty-nine residents volunteered and were assessed for eligibility. Forty-five residents were enrolled in the study, whereas 24 residents were excluded due to various reasons. Enrolled residents were randomized into three groups. Details of the CONSORT diagram is presented on Figure 2.

FIG. 2.

CONSORT diagram. 2D HD, two-dimensional high-definition Endovision system; 3D HD, three-dimensional high-definition Endovision system; 4K HD, ultra-high-definition Endovision system. Color images are available online.

The mean age of residents was 26.1 years. Majority of the participants were male. Other baseline variables are shown in Table 1.

Table 1.

Demographic and Surgical Experience Details of Participants

	2D HD (n = 15)	3D HD (n = 15)	4K HD (n = 15)	P
Age (years)
Mean	26.1	26.4	26.0	.736
Gender, n (%)
Male	9 (60)	11 (73.3)	13 (86.6)	.076
Female	6 (40)	4 (26.6)	2 (13.3)
Dexterity, n (%)
Right handed	15 (100)	14 (93.3)	15 (100)	.631
Left handed	0	1 (6.6)	0
Mean training period, months (±SD)	12.84 (±6.12)	15.6 (±9.2)	12.75 (±9.76)	.158
Specialty, n (%)
Surgical disciplines	11 (73.3)	12 (80)	13 (86.6)
Obstetrics and gynecology	4 (26.6)	3 (20)	2 (13.3)	.244
Open surgery experience, n (%)
<20	14 (93.3)	0	14 (93.3)
>20	1 (6.6)	15 (100)	1 (93.3)	.541
Laparoscopic surgery experience, n (%)
<20	11 (73.3)	11 (73.3)	9 (60)
>20	4 (26.6)	4 (26.6)	6 (40)	.091
Assisted surgery using 3D-HD system previously, n (%)	9 (60)	10 (66.6)	11 (73.3)	.740

2DHD, two-dimensional high-definition Endovision system; 3DHD, three-dimensional high-definition Endovision system; 4K HD, ultra-high-definition Endovision system; SD, standard deviation.

The execution time for each task after increasing number of repetitions are presented in Table 2.

Table 2.

Mean Execution Time in Different Endovision Systems

Task	Technology	Execution time in seconds, mean (±SD)
Task	Technology	Block 1 (R_1–4)	Block 2 (R_5–8)	Block 3 (R_9–12)	Block 4 (R_13–16)	Block 5 (R_17–20)
Task 1	2D HD	67.78 (±31.50)	55.70 (±27.89)	49.63 (±26.56)	55.65 (±23.30)	53.95 (±21.89)
	3D HD	52.78 (±15.43)	43.66 (±14.38)	37.91 (±14.93)	42.50 (±19.14)	32.58 (±14.07)
	4K HD	68.61 (±39.44)	51.85 (±22.26)	45.93 (±20.76)	46.26 (±16.80)	41.71 (±11.54)
Task 2	2D HD	80.98 (±33.94)	69.31 (±31.07)	66.31 (±30.77)	69.43 (±28.73)	66.73 (±27.33)
	3D HD	56.08 (±18.07)	47.75 (±12.79)	44.18 (±12.79)	41.36 (±13.38)	35.56 (±14.01)
	4K HD	69.66 (±20.52)	64.28 (±24.99)	63.21 (±16.62)	55.11 (±8.58)	46.80 (±8.91)
Task 3	2D HD	499.05 (±278.22)	416.40 (±299.88)	404.73 (±308.49)	391.50 (±291.25)	346.10 (±268.03)
	3D HD	271.83 (±44.91)	244.33 (±29.02)	133.16 (±34.87)	110.94 (±16.47)	105.26 (±15.90)
	4K HD	395.50 (±160.57)	370.16 (±69.28)	307.01 (±86.33)	271.23 (±83.95)	258.93 (±74.08)

2DHD, two-dimensional high-definition Endovision system; 3DHD, three-dimensional high-definition Endovision system; 4K HD, ultra-high definition Endovision system; R, repetition; SD, standard deviation.

Task 1, on 2D HD group, on comparing the execution time among the blocks, learning was stabilized after Block 3 (P = .001, .001, .510, .059 in B1 versus B2, B2 versus B3, B3 versus B4, and B4 versus B5, respectively). On 3D HD group, learning was not stabilized until completion of fifth blocks of repetitions (P = .002, .012, .307, .029 in B1 versus B2, B2 versus B3, B3 versus B4, B4 versus B5, respectively). On 4K HD, the learning was continuous. (P = .002, .012, .307, .029 in B1 versus B2, B2 versus B3, B3 versus B4, and B4 versus B5, respectively) (Fig. 3a).

FIG. 3.

Learning pattern on execution time. (a) Task 1, (b) Task 2, (c) Task 3. 2D HD, two-dimensional high-definition Endovision system; 3D HD, three-dimensional high-definition Endovision system; 4K HD, ultra-high-definition Endovision system. Color images are available online.

Task 2, Learning was not stabilized in either of the groups, P = .015, .009, .955, .031 on 2D HD group, P = .004, .102, .149, .003 on 3D HD group, P = .041, .955, .029, .015 on 4K HD group between B1 versus B2, B2 versus B3, B3 versus B4, and B4 versus B5, respectively (Fig. 3b).

Task 3, on 2D HD learning was stabilized after third block repetitions (P = .001, .008, .281, .061 between B1 versus B2, B2 versus B3, B3 versus B4, and B4 versus B5, respectively). On 3D HD, there was continuous learning. (P = .055, .001, .001, .002 between B1 versus B2, B2 versus B3, B3 versus B4, and B4 versus B5, respectively). In 4K HD learning was stabilized after fourth block of repetition (P = .609, .363, .018, .256 between B1 versus B2, B2 versus B3, B3 versus B4, and B4 versus B5, respectively) (Fig. 3c).

Error score during 20 repetitions (Table 3)

Task 1, on 2D HD, learning was stabilized on second block of repetitions (P = .029, .285, .284, .270). On 3D HD it was stabilized on completion of the second block (P = .007, .303, .714, .176). On 4K HD, there was no statistically significant difference among the sequential blocks (P = .461, .377, .785, .130) (Fig. 4a).

FIG. 4.

Learning pattern on error score. (a) Task 1, (b) Task 2, (c) Task 3. 2D HD, two-dimensional high-definition Endovision system; 3D HD, three-dimensional high-definition Endovision system; 4K HD, ultra-high-definition Endovision system. Color images are available online.

Table 3.

Mean Error Scores in Different Endovision System

Task	Technology	Error score, mean ± SD
Task	Technology	Block 1 (R_1–4)	Block 2 (R_5–8)	Block 3 (R_9–12)	Block 4 (R_13–16)	Block 5 (R_17–20)
Task 1	2D HD	0.51 ± 0.38	0.33 ± 0.15	0.26 ± 0.19	0.18 ± 0.17	0.24 ± 0.22
	3D HD	1.08 ± 0.75	0.66 ± 0.47	0.58 ± 0.63	0.56 ± 0.67	0.33 ± 0.20
	4K HD	0.38 ± 0.44	0.28 ± 0.43	0.36 ± 0.28	0.38 ± 0.43	0.14 ± 0.23
Task 2	2D HD	0.91 ± 0.47	0.55 ± 0.40	0.65 ± 0.46	0.41 ± 0.56	0.36 ± 0.51
	3D HD	1.11 ± 0.61	0.91 ± 0.55	0.611 ± 0.51	0.66 ± 0.56	0.50 ± 0.41
	4K HD	1.10 ± 0.55	0.96 ± 0.41	0.73 ± 0.66	0.81 ± 0.62	0.46 ± 0.28
Task 3	2D HD	49	44	38	20	10
	3D HD	38	30	26	11	6
	4K HD	34	26	18	17	9

2DHD, two-dimensional high-definition Endovision system; 3DHD, three-dimensional high-definition Endovision system; 4K HD, ultra-high-definition Endovision system; R, repetition; SD, standard deviation.

Task 2, on 2D HD, learning was stabilized after second block of repetitions (P = .004, .271, .072, .477). On 3D HD, learning was stabilized after third block (P = .192, .012, .441, .284). On 4K HD, no statistically significant difference was seen among the sequential blocks (P = .634, .168, .455, .084) (Fig. 4b).

Task 3, on 2D HD, cumulative execution error was 49 in the first block, which reduced to 10 in the last block. On 3D HD, the cumulative execution errors were 38 in first block which reduced to 6 in the last block. On 4k HD, cumulative execution errors were 34 in first block, which reduced to 9 in the last block. Due to the categorical nature of data, learning pattern was not statistically evaluated in this task (Fig. 4c).

Comparison among the three technologies for execution time and error score in first and fifth block of repetition

Task 1

In Block 1, the time taken to perform the task was uniform across the Endovision systems, but 3D HD group made significantly more errors than the 2D HD and 4K HD group. On Block 5, the error scores of 3D HD group became similar with 2D HD but remained significantly more than the 4K HD group. Details are presented on Table 4.

Table 4.

Pair-Wise Analysis of Execution Time and Error Score Between Two-Dimensional High-Definition Endovision System, Three-Dimensional High-Definition Endovision System and Ultra-High-Definition Endovision System Groups

Tasks	Repetition	Execution time, P =			Error score, P =
Tasks	Repetition	2D HD versus 3D HD	2D HD versus 4K HD	4K HD versus 3D HD	2D HD versus 3D HD	2D HD versus 4K HD	4K HD versus 3D HD
Task 1	Block 1	.191	.756	.351	.028	.177	.009
Task 1	Block 5	.007	.178	.054	.230	.139	.017
Task 2	Block 1	.038	.507	.040	.394	.778	.772
Task 2	Block 5	<.001	.016	.021	.460	.665	.881
Task 3	Block 1	.001	.254	.036	.024	.003	.452
Task 3	Block 5	<.001	.575	<.001	.282	.802	.407

Pair-wise analysis between Endovision systems were done using Mann–Whitney test.

2DHD, two-dimensional high-definition Endovision system; 3DHD, three-dimensional high-definition Endovision system; 4K HD, ultra-high definition Endovision system; SD, standard deviation.

There were no postprocedural adverse events like vomiting or dizziness or headache.

Discussion

Our study evaluated the impact of 2D HD, 3D HD, and 4K HD Endovision systems on performance of resident doctors in three standardized tasks. Tasks 1 and 2 were basic laparoscopic tasks, whereas Task 3 was an advanced task. Both procedural time (speed) and error scores (safety) were used to appraise the learning process, which differed among the Endovision systems.

The learning was stabilized for execution time after third block of repetitions in 2 out of 3 tasks in the 2D Group. There was continuous learning effect in 3D group in most of the tasks for execution time. In the 4K group, a similar learning effect was seen in two out of three tasks. In Task 1, analysis of inter groups' performance revealed that there was no significant difference in execution time among the groups in the first block of repetition, however, 3D was faster than 2D and 4K in the fifth block. In Tasks 2 and 3, 3D was significantly faster on Block 1 and also on Block 5. It was interesting to note that the improvement in execution time with repeated practice was more marked in 3D HD than with 2D HD and 4K HD groups. There was continuous improvement on the 3D HD group and was significantly faster at fifth block compared with 2D HD and 4K HD. These findings might imply better learning potential of 3D and possibility of faster execution with longer learning duration.

Earlier researchers suggested about 11 repetitions to achieve a learning plateau in similar tasks.¹⁴ We undertook 20 repetitions, although learning was stabilized for operating time with 2D HD, but not with 3D HD and with 4K HD. Kunert et al. found that the performance time and error rates were lesser in the 3D HD group,¹⁴ whereas Kang et al. demonstrated that speed of acquisition of skills was faster in 3D as compared with 2D, especially for complex tasks.¹⁵ A few other studies have favored the 3D technology over 2D HD/4K HD technology for execution time.^1,2,7–9,11 The only study looking at the learning process on 3D and 4K Endovision systems suggests that the basic skill can be learned quicker using 3D system, whereas, for tasks in confined space, learning speed was comparable between the systems.¹⁰

With experience, surgeons learn to decipher indirect cues obtained from color, contrast, shadow from surrounding, or motion parallax and gain ability for mono-ocular depth perception.¹⁶ The advantages shown by 3D HD Endovision system over 2D HD and 4K HD Endovision systems, in our study, may be due to the combination of binocular vision of 3D and relatively inexperienced status of our study participants.

It was interesting to note that in Task 1, the 3D HD group committed significantly more errors than 2D HD and 4K HD groups, whereas 2D HD and 4K HD groups had similar error score in the first block. After repeated practices, difference was similar between 2D HD and 3D HD, 2DHD and 4K HD, although 3D HD continues to make more mistakes than 4K HD in Task 1. On subsequent tasks, as duration of practice in 3D environment had increased, the 3D HD group had similar or less error scores in most of the tasks. Task 1 was the first task to be performed by the residents. Although, we did an adaptation task twice for the 3D HD group, it is likely that there is need for more practice to safely adapt to the endoscopic 3D working environment.

To start clinical activity with the 3D Endovision system, it will be prudent to get accustomed to the 3D endoscopic environment upfront before attempting clinical cases. These patterns of error score among the technologies might suggest the varying need for training or adaptation task for safe execution, which is least with 4K HD. A few other experimental studies have also shown lesser error rates with the 4K technology.^4,5

The main strength of our study lies in its methodology. It was a randomized trial. Unlike medical students being the study population in many of the studies, our study participants bear significant relevance as they are the target population for laparoscopic skill training in clinical practice. All participants had exposure to laparoscopic surgery and were tested for stereo ability. The standardized task requiring both basic and advanced skills were used. We anchored both operating time and safety parameters for appraisal of learning process, which is desirable for expression of learning process.¹⁷

Our study also had a few limitations. First, we did 20 repetitions for each task. However, learning was not stabilized for 3D HD and 4K HD groups. Addition of a few more repetitions probably would have given us definite result on stabilization of operative time. Analysis of parameters like path length traversed by the instrument, may have given us more information on economy of movements.

In conclusion, our study suggested that 3D HD Endovision system has more potential for faster execution of a task, but need more practice to reach similar safety profile. The safety parameter was stabilized with minimal repetitions in 4K HD Endovision system. Execution time was stabilized after third block of repetition in majority of tasks in 2D HD Endovision system. In 3D HD and 4K HD groups, execution time was not stabilized and would need more than 20 repetitions to define it clearly.

Footnotes

Acknowledgments

All authors acknowledge Mr. Nitin Srivastava, Mr. Manpreet Singh Sudan, and Mr. Bhupender Kumar of Electronics Section, Central Workshop, All India Institute of Medical Sciences, New Delhi, for their help in preparing the electronic digital error counter.

Ethics Approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the Institutional and/or National Research Committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. A formal ethical clearance was taken for this study.

Informed Consent

Informed consent was obtained from all individual participants included in the study.

Disclosure Statement

The authors declare that they have no conflicts of interests or financial ties to expose.

Funding Information

No funding was received for this article.

References

Arezzo

, Vettoretto

, Francis

, et al. The use of 3D laparoscopic imaging systems in surgery: EAES consensus development conference 2018. Surg Endosc, 2019; 33:3251–3274.

Pantalos

, Patsouras

, Spartalis

, Dimitroulis

, Tsourouflis

, Nikiteas

. Three-dimensional versus two-dimensional laparoscopic surgery for colorectal cancer: Systematic review and meta-analysis. In Vivo, 2020; 34:11–21.

Chen

, Li

, Zeng

, et al. Three-dimensional vs 2-dimensional laparoscopic gastrectomy for gastric cancer. Medicine (Baltimore), 2019; 98:e18222.

Abdelrahman

, Belramman

, Salem

, Patel

. Acquiring basic and advanced laparoscopic skills in novices using two-dimensional (2D), three-dimensional (3D) and ultra-high definition (4K) vision systems: A randomized control study. Int J Surg Lond Engl, 2018; 53:333–338.

Dunstan

, Smith

, Schwab

, et al. Is 3D faster and safer than 4K laparoscopic cholecystectomy? A randomised-controlled trial. Surg Endosc, 2020; 34:1729–1735.

Parshad

, NanjaklaJayappa

, Bhattacharjee

, et al. Comparison of three-dimensional (3D) endovision system versus ultra-high-definition 4K endovision system in minimally invasive surgical procedures: A randomized-open label pilot study. Surg Endosc, 2021 [Epub ahead of print]; https://doi.org/10.1007/s00464-021-08377-2.

Wahba

, Datta

, Bußhoff

, et al. 3D versus 4K display system—Influence of “state-of-the-art”-display technique on surgical performance (IDOSP-study) in minimally invasive surgery: A randomized cross-over trial. Ann Surg, 2020; 272:709–714.

Kanaji

, Watanabe

, Mascagni

, et al. Three-dimensional imaging improved the laparoscopic performance of inexperienced operators: A prospective trial. Surg Endosc, 2019; 34:5083–5091.

Rana

, Rana

, Hewett

. Comparison of three-dimensional and 4K imaging systems in novice surgeons: A cross-over study. ANZ J Surg, 2020; 90:1009–1013.

10.

Thomaschewski

, Jürgens

, Keck

, Laubert

, Benecke

Learning laparoscopic surgery skills with a 4K ultra-high definition 2D vs a three-dimensional HD laparoscopic system: Results from a prospective randomized trial. Surg Innov, 2021 [Epub ahead of print]; https://doi.org/10.1177/1553350621991224.

11.

Bhattacharjee

, Chaliyadan

, Mishra

, et al. Comparison of two-dimensional high-definition, ultra high-definition and three-dimensional endovision systems: An ex-vivo randomised study. Surg Endosc, 2020; 35:5328–5337.

12.

Storz

, Buess

, Kunert

, Kirschniak

. 3D HD versus 2D HD: Surgical task efficiency in standardised phantom tasks. Surg EndoscInterv Tech, 2012; 26:1454–1460.

13.

Fried

, FLS Program. FLS Manual Skills Written Instructions and Performance Guidelines Important Scoring Information Task One: Peg Transfer FLS Manual Skills Written Instructions and Performance Guidelines. 2014. https://www.flsprogram.org/wp-content/uploads/2014/03/Revised-Manual-Skills-Guidelines-February-2014.pdf

14.

Kunert

, Storz

, Dietz

, et al. Learning curves, potential and speed in training of laparoscopic skills: A randomised comparative study in a box trainer. Surg Endosc, 2020; 35:3303–3312.

15.

Kang

, Wong

CMJ

, Tan

, Bohari

, Han

, Soon

. A secondary learning curve in 3D versus 2D imaging in laparoscopic training of surgical novices. Surg Endosc, 2021; 35:1046–1051.

16.

Pierre

, Ferrandino

, Simmons

, et al. High definition laparoscopy: objective assessment of performance characteristics and comparison with standard laparoscopy. J Endourol, 2009; 23:523–528.

17.

Barrie

, Jayne

, Wright

, Murray

CJC

, Collinson

, Pavitt

. Attaining surgical competency and its implications in surgical clinical trial design: A systematic review of the learning curve in laparoscopic and robot-assisted laparoscopic colorectal cancer surgery. Ann Surg Oncol, 2014; 21:829–840.