Improving Robotic Skills by Video Review

Abstract

Background:

Since the transition of surgical robot systems into the medical field, physicians have had to develop new dexterity skills. The ideal learning environment for robotic surgery has yet to be discovered. Virtual reality (VR) simulation is a possible safe and economic method. In VR simulator training human feedback is hardly used, and an alternative may be found in video revision. The purpose of this study is to investigate whether adding video review to VR simulation-based training in novice physicians improves their ability to complete a complex robot task. In addition, the secondary goal is to investigate whether the skills learned on the robotic simulator can be transferred to a real robotic system.

Materials and Methods:

Forty participants, medical students and, medical-PhD candidates, from one university hospital were included. Baseline dexterity skills were measured through completion of a vesicourethral anastomosis on a VR robot simulator and the da Vinci robot. Participants were randomized into a video and control group. The video group practiced skills on the robot simulator with intermediate video revision, whereas the control group had intermediate pause instead. Postintervention dexterity skills were measured using the same exercises as the baseline tests.

Results:

No significant differences were found in baseline performance. Postintervention results on the VR simulator show that the video group commits significantly fewer injuries to the urethra and sutures at a greater optimal depth. The control group was significantly faster, had less camera travel, and had their instruments less out of view. On the da Vinci robot, participants in both groups performed significantly faster and had better global evaluative assessment of robotic skill score after the training sessions on the VR simulator.

Conclusions:

Video revision significantly improves the quality of robotic skills in novice surgeons on the VR simulator, although at the expense of time. Furthermore, both groups demonstrated enhanced skills on the da Vinci robot after training sessions, which advocates transferability of skill.

Introduction

Since its introduction, robotic surgery has gained an important place in modern surgery.

Today, a large number of minimally invasive surgical procedures can be performed using robotic surgery.

This innovation has required surgeons to learn a completely different skill set. Until now no uniform accepted curriculum for robotic surgery exists and the ideal learning environment is an ongoing search. Possibilities are web-based training, on-site robotic training, and virtual reality (VR) simulation training. VR simulation provides a safe, convenient, and economical method of training and assessment of robotic skills without the risks associated with training involving real-life patients.¹ Furthermore, it has been proven that VR simulation training improves both basic robotic surgical skills and more complex procedural skills.^2

–5

The most common approach to training surgical skills remains the master-apprentice system, in which trainees receive real-time feedback from a supervisor during a procedure. However, it is a very time-consuming way of teaching, and robot-assisted training in an operating room is particularly expensive and very lengthy.⁶ Moreover, due to time constraints in surgery, real-surgery teaching opportunities are limited in robotic surgery. Therefore, a possible alternative to human feedback in robot-assisted training is by using video as a teaching and learning modality.^7,8

Video-based learning has been adopted across various sectors such as sports, aviation, and business management. In the medical field, it is used in a wide range of educational settings for novices, residents, and experienced surgeons alike. It seems a feasible way to become familiar with complicated procedures.^9

–13

Since the 1960s, video assessment has proven to improve surgical techniques and clinical skills.^13
–15 More recent studies have concluded that video self-assessment improves laparoscopic skills on a VR simulator and that a video review program is beneficial in training laparoscopic inguinal hernia operations.^16,17

However, the majority of the evidence on the effect of video-based training derives from observational studies. Furthermore, we are not aware of research about the effect of video review on learning robotic surgery. Therefore, this randomized controlled trial aims to investigate whether adding video review to VR simulation-based training in novices improves the learning of robotic surgery skills. In addition, the secondary aim is to explore whether the skills learned on the robot simulator are transferrable to a real robot system.

Based on these aims, the following research questions were generated:

What is the effect of video revision on the quality of robotic surgery skills learned by novice surgeons?

Is there transferability of skill from a robot simulator to a real robot system?

Materials and Methods

Study design

This prospective single-blinded randomized controlled trial was conducted at the University Hospital of Utrecht between March 2021 and May 2021. All procedures were in accordance with the ethical standards of Dutch Law, through formal evaluation by the Medical Research Ethics Committee of the University Hospital of Utrecht, under case number 20/573.

Study design is presented in Figure 1.

FIG. 1.

Flowchart of study design. Color images are available online.

Participants

Prior power analysis with the G*Power software calculated a minimal sample size of 26 participants in total, to measure an intended difference for a large size effect (0.5) with a power of 0.8 based on the independent-sample t-test.

Included were medical-PhD students and medical students. Participants with more than 1 hour of surgical robot or robotic simulator experience were excluded. Upon registration, participants received a participant number based on enrollment order. Based on this number, participants were randomly assigned to a study group: odd numbers in the video group, even numbers in the control group.

All data were anonymized. Participants completed a pretraining questionnaire to obtain basic characteristics, such as age, gender, dominant hand, profession, and laparoscopic or robotic experience.

VR robot simulator

For the VR robotic simulation, we used the Robotix mentor learn systems designed by 3D systems (Fig. 2).

FIG. 2.

Participant training on the VR robot simulator. Color images are available online.

This system gives participants the opportunity to develop basic and advanced skills for surgical robotics in different medical specialties such as General Surgery and Urology.

Before baseline tests, participants followed five basic skill exercises for the acquisition of basic dexterity.

There was no time limit for the completion of these exercises.

The exercises were:

Camera 0

Wrist articulation 1

Wrist articulation 2

Railroad track

Knot tying

Baseline test consisted of the guided vesicourethral anastomosis exercise, in which the aim is to connect the bladder neck to the urethral stump. The simulator program displayed the suturing direction with entrance and exit points and provided simple feedback concerning suturing position accuracy and injuries to specific anatomic structures. Before starting the exercise, all participants were given a single oral instruction on basic anatomy and how to complete the exercise. During the exercise, no further help was provided.

The exercise started with two barbed sutures, one for each half of the anastomosis. Starting at the posterior aspect of the bladder neck, five continuous stitches were used to complete one side of the anastomosis. Stitches through the bladder neck were performed in an outside-in direction, and stitches through the urethral stump in an inside-out direction. Using an “in” and “out” button, participants were able control the urethral catheter. Pulling the catheter aside allowed space at the inner surface of the urethra for the suture to be placed.

To limit time, participants were only required to fulfill the right-sided half of the anastomosis. When the simulator indicated to use the second barbed suture, the test was finished. In the event of a ruptured wire (due to excessive force), the exercise was restarted. There was no time limit for completion of the baseline test.

Da Vinci robot

For the baseline vesicourethral anastomosis exercise on the surgical robot, we used the da Vinci model X. Large needle drivers were installed in the first and third arm; the video camera was installed in the second arm. The anastomosis was performed on a Vesicourethral Anastomosis Kit consisting of a synthetic bladder and urethra, manufactured by Skills Meducation (Fig. 3).

FIG. 3.

(a) Set up da Vinci robot in the operating room. (b) Camera view of da Vinci robot. Color images are available online.

Participants were instructed to suture the right-sided half of the vesicourethral anastomosis by following the same technique as on the robot simulator.

Instructions were provided orally in advance, and no further instructions were given during the exercise. In case of a torn suture, the participants were given a new suture and asked to continue the exercise where they left off. No urethral catheter was available in this model.

Training

Video group

After the baseline test, the video group received a 1-hour training program on the VR robot simulator twice, on two separate days. During this hour, participants were given 24 minutes to train the vesicourethral anastomosis with the same exercises used in the baseline test. All exercises were videorecorded by the simulator. The exercise was followed by 6 minutes of video revision. The first 3 minutes consisted of their own performance, during which they could select the parts they wanted to review. The last 3 minutes consisted of an expert video, demonstrating how to perform the anastomosis correctly as well as frequent errors such as tearing of the urethra. In this way, participants could compare their own performance with that of an expert to learn from it. No additional oral feedback was given. After this, the above was repeated to complete the hour. We chose a maximum of 1-hour training session per day, the estimated maximum time available besides the mandatory university/clinical attendance. The expert video used was included in the software program of the simulator.

Control group

The control group received the same 2-day training program as the video group. Instead of 6 minutes of video revision, they received 6 minutes of pause.

Outcome

Postintervention tests were identical to the baseline tests on the VR simulator and the da Vinci robot. Before starting the postintervention tests on the VR simulator, the video group received one more video revision opportunity of 6-minute video, the first 3 minutes were of their last training. No oral feedback was given.

The primary outcome measure was the quality of robotic skills measured by the VR simulator through parameters and errors such as time duration and injuries to anatomical structures. Secondary outcome measure was the transferability of skills to the da Vinci robot. For this purpose, all da Vinci baseline and postintervention test videos were assessed by an urologist with robotic surgery experience (W.M.B.), using the global evaluative assessment of robotic skills (GEARS), a validated assessment tool measuring depth perception, bimanual dexterity, efficiency, force sensitivity, and robotic control, all graded on a scale from 1 to 5.¹⁸ In addition to the GEARS, total time, number of camera movements, breaking of sutures, and grasping of the urethra were recorded.

Data analysis

Data analysis was performed using SPSS statistics version 26.0. Frequency statistics were used to present categorical variables. Statistical significance regarding difference between the video and control group was calculated using the chi-square test or Fisher exact test for categorical variables, as appropriate.

The independent sample t-test was used for normally distributed continuous variables. The Mann–Whitney U test was used in case of non-normal distributed data.

Statistical significance regarding differences between baseline and post-training outcomes were calculated using the paired sample T-test for normally distributed data, and the Wilcoxon signed rank test for non-normal distributed data. All tests were two-sided and statistical significance was set at p-values <0.05.

Results

A total of 40 participants were included. Data of the postintervention test on the VR simulator of one participant of the control group were lost due to software failure.

Demographics

Baseline characteristics are displayed in Table 1. The majority of the participants were medical students (n = 31), followed by MD-PhD candidates (n = 9). Most of the participants were right handed; only one left-handed participant, randomly assigned to the control group, was included. There were no significant differences regarding baseline characteristics between the groups.

Table 1.

Participants' Baseline Characteristics

	Control group (n = 20)	Video group (n = 20)	p
Age	24.40 ± 2.817	23.95 ± 2.762	0.613^a
Gender, n (%)			0.204^b
Female	13(65)	9 (45)
Male	7 (35)	11 (55)
Profession, n (%)			1.000^c
Medical student	16 (80)	15 (75)
Medical PhD candidate	4 (20)	5 (25)
Dominant hand, n (%)			1.000^c
Left	1 (5)	0 (0)
Right	19 (95)	20 (100.0)
Current field of expertise, n (%)			0.407^c
General surgery	2 (10)	5 (25)
Urology	1 (5)	0 (0)
Other/inter	17 (85)	15 (75)
Laparoscopic experience, n (%)			0.288^b
Yes	4 (20)	7 (35)
No	16 (80)	13 (65)
Level of laparoscopic experience, n (%)			0.458^c
Participated as assistant surgeon	3 (15)	3 (15)
Participated in simulator based Training	1 (5)	4 (20)
Limited or no experience, n (%)	16 (80)	13 (65)

Calculated using Independent T-test (mean ± SD).

Calculated using Chi-square test.

Calculated using Fisher's exact test.

SD = standard deviation.

Baseline performance

Baseline performance between the groups showed no significant difference on the VR simulator (Table 2) and the da Vinci Robot (Table 3).

Table 2.

Baseline Test Virtual Reality Robot Simulator

	Control group (n = 20), median (IQR)	Video group (n = 20), median (IQR)	p ^a
Total time (seconds)	1025.94 (288.78)	1260.82 (680.91)	0.052
Movements L instrument (n)	722.00 (325.00)	907.00 (599.00)	0.221
Movements R instrument (n)	710.00 (224.00)	979.00 (542)	0.127
Path length L instrument (mm)	6121.33 (4617.22)	7958.41 (5454.14)	0.398
Path length R instrument (mm)	5683.34 (2871.39)	7367.91 (5467.23)	0.341
Distance by camera (mm)	748.18 (1344.94)	776.48 (1385.39)	0.779
Instrument collisions (n)	58.00 (37.00)	73.00 (60)	0.265
Total path instruments traveled out of view (mm)	380.32 (1461.97)	855.40 (1432.18)	0.478
Instruments out of view (n)	22.00 (778.25)	47.00 (78.5)	0.445
Total time instruments out of view (seconds)	26.03 (160.62)	42.88 (128.27)	0.495
Clutch usage (n)	5.50 (15)	7.00 (13)	0.883
Injury to the urethra (n)	3.00 (5.00)	4.00 (5.00)	0.547
Injury to the bladder neck (n)	0.50 (1.00)	0.00 (1.00)	0.738
Suspected injury to the endopelvic fascia/urethral sphincter (n)	4.00 (5.00)	5.50 (8.00)	0.192
Suspected injury to the bladder (n)	0.00 (0.00)	0.00 (0.00)	0.989
Suspected injury to the neurovascular bundle (n)	0.00 (1.00)	0.00 (1.00)	0.414
Suspected injury to the orifices (n)	1.00 (1.00)	1.00 (3.00)	1.000
Wound separation (mm)	11.52 (1.00)	12.04 (4.00)	0.583
Improper suturing technique (n)	0.00 (1.00)	0.00 (1.00)	0.947
Stitches within optimal depth (%)	80.00 (29.00)	80.00 (73.00)	0.758
Unnecessary needle piercing points (n)	47.50 (36.00)	73.50 (65.00)	0.183
Total entrance and exit points, through which needle has passed (n)	20.00 (2.00)	20.00 (2.00)	0.620
Excessive force (n)	0.00 (0.00)	0.00 (0.00)	1.000

Calculated using Mann–Whitney U test.

IQR = interquartile range.

Table 3.

Baseline Test on Da Vinci Robot

		Control group (n = 20)	Video group (n = 20)	p ^a
Depth perception		2.00 (1.00)	2.00 (1.00)	0.151
Bimanual dexterity		1.50 (1.00)	2.00 (1.00)	0.195
Efficiency		1.00 (1.00)	1.00 (1.00)	0.437
Force sensitivity	2.00 (1.00)		2.00 (1.00)	0.841
Robotic control		1.00 (1.00)	1.00 (1.00)	0.589
Total Gears score		7.00 (3.00)	8.00 (3.00)	0.379
Grasping of the urethra (n)		2.00 (5.00)	0.5 (4.00)	0.314
Excessive force–suture breakage (n)		0.00 (1.00)	0.0.0 (1.00)	0.678
Camera movements (n)		0.50 (4.00)	0.00 (2.00)	0.352
Total time (seconds)		886.25 ± 150.36	876.55 ± 307.69	0.900^b

All calculated using Mann–Whitney U test (median [IQR]), except ^bIndependent-sample t test (mean ± SD).

Postintervention performance

VR robot simulator

Results of the postintervention tests are shown in Table 4. The control group performed the vesicourethral anastomosis significantly faster (p = 0.016), with less distance moved by camera (p = 0.044). Regarding the movement of the instruments, the control group made fewer movements with the right instrument (p = 0.008). Furthermore, instruments were out of view less frequently (p = 0.019), out of view a shorter period of time (p = 0.08), and out of view a shorter distance (p = 0.044).

Table 4.

Postintervention Test Virtual Reality Robot Simulator

	Control group (n = 19), median (IQR)	Video group (n = 20), median (IQR)	p ^a
Total time (seconds)	592.25 (211.64)	717.75 (215.26)	0.016
Movements L instrument (n)	391.00 (160.00)	473.50 (212.00)	0.204
Movements R instrument (n)	469.00 (165.00)	571.00 (233.00)	0.008
Path length L instrument (mm)	3855.02 (2124.00)	4163.32 (2121.42)	0.380
Path length R instrument (mm)	4016.13 (1276.50)	4575.92 (2034.73)	0.120
Distance by camera (mm)	247.97 (289.78)	497.37 (410.25)	0.044
Instrument collisions (n)	30.00 (20.00)	27.00 (27.00)	0.879
Total path instruments traveled out of view (mm)	242.12 (420.21)	471.89 (435.75)	0.044
Instruments out of view (n)	11.00 (14.00)	27.00 (30.00)	0.019
Total time instruments out of view (seconds)	9.10 (35.78)	34.07 (57.58)	0.008
Clutch usage (n)	0.00 (1.00)	2.00 (4.00)	0.041
Injury to the urethra (n)	1.00 (3.00)	0.00 (1.00)	0.028
Injury to the bladder neck (n)	0.00 (0.00)	0.00 (1.00)	0.428
Suspected injury to the endopelvic fascia/urethral sphincter (n)	1.00 (3.00)	0.00 (1.00)	0.247
Suspected injury to the bladder (n)	0.00 (0.00)	0.00 (0.00)	1.000
Suspected injury to the neurovascular bundle (n)	0.00 (0.00)	0.00 (0.00)	0.835
Suspected injury to the orifices (n)	0.00 (1.00)	0.00 (1.00)	0.607
Wound separation (mm)	11.52 (2.00)	11.52 (2.00)	0.531
Improper suturing technique (n)	0.00 (0.00)	0.00 (00.0)	0.792
Stitches within optimal depth (%)	80.00 (20.0)	100.00 (10.00)	0.001
Unnecessary needle piercing points (n)	18.00 (18.00)	15.50 (20.00)	0.813
Total entrance and exit points, through which needle has passed (n)	20.00 (0.00)	20.00 (0.00)	0.771
Excessive force (n)	0.00 (0.00)	0.00 (0.00)	0.411

Significant differences are in bold.

All calculated using Mann–Whitney U test.

The video group had significantly fewer injuries to the urethra (p = 0.028) and a greater percentage of sutures within optimal depth (p = 0.001).

Da Vinci robot

No significant differences were found on the da Vinci robot between the control and video groups (Table 5).

Table 5.

Postintervention Test on Da Vinci Robot

	Control group (n = 20)	Video group (n = 20)	p ^a
Depth perception	2.00 (1.00)	2.00 (1.00)	0.912
Bimanual dexterity	2.00 (1.00)	2.50 (1.00)	0.277
Efficiency	2.00 (1.00)	2.00 (1.00)	0.768
Force sensitivity	2.00 (1.00)	2.00 (1.00)	0.807
Robotic control	2.00 (1.00)	2.00 (0.75)	0.555
Total GEARS score	11.50 (3.75)	12.00 (3.00)	0.935
Grasping of the urethra (n)	0.00 (4.00)	0.00 (2.00)	0.478
Excessive force–suture breakage (n)	0.00 (0.00)	0.00 (0.00)	1.00
Camera movements (n)	0.00 (2.00)	1.00 (2.00)	0.479
Total time (seconds)	694.10 ± 192.06	721.95 ± 162.67	0.624^b

All calculated using Mann–Whitney U test (median [IQR]), except ^bIndependent-sample t test (mean ± SD).

GEARS = global evaluative assessment of robotic skills.

Table 6 demonstrates significant differences in total time (p ≤ 0.01) (Fig. 2) grasping of the urethra (p = 0.05), excessive force (p = 0.042), and total GEARS score (p ≤ 0.0005) (Fig. 3) comparing the baseline test with the postintervention test.

Table 6.

Preintervention vs Postintervention Test on the Da Vinci Robot

	Preintervention (n = 40)	Postintervention (n = 40)	p
Total GEARS score	7.67 ± 0.283	11.64 ± 0.351	<0.0005^a
Grasping of the urethra (n)	1.00 (4.00)	0.00 (2.75)	0.05^b
Excessive force–suture breakage (n)	0.00 (1.00)	0.00 (0.00)	0.042^b
Camera movements (n)	0.00 (3.00)	1.00 (2.00)	0.611^b
Total time (seconds)	844.00 (281.5)	719.00 (224,25)	<0.01^b

Calculated using paired sample T-test (mean ± SD).

Calculated using Wilcoxon signed-rank test (median [IQR]).

Discussion

This study demonstrates that video revision enhances robotic surgical skills in novices. The video group showed both a significant decrease in damage to the urethra as well as a significantly higher percentage of sutures within optimal depth compared with the control group. In addition, this study proves that improvement of skills on the simulator can be translated into improvement on the actual robot, comparable to previous studies.⁵ After the training sessions on the VR simulator robot, all participants performed significantly faster, had greater total GEARS scores, and made fewer errors on the da Vinci robot, compared with the baseline tests. (Figs. 4 and 5).

FIG. 4.

Box and Whiskers plot comparing the total GEARS scores of the anastomosis on the da Vinci robot. Color images are available online.

FIG. 5.

Box and Whiskers plot comparing the total time needed to complete half anastomosis on the da Vinci robot. Color images are available online.

The enhanced quality of robotic surgical skill on the VR simulator in the video group came at the expense of significantly increased total time to complete the exercise and increased dexterity metrics such as distance of camera movements, number of instrument movements (right), and instruments out of view both in time, frequency, and distance.

Prior studies on laparoscopic VR simulation have reported similar findings in which there was no decrease¹⁹ or even an increase^20,21 in time and dexterity parameters in the video group compared with the control group, despite statistically significant improvement in the quality of the performance.

A plausible explanation for differences between the video and control group is that novices who start learning a new complex skill, are unconsciously incompetent. They are unaware of how to correctly perform the task and are also unaware of the errors they make. When receiving feedback, in our study, by watching a video of their own performance in combination with an expert video, they become consciously incompetent. They become aware of how the task should be performed and of the mistakes they make. To become consciously competent in a task requires concentration and attentiveness, as nothing goes automatically. The focus of attention is on performing the task correctly, not on performing the task in itself. This may lead to longer duration and more movement with instruments.²² The overall improvement between the pre- and post-test seen in both groups can likely be attributed to repetition.

In addition, for optimum results, there is the need for optimal visibility. Therefore, the camera must be positioned correctly throughout the exercise, as shown in the expert video. The greater distance in camera movement in the intervention is presumably because they are more aware of this necessity. A consequence of camera movement is that the instruments are out of view more often and longer periods, especially when zooming in, which reduces the overall view. This occurrence is shown in multiple studies, in which experienced surgeons compared with novices had more camera movements and had instruments longer out of view due to their camera's close proximity to the tissue.^23,24

To the best of our knowledge, this is the first randomized controlled trial to investigate the influence of video-based learning on robotic surgery skills. Additional strengths of our study are the large sample size and the use of the VR simulator in combination with the actual robot.

Our study has some limitations. First of all, it could be argued that watching the expert's video and their own video are two different interventions, making it unclear which intervention has resulted in the outcome. However, we deliberately chose this combination of interventions because research shows novices to be inaccurate in self-assessment. Nevertheless, self-assessment accuracy can be significantly improved by combining it with benchmarks video.²⁵

Second, the use of the guided version of the exercise may have distorted the results. On the other hand, our participants had limited to no knowledge of the operation and anatomy. Due to limited time, it was not achievable to teach the participants enough knowledge to perform this exercise without any form of guidance.

This research shows that the content of the instruction video is of vital importance. The video should focus on important aspects but should not neglect other important safety instructions such as instruments in view. Future research could focus on different instruction videos and the optimal use in learning robotic surgery.

Conclusions

The results of the study demonstrate that novice surgeons can significantly enhance the quality of robotic surgical skills by video revision. Second, although we found no significant difference between the video and the control group on the da Vinci robot, the improvement after the VR training sessions in both groups, proves a degree of basic skill transferability.

Footnotes

Authors' Contributions

J.A.v.d.L. and G.S. contributed to study planning and data collection. J.A.v.d.L., G.S., and W.A.B. are responsible for the overall content. R.P.M. contributed to the development and critical revision of article. J.A.v.d.L. submitted the study.

Acknowledgments

The authors thank 3D systems Simbionix for loaning the RobotiX Mentor simulator and Skills Meducation for the silicon Vesicourethral Anastomosis Kit.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

Abbreviations Used

References

Noureldin

, Stoica

, Kassouf

, Tanguay

, Bladou

, Andonian

. Incorporation of the da Vinci Surgical Skills Simulator at urology Objective Structured Clinical Examinations (OSCEs): A pilot study. Can J Urol, 2016; 23:8160–8166.

Chen

, Rodrigues Armijo

, Krause

, Siu

, Oleynikov

. A comprehensive review of robotic surgery curriculum and training for residents, fellows, and postgraduate surgical education. Surg Endosc, 2020; 34:361–367.

Raison

, Harrison

, Abe

, Aydin

, Ahmed

, Dasgupta

. Procedural virtual reality simulation training for robotic surgery: A randomised controlled trial. Surg Endosc, 2021; 35:6897–6902.

Mangano

, Gheza

, Giulianotti

. Virtual reality simulator systems in robotic surgical training. Surg Technol Int, 2018; 32:19–23.

Almarzouq

, Hu

, Noureldin

, et al. Are basic robotic surgical skills transferable from the simulator to the operating room? A randomized, prospective, educational study. Can Urol Assoc J, 2020; 14:416.

Steinberg

, Merguerian

, Bihrle

, Seigne

. The cost of learning robotic-assisted prostatectomy. Urology, 2008; 72:1068–1072.

Shim

, Kim

, Pyun

, et al. Comparison of effective teaching methods to achieve skill acquisition using a robotic virtual reality simulator expert proctoring versus an educational video versus independent training. Medicine (Baltimore), 2018; 97:e13569.

Schreuder

HWR

, Wolswijk

, Zweemer

, Schijven

, Verheijen

RHM

. Training and learning robotic surgery, time for a more structured approach: A systematic review. BJOG, 2012; 119:137–149.

McQueen

, McKinnon

, VanderBeek

, McCarthy

, Sonnadara

. Video-based assessment in surgical education: A scoping review. J Surg Educ, 2019; 76:1645–1654.

10.

Kreutzer

, Hu

, Stulberg

, Greenberg

, Bilimoria

, Johnson

. Formative evaluation of a peer video-based coaching initiative. J Surg Res, 2021; 257:169–177.

11.

Isreb

, Attwood

, Hesselgreaves

, McLachlan

, Illing

. Synchronized video-review as a tool to enhance reflection and feedback: A design-based feasibility study. J Surg Educ, 2021; 78:1–8.

12.

Mota

, Carvalho

, Carvalho-Dias

, João Costa

, Correia-Pinto

, Lima

. Video-based surgical learning: Improving trainee education and preparation for surgery. J Surg Educ, 2018; 75:828–835.

13.

Stranc

, McDiarmid

JGM

, Stranc

. Video assessment of surgical technique. Br J Plast Surg, 1991; 44:65–68.

14.

Goldman

, Maier

, Rosemond

, Saltzman

, Cramer

. Teaching surgical technique by the critical review of videotaped performance—The surgical instant replay. Surgery, 1969; 66:237–241.

15.

Mir

, Evans

, Marshall

, Newcombe

, Hayes

. The use of videorecordings of medical postgraduates in improving clinical skills. Med Educ, 1989; 23:276–281.

16.

Netter

, Schmitt

, Agostini

, Crochet

. Video-based self-assessment enhances laparoscopic skills on a virtual reality simulator: A randomized controlled trial. Surg Endosc, 2021; 35:6679–6686

17.

Tanaka

, DeAsis

, Vigneswaran

, et al. Video review program enhances resident training in laparoscopic inguinal hernia: A randomized blinded controlled trial. Surg Endosc 2017 326, 2017; 32:2847–2851.

18.

Goh

, Goldfarb

, Sander

, Miles

, Dunkin

. Global evaluative assessment of robotic skills: Validation of a clinical assessment tool to measure robotic surgical skills. J Urol, 2012; 187:247–252.

19.

Arora

, Aggarwal

, Sirimanna

, et al. Mental practice enhances surgical technical skills: A randomized controlled study. Ann Surg, 2011; 253:265–270.

20.

Singh

, Aggarwal

, Tahir

, Pucher

, Darzi

. A randomized controlled study to evaluate the role of video-based coaching in training laparoscopic skills. Ann Surg, 2015; 261:862–869.

21.

Crochet

, Aggarwal

, Dubb

, et al. Deliberate practice on a virtual reality laparoscopic simulator enhances the quality of surgical technical skills. Ann Surg, 2011; 253:1216–1222.

22.

Vaporciyan

. Teaching and learning surgical skill. Ann Thorac Surg, 2016; 101:12–14.

23.

Hovgaard

, Andersen

SAW

, Konge

, Dalsgaard

, Larsen

. Validity evidence for procedural competency in virtual reality robotic simulation, establishing a credible pass/fail standard for the vaginal cuff closure procedure. Surg Endosc, 2018; 32:4200–4208.

24.

Leijte

, de Blaauw

, Rosman

, Botden

SMBI

. Assessment of validity evidence for the RobotiX robot assisted surgery simulator on advanced suturing tasks. BMC Surg, 2020; 20:183.

25.

Scaffidi

, Walsh

, Khan

, et al. Influence of video-based feedback on self-assessment accuracy of endoscopic skills: A randomized controlled trial. Endosc Int Open, 2019; 07:E678-E684.