Virtual Reality Training Improves da Vinci Performance: A Prospective Trial

Abstract

Introduction:

The DV-Trainer™ (a virtual reality [VR] simulator) (Mimic Technologies, Inc., Seattle, WA) is one of several different robotic surgical training methods. We designed a prospective study to determine whether VR training could improve da Vinci^® Surgical System (Intuitive Surgical, Inc., Sunnyvale, CA) performance.

Subjects and Methods:

Surgeons (n=12) were enrolled using a randomized protocol. Groups 1 (VR training) and 2 (control) participated in VR and da Vinci exercises. Participants' time and moving distance were combined to determine a composite score: VR index=1000/(time×moving distance). The da Vinci exercises included needle control and suturing. Procedure time and error were measured. A composite index (DV index) was computed and used to measure da Vinci competency. After the initial trial with both the VR and da Vinci exercises, only Group 1 was trained with the VR simulator following our institutional curriculum for 3 weeks. All members of both groups then participated in the second trial of the VR and da Vinci exercises and were scored in the same way as in the initial trial.

Results:

In the initial trial, there was no difference in the VR index (Group 1 versus Group 2, 8.9±3.3 versus 9.4±3.7; P=.832) and the DV index (Group 1 versus Group 2, 3.85±0.73 versus 3.66±0.65; P=.584) scores between the two groups. At the second time point, Group 1 showed increased VR index scores in comparison with Group 2 (19.3±4.5 versus 9.7±4.1, respectively; P=.001) and improved da Vinci performance skills as measured by the DV index (5.80±1.13 versus 4.05±1.03, respectively; P=.028) and by suturing time (7.1±1.54 minutes versus 10.55±1.93 minutes, respectively; P=.018).

Conclusions:

We found that VR simulator training can improve da Vinci performance. VR practice can result in an early plateau in the learning curve for robotic practice under controlled circumstances.

Introduction

Robotic surgery is an emerging area in the surgical field, and robotic surgical systems have been approved for use in the treatment of various surgical diseases. The da Vinci^® (Intuitive Surgical, Inc., Sunnyvale, CA) console has a binocular stereoscopic vision system for displaying three-dimensional images of the surgical field, two handles or joysticks that transfer the surgeon's hand movements to the instrument, a pedal unit for controlling camera movements and working the different types of cautery, and a clutch device for repositioning the instruments.¹ During a robotic procedure, particularly one involving several quadrants of the abdomen such as colorectal surgery, the surgeon uses both his or her feet and hands, substituting for the surgical assistant and camera assistant who would be used for the same procedure using laparoscopic techniques. This is the key difference between robotic and laparoscopic surgical approaches. Laparoscopic surgery has increased need for good “hand–eye coordination” in performing tasks via a video screen rather than under direct vision in open surgery,² while “foot–hand–eye coordination” represents an extra psychomotor skill for robotic console practice, especially for the more recent robot models having four arms (da Vinci S^® and da Vinci Si^®; Intuitive Surgical).

There are some restrictions with robotic surgical training in comparison with laparoscopy. Robotic surgical training with live animals and human cadavers is too expensive for regular practice.^3,4 Dry lab practice with robotic instruments for exercise is an option but requires the purchase of a separate robot system for training purposes. Therefore, the use of a virtual reality (VR) simulator is an appealing option for robotic surgical training.⁵

Some research has been conducted concerning face, content, and construct validity of VR simulators, particularly in urology.^6–9 However, there has been little research done to determine if robotic VR training can actually improve da Vinci skills. The aim of the present study was to investigate whether robotic skills acquired in a virtual environment can be applied to an actual complex da Vinci procedure.

Subjects and Methods

The robotic simulator, the dV-Trainer™ (Mimic Technologies, Inc., Seattle, WA), is a commercially available VR simulator. This simulator is based on a complete kinematic representation of the da Vinci Surgical System, and the hardware design and main components of the VR simulator function in a manner similar to the actual da Vinci console.⁸ Thirty tasks involved in robotic surgical procedures are simulated. The VR simulator provides both immediate performance feedback and retrospective assessment of an individual's scores after consecutive trials.

In February 2011, we enrolled 11 general surgeons who had obtained a certificate for Specialist in General Surgery in the Republic of Korea and one colorectal surgeon from Malaysia attending Korea University Anam Hospital as a surgical fellow. The surgeons had similar experience in terms of the numbers of previous laparoscopic surgeries performed, including laparoscopic appendectomy, cholecystectomy, and colectomy, and none had prior experience with the da Vinci console. All members were randomly assigned to one of two groups (Group 1, VR training; Group 2, control) (Table 1). Participants had volunteered, and all were highly motivated.

Table 1.

Participant Demographic Data

	Group 1 (n=6)	Group 2 (n=6)
Age (years)	35.8 (33–37)	35 (31–41)
Sex ratio (M:F)	4:2	5:1
Resident postgraduate year	7.6 (2–8)	8 (2–10)
Number of laparoscopic surgeries performed (n)^a	40 (25–60)	55.8 (30–200)

Laparoscopic appendectomy, laparoscopic cholecystectomy, and laparoscopic colectomy.

F, female; M, male.

All members of both groups received an introduction to the dV-Trainer and the da Vinci console. The initial trial included a VR simulator exercise session and a da Vinci exercise session. The VR simulator exercise included the following: (1) pick and place, which requires the user to manipulate a pin over a series of cone-shaped targets; (2) peg board, in which the user picks up a sequence of rings from pegs, performs a transfer from one instrument to another, and places the ring on a new peg; and (3) needle driving, in which the user picks up a curved needle and passes it through ring-shaped pins (Fig. 1). The three tasks were of progressive complexity and were designed to simulate the robotic EndoWrist^® (Intuitive Surgical, Inc.) and the pedal used for clutch and camera movement. The order in which participants practiced with the VR simulator was randomized, and scores were recorded based on the metrics of time to complete the session, economy of motion, and working space range.

FIG. 1.

The virtual reality simulator exercise: (A) pick and place, (B) peg board, and (C) needle driving.

All participants started the da Vinci exercise in the afternoon (dry lab platform; Intuitive Surgical, Inc.). As shown in Figure 2, this exercise included (1) needle control, in which 10 pods with fine metal pins are placed on a plate in an “S” configuration and the user must use the robot arm to pass a curved suture needle with thread through all of the small holes on the pod, and (2) suture and tying, in which a rubber plate is prepared and the user must insert three interrupted sutures and tie appropriate surgical knots at indicated points using 3-0 polyglactin 910 suture (Vicryl™; Ethicon, Somerville, NJ).

FIG. 2.

The da Vinci exercise: (A) needle control and (B) suture and tying.

An experienced robotic surgeon (S.H.K.) and a junior surgeon (J.S.C.) supervised this robotic exercise and recorded the total time to complete the two tasks and the number of errors. We assessed suturing performance using a standardized objective method modified by addition of task completion time and knot error, as described previously.^10,11

Within a month after the initial trial, the members of Group 1 were trained with the VR simulator under the supervision of an experienced trainer at the Korea University Anam Hospital Robotic Training Center. One trainee in Group 1 dropped out of the study because of moving to another hospital after the initial trial and thus was unable to participate in the VR training program. Our institutional curricula included 1 hour each of (1) EndoWrist manipulation, (2) camera and clutching, (3) electrocautery using the energy device pedals, (4) needle driving, (5) suturing, and (6) a self-determined task. All of these sessions were repeated twice within 3 weeks. The members of Group 2 did not practice with the VR simulator or the da Vinci system during this period.

All participants from both groups (Group 1 and Group 2) participated in the second trial 7 weeks after the initial trial in April 2011. The second trial included VR exercises over three sessions and two da Vinci exercises that were the same as those in the initial tests. The practice order was randomized. Participants' VR index and da Vinci exercise scores (needle control time, suture and tie time, error numbers, and DV index) were recorded.

VR index

We computed an overall VR index to objectively evaluate VR simulator competency. Scoring the performance of novice participants using the dV-Trainer had some limitations and biases because the dV-Trainer computes a relative percentage of competences based on the performance of a highly experienced trainee.

Based on repeated observations of trainees, including surgical residents and medical students (authors' unpublished data), we concluded that total time to complete task, economy of motion (distance covered in instrument movement), and working space range (distance of trainee hand movement) are conclusive metrics for accurately evaluating performance.

Therefore, we calculated the VR index as follows: \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}\hbox { VR index } = 1000 \times \frac { 1 } { { \rm Time } } \times \frac { 1 } { { \rm Distance } } \end{align*} \end{document}

where Time represents the time to complete the exercise (in minutes) and Distance represents the economy of motion+working space range (in m).

Da Vinci exercise scores

We evaluated trainees' competency at performing da Vinci procedures by analyzing the time required for needle control, the time required to suture and tie, and the number of errors. The errors for each exercise are as follows.

1. Needle control

1. Moving the dry lab plate when the curved needle passed through a small hole

2. Missing a pin or skipping to another site

3. Stoppage of the robotic system due to excessive force on the grips or instrument collision

4. Being out of view (the field) or camera readjustment outside the system

5. Dropping the needle during the procedure

2. Suture and tying

1. Loosening of the knot tie or space greater than 1.0 mm between the two edges

2. Broken Vicryl thread because of excessive mechanical force

3. Lacerated rubber surface during suture or tying

4. Stoppage of the robotic system due to excessive force on the grips or instrument collision

5. Being out of view (the field) or camera readjustment outside the system

DV index

Competency scores for the da Vinci exercise needed to be expressed numerically to allow for objective comparison and easy analysis of data. Numerous studies on surgical training with various simulators of various laparoscopic areas have demonstrated that the time and accuracy of each performance are important factors for evaluating competency.² Quicker performance and fewer errors are indicators of a higher level of skill. Thus we calculated a composite score that included time and error metrics.

We formulated the DV index as follows: \documentclass{aastex}\usepackage{amsbsy}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{bm}\usepackage{mathrsfs}\usepackage{pifont}\usepackage{stmaryrd}\usepackage{textcomp}\usepackage{portland, xspace}\usepackage{amsmath, amsxtra}\pagestyle{empty}\DeclareMathSizes{10}{9}{7}{6}\begin{document} \begin{align*}\hbox { DV index } = \frac { 10 } { { \rm Time } } \times { \rm Accuracy } \end{align*} \end{document}

where Time represents time in minutes and Accuracy equals 10 – α×(error numbers), such that 1≤error numbers ≤10 and 0<α ≤1, with α=0.7 if the time and accuracy are weighted at a ratio of 7:3.

Statistical analysis

A nonparametric test (the Mann–Whitney U test) was used for comparison between groups, and a P value of <.05 was considered statistically significant. Data are mean±standard error values. Statistical analysis was performed using SPSS version 12.0 software (SPSS Inc., Chicago, IL).

Results

VR score

In our initial trial, there was no difference in the VR index between the two groups (Group 1 versus Group 2, 8.9±3.3 versus 9.4±3.7; P=.832). However, in the second trial, the VR index of Group 1 was significantly greater than that of Group 2 (19.3±4.5 versus 9.7±4.1, respectively; P=.001). Additionally, the VR index of Group 1 increased significantly in the second trial (P=.010), whereas the VR index of Group 2 did not change (P=.871) (Fig. 3).

FIG. 3.

The virtual reality (VR) index of the two groups in the two consecutive trials. Group 1 showed a significantly improved VR index in the second trial.

Da Vinci exercise score

Objective learning curve

The mean time required for needle control was reduced in Group 1 by about 63 seconds after VR training, although the difference was not significant. (initial trial versus second trial, 7.05±2.46 minutes versus 5.99±1.58 minutes; P=.130) (Fig. 4). However, there was a significant decrease in the time it took to perform the robotic exercises in the suture and tying session (initial trial versus second trial, 9.56±1.41 minutes versus 7.1±1.54 minutes; P=.031) (Fig. 5). The performance of Group 2 did not improve significantly between the sessions.

FIG. 4.

The mean time for needle control in the da Vinci exercise did not differ between the two consecutive trials in either group.

FIG. 5.

The mean time for suturing and tying in the da Vinci exercise significantly decreased in Group 1 in the second trial.

Although both groups showed a decrease in the number of errors in the second trial compared with the first trial, the difference was not statistically significant (Fig. 6). There was a significant increase in the DV score index of Group 1 (initial trial versus second trial, 3.85±0.73 versus 5.80±1.13; P=.016) between the two trials but not of Group 2 (initial trial versus second trial, 3.66±0.65 versus 4.05±1.03; P=.262) (Fig. 7).

FIG. 6.

The number of errors in each da Vinci performance in the two consecutive trials did not show significant differences.

FIG. 7.

The composite DV index between the two consecutive trials demonstrated significant improvement of da Vinci performance in Group 1 but not in Group 2.

Comparison of da Vinci performance between Group 1 and Group 2

In the initial trial, there was no significant difference in the DV index between Group 1 and Group 2. However, the results were significantly different between the two groups in the second trial. The time for suture and tying in Group 1 was 7.1±1.54 minutes and for Group 2was 10.55±1.93 minutes (P=.018). The DV index was higher in Group 1 (5.80±1.13) compared with Group 2 (4.05±1.03) (P=.028). Assuming that the DV index is an accurate indicator of skill, Group 1 showed improved da Vinci performance after the VR training in the second trial compared with Group 2 (Table 2).

Table 2.

Measures of da Vinci Performance Between Groups

Trial, parameter	Group 1 (n=6)	Group 2 (n=6)	P value
Initial
Time (minutes)
Needle control	7.05±2.46	7.45±2.88	.264
Suture and tying	9.56±1.41	10.21±2.63	.351
Number of errors
Needle control	2.20±0.44	2.50±0.54	.343
Suture and tying	2.61±0.54	2.83±0.408	.438
DV index	3.85±0.73	3.66±0.65	.584
Second
Time (minutes)
Needle control	5.99±1.58	6.8±1.68	.361
Suture and tying	7.1±1.54	10.55±1.93	.018
Number of errors
Needle control	1.60±0.89	2.50±0.81	.070
Suture and tying	2.00±0.72	2.33±0.51	.317
DV index	5.80±1.13	4.05±1.03	.028

Data are mean±standard deviation values.

DV index, composite index.

Discussion

Despite the widespread use and interest in robotic surgery, incorporating this technology into surgical education and developing a training device for future robotic surgeons has remained a challenge.^12,13 Training methods for laparoscopic surgery include live animal training, human and animal cadaver training, training using a box trainer, and VR training.^2,14 However, a consensus about the proper tool for robotic surgical training has not yet been reached.

Multiple factors in the current surgical environment have led to the development of simulators. Operative time and economic issues limit surgical training in the operating room.¹⁵ Having the novice surgeon achieve a certain level of competence before attending in the actual operating room has been proposed as a method of improving efficiency and safety in the operating room.^16–18 Other issues surrounding surgical training include legal complications, limitations in trainee work hours, and ethical considerations related to learning basic skills on humans and animals.¹⁹ In addition, developments in computer technology have led to the introduction of VR simulators that can allow for the training and evaluation of surgical skills while avoiding many of these issues.²⁰

Until now, there has been limited work describing the initial validation (face, content, and construct validity) of robotic VR simulators (the dV-Trainer) and the Robotic Surgery Simulator (RoSS; Roswell Park Cancer Institute and the State University of New York at Buffalo, NY).^6–9,12 However, beyond assessing the basic validity, only a few controlled studies have been conducted to assess the impact of VR simulator training on robot-assisted performance competency. In a study by Halvorsen et al.,²¹ groups training on the SimLap-Zeus VR simulator (SimSurgery^®, Oslo, Norway) showed similar robot-assisted suture skills (without knot-tying) to groups that trained using a mechanical simulator (ZEUS^® Surgical System; Computer Motion Inc., Santa Barbara, CA). They concluded that the VR simulator is a suitable alternative to mechanical training models and suggested that it might be superior because mechanical training requires a full setup, which is time consuming, requires technicians, and has the potential risk of harming the robotic system if used improperly. They also emphasized that the ability to record all types of performance metrics during training was an another advantage of the VR simulator.²¹ Lerner et al.²² compared a VR simulator training group (medical students) with a dry lab training group (urology residents) performing five exercises on the da Vinci. They demonstrated that training with the VR simulator (dV-Trainer) and da Vinci dry lab training yielded similar improvements in da Vinci performance.

In this study, we used a randomized controlled method with a homogeneous cohort composed of trainees with similar laparoscopic surgical experience and demographics. The enrolled individuals were all enthusiastic about acquiring robotic surgical skills. We computed two indices for assessing objective individual performance scores on both the VR simulator and da Vinci Surgical System. The different metrics used to evaluate competency in each task were combined, which made it possible to objectively and quantitatively determine the performance level of each student and assess change over time.

After completing VR simulator training with an experienced trainer, Group 1 showed improved da Vinci performance, as demonstrated by a decrease in suturing time and an increase in DV index. During our two consecutive VR and da Vinci trials, both groups exhibited a learning curve that showed improved competency in the VR training group and significantly less improvement in the control group. The learning curve in the context of surgical skills is the time taken and/or the number of procedures an average surgeon needs to enable him or her to perform a procedure independently with a reasonable outcome.¹⁶ There are several factors that influence the learning curve that should be taken into account. The manual dexterity and anatomical knowledge of the surgeon play a considerable role in determining the learning curve. The use of a structured training scheme and mentoring also has an effect, as does the nature of the procedure. Other important factors are the frequency of procedures performed in a specific time period. Patient factors such as anatomic complexity and disease variation can also affect the learning process.^15,16

Improved da Vinci performance does not directly imply an acceleration of the learning curve in real robotic surgery. As mentioned previously, the role of manual dexterity in successful surgical performance is critical. In this study, needle control and knot tying were not simple tasks and required concentration. For example, in the needle control session, a small hole of about 2.0 mm within a pin made the passing of a curved needle difficult even with the use of the robotic EndoWrist function. Suturing and tying are known to be easier using a robotic system compared with the same skills in a laparoscopic procedure.¹¹ However, in a study by Chang et al,²³ experienced laparoscopists showed initially inferior performance using the robotic system compared with conventional laparoscopy when performing intracorporeal suturing and knot tying. Novices of robotic surgery struggle because of the lack of tactile feedback from the da Vinci system during suture and tying with thin suture materials. Breaking thread and needle bending were the main causes of error detected during the robotic exercise in this study, likely due to the fact that there is less visual feedback. These issues increased the time it took to complete the exercise and the number of errors. However, after training with the VR simulator, it appeared that trainees performed these complex robotic tasks with less psychomotor stress and actively manipulated the clutch and camera pedals of the robotic system.

There were some limitations to this study. The small sample size is a factor that may have resulted in the lack of statistical significance in the comparisons between the two groups. Although we randomly assigned the participants, who had similar laparoscopic surgical experiences and demographics, into two groups, the difference in their baseline surgical experiences or skills is an inevitable limitation to this study. Another issue is that da Vinci performance included only an inanimate platform. However, the use of living training conditions such as animal models or human surgery has its own drawbacks, including difficulty in evaluating an individual's performance level objectively due to anatomical variations and lack of concrete metrics.

It may be difficult to draw solid conclusions from this type of comparison because it does not define the role of the VR device on a real operation at the da Vinci console and does not provide the information on which lab-based module is the best for training purposes. A more clinically relevant study may be a comparison between a VR simulator and other robotic training tools such as an animal or cadaveric lab.

In this study, we computed composite scores including time and error after several experimental trials for robotic performance, anticipating a standardized assessment for certification of real da Vinci surgery. Based on our results, we can conclude that VR simulator training helps to expedite the acquisition of sophisticated da Vinci skills and that this training could improve the manual dexterity of surgeons for use in a real robotic surgery.

Footnotes

Acknowledgments

We would like to thank Drs. Dong-Nyung Son, Dong-Woo Lee, Young-Dong Yoo, Gun-Young Byun, Yoon-Jung Boo, Sung-Bae Hwang, Hye-Yoon Lee, and Azali H. Amar for participating in this study as volunteer examiners.

Disclosure Statement

No competing financial interests exist.

Author Contributions

J.S.C. and S.H.K. are responsible for study concept and design. J.S.C., K.Y.H., J.M.K., J.K., S.J.B., and J.W.S. are responsible for acquisition of data. J.S.C., K.Y.H., and S.H.K. are responsible for analysis and interpretation of data. Jae Sung Cho drafted the manuscript. J.M.K., J.K., S.J.B., J.W.S., and S.H.K. are responsible for critical revision.

References

Pugin

, Bucher

, Morel

. History of robotic surgery: From AESOP^® and ZEUS^® to da Vinci^®. J Visc Surg, 2011; 148:e3–e8.

Gurusamy

, Aggarwal

, Palanvelu

, Davidson

. Systemic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. Br J Surg, 2008; 95:1088–1097.

Mahadevan

. The Human Tissue Act 2004: Implications for anatomical work at the College. Bull R Coll Surg Engl, 2006; 88:264–265.

Pellen

, Horgan

, Barton

, Attood

. Construct validity of the ProMIS laparoscopic simulator. Surg Endosc, 2009; 23:130–139.

Gilley

, Sundaram

. Impact of virtual reality simulators in training of robotic surgery. In: Hemal

, Menon

, eds. Robotics in Genitourinary Surgery. London: Springer-Verlag, 2011:163–168.

Lendvay

, Casale

, Sweet

, Peters

. VR robotic surgery: Randomized blinded study of the Dv-Trainer robotic simulator. Stud Health Technol Inform, 2008; 132:242–244.

Kenney

, Wszolek

, Gould

, Libertino

, Moinzadeh

. Face, content, and construct validity of dV-Trainer™, a novel virtual reality simulator for robotic surgery. Urology, 2009; 73:1288–1292.

Sethi

, Peinie

, Mohammadi

, Sundaram

. Validation of novel virtual reality robotic simulator. J Endourol, 2009; 23:503–508.

Hung

, Zehnder

, Patil

, Cai

, Nq

, Aron

, Gill

, Desai

. Face, content and construct validity of a novel robotic surgery simulator. J Urol, 2011; 186:1019–1024.

10.

Korndorffer

Jr , Dunne

, Sierra

, Stefanidis

, Touchard

, Scott

. Simulator training for laparoscopic suturing using performance goals translates to the operating room. J Am Coll Surg, 2005; 201:23–29.

11.

Stefanidis

, Wang

, Dune

, Scott

. Robotic assistance improves intracorporeal suturing performance and safety in the operating room while decreasing operator workload. Surg Endosc, 2010; 24:377–382.

12.

Seixas-Mikelus

, Kesavadas

, Srimathveeravalii

, Chandrasekhar

, Wilding

, Guru

. Face validation of a novel robotic surgical simulator. Urology, 2010; 76:357–360.

13.

Katsavelis

, Siu

, Brown-Clerk

, et al. Validated robotic laparoscopic surgical training in a virtual-reality environment. Surg Endosc, 2009; 23:66–73.

14.

Munz

, Kumar

, Moorhy

, Bann

, Darzi

. Laparoscopic virtual reality and box trainers: Is one superior to the other?. Surg Endosc, 2004; 18:485–494.

15.

Barreq

, Jayarman

, Kiaii

, Schlachta

, Pautler

. The role of surgical simulation and the learning curve in robot-assisted surgery. J Robotic Surg, 2008; 2:11–15.

16.

Lee

, Caddede

, Janetscek

, et al. International surgical telemonitoring: Our initial experience. Stud Health Technol Inform, 1998; 50:41–47.

17.

Caddedu

, Kondraske

. Human performance testing and simulators. J Endourol, 2007; 21:300–304.

18.

Rehring

, Powers

, Jones

. Integrating simulation in surgery as a teaching tool and credentialing standard. J Gastrointest Surg, 2008; 12:222–233.

19.

Subramonian

, Muir

. The ‘learning curve’ in surgery: What is it, how do we measure it and can we influence it?. BJU Int, 2004; 93:1173–1174.

20.

Grantcharov

, Kristiansen

, Bendix

, Bardram

, Funch-Jensen

. Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg, 2004; 91:146–150.

21.

Halvorsen

, Elle

, Dalinin V

, Merk

, Sorhus

, Rotnes

, Fosse

. Virtual reality simulator training equals mechanical robotic training in improving robot-assisted basic suturing skills. Surg Endosc, 2006; 20:1565–1569.

22.

Lerner

, Ayalew

, Peine

, Sundaram

. Dose training on a virtual reality robotic simulator improve performance on the da Vinci^® surgical system?. J Endourol, 2010; 24:467–472.

23.

Chang

, Satava

, Pellefrinin

, Sinannan

. Robotic surgery: Identifying the learning curve through objective measurement of skill. Surg Endosc, 2003; 17:1744–1748.