Abstract
Purpose:
We present our experience of training residents in a weekend robotic training program to assess its effectiveness and perceived usefulness.
Methods:
Bimonthly training sessions were arranged such that residents could sign up for hour-long, weekend training sessions. They are required to complete four training sessions. Five tasks were scored for time and accuracy: Peg-Board, checkerboard, string running, pattern cutting, and suturing. Participants completed surveys (5-point Likert scale) regarding program utility, ease of attendance, and interest in future weekend training sessions.
Results:
Mean number of trials completed by 19 residents was >4, and 16 completed the trials within an average of 13.7±8.1 mos. Significant improvements (P<0.05) were seen in final trials for Peg-Board accuracy (95.8% vs 79.0%), checkerboard deviation (4.8% vs 18.2%), and time (293 s vs 404 s), pattern-cutting time (257 s vs 399 s), and suture time (203 s vs 305 s). Time to previous session correlated with relative improvement in Peg-Board and pattern-cutting time (r=0.300 and 0.277, P=0.021 and 0.041), but no specific training interval was predictive of improvement. Residents found the course easy to attend (3.6), noted skills improvement (4.1), and found it useful (4.0).
Conclusion:
Training in the weekend sessions improved performance of basic tasks on the robot. Training interval had a modest effect on some exercises and may be more important for difficult tasks. This training program is a useful supplement to resident training and would be easy to implement in most programs.
Introduction
There are several inanimate models and virtual reality models that have demonstrated improvement in performance in a live surgical scenario for laparoscopy.
7
–12
More recently, it has been suggested that training on robotic simulators can also translate to improved performance of a live surgical task.
13
–18
At our institution, we have required residents to perform at least four separate training sessions on the da Vinci robot as part of our robotics curriculum. Briefly, the curriculum includes the following components: (1) Online da Vinci training modules (
In this study, we evaluate improvement, including the impact of training frequency, of basic robotic tasks during training on inanimate models with the da Vinci robot via time and accuracy parameters, and with the use of questionnaires, evaluate resident perception of this training course.
Methods
A total of 19 residents and fellows were followed prospectively. Training sessions with the da Vinci robot were arranged on Saturday morning for two weekends each month. Residents were notified of the days and were asked to choose 1-hour time sessions to allow for an individualized training session. During the first training session, manipulation of the patient-side cart arms for tasks such as docking, changing instruments, and changing the camera angle were demonstrated to the trainee. Then, the controls of the console were shown, including the function of the pedals and the importance of matching the camera angle with the angle set at the patient-side cart.
Description of inanimate simulator tasks
The residents were asked to complete five tasks that have been described previously, but are described briefly below:
7,15
1. Peg-Board—four pegs are moved from one side to the other of a Peg-Board, while transferring hands before placement. The task is scored for time and the number of pegs successfully transferred. 2. Letterboard—nine wooden letters and numbers are placed in a standardized fashion alongside a Peg-Board, which has squares containing matching outlines of numbers. The wooden numbers are placed on the appropriate outlines as accurately as possible. The task is scored for time and accuracy (total number of mm deviated divided by total mm of the numbers and letters in the X and Y dimensions). 3. String running—the trainee must grasp marks with alternating hands along the length of an umbilical tape. The task is scored for time. 4. Pattern cutting—the trainee must cut a circle of uniform radius off the surface of a Kendall Telfa® pad. The task is scored for time and accuracy (number of square mm blocks of deviation from the same circle marked on a graph paper divided by the total area of the circle). 5. Suturing—a uniform rectangular-shaped hole is cut from one side of the Penrose drain. A dot is placed on either side of the hole. The trainee sutures from one dot to the other and ties three knots. The task is scored for time.
Assessment of perceived usefulness
Surveys were sent to each resident who had completed the robotic training course. The surveys comprised a total of nine questions scored on a 5-point Likert scale (Fig. 1). The questions asked residents whether they thought that they had improved as a result of training, their perceived usefulness of the training, interest in further training, and ease of participating in the training program.

Resident perception of the robotics training curriculum.
Statistical analysis
Statistical comparisons were conducted using PASW 17® (SPSS Inc, Chicago, IL) software. Construct validity was analyzed by comparing mean scores for each task by year of training and number of previous cases. Comparison of mean final training performances to initial training performances was conducted via paired t tests. Pearson correlations coefficients were used to determine relationships between training intervals and percent improvement in each scored dimension of all training tasks. Additional correlations between number of robotic cases performed during the training period and the relative trainee improvement were calculated to assess the effect of clinical robot experience on improvement in training scores. Finally, the mean scores for each task were compared by training year and robotic experience, using one-way analysis of variance and independent t tests. Significance level for type 1 error was set at 0.05.
Results
One fellow and 18 residents performed a baseline training session, where each of the tasks was scored. Based on year of surgical training, more advanced residents performed better in mean Peg-Board time (81 s vs 133 s), and pattern-cutting time (258 s vs 555 s), P<0.05 (Table 1). Participants with previous robotic experience performed suturing quicker than those who did not (201 s vs 370 s, P=0.029).
A total of 18 residents and 1 fellow completed a mean of 4.3 trials. Three residents were at the beginning of the training and have not yet completed the four sessions or had time on the robotic console during a live procedure. Trainees who performed four or more trials (16) completed them over an average time span of 13.7±8.1 mos. Trainees demonstrated significant improvements (P<0.05) from their initial training sessions to their final session in the following tasks (Table 2): Peg-Board accuracy (95.8% vs 79.0%), checkerboard deviation (4.8% vs 18.2%) and time (293 s vs 404 s), string-running time (110.2 s vs 70.7 s), pattern-cutting time (257 s vs 399 s), and suture time (203 s vs 305 s).
Median interval between training sessions was 2.3 mos (interquartile ratio=4.0 mos). Time to previous session correlated with relative improvement in Peg-Board and pattern-cutting time (r=−0.300 and −0.277, P=0.021 and 0.041) from the previous session, but no significant correlations were seen with the other training exercises (Table 3). No specific training interval (using 1 mo, 2 mos, 3 mos, 4 mos, 5 mos, and 6 mos as specific cut points) demonstrated a significant change in the degree of improvement from one session to the next for any of the training parameters studied. Number of robotic cases in which the resident participated did not correlate with improvement in scores.
Finally, year of training and number of robotic cases in which the trainee participated was compared to the final scores for each training exercise. No significant differences were noted between inexperienced and more experienced residents at the end of training (P>0.05, Table 4).
Survey responses
The trainees reported a high mean score (4.13, confidence interval [CI]95%=3.67–4.60) for interest in minimally invasive surgery, Figure 1. Trainees significantly preferred weekend sessions to weekday sessions (mean score 4.57, CI95%=4.20–4.90, vs 2.36, CI95%=1.55–3.16, P<0.001), and reported relative ease in attending the weekend sessions (3.71, CI95%=3.24–4.19). In addition, they demonstrated high interest in attending future weekend sessions (mean=4.60, CI95%=4.32–4.81). Subjective degree of improvement also scored high (4.11, CI95%=3.65–4.57). Mean preparedness for live surgery was 3.54 (CI95%=3.01–4.07), while mean perceived overall usefulness of the training program was 3.71 (CI95%=3.44–3.98). There were no significant correlations between training year and reported improvement, translation to live surgery, or overall usefulness.
Discussion
In this study, we followed residents and fellows who participated in a weekend robotic training session. Basic robotic tasks showed construct validity for Peg-Board, cutting, and suturing exercises at baseline performance. Trainees demonstrated significant improvements in the Peg-Board accuracy, checkerboard time and deviation, string-running time, pattern-cutting time, and suturing time after a mean of more than four total training sessions. Although improvements were noted in the pattern-cutting deviation and Peg-Board time, these were not statistically significant. The length of time between training exercises had a moderate effect on objective interval improvement between training sessions, with regard to pattern-cutting time and Peg-Board time, but not other elements scored in the training program.
Resident-reported satisfaction, according to the nine-item survey, was high. Weekend training sessions were preferred, and it was thought that these were easy to attend. The training program was described as very useful, with many trainees reporting improvement in their performance of robotic tasks and increased preparedness for live robotic surgery. Although the training program was rated generally favorable for preparing residents to perform live robotic surgery, this rating was somewhat lower than other dimensions (median 3.54). This may be explained by the limitation of most robotic surgical training tasks in general. Currently, inanimate robotic laboratories and robotic virtual reality simulators are limited to simulating basic robotic tasks, whereas other complex tasks such as a nerve-sparing dissection or apical dissection cannot be adequately replicated. Thus, while the training may improve the familiarity of the use of the robot, it does not offer experience in complex dissections needed in live surgery. This also contributes to the failure of robotic experience during the training period to influence the scores of these basic tasks, which can result from either not performing the procedures on the console or performing things that are more advanced than the basic tasks.
In addition, while surgical experience did initially influence scores on the training exercises, the differences attenuated by the end of the training period. This suggests that basic skills are attained by trainees to a similar level, regardless of experience. A potential use for this training program then is to ensure that residents learn basic robotic tasks early in the training program to prepare them for more advance robotic tasks. Additional studies would need to determine whether performance in live surgery is improved after the training program.
With the emerging popularity and reliance of many urology training programs on robot-assisted surgery, it is necessary to delineate a proven system for training residents on these complex surgical systems. 19 –21 Training models have previously been described by Scott and associates 7 and modified for comparison with a virtual reality robotic simulator by Lerner and colleagues. 15 These training exercises have shown some benefit, because they addressed the development of technical skills using inanimate objects without exposing the trainee to the risks associated with live surgery. 15,18,22
While these studies discuss the formulation and validation of these models as acceptable training adjuncts, much is still unknown about how to best use the exercises for optimal training outcomes. As of yet, no study addresses the amount of time learners should spend with the training exercises and with what frequency training should be conducted. In this manner, our study is unique in that it assesses objective and subjective utility of a robotic training program within a practical, realistic context in a residency program. Further, we noted only a small influence of frequency on interval improvement in the performance of basic robotic tasks.
The value of inanimate models in minimally invasive surgical skills acquisition is more established with laparoscopic training tasks. Scott and coworkers 7 compared 13 residents who were undergoing training on inanimate surgical task with 14 untrained residents, demonstrating improved performance of a laparoscopic cholecystectomy assessed with the GOALS structured evaluation. Korndorffer and colleagues 9 randomized 17 residents to demonstrate that trained (8) residents demonstrated significant improvement in suturing during a live laparoscopic case than untrained residents. The Mistels training program has been developed as a standardized training curriculum for laparoscopic surgical training because it combines both basic training tasks and suturing tasks. 23 Similarly, laparoscopic virtual reality trainers have been shown to improve performance in live laparoscopic surgery. 24 –26
Robotic skills training has demonstrated some effectiveness in previous reports. In a study of 43 medical students, Benson and associates 27 demonstrated that training improved performance of two basic skills tasks and one suturing task. This benefit was augmented when the training sessions were more formal, including demonstration of the tasks by an experienced surgeon and allowance for trainees to ask questions or receive feedback. 27 Basic skills tasks in inanimate models used for laparoscopic training translate to robotic training, and it has been shown that naïve trainees perform the tasks faster with the robot. 3,4,17 A virtual reality simulator of the robot, which includes performance metrics, reliably distinguished seven expert surgeons from nine novices. 18 Most participants found it realistic and reliable for basic tasks, although suturing was less realistic. 18 Furthermore, training on the virtual reality simulator produced similar improvements in basic tasks performed on the da Vinci system when compared with training on the da Vinci system itself, in 23 trainees. 15
Although a laparoscopic training program is perceived to be an effective tool in training by 88% of 162 responding program directors of a survey of 253 programs, only 55% had laparoscopic skills training laboratories. 23 Nearly all programs reporting a training laboratory had video trainers, but only 46% had virtual reality equipment. 23 Furthermore, in a survey of 1056 urology residents, only 38% perceived their training to be satisfactory, with only 31% stating that they would perform robotic surgery after residency. 28 Although this may have increased since 2005, the year of this report, increased use of simulator programs could help improve this.
Because robotic training laboratories require either a dedicated training robot or a virtual reality trainer, there are substantial financial barriers to acquiring a robot training laboratory. An alternative is to use the robots normally used in the operating room to train with specific training instruments that have more use lives than normal instruments. This can limit the ability to train during the weekday, often making weekday sessions difficult to arrange or attend. This may explain why weekend training sessions are preferred. Furthermore, the training sessions are supervised by the fellow in minimally invasive surgery. The system is set up by a surgical technician who is trained to set up the da Vinci surgical system. This helps keep costs of the robotics course low.
In addition, previous data on laparoscopic and robotic skills training has not investigated whether the acquisition of skills is dependent on training frequency. In this study, we demonstrate a shorter interval between training sessions correlated modestly with the performance of two separate tasks: Peg-Board and pattern cutting. A specific length of time, however, did not significantly distinguish training performance. With more difficult tasks, such as completing a model of the anastomosis, training frequency may become more important.
This study had several limitations, of which we discuss some. One limitation is that the study did not have a comparison group. Having a group whose members did not train, however, would have conflicted with resident training because we were evaluating the improvement of skills in a practical setting as part of a standard resident training program. Along with using the same exercises for pretraining evaluation, training, and post-training evaluation, the absence of a control makes it difficult to determine whether the improvement actually represented the acquisition of skills or was solely from repetition. Also, the cohort studied is small (although it is comparable to other series evaluating robotic trainers), thus evaluating factors that may influence training performance is difficult. A third limitation was that there was no proficiency level set for the residents to attain. More experience with robotic training will hopefully allow some basic proficiency criteria to be established, which could provide a benchmark for residents to achieve before beginning live surgery.
The focus of this study was to determine whether a regular, weekend robotic training session was practical and effective in a residency training program, given both financial and time constraints in a training institution. This was, in fact, demonstrated through observation of those involved in the course.
In the future, the development of standardized training programs that develop basic and advanced robotic skills will be necessary. These programs will also let trainees and mentors evaluate the progress of the trainee to allow safe progression to live surgery at the console. This curriculum should be validated in a multi-institutional setting and supported by a national society such as the American Urological Association, similar to the Fundamentals of Laparoscopic Surgery program of the Society of American Gastrointestinal and Endoscopic Surgeons. 23
Conclusion
Attending regularly scheduled weekend training sessions involving the use of bench models and a da Vinci robot led to improved performance of five basic robotic tasks. Subjective assessment of the program demonstrated that the program was useful to the residents and that weekends were more easily attended. Future developments of both inanimate and virtual reality simulators should focus on more complex tasks to allow continued benefit in simulated training throughout one's training.
Footnotes
Acknowledgments
This study was partially funded by the Clarian Values Fund Grant VFE-147.
The authors would like to acknowledge Drs. Amanjot S. Sethi and Michelle Learner for previously conducting training exercises with the residents.
Disclosure Statement
No competing financial interests exist.
