Robotics Training Program: Evaluation of the Satisfaction and the Factors That Influence Success of Skills Training in a Resident Robotics Curriculum

Introduction

S ince the first publication of a robot-assisted prostatectomy in 2001, ¹ the use of the robot has increased dramatically in the field of urology, as well as other fields. ² The use of the robot has helped to facilitate the performance of complex urologic procedures in a minimally invasive manner. Although it may have a shorter learning curve than laparoscopy, ^{3 –6} initial training on the robot is difficult because the teaching surgeon has reduced direct control. Because robotic surgery obligates the operating surgeon to be removed from the patient, the teaching surgeon is less able to correct errors in operative technique, when not near the console, and is farther away from the patient to control or repair significant injuries, when near the console. Thus, training on the robot before experience on the console in a clinical scenario becomes vastly important. Adequate training on a robotic surgical system or simulator, however, can be difficult to implement in a training institution given the expense of simulators and the inability to have available time on the robot in an active clinical setting.

There are several inanimate models and virtual reality models that have demonstrated improvement in performance in a live surgical scenario for laparoscopy. ^{7 –12} More recently, it has been suggested that training on robotic simulators can also translate to improved performance of a live surgical task. ^{13 –18} At our institution, we have required residents to perform at least four separate training sessions on the da Vinci robot as part of our robotics curriculum. Briefly, the curriculum includes the following components: (1) Online da Vinci training modules (www.davincisurgerycommunity.com); (2) learn details of patient-side cart maneuvers, such as docking and instrument changes during the first weekend session; (3) complete four weekend training sessions at the console; (4) participate as a bedside assistant in cases; and (5) gradually increase time on the console performing increasingly complex portions of the procedure.

In this study, we evaluate improvement, including the impact of training frequency, of basic robotic tasks during training on inanimate models with the da Vinci robot via time and accuracy parameters, and with the use of questionnaires, evaluate resident perception of this training course.

Methods

A total of 19 residents and fellows were followed prospectively. Training sessions with the da Vinci robot were arranged on Saturday morning for two weekends each month. Residents were notified of the days and were asked to choose 1-hour time sessions to allow for an individualized training session. During the first training session, manipulation of the patient-side cart arms for tasks such as docking, changing instruments, and changing the camera angle were demonstrated to the trainee. Then, the controls of the console were shown, including the function of the pedals and the importance of matching the camera angle with the angle set at the patient-side cart.

Assessment of perceived usefulness

Surveys were sent to each resident who had completed the robotic training course. The surveys comprised a total of nine questions scored on a 5-point Likert scale (Fig. 1). The questions asked residents whether they thought that they had improved as a result of training, their perceived usefulness of the training, interest in further training, and ease of participating in the training program.

OPEN IN VIEWER

FIG. 1.

Resident perception of the robotics training curriculum.

Results

One fellow and 18 residents performed a baseline training session, where each of the tasks was scored. Based on year of surgical training, more advanced residents performed better in mean Peg-Board time (81 s vs 133 s), and pattern-cutting time (258 s vs 555 s), P<0.05 (Table 1). Participants with previous robotic experience performed suturing quicker than those who did not (201 s vs 370 s, P=0.029).

A total of 18 residents and 1 fellow completed a mean of 4.3 trials. Three residents were at the beginning of the training and have not yet completed the four sessions or had time on the robotic console during a live procedure. Trainees who performed four or more trials (16) completed them over an average time span of 13.7±8.1 mos. Trainees demonstrated significant improvements (P<0.05) from their initial training sessions to their final session in the following tasks (Table 2): Peg-Board accuracy (95.8% vs 79.0%), checkerboard deviation (4.8% vs 18.2%) and time (293 s vs 404 s), string-running time (110.2 s vs 70.7 s), pattern-cutting time (257 s vs 399 s), and suture time (203 s vs 305 s).

Median interval between training sessions was 2.3 mos (interquartile ratio=4.0 mos). Time to previous session correlated with relative improvement in Peg-Board and pattern-cutting time (r=−0.300 and −0.277, P=0.021 and 0.041) from the previous session, but no significant correlations were seen with the other training exercises (Table 3). No specific training interval (using 1 mo, 2 mos, 3 mos, 4 mos, 5 mos, and 6 mos as specific cut points) demonstrated a significant change in the degree of improvement from one session to the next for any of the training parameters studied. Number of robotic cases in which the resident participated did not correlate with improvement in scores.

Finally, year of training and number of robotic cases in which the trainee participated was compared to the final scores for each training exercise. No significant differences were noted between inexperienced and more experienced residents at the end of training (P>0.05, Table 4).

Discussion

In this study, we followed residents and fellows who participated in a weekend robotic training session. Basic robotic tasks showed construct validity for Peg-Board, cutting, and suturing exercises at baseline performance. Trainees demonstrated significant improvements in the Peg-Board accuracy, checkerboard time and deviation, string-running time, pattern-cutting time, and suturing time after a mean of more than four total training sessions. Although improvements were noted in the pattern-cutting deviation and Peg-Board time, these were not statistically significant. The length of time between training exercises had a moderate effect on objective interval improvement between training sessions, with regard to pattern-cutting time and Peg-Board time, but not other elements scored in the training program.

Resident-reported satisfaction, according to the nine-item survey, was high. Weekend training sessions were preferred, and it was thought that these were easy to attend. The training program was described as very useful, with many trainees reporting improvement in their performance of robotic tasks and increased preparedness for live robotic surgery. Although the training program was rated generally favorable for preparing residents to perform live robotic surgery, this rating was somewhat lower than other dimensions (median 3.54). This may be explained by the limitation of most robotic surgical training tasks in general. Currently, inanimate robotic laboratories and robotic virtual reality simulators are limited to simulating basic robotic tasks, whereas other complex tasks such as a nerve-sparing dissection or apical dissection cannot be adequately replicated. Thus, while the training may improve the familiarity of the use of the robot, it does not offer experience in complex dissections needed in live surgery. This also contributes to the failure of robotic experience during the training period to influence the scores of these basic tasks, which can result from either not performing the procedures on the console or performing things that are more advanced than the basic tasks.

In addition, while surgical experience did initially influence scores on the training exercises, the differences attenuated by the end of the training period. This suggests that basic skills are attained by trainees to a similar level, regardless of experience. A potential use for this training program then is to ensure that residents learn basic robotic tasks early in the training program to prepare them for more advance robotic tasks. Additional studies would need to determine whether performance in live surgery is improved after the training program.

With the emerging popularity and reliance of many urology training programs on robot-assisted surgery, it is necessary to delineate a proven system for training residents on these complex surgical systems. ^{19 –21} Training models have previously been described by Scott and associates ⁷ and modified for comparison with a virtual reality robotic simulator by Lerner and colleagues. ¹⁵ These training exercises have shown some benefit, because they addressed the development of technical skills using inanimate objects without exposing the trainee to the risks associated with live surgery. ^15,18,22

While these studies discuss the formulation and validation of these models as acceptable training adjuncts, much is still unknown about how to best use the exercises for optimal training outcomes. As of yet, no study addresses the amount of time learners should spend with the training exercises and with what frequency training should be conducted. In this manner, our study is unique in that it assesses objective and subjective utility of a robotic training program within a practical, realistic context in a residency program. Further, we noted only a small influence of frequency on interval improvement in the performance of basic robotic tasks.

The value of inanimate models in minimally invasive surgical skills acquisition is more established with laparoscopic training tasks. Scott and coworkers ⁷ compared 13 residents who were undergoing training on inanimate surgical task with 14 untrained residents, demonstrating improved performance of a laparoscopic cholecystectomy assessed with the GOALS structured evaluation. Korndorffer and colleagues ⁹ randomized 17 residents to demonstrate that trained (8) residents demonstrated significant improvement in suturing during a live laparoscopic case than untrained residents. The Mistels training program has been developed as a standardized training curriculum for laparoscopic surgical training because it combines both basic training tasks and suturing tasks. ²³ Similarly, laparoscopic virtual reality trainers have been shown to improve performance in live laparoscopic surgery. ^{24 –26}

Robotic skills training has demonstrated some effectiveness in previous reports. In a study of 43 medical students, Benson and associates ²⁷ demonstrated that training improved performance of two basic skills tasks and one suturing task. This benefit was augmented when the training sessions were more formal, including demonstration of the tasks by an experienced surgeon and allowance for trainees to ask questions or receive feedback. ²⁷ Basic skills tasks in inanimate models used for laparoscopic training translate to robotic training, and it has been shown that naïve trainees perform the tasks faster with the robot. ^3,4,17 A virtual reality simulator of the robot, which includes performance metrics, reliably distinguished seven expert surgeons from nine novices. ¹⁸ Most participants found it realistic and reliable for basic tasks, although suturing was less realistic. ¹⁸ Furthermore, training on the virtual reality simulator produced similar improvements in basic tasks performed on the da Vinci system when compared with training on the da Vinci system itself, in 23 trainees. ¹⁵

Although a laparoscopic training program is perceived to be an effective tool in training by 88% of 162 responding program directors of a survey of 253 programs, only 55% had laparoscopic skills training laboratories. ²³ Nearly all programs reporting a training laboratory had video trainers, but only 46% had virtual reality equipment. ²³ Furthermore, in a survey of 1056 urology residents, only 38% perceived their training to be satisfactory, with only 31% stating that they would perform robotic surgery after residency. ²⁸ Although this may have increased since 2005, the year of this report, increased use of simulator programs could help improve this.

Because robotic training laboratories require either a dedicated training robot or a virtual reality trainer, there are substantial financial barriers to acquiring a robot training laboratory. An alternative is to use the robots normally used in the operating room to train with specific training instruments that have more use lives than normal instruments. This can limit the ability to train during the weekday, often making weekday sessions difficult to arrange or attend. This may explain why weekend training sessions are preferred. Furthermore, the training sessions are supervised by the fellow in minimally invasive surgery. The system is set up by a surgical technician who is trained to set up the da Vinci surgical system. This helps keep costs of the robotics course low.

In addition, previous data on laparoscopic and robotic skills training has not investigated whether the acquisition of skills is dependent on training frequency. In this study, we demonstrate a shorter interval between training sessions correlated modestly with the performance of two separate tasks: Peg-Board and pattern cutting. A specific length of time, however, did not significantly distinguish training performance. With more difficult tasks, such as completing a model of the anastomosis, training frequency may become more important.

This study had several limitations, of which we discuss some. One limitation is that the study did not have a comparison group. Having a group whose members did not train, however, would have conflicted with resident training because we were evaluating the improvement of skills in a practical setting as part of a standard resident training program. Along with using the same exercises for pretraining evaluation, training, and post-training evaluation, the absence of a control makes it difficult to determine whether the improvement actually represented the acquisition of skills or was solely from repetition. Also, the cohort studied is small (although it is comparable to other series evaluating robotic trainers), thus evaluating factors that may influence training performance is difficult. A third limitation was that there was no proficiency level set for the residents to attain. More experience with robotic training will hopefully allow some basic proficiency criteria to be established, which could provide a benchmark for residents to achieve before beginning live surgery.

The focus of this study was to determine whether a regular, weekend robotic training session was practical and effective in a residency training program, given both financial and time constraints in a training institution. This was, in fact, demonstrated through observation of those involved in the course.

In the future, the development of standardized training programs that develop basic and advanced robotic skills will be necessary. These programs will also let trainees and mentors evaluate the progress of the trainee to allow safe progression to live surgery at the console. This curriculum should be validated in a multi-institutional setting and supported by a national society such as the American Urological Association, similar to the Fundamentals of Laparoscopic Surgery program of the Society of American Gastrointestinal and Endoscopic Surgeons. ²³

Conclusion

Attending regularly scheduled weekend training sessions involving the use of bench models and a da Vinci robot led to improved performance of five basic robotic tasks. Subjective assessment of the program demonstrated that the program was useful to the residents and that weekends were more easily attended. Future developments of both inanimate and virtual reality simulators should focus on more complex tasks to allow continued benefit in simulated training throughout one's training.