Baseline Laparoscopic Skill May Predict Baseline Robotic Skill and Early Robotic Surgery Learning Curve

Introduction

While not as prevalent as in the United States, robotic surgery is becoming increasingly popular in Canada. In addition to patient-driven demands, it is anticipated that in the coming years, the cost of robotic surgery will become more affordable, ¹ which will allow for further growth in our healthcare system. Along with this growth, there is a need for comprehensive training. In this context, we have previously developed and implemented a multidisciplinary simulation-based robotic surgery basic skills training course (BSTC) ² at our institution.

Simulation-based training is an invaluable component of modern surgical education, particularly for novice surgeons in the early part of their learning curve. ^{3 –5} The opportunity for deliberate practice, in a low-stakes environment, with timely provision of expert feedback permits the trainee to learn a new skill or procedure while limiting the impact of training on patient outcomes, thus making simulation-based training an integral tool in the education of contemporary surgeons. ^{6 –9}

Trainees frequently begin educational curricula with differing levels of ability and experience and we know from the experiential learning theory ¹⁰ that a trainee's prior knowledge and experience can directly impact their future skills learning. As such, whenever possible, it is most effective when the course learning objectives and teaching methods are matched to the participants' baseline skill level and learning curve. Kneebone's eminent conceptual framework ⁷ of simulation-based training curricula impresses upon the educator the importance of facilitating individualization of the curriculum based on the trainee's personal needs, innate skills, and learning curve.

The use of simulation-based training methods to improve procedural skills is important and has been demonstrated to be effective. ^7,8 There are many studies that demonstrate improved performance in an ex-vivo surgical simulation training setting. ⁹ More importantly, an increasing number of studies are demonstrating transference between simulation-based training and intraoperative performance and clinical outcomes; the intended definitive goal of surgical simulation-based training. ¹⁰

Regardless of training methodology, however, it is well recognized that surgical trainees acquire technical skills at different rates and that the time required to achieve competency can vary widely among trainees. The reason for this is multifactorial, relating not only to the fact that trainees start with different prior baseline knowledge but also that each trainee has a unique set of innate skills and potential for learning. In essence, each surgical trainee has a unique learning curve for any new technical skill. Knowledge of, or the ability to anticipate or predict, an individual trainee's personal learning curve would facilitate the development of a more individualized training curriculum, an important principle in the design of a comprehensive simulation-based training curriculum. ³

While robotic surgery programs continue to grow, at our institution, pure laparoscopy remains the more prevalent surgical platform. In an effort to improve our robot surgery BSTC, this study aims to examine whether there is any relationship between a trainee's laparoscopic skills and the baseline robotic skill or the robotic surgery learning curve.

Materials and Methods

Our study was conducted from November to December 2014 at two tertiary academic medical centers in Toronto, Canada: St Michael's Hospital Allan Waters Family Simulation Centre and Mount Sinai Hospital Surgical Skills Centre. Research ethics board approval was obtained for the study.

Residents and fellows across various surgical specialties (urology, gynecologic oncology, thoracic surgery, and general surgery) at the University of Toronto were invited to participate in a robotic surgery BSTC. The details of the 4-week curriculum have been previously published. ² Before the hands-on introduction and training portion of the BSTC, each participant underwent an assessment of the baseline laparoscopic skill. This involved completing a basic laparoscopic peg transfer (PT) task and a more advanced intracorporeal suturing and knot tying (ISKT) task using standardized instruments and a Laprotrain (Endosim™, Belfast, NI) box trainer. These tasks have been previously validated as part of the Fundamentals of Laparoscopic Surgery Curriculum. ⁸

Following the initial hands-on training portion of the course, each participant's baseline robotic skill was assessed on the da Vinci^® (S model) robot (Intuitive Surgical, Sunnyvale, CA), using the same two validated tasks utilized for the laparoscopic skill assessments (PT and ISKT).

One week after completion of the 4-week, standardized, simulation-based robotic surgery BSTC, each participant was tested again using the same da Vinci robot on the same PT and ISKT tasks.

Each trainee's performance of the PT and ISKT tasks for the laparoscopic, baseline robotic, and postcourse robotic assessments was video recorded and deidentified. At a later date, two blinded, laparoscopic/robotic experts scored all performances. For each task, participants' time to completion and number of errors were recorded. Each error was given a time penalty, and the total time to completion was calculated for each task. Two blinded content experts then provided a global rating score (GRS), using a Likert scale from 1 to 5, for each task performed by all participants.

Paired sample t-tests, Wilcoxon matched-pairs tests, and analysis of variance were used to compare laparoscopic, baseline robotic, and postcourse robotic task performance metrics; task time to completion; and GRS. Pairwise comparisons were conducted to analyze the specialty groups with appropriate p-value corrections. Pearson and Spearman correlation analyses were used for continuous and categorical variables, respectively. SPSS v21 software was used for all data analysis.

Results

Of the total 43 participants in the course, complete pre- and postcourse data were available for 32 trainees (14 urology, 7 gynecology, 8 thoracic surgery, 3 general surgery). The majority were clinical fellows (66%) or senior residents (25%). Half (16/32) of the participants had no prior clinical robotic surgery experience. While 28% (9/32) of participants had previous robotic surgeon console experience, no participant had >10 cases of console experience (Table 1).

There was internal validity demonstrated in the GRS between blinded experts, with an excellent inter-rater reliability of 0.90 (p < 0.05).

Interestingly, there was no significant difference in baseline laparoscopic and robotic performances between groups based on reported robotic experience. Also, no differences were seen between surgical subspecialty groups on baseline laparoscopic performance (Table 2). There were small differences in baseline, precourse robotic skill (Table 3); urology trainees completed the robotic PT task significantly faster than the general surgery trainees (p = 0.03), while the gynecology trainees completed the robotic ISKT task significantly slower than the general surgery trainees (p = 0.01). Mean laparoscopic PT task time correlated with mean laparoscopic PT task GRS (r = −0.942) and mean laparoscopic ISKT task time correlated with mean laparoscopic ISKT task GRS (r = 0.554). All correlations were statistically significant (p < 0.001), demonstrating that assessments made using task completion time and GRS were comparable.

Baseline robotic PT task time did not correlate with laparoscopic performance (neither PT nor ISKT tasks), however, baseline robotic ISKT task time correlated with both mean laparoscopic PT task time (r = 0.404, p = 0.01) and ISKT task time (r = 0.503, p = 0.01, Table 4A). Similar findings were found for GRS (Table 4B).

While laparoscopic PT task time did not correlate with postcourse robotic performance, laparoscopic ISKT task time correlated with both postcourse robotic PT task time (r = 0.480, p = 0.01) and ISKT time (r = 0.529, p < 0.01, Table 5A). Again, similar findings were seen with GRS (Table 5B).

Baseline robotic PT task time did not predict postcourse robotic performance, however, baseline robotic ISKT task time correlated with both postcourse robotic PT task time (r = 0.459, p = 0.02) and ISKT task time (0.536, p < 0.01).

There were no differences in postcourse robotic performance on the PT task, however, for the ISKT task, general surgery trainees were consistently better than gynecology and thoracic surgery trainees (Table 6). Prior robotic experience did not correlate with postcourse robotic performance.

Discussion

While there is some colorectal surgery literature that intimates that no prior laparoscopic experience is necessary for the adoption of robotic surgery, ¹¹ the transferability of laparoscopic skills to robotics has been previously validated. ^12,13 For example, Panait and colleagues concluded that even in a group of subjects naive to robotic surgery, laparoscopic skill appears to readily transfer to a robotic platform, and performance of difficult tasks such as intracorporeal suturing is further enhanced with more laparoscopic experience. ¹⁴ Obek et al. similarly demonstrated reciprocal transfer of skill between conventional laparoscopy and robotically assisted surgery. ¹⁵

We found that baseline laparoscopic performance on an advanced task (ISKT) correlated with both baseline and postcourse robotic performance, while baseline laparoscopic performance on a basic skill task (PT) was less predictive. This finding suggests that performance on a complex, as opposed to a basic, laparoscopic skill task may be a better predictor of both baseline robotic skill and capacity for learning basic robotic surgery skills. Similarly, baseline robotic performance on the more difficult robotic ISKT task also better predicted postcourse performance than performance on robotic PT.

The ISKT and PT tasks are two ex-vivo skills that have demonstrated strong construct validity evidence and the ability to distinguish novice from expert surgeons. ⁶ Intracorporeal suturing has been recognized as among the more challenging technical laparoscopic skills across specialties ^16,17 and such so-called high-level skills have been shown to be a more reliable way of differentiating skill sets among senior residents. ¹⁶

There was no statistical difference between the subspecialty groups in baseline laparoscopic skill. However, for baseline robotic skill, there were a few subtle differences; urology trainees demonstrating better scores for baseline PT task performance while general surgery trainees scored significantly better on baseline ISKT than gynecologic trainees. On postcourse analysis, there were no differences in performance of the PT task, but for the more complex ISKT task, general surgery trainees scored significantly better than gynecology and thoracic surgery trainees on GRS. This demonstrates that perhaps the general surgery trainees had the shortest learning curve during the simulation-based robotic BSTC.

There were no differences in postcourse performance based on prior robotic experience, suggesting that our study group were all relatively novice robotic surgery trainees and the heterogeneity of baseline robotic skill level in our study group further suggests a need for training programs to robustly address the learning curve for individual procedures and for individual surgeons.

By better assessing baseline trainee knowledge and skill, educators can develop individualized curricula that not only focuses on what educators want to teach but more so on what the learner needs or hopes to achieve, using the tools most suitable to their learning abilities.

This study has several limitations. The sample size is small and probably disproportionately represents those trainees with an interest in robotic surgery rather than trainees in general given the voluntary nature of the course. In addition, the significance of subspecialty comparisons may be limited by small group samples. This study examines the learning curve for a group of novice robotic surgeons over a short period of time, and as such, the durability of learning was not addressed. In addition, a study of comparative effectiveness between this robotic surgery BSTC and another mode of training would better assess the value of the course. The study participants themselves may have also introduced another form of bias into the results that cannot be accounted for. The trainees in our study are adult learners, ¹⁸ with external factors affecting their lives. Given that learning is the voluntary, active assimilation of knowledge so that it becomes useful in the future, one potential explanation for the differences in skills level is the perception of the usefulness of robotic skills in the Canadian context, where robotic usage is exclusive to leading academic centers. Perhaps those who performed better believed it to be more relevant to their future careers.

Conclusions

Among a multidisciplinary group of senior level trainees, with limited prior robotic experience, laparoscopic skill may predict baseline robotic skill and may also influence a trainee's basic robotic skills learning curve. Baseline laparoscopic and robotic performance on an advanced suturing and knot tying task, but not a basic PT task, may predict a shorter learning curve for basic robotic skills training. Further exploration of this finding may facilitate the development of more individualized training curricula.