Efficiency and Accuracy of Robotic Surgical Performance Decayed Among Urologists During COVID-19 Shutdown

Introduction

The COVID-19 pandemic has severely impacted health care delivery worldwide. In March 2020, the United States Surgeon General recommended postponing all elective procedures in the United States, resulting in a dramatic decrease in caseloads across surgical specialties. ¹ The potential degradation of surgical skills during the pandemic and this surgical hiatus has not yet been studied. The association between surgeon performance and clinical outcomes is strong. ² Our own study has demonstrated that suturing performance is associated with long-term recovery of urinary continence after robotic prostatectomy. ³ Herein, we evaluate suturing skill degradation succeeding the COVID-19 shutdown.

Methods

Surgeons completed a series of robotic virtual reality (VR) exercises immediately before and after a mandated 6-week surgical hiatus (Fig. 1a). Assessment of computer-generated metrics and objective robotic surgical skills has been extensively studied and validated in VR simulation. ⁴ We collected simulator generated metrics of surgeon efficiency and precision (Fig. 1b). In addition, three independent and blinded reviewers (B.D., D.S., and R.H.) evaluated suturing technical skill utilizing a validated assessment tool. ⁵ Reviewers initially assessed cases independently before reaching a consensus score for discrepant gradings. Surgeons reported their perception of skills based on a survey questionnaire postshutdown. Depending on the results of the D'Agostino–Pearson normality test, comparisons of surgical performance before/after the shutdown were performed with either paired Student's t-test or Wilcoxon-matched signed-rank test. The graphical presentations of subgroup analyses were omitted where p > 0.05.

OPEN IN VIEWER

FIG. 1.

Study overview and assessment parameters. (a) The study timeline indicating the periods of data collection and the duration of the COVID-19 shutdown. (b) Study design and representative image of simulator exercise. (c) COVID-19-shutdown-related surgical caseload drop-off. (d) Time to complete exercise in all participants, experts, and trainees. (e) The comparison of needle handling time. (f) The comparison of needle targeting accuracy. (g) The comparison of instrument collisions. (h) Qualitative assessment of suturing skill. (i) Self-rated assessment of skills before and after the COVID-19-shutdown. NS p > 0.05, *p < 0.05, **p < 0.01, ****p < 0.0001 column heights represent averages. NS, not significant.

Results

Sixteen surgeons completed the VR sessions pre- and post-COVID-19 shutdown. Urologic surgeons were classified a priori as trainee (<300 cases, n = 7, residents+junior fellows) or expert (≥300 cases, n = 9, senior fellows+faculty). Weekly robotic caseload decreased from median 5.5 (1–12.5) to 1 (0–3) cases during the shutdown (p < 0.0001; Fig. 1c).

Overall, exercise completion time increased by 17% for the entire cohort as compared with baseline (p = 0.0056; Fig. 1d). Specifically, trainees required more time to finish the exercise (p = 0.0313). Within the suturing sequence (needle handling, targeting, driving), the needle handling component accounted for the increased task completion time (experts+trainees: p = 0.0459; experts: p = 0.0469, Fig. 1e). Targeting accuracy and instrument collisions (precision metrics) declined during the shutdown. Overall, needle targeting accuracy decreased from median 80% (44–100) to 67% (46–80) (p = 0.0148; Fig. 1f); however, experts accounted for a more robust decrease from median 84% (63–100) to 68% (60–80) (p = 0.0313). Instrument collision events showed no overall increase; however, experts had a significant increase in collisions (p = 0.0497; Fig. 1g). Technical skill evaluation revealed that experts' ability to achieve optimal needle entry angle declined (p = 0.0112; Fig. 1h). Finally, surgeons reported a significant decrease in their perception of robotic surgical skills after the shutdown (trainees: from median 7 (7–9) to 6 (4–9), p = 0.125; experts: from median 8.5 (7–10) to 8.5 (7–9), p = 0.3506, experts+trainees: p = 0.0335; Fig. 1i).

Discussion

Our study collected detailed surgical performance data in a controlled environment before and after the COVID-19 shutdown. A profoundly disruptive event such as a pandemic that has impacted all surgeons simultaneously has not been seen previously. In just 6 weeks, we show significant decay in efficiency and precision metrics alongside technical surgical skills. This is the first study to evaluate surgical skills degradation in expert surgeons. Of note, experts demonstrated a significant decrease in precision similarly to newly trained surgeons. ⁶ The decreased accuracy, efficiency, and perception of skills warrant further investigation on how the COVID-19 shutdown may be impacting clinical outcomes. This study is potentially limited by its focus on suturing performance without evaluation of other technical skills used in surgery. Future study should focus on methods of retaining surgical skills during more common periods of inactivity (i.e., regular skill maintenance training on VR simulator ⁷ during sabbatical, maternity/paternity leave).

IRB Approval and Human and Animal Rights

Our study complied with protocols approved by the University of Southern California's IRB. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. This article does not contain any studies with animals performed by any of the authors.

Informed Consent

Informed consent was obtained from all individuals included in the study.