Validation of the RobotiX Mentor Robotic Surgery Simulator

Abstract

Objectives:

To assess face, content, and construct validity of the RobotiX Mentor virtual reality simulator, to assess its acceptability as a robotic surgery training tool and feasibility of its use, and to develop a supplementary training curriculum.

Subjects and Methods:

This prospective, observational, and comparative study recruited novice (n = 20), intermediate (n = 15), and expert (n = 11) robotic surgeons as participants from institutions across the United Kingdom and at the 30th European Association of Urology Annual Meeting. Each participant completed nine surgical tasks across two modules on the simulator, followed by a questionnaire to evaluate subjective realism (face validity), task importance (content validity), feasibility, and acceptability. Outcome measures of novice, intermediate, and expert groups were compared using Mann–Whitney U-tests to assess construct validity.

Results:

Construct validity was demonstrated in a total of 17/25 performance evaluation metrics (p < 0.001). Experts performed better than intermediates with regard to time taken to complete the first (p = 0.002) and second (p = 0.043) module, number of instrument collisions (p = 0.040), path length (p = 0.049), number of cuts >2 mm deep (p = 0.033), average distance from suture target (p = 0.015), and number of suture breakages (p = 0.038). Participants determined both the simulator console and psychomotor tasks as highly realistic (mean: 3.7/5) and important for surgical training (4.5/5), with system pedals (4.2/5) and knot tying task (4.6/5) scoring highest, respectively. The simulator was also rated as an acceptable (4.3/5) tool for training and its use highly feasible (4.3/5).

Conclusion:

Construct, face, and content validity was established for the RobotiX Mentor, and feasibility and acceptability of incorporation into surgical training were ascertained. The RobotiX Mentor shows potential as a valuable tool for training and assessment of trainees in robotic skills. Investigation of concurrent and predictive validity is necessary to complete validation, and evaluation of learning curves would provide insight into its value for training.

Introduction

The practice of robot-assisted surgery has increased exponentially over recent years with no sign of growth slowing. Performing surgery with the da Vinci^® Surgical System (Intuitive Surgical, Sunnydale, CA) has several key advantages over traditional laparoscopic surgery, including reduced blood loss,^1,2 shorter hospital stay,^1,2 and reduced risk of nerve and rectum injury.² However, a major limitation is the considerable learning curve associated with performing laparoscopic procedures with robotic assistance.³ Acquiring technical skills in a controlled environment to reduce the initial learning curve is therefore a desirable aspect of training, which may be achieved with simulation.

Virtual reality simulation has gained widespread approval within surgical training for several reasons. Principally, simulation allows trainees to acquire technical skills proficiency in a safe environment without risk of harm to patients.⁴ In addition, performance can be objectively evaluated and measured against predetermined benchmarks, which allows for assessment and monitoring of training progression. The rapid growth of robot-assisted surgery has triggered extensive research into the development and validation of numerous robotic simulators. Robotic simulation training has been shown to significantly improve robotic technical performance,⁴ which may help to reduce the initial learning curve before gaining live console experience.

Presently, a total of five virtual reality robotic simulators are commercially available, each with different characteristics and differing levels of validation.⁵ The most widely used simulator is the da Vinci Skills Simulator, which integrates virtual reality training tasks with the real da Vinci console for maximal realism. The platform has inherently excellent face validity and has demonstrated construct and content validity.⁶ Although it is an exceptional training tool, the simulator requires the actual da Vinci console to operate, limiting the opportunity for use to select institutions. The SimSurgery Educational Platform is a modified laparoscopic simulator, which has face, content, and construct validity established.⁷ However, the value of this simulator is limited due to lack of stereoscopic vision and unrealistic hand controls. The Mimic dV-Trainer (MdVT) is a novel simulator that provides an authentic representation of the da Vinci surgical robot with realistic workspace and hand controls. Simulation training with the MdVT has been shown to significantly improve task completion time on the da Vinci Surgical System,⁸ and face, content, and construct validity has been proven,⁹ although tasks are limited to basic psychomotor skills. The ProMIS is a hybrid simulator that combines real instruments with virtual reality and has face, content, and construct validity established.¹⁰ However, its value in training of robotic technical skills is limited by the dissimilar hand controls and two-dimensional vision. Finally, the robotic surgical simulator is a stand-alone system that incorporates modular training with guided procedures, although only face¹¹ and content¹² validity has been investigated.

The RobotiX Mentor™ (3D Systems; Simbionix Products, Cleveland, OH) system is a new virtual reality simulator, developed to train surgeons for robotic surgery performed using the da Vinci interface. The platform consists of an adjustable headset with integrated stereoscopic visors, nonfixed hand controls, and adjustable foot pedals integrated into a single console (Fig. 1). Four training modules encompassing numerous tasks and two gynecologic operative cases are currently available, with other training modules and urologic procedures in development.

FIG. 1.

RobotiX Mentor surgical simulator in use. Color images available online at www.liebertpub.com/end

Validity of the RobotiX Mentor has not yet been evaluated in the current literature. However, the face and content validity of the Robotic Suturing module has been investigated in a previous study.¹³ Participants determined most aspects of the module as realistic and useful for training, proving face and content validities, respectively. However, the Robotic Suturing training module was completed on the da Vinci Surgical System, and therefore, validity of the RobotiX Mentor has not been investigated before the current study.

This prospective study was performed to demonstrate construct validity of the RobotiX Mentor simulator by comparing performance between expert, intermediate, and novice groups. It also aimed to ascertain the face validity, content validity, acceptability, and feasibility through a quantitative questionnaire. In addition, the present study aimed to develop a training curriculum to reach proficiency based on the evidence obtained.

Subjects and Methods

Study design and participants

The prospective, observational, and comparative study was conducted at the Guy's and St Thomas' NHS Foundation Trust, London. Participants for this multi-institutional validation were recruited from Guy's Hospital, King's College Hospital, and King's College London. In addition, several participants were recruited from across Europe at the 30th European Association of Urology Annual Meeting.

Subjects were recruited and subsequently categorized into one of three groups (expert, intermediate, or novice) depending on experience. The expert group consisted of consultants who had performed at least 25 robotic procedures independently, the number of cases necessary to reach proficiency in robot-assisted radical prostatectomy.¹⁴ The intermediate group was defined as surgeons who had some dual-console experience but had not completed any robotic cases independently. The novice group had no previous operative experience and comprised medical students.

Process

Two training modules were evaluated in the present study. The Fundamentals of Robotic Surgery (FRS) module is based on the physical training model used for the FRS psychomotor skills curriculum, which was developed by international experts through multiple conference meetings.¹⁵ This module contains six tasks: Ring Tower Transfer, Knot Tying, Railroad Suturing, 4th Arm Cutting, Puzzle Piece Dissection, and Vessel Energy Dissection. The Robotic Suturing module includes four tasks: Vertical Defect Suturing, Horizontal Defect Suturing, Continuous Suturing, and Interrupted Suturing (Fig. 2).

FIG. 2.

List of tasks included in the (A) Fundamentals of Robotic Surgery and (B) Robotic Suturing modules.

All participants were guided through the simulator controls and allowed 5 minutes of free practice on the first task (Ring Tower Transfer) as a familiarization session before starting the other modular tasks. Participants had no prior experience with this simulator, and no data were collected from this initial task. Once oriented with the simulator controls, participants completed the remaining five FRS tasks followed by all four Robotic Suturing tasks (Fig. 3).

FIG. 3.

Flowchart depicting study process through which participants travel. Color image available online at www.liebertpub.com/end

After completing both modules, the subjects were given a short structured questionnaire, which included basic demographics and previous experience. This questionnaire was derived from a previously validated questionnaire, which was used by the authors' institution to validate the GreenLight™ laser simulator.¹⁶ Face validity was explored using quantitative questions regarding the perceived realism of the simulator and similarity to the da Vinci surgical robot. The questionnaire also evaluated content validity, feasibility, and acceptability through multiple questions, with a mean score of ≥3 confirming the parameter as valid.

Performance evaluation and outcome measures

Following each task, an instant performance evaluation report was generated by the internal objective assessment tool. The report incorporated various generic and task-specific metrics, which were accepted as outcome measures and subsequently analyzed.

General parameters for the FRS module included time to complete task, total errors, instrument collisions, number of times instrument is out of view, clutch usage, distance by camera, number of movements, and path length. Total errors were defined as the sum of instrument collisions, instrument out of view count, and task-specific errors.

Task-specific metrics in the module encompassed critical errors (Knot Tying), inaccurate punctures (Railroad Suturing), inaccurate targeting (Railroad Suturing), precision (Railroad Suturing), cuts outside marked line (Puzzle Piece Dissection), and cuts >2 mm deep (Puzzle Piece Dissection).

Parameters in the Robotic Suturing module contained time to complete task, unnecessary piercings, average distance from target, suture breakage, needle drops, knot tail length deviation, time needle is out of view, correct suturing angle, needle passage accuracy, suture overstretch, and new suture requests.

Statistical analysis

Performance metrics were analyzed with GraphPad Prism v5.03 (GraphPad Software, Inc., La Jolla, CA) and Microsoft Excel 2013 (Microsoft Corporation, Redmond, WA). Differences between groups were analyzed using two-tailed Mann–Whitney U-tests to assess construct validity. Statistical significance was established with a p-value of <0.05.

Results

Demographics

A total of 46 subjects participated in the study consisting of 20 novices, 15 intermediates, and 11 experts (Table 1). All participants within the novice category were medical students who had no previous surgical experience. The intermediate cohort comprised 2 consultant urologists and 13 urologic trainees, who were in their first to seventh year of surgical training (median: 3) and had assisted in a mean of 81 robotic procedures through console (range: 4–680). The expert group consisted of 11 urologic consultants who had a median independent practice length of 5 to 10 years and had performed a mean of 271 robotic cases (range: 70–1160). The intermediate group had console assisted in a median of 26 cases (interquartile range [IQR]: 11–110) compared with the expert group who had assisted in a median of 114 cases (IQR: 50–145). The expert group had completed a median of 85 cases independently (IQR: 70–150).

Table 1.

Demographics and Previous Experience of Participants (n = 46)

	Novice	Intermediate	Expert
No. of participants	20	15	11
No. of grade/specialty
Medical student	20	—	—
Urology trainee	—	13	—
Urology consultant	—	2	11
Mean age (range)	22 (19–26)	34 (26–51)	47 (41–60)
Median year of study/training (range)	3 (2–4)	3 (1–7)	—
Median years of independent practice (range)	—	1 (0–3)	5–10 (0–>20)
Mean robotic procedures observed (range)	0	37 (4–230)	157 (25–290)
Mean robotic procedures assisted (range)	0	81 (4–680)	112 (25–255)
Mean robotic procedures performed (range)	0	0	271 (70–1160)

Construct validity

The mean and range of participant metric scores are displayed for each group in Table 2, with results of statistical calculations presented in Table 3. Experts significantly outperformed novices in the majority of performance evaluation metrics (17/25), including FRS and Robotic Suturing module completion times, instrument collisions, distance by camera, path length, inaccurate punctures, number of cuts >2 mm deep, number of unnecessary piercings, average distance from target, and needle passage accuracy. However, there was no significant difference between the two groups in numerous metrics, including total errors, number of times instruments are out of view, number of cuts outside the marked line, and number of needle drops.

Table 2.

Comparison of Performance Metrics Between Groups

Module & performance metric	Novice	Intermediate	Expert
FRS
Generic
Time to complete module (seconds)	1832 (1238–2361)	1898 (971–2792)	1223 (893–1733)
Total errors (n)	133 (42–269)	121 (49–218)	113 (56–211)
Instrument collisions (n)	51 (13–121)	38 (7–66)	27 (13–87)
Instruments out of view (n)	43 (6–143)	45 (3–153)	56 (11–109)
Clutch usage (n)	15 (0–53)	12 (0–33)	20 (1–44)
Distance by camera (mm)	345 (0–3369)	713 (0–1987)	1165 (18–2285)
Number of movements (n)	4246 (2002–22,148)	4166 (1684–8779)	2887 (2149–4183)
Path length (mm)	41,732 (20,620–121,722)	30,808 (20,014–46,389)	24,894 (16,065–41,030)
Task specific
Critical errors (n)	3.4 (0–11)	0.9 (0–2)	0.4 (0–1)
Inaccurate punctures (n)	18 (5–43)	10 (2–29)	7 (0–17)
Inaccurate targeting (mm)	90 (7–232)	31 (0–170)	17 (0–46)
Precision (%)	40 (22–64)	56 (28–80)	59 (39–100)
Cuts outside marked line (n)	4.8 (1–15)	3.7 (0–8)	3.7 (1–12)
Cuts >2 mm deep (n)	53 (3–114)	48 (15–92)	29 (14–61)
Robotic suturing
Time to complete module (seconds)	2027 (1297–3099)	1490 (753–1958)	1002 (782–1456)
Unnecessary piercings (n)	65 (11–144)	41 (14–86)	34 (13–69)
Average distance from target (mm)	41 (10–104)	23 (7–142)	8 (7–10)
Suture breakage (n)	3 (0–8)	1 (0–4)	0 (0)
Needle drops (n)	26 (2–79)	25 (6–61)	26 (15–36)
Knot tail length deviation (mm)	224 (99–402)	177 (101–329)	126 (64–178)
Time needle is out of view (seconds)	196 (3–520)	35 (3–131)	30 (1–86)
Correct suturing angle (%)	48 (32–61)	45 (23–69)	49 (23–80)
Needle passage accuracy (%)	90 (79–96)	95 (87–100)	97 (93–100)
Suture overstretch (n)	4731 (33–50,988)	5292 (5–45,122)	1098 (10–5446)
New suture requests (n)	2 (0–5)	2 (0–4)	1 (0–2)

Values displayed are mean scores (range).

FRS = Fundamentals of Robotic Surgery.

Table 3.

Statistical Analysis of Performance Metrics Between Groups

Module & performance metric	Expert vs novice	Expert vs intermediate	Intermediate vs novice
FRS
Generic
Time to complete module (seconds)	<0.001^***	0.002^**	0.629
Total errors (n)	0.397	0.876	0.395
Instrument collisions (n)	0.008^**	0.040^*	0.177
Instruments out of view (n)	0.112	0.253	0.960
Clutch usage (n)	0.137	0.051	0.828
Distance by camera (mm)	0.001^**	0.161	0.004^**
Number of movements (n)	0.332	0.177	0.689
Path length (mm)	0.002^**	0.049^*	0.106
Task specific
Critical errors (n)	<0.001^***	0.066	0.001^**
Inaccurate punctures (n)	<0.001^***	0.447	0.011^*
Inaccurate targeting (mm)	<0.001^***	0.876	0.002^**
Precision (%)	0.001^**	0.917	0.004^**
Cuts outside marked line (n)	0.269	0.814	0.364
Cuts >2 mm deep (n)	0.005^**	0.033^*	0.560
Robotic suturing
Time to complete module (seconds)	<0.001^***	0.043^*	0.023^*
Unnecessary piercings (n)	0.011^*	0.483	0.020^*
Average distance from target (mm)	<0.001^***	0.015^*	0.009^**
Suture breakage (n)	<0.001^***	0.038^*	0.017^*
Needle drops (n)	0.770	0.591	0.813
Knot tail length deviation (mm)	0.005^**	0.052	0.156
Time needle is out of view (seconds)	0.001^**	0.837	<0.001^***
Correct suturing angle (%)	0.690	0.649	0.262
Needle passage accuracy (%)	<0.001^***	0.381	0.005^**
Suture overstretch (n)	0.036^*	0.173	0.439
New suture requests (n)	0.118	0.089	0.877

p-Values are displayed.

p < 0.05; ^** p < 0.01; ^*** p < 0.001.

Experts additionally outperformed intermediates in several (7/25) parameters, including time to complete the FRS and Robotic Suturing modules, instrument collisions, and number of cuts >2 mm deep.

Significant differences were observed between intermediates and novices in 44% (11/25) of metrics, including distance by camera, inaccurate punctures, inaccurate targeting, time taken to complete the Robotic Suturing module, and unnecessary piercings.

Face validity

Face validity was assessed by experts and intermediates with questions relating to the perceived realism of the simulator console and virtual reality tasks. Each element was scored on a 5-point Likert scale (Fig. 4). Overall, the simulator was rated as highly realistic with a mean score of 3.7 (range: 3–5). Regarding the physical console, clutch and diathermy pedals were rated highest (4.2) and graphics were scored lowest (3.5). Participants also deemed the “4th Arm Cutting” (3.9) and “Knot Tying” (3.1) exercises as the most and least realistic tasks, respectively. Furthermore, 73% of experts scored ≥4 on a 5-point Likert scale evaluating similarity to the da Vinci system.

FIG. 4.

Assessment of face validity by novice, intermediate, and expert groups. Each element was scored on a Likert scale between 1 (not realistic) and 5 (very realistic). Color image available online at www.liebertpub.com/end

Content validity

Experts and intermediates evaluated content validity with questions pertaining to the importance of each task for robotic training, rated on a 5-point Likert scale (Fig. 5). All modular tasks were scored between 4.3 and 4.6, demonstrating these tasks as important for robotic training. “Knot Tying” (4.62) and “Vertical Defect Suturing” (4.63) were rated as the most essential tasks, while “4th Arm Cutting” (4.27) was graded as least important. Furthermore, 100% of intermediates and experts agreed (≥4/5) that there is a role for the RobotiX Mentor in training.

FIG. 5.

Assessment of content validity by intermediate and expert groups. Each task was scored on a numerical scale between 1 (not important) and 5 (very important). Color image available online at www.liebertpub.com/end

Feasibility and acceptability

Intermediate and expert groups rated the feasibility and acceptability of integrating the RobotiX Mentor into surgical training using a 5-point Likert scale. The simulator was rated as highly feasible (4.31/5) and highly acceptable (4.27/5). Furthermore, 100% of intermediates and experts believed that simulation should be implemented into surgical training and 63% felt it should form part of accreditation.

Discussion

Virtual reality simulation has been an effective adjunct to surgical training in a number of specialties, including urology,¹⁷ gynecology,¹⁸ and general surgery.¹⁹ Recent technologic advances have allowed the development of numerous virtual reality simulators, which provide training in a variety of procedures, including laser prostatectomy,²⁰ transrectal ultrasound-guided biopsy,²¹ and laparoscopic nephrectomy.²²

Robotic surgical training is particularly challenging due to the substantial accompanying steep learning curve coupled with limited opportunities for console experience.³ To address this issue, extensive research into robotic simulation has been undertaken in an attempt to provide unrestricted psychomotor training without risk of harm to the patient. Virtual reality robotic simulation curricula have previously been shown to improve technical skill proficiency,^23
–25 and live surgical performance has been shown to improve following deliberate practice²⁶ or preoperative warm-up²⁷ with virtual reality simulation. Evaluation of a virtual reality robotic surgical skills curriculum revealed that performance on the da Vinci system increased after simulation training, with trainees reaching proficiency levels in an average of ∼70 attempts.²⁵ Furthermore, it has been suggested that robotic simulation may increase the effectiveness of current training programmes as it allows trainees to mount the learning curve in a laboratory environment before live operating.⁴ The RobotiX Mentor may therefore serve as a useful virtual reality training tool for novice robotic surgeons.

Evidence of construct validity was demonstrated in the present study. Experts and intermediates significantly outperformed novices in the majority of metrics, and experts outperformed intermediates with regard to crucial metrics. Validity was demonstrated in 9 of 14 (64%) FRS metrics and 8 of 11 (73%) Robotic Suturing metrics. However, the number of metrics showing significant differences between expert/novice (n = 7/25) and expert/intermediate (n = 11/25) groups is relatively low, providing inadequate evidence for construct validity. This apparent lack of significance across the remaining tasks may be due to a variety of factors, most importantly the vast range of experience between and within groups, the impact of which may undermine construct validity.

The lack of significance regarding the total number of errors in the FRS module (p = 0.395) may be attributable to the inclusion of “number of times instruments are out of view” metric in the calculation process. Novices and intermediates tended to commit a high number of “cuts >2 mm deep” errors during the Puzzle Piece Dissection task (mean: 53 and 48, respectively), whereas experts tended to accumulate errors throughout by advancing the camera and subsequently incurring penalties each time the instruments are out of the limited view (mean: 56), resulting in similar error rates overall. This explanation is supported by the significant differences observed between expert and novice groups regarding “number of cuts >2 mm deep” (p = 0.005) and “distance by camera” (p = 0.001). However, experts did not provoke the “instruments out of view” error significantly more than other groups (p = 0.112). The absence of significance may be due to the wide range of results within experts (11–109), intermediates (3–153), and novices (6–143).

Metrics relating to economy of motion produced varying results. The total number of movements required to complete the FRS module failed to reach significance (p = 0.177) despite path length between groups showing significant differences (p = 0.002). An explanation for this discrepancy may be that experts moved the camera into close proximity of the working area, and hence, a smaller path length can be achieved with a similar number of movements.

“Number of cuts outside marked line” was the only task-specific metric that was not statistically significant (p = 0.269). This unexpected result may be owing to experts sacrificing psychomotor precision in favor of a shorter task completion time, evidenced by the high significance shown between expert and novice (p < 0.001) and expert and intermediate (p = 0.002) FRS completion times. In addition, no significant differences were detected for number of needle drops (p = 0.591) and correct suturing angle (p = 0.262), the former possibly due to experts placing needles down to tie the suture as is commonly practiced. The inability of these metrics to differentiate between expertise levels suggests these metrics are unsuitable for performance evaluation.

Face validity of the simulator was also confirmed in the present study. Participants rated the RobotiX Mentor hand controls, pedals, and overall experience as highly realistic, reflecting similarity between the simulator and the da Vinci console. All nine tasks were perceived as realistic with participants scoring in most tasks an average of ≥3.5. Moreover, 73% of experts scored ≥4 for similarity to the da Vinci system, further reinforcing face validity. Content validity was additionally ascertained, demonstrated by high scores specified by participants. All tasks were rated as important for robotic training and are hence relevant to the acquisition and development of required psychomotor skills. Subjects rated implementation of the simulator into surgical training as highly feasible and acceptable, and all participants felt that practice with the RobotiX Mentor should be an integral element of robotic surgical training.

The present study proposes a supplementary RobotiX Mentor training curriculum based on the data acquired (Fig. 6). Performance evaluation metrics, which demonstrated construct validity, were selected for inclusion in the curriculum, and mean scores of the expert group were used as benchmark targets to indicate competency. These values were selected for benchmark scores as a wide range of console experience was represented within the expert group, and the targets are achievable with training. Trainees begin with a 5-minute familiarization session to acquaint with the simulator controls followed by completion of all six FRS tasks. After each exercise has been performed twice, trainees repeat the five validated FRS tasks as necessary until benchmark proficiency scores are achieved for both essential skills and task-specific parameters. Once proficiency levels are reached for the FRS module, all four Robotic Suturing tasks are repetitively practiced until each performance objective is accomplished satisfactorily. Trainees may progress onto virtual operative cases such as robotic hysterectomy or vaginal cuff closure after competency is achieved in both training modules.

FIG. 6.

Recommended basic training curriculum for the RobotiX Mentor. All proficiency targets are based on mean scores of the expert group. Color image available online at www.liebertpub.com/end

The RobotiX Mentor is a new stand-alone surgical simulator, which replicates the da Vinci system user interface to provide a highly realistic experience. This novel simulator addresses many limitations described in other simulators, such as inclusion of modular courses and complete virtual procedures (e.g., hysterectomy, vaginal cuff closure), stereoscopic vision, authentic hand controls and pedals, and a realistic workspace. Furthermore, performance reports detail specific parameters to improve and training progression can be monitored with an integrated online platform.

Several limitations were identified in this study. First, the sample size was fairly restricted due to the limited number of practicing robotic surgeons available to participate. Data from other institutions would be required to reinforce our results as all experts at the authors' institutions were utilized. Furthermore, the loss of seven experts from the Robotic Suturing tasks, due to participant time constraints, meant that the sample size was further reduced for this module. This limits the strength of our conclusion regarding the construct validity of the simulator, particularly for the Robotic Suturing module. The majority of significant results were between expert and novice or intermediate and novice groups, with statistically indistinguishable performance shown between experts and intermediates in 72% of metrics. This may be explained by the heterogeneity of the intermediate group that comprised surgeons who varied widely in their year of training but tended to be in the latter stages. Another possible explanation is that senior intermediates tended to have simulation experience and had assisted in numerous robotic procedures (maximum: 680), showing high levels of baseline experience, which may be similar to that of experts. Simulation training with the RobotiX Mentor may therefore be most beneficial to novices who are initializing the learning curve. Moreover, the majority of participants (39/46) completed the tasks on a simulator prototype, which may have negatively affected results relating to the face validity of the console.

Conclusions

In summary, some evidence of construct validity was demonstrated within the majority of performance evaluation metrics across the FRS and Robotic Suturing modules. However, several generic and economy of motion parameters such as number of movements and clutch usage seemed unhelpful. High levels of face and content validity were also established for the console interface and virtual reality tasks. Furthermore, the simulator was shown as highly feasible and acceptable to integrate into surgical training. The RobotiX Mentor is therefore a potentially useful supplementary training tool that can help novices acquire robotic skills in a safe and controlled environment. Investigation of concurrent and predictive validity is essential for further validation, and construct validity of complete simulated procedures is required. Furthermore, establishment of the learning curve to reach proficiency with the RobotiX Mentor would provide insight into its value for training.

Footnotes

Acknowledgments

The authors thank 3D Systems Simbionix Products for loan of the RobotiX Mentor simulator. However, the authors received no financial payments of any kind and declare no conflicts of interest. The authors acknowledge support from the NIHR Biomedical Research Centre, MRC Centre for Transplantation, King's Health Partners, Guy's and St. Thomas' Charity, School of Surgery, London Deanery, Royal College of Surgeons of England, Intuitive Surgical, The Urology Foundation, Olympus, EU-FP7, Prostate Cancer UK, Technology Strategy Board, and The Vattikuti Foundation.

Author Disclosure Statement

All authors report that no competing financial interests exist.

Abbreviations Used

References

, Tsakonas

, Tran

, et al. Robot-Assisted Surgery Compared with Open Surgery and Laparoscopic Surgery: Clinical Effectiveness and Economic Analyses. Ottawa, Canada: CADTH, 2011.

Tewari

, Sooriakumaran

, Bloch

, Seshadri-Kreaden

, Hebert

, Wiklund

. Positive surgical margin and perioperative complication rates of primary surgical treatments for prostate cancer: A systematic review and meta-analysis comparing retropubic, laparoscopic, and robotic prostatectomy. Eur Urol, 2012; 62:1–15.

Murphy

, Bjartell

, Ficarra

, et al. Downsides of robot-assisted laparoscopic radical prostatectomy: Limitations and complications. Eur Urol, 2010; 57:735–746.

Balasundaram

, Aggarwal

, Darzi

. Short-phase training on a virtual reality simulator improves technical performance in tele-robotic surgery. Int J Med Robot, 2008; 4:139–145.

Abboudi

, Khan

, Aboumarzouk

, et al. Current status of validation for robotic surgery simulators—A systematic review. BJU Int, 2013; 111:194–205.

Hung

, Zehnder

, Patil

, et al. Face, content and construct validity of a novel robotic surgery simulator. J Urol, 2011; 186:1019–1024.

Gavazzi

, Bahsoun

, Van Haute

, et al. Face, content and construct validity of a virtual reality simulator for robotic surgery (SEP Robot). Ann R Coll Surg Engl, 2011; 93:152–156.

Lerner

, Ayalew

, Peine

, Sundaram

. Does training on a virtual reality robotic simulator improve performance on the da Vinci surgical system?. J Endourol, 2010; 24:467–472.

Sethi

, Peine

, Mohammadi

, Sundaram

. Validation of a novel virtual reality robotic simulator. J Endourol, 2009; 23:503–508.

10.

Kumar

, Smith

, Patel

. Current status of robotic simulators in acquisition of robotic surgical skills. Curr Opin Urol, 2015; 25:168–174.

11.

Seixas-Mikelus

, Kesavadas

, Srimathveeravalli

, Chandrasekhar

, Wilding

, Guru

. Face validation of a novel robotic surgical simulator. Urology, 2010; 76:357–360.

12.

Seixas-Mikelus

, Stegemann

, Kesavadas

, et al. Content validation of a novel robotic surgical simulator. BJU Int, 2011; 107:1130–1135.

13.

Amirian

, Lindner

, Trabulsi

, Lallas

. Surgical suturing training with virtual reality simulation versus dry lab practice: An evaluation of performance improvement, content, and face validity. J Robot Surg, 2014; 8:329–335.

14.

Sammon

, Perry

, Beaule

, Kinkead

, Clark

, Hansen

. Robot-assisted radical prostatectomy: Learning rate analysis as an objective measure of the acquisition of surgical skill. BJU Int, 2010; 106:855–860.

15.

Smith

, Patel

, Satava

. Fundamentals of robotic surgery: A course of basic robotic surgery skills based upon a 14-society consensus template of outcomes measures and curriculum development. Int J Med Robot, 2014; 10:379–384.

16.

Aydin

, Muir

, Graziano

, Khan

, Dasgupta

, Ahmed

. Validation of the GreenLight Simulator and development of a training curriculum for photoselective vaporisation of the prostate. BJU Int, 2015; 115:994–1003.

17.

Kallstrom

, Hjertberg

, Svanvik

. Impact of virtual reality-simulated training on urology residents' performance of transurethral resection of the prostate. J Endourol, 2010; 24:1521–1528.

18.

Burden

, Oestergaard

, Larsen

. Integration of laparoscopic virtual-reality simulation into gynaecology training. BJOG, 2011; 118 Suppl 3:5–10.

19.

Larsen

, Soerensen

, Grantcharov

, et al. Effect of virtual reality training on laparoscopic surgery: Randomised controlled trial. BMJ, 2009; 338:b1802.

20.

Aydin

, Muir

, Graziano

, Khan

, Dasgupta

, Ahmed

. Validation of the GreenLight Simulator and development of a training curriculum for photoselective vaporisation of the prostate. BJU Int, 2015; 115:994–1003.

21.

Matsumoto

. Development and validation of a virtual reality transrectal ultrasound guided prostatic biopsy simulator. Can Urol Assoc J, 2011; 5:27.

22.

Brewin

, Nedas

, Challacombe

, Elhage

, Keisu

, Dasgupta

. Face, content and construct validation of the first virtual reality laparoscopic nephrectomy simulator. BJU Int, 2010; 106:850–854.

23.

Gomez

, Willis

, Van Sickle

. Development of a virtual reality robotic surgical curriculum using the da Vinci Si surgical system. Surg Endosc, 2015; 29:2171–2179.

24.

Foell

, Finelli

, Yasufuku

, et al. Robotic surgery basic skills training: Evaluation of a pilot multidisciplinary simulation-based curriculum. Can Urol Assoc J, 2013; 7:430–434.

25.

Bric

, Connolly

, Kastenmeier

, Goldblatt

, Gould

. Proficiency training on a virtual reality robotic surgical skills curriculum. Surg Endosc, 2014; 28:3343–3348.

26.

Palter

, Grantcharov

. Individualized deliberate practice on a virtual reality simulator improves technical performance of surgical novices in the operating room: A randomized controlled trial. Ann Surg, 2014; 259:443–448.

27.

Lendvay

, Brand

, White

, et al. Virtual reality robotic surgery warm-up improves task performance in a dry laboratory environment: A prospective randomized controlled study. J Am Coll Surg, 2013; 216:1181–1192.