Robotic Surgery Training with Commercially Available Simulation Systems in 2011: A Current Review and Practice Pattern Survey from the Society of Urologic Robotic Surgeons

Abstract

Objectives:

Virtual reality (VR) simulation has the potential to standardize surgical training for robotic surgery. We sought to evaluate all commercially available VR robotic simulators.

Materials and Methods:

A MEDLINE^® literature search was performed of all applicable keywords. Available VR simulators were evaluated with regard to face, content, and construct validation. Additionally, a survey was e-mailed to all members of the Endourological Society, querying the pervasiveness of VR simulators in robotic surgical training. Finally, each company was e-mailed to ask for a price quote for their respective system.

Results:

There are four VR robotic surgical simulators currently available: RoSS™, dV-Trainer™, SEP Robot™, and da Vinci ^® Skills Simulator™. Each system is represented in the literature and all possess varying degrees of face, content, and construct validity. Although all systems have basic skill sets with performance analysis and metrics software, most do not contain procedural components. When evaluating the results of our survey, most respondents did not possess a VR simulator although almost all believed there to be great potential for these devices in robotic surgical training. With the exception of the SEP Robot, all VR simulators are similar in price.

Conclusions:

VR simulators have a definite role in the future of robotic surgical training. Although the simulators target technical components of training, their largest impact will be appreciated when incorporated into a comprehensive educational curriculum.

Introduction

E valuation of skills in surgical training has been re-energized in the modern era of medicine, where quality improvement and patient safety have become driving forces behind change. Indeed, advancement of technology has opened the doors to new possibilities for trainee assessment. The ultimate goal is a more objective means by which to evaluate a trainee's performance in anticipation of their ultimate graduation.

It has been recognized in other professional arenas such as aviation and the military that simulation-based training can provide a more consistent mode of skill acquisition and mastery. Accordingly, in response to the introduction of rapidly evolving minimally invasive technologies and external pressures from the public and lawmakers alike to ensure patient safety, the surgical Residency Review Committee of the Accreditation Council on Graduate Medical Education issued a mandate in 2008 that all surgical residency programs must have access to simulation and skills training tools.¹ One year later, the American Board of Surgery began to require that all general surgery graduates provide documentation of successful completion of the Fundamentals of Laparoscopic Surgery (FLS) course before sitting for their certification boards.² FLS is a validated, standardized education module designed to teach the physiology, fundamental knowledge, and technical skills required in basic laparoscopic surgery. It includes a simulation-based skills laboratory with uniform metrics and assessment criteria.³ Although such training has not become consistent across all genres of surgical residencies, it stands to reason that surgical subspecialties will follow this lead.

“Simulation” is a broad-based term that incorporates standardized teaching and assessment tools designed to replicate a realistic environment where trainees are forced to act upon a variety of scenarios so that their performance during the actual event is more constant, controlled, and automatic. A variety of simulators exist, all designed to hone performance through promoting the interaction of cognitive, communicative, and technical components (i.e., think about the task, communicate findings, and perform the task), as described by Sweet.⁴ We focused our evaluation on the technical component of performance and the impact of virtual reality (VR) surgical simulators.

VR-based simulation provides a useful training tool, particularly for minimally invasive surgical techniques such as robotic surgery. Indeed, it has been proven with level-1 evidence that VR simulation-based training can improve operating-room performance in surgical residents preparing for laparoscopic cholecystectomy.⁵ Some of the advantages of utilizing these trainers are (1) realistic surgical platforms and basic skill sets (face and content validation); (2) standardized performance analysis and metrics that potentiate improvement (construct validation); and (3) the potential of procedural modules that allow the trainee to practice a procedure before entering the operating room. We evaluated the VR robotic surgical simulators that are currently available on the market and gave an objective analysis of each with regard to the above-mentioned criteria. Additionally, we sent a survey to all active members of the Endourological Society, querying the pervasiveness of simulation in surgical training and utilization of VR simulators.

Materials and Methods

We focused our evaluation on four separate VR-based robotic surgical simulators: (1) Robotic Surgical Simulator (RoSS™; Simulated Surgical Systems, Buffalo, NY; www.simulatedsurgicals.com); (2) dV-Trainer™ (Mimic Technologies, Inc., Seattle, WA; www.mimictech.net); (3) SEP Robot™ (SimSurgery^®, Norway; www.simsurgery.com/web/home); and (4) the da Vinci Skills Simulator (Intuitive Surgical, Sunnyvale, CA; www.intuitivesurgical.com/products/skills_simulator). For each system, a thorough MEDLINE^® literature search was performed. Search terms utilized were the names of the individual systems, “robotic surgery simulation,” “VR robotic surgery,” and “validation studies.” For each surgical simulation system, published validation studies were evaluated with regard to face and content validation, and construct validation. Additionally, each company's specific Web site was reviewed for supplementary information, including future releases, most particularly procedural components to their simulators. Finally, each company was contacted via electronic mail and asked for a price quote for purchasing their system (results reflect September 2010 pricing).

Face validation is defined as the degree of approximation between a given system and the actual activity being emulated. With respect to robotic VR simulators, some of the criteria that are applicable to face validation include clutch and camera movement, feel of the pinch device, and visualization. Face validation is often evaluated in the same setting as content validation, or the level to which a given system copies the subject matter (i.e., operative conditions) of a given activity. Construct validation evaluates a system's assessment tools, or software, and whether they can discriminate different abilities or experience levels.⁶

As an adjunct to our literature search, a survey was sent out to all members of the Endourological Society, querying their training practices for robotic surgery, exposure to robotic VR simulation, and whether VR simulation was part of their curriculum (see Appendix I). The contents of this survey were constructed by the authors (C.L. and J.D.) based upon discussions held at the 2010 Society of Urologic Robotic Surgeons meeting after this material was presented. The survey was reviewed and approved by the editors of the Survey Section of the Journal of Endourology. The survey was sent out in English as an e-mail blast to all members of the Endourological Society, and responses were blinded.

Results

RoSS

The RoSS system has currently been validated with respect to face and content.^7,8 All published validation studies have come from a single institution, Roswell Park Cancer Institute (Buffalo, NY). It has both validated basic orientation modules (camera and arm movement, clutch, and fourth arm manipulation) as well as basic skill modules (suturing and tying, vessel clipping, and suture cutting). Although the Simulated Surgical Systems Web site reports that their data management system software contains performance analysis and measurement tools that objectively measure and record performance, as of the writing of this article there have been no published construct validation studies. The RoSS system currently has a robotic prostatectomy procedural component that is patent pending. This component, called Hands-On Surgical Training or HOST™, reverses the master–slave relationship of the robot so that the trainee can sit at the console with his or her fingers relaxed in the pinchers, and watch a robotic prostatectomy being performed in the console while the console arms move in concert with the surgeon who is operating, so that the trainee can perform the procedure in real time along with the surgeon, mimicking the surgeon's every movement and having the performance evaluated and measured by the RoSS software. In addition to robotic prostatectomy, Simulated Surgical Systems also plans to add other HOST procedural components in the future, including robotic cystectomy, partial nephrectomy, and hysterectomy. The cost of the RoSS ranges from $95 to $125,000 with a negotiable annual service plan. The size of the system is comparable to an actual console, and therefore consideration must be given to usage of space and storage.

dV-Trainer

Four validation studies have been published from a variety of institutions for this system.^9

–12 Combined, these studies provide proof of face, content, and construct validation. The system has validated orientation modules and skill modules with simulated training scenarios (MSim™). Additionally, its performance analysis software and metrics (MScore™) have also been independently validated. Currently, there are no commercially available procedural components to the Mimic Technology software. The cost of the system is $85 to $100,000 with an annual service plan. The system is fairly compact in that it can fit on a desktop with simulated pedals placed on the floor, and requires a separate desktop computer nearby to run the applications.

SEP Robot

There is one published study for this system that tested face and construct validity.¹³ This system has a full complement of basic skill sets that are based on its validated SimSurgery Education Platform, although no content validation studies have been published for SEP Robot alone. From a procedural component standpoint, although SimSurgery does not offer any VR urologic surgery modules, they do have significant experience with VR for minimally invasive surgeries, producing both a laparoscopic cholecystectomy and laparoscopic ectopic pregnancy learning module that are available with their laparoscopic VR simulator. SEP Robot is the least expensive of all available VR robotic simulators, at approximately $40 to 45,000 with an annual service plan. There is no actual console for this simulator: only a monitor, armboard, and central processing unit that is akin to a desktop computer.

da Vinci Skills Simulator

Intuitive Surgical, Inc., manufacturer of the da Vinci™ robotic surgical system, has partnered with Mimic Technologies to offer its own VR simulator. Now available in early 2011, the system integrates the software of the dV-Trainer into the da Vinci Si console through a backpack (roughly the size of a desktop computer) that sits on the back panels of an actual Si robotic console. Thus, the actual hardware of this VR simulator is not a freestanding component as with the other simulators mentioned, but the actual Si console. The software of the simulator tallies and records the metrics for each procedure and user, and these data can be retrieved and reviewed. Although there are no validation studies currently published for this simulator, those available for the dV-Trainer can be extrapolated to this system, especially studies evaluating content and construct validation of the Mimic Technologies software. Additionally, as with dV-trainer, there are no procedural components. The cost of this simulator is similar to that of the dV-Trainer, although this does not include the Si console or upgrade of the da Vinci surgical system to the Si model, which would be significantly more costly. This simulator also has an annual service fee.

Comparison of the VR simulators is included in Table 1.

Table 1.

Comparison of Virtual Reality Simulators

	Face validation	Skill exercises with content validation	Performance metrics with construct validation	Procedural modules	Cost (approximate)
RoSS	√	√		√	$100–125,000
dV-Trainer	√	√	√		$85–100,000
SEP Robot			√	+/−	$40–45,000
da Vinci Skills Simulator		√	√		$85–90,000

+/−=capability present, but no urologic procedural components available.

Results of survey

In total, approximately 1000 to 1250 total surveys were e-mailed with 65 responses. Although the vast majority of the respondents were male, 20% were aged ≥55 years and nearly 40% have been in practice for over 15 years. Just over half had completed a fellowship in endourology, laparoscopic/robotic surgery, or urologic oncology, and about 75% of respondents currently were involved with resident or fellow training, either in an academic setting or in private practice. Most of these respondents indicated that trainees at their institution could independently perform some of a robotic prostatectomy procedure, with just a minority able to perform over 75%.

With regard to VR simulation, although approximately 25% of those who responded have access to a simulator at their institution, 40% have neither seen nor heard of VR simulators. Interestingly, although most respondents believe that there is an expanding role for VR simulators in surgical training, over 70% have not purchased a simulator and a significant majority does not believe that the current cost for most simulators is reasonable. For those who have purchased a simulator, the source of funding is primarily departmental, although there are some who have been able to take advantage of grant-awarded and philanthropic subsidies. Of note, several respondents wrote in that a simulator should be included in the price of purchase of a new da Vinci surgical system (question #15).

For results of the survey, please see Appendix II.

Discussion

The evolving field of minimally invasive surgery and external pressures for patient safety has shifted the paradigm for surgical training, with the general surgeons taking the lead in promoting standardized curricula with simulation-based components.^14
–16 Additionally, the field of urology has recognized the importance of simulation in training for other surgical techniques.^17

–21 With regard to technical surgical training for laparoscopic and robotic surgery, most staff at academic centers would agree that they cringe at the thought of a novice trainee first undertaking particular surgical skills on an anesthetized patient with whom only the staff surgeon has made a connection, not to mention the relationship with the family and loved ones. For minimally invasive surgery, there are inanimate modes (box trainers) that can be helpful to trainees. An inherent problem with training on these models, however, is that it is heavily dependent on the trainee. Unless a mentor is always available to monitor a given trainee's progress on an inanimate trainer, which is usually not the case, the trainees are left to their own devices. It has been demonstrated that surgical trainees lack the ability to assess their own surgical skills, with overestimation being the trend.²² Additionally, training on these models can be stale and self-limited.

One simulation training tool that is considered high-fidelity, that is, it more closely replicates intraoperative conditions, is the animal model. This is in contrast to the box trainer, which is considered a low-fidelity simulator. Unfortunately, wet laboratories are limited and in reality many surgical trainees will complete a residency without operating on an animal model more than a handful of times. VR simulation lies somewhere between the animal model and the box trainer; although it does not provide live tissue for dissection, it can provide anatomy and anatomic variants specific to certain operations. In addition, unlike either the box trainer or animal laboratory, VR can provide a virtual instructor with standardized metrics that can assess performance and identify errors and areas of improvement to foster proficiency. In fact, the one property that prevents VR simulators from essentially replacing wet laboratories is the ability to replicate the viscoelastic properties of tissue and its response to manipulation, dissection, ligation, and other operative maneuvers. This explains the sometimes cartoonish appearance of current VR simulation software and thus the low face and content validation scores. VR soft tissue behavior is one of the focuses in the development of a novel VR transperitoneal laparoscopic nephrectomy simulator and is a barrier for all VR surgical simulators that will need to be overcome if these training modules are to gain widespread acceptance and utilization.²³

When comparing the commercially available VR robotic simulators, all contain certain attributes: (1) a stand-alone surgical simulator that is independent of a surgical robot or vision cart; (2) a vision monitor that allows a user or tutor to observe and critique the procedure; (3) minimal cost of operation as no consumables or disposables are required; and (4) relative freedom from an operating room environment for training purposes. In fact, with the exception of the HOST technology of RoSS and the utilization of the actual da Vinci Si console in the da Vinci Skills Simulator, all of these simulators essentially offer the same features with minor differences. The only objective data that are available to distinguish one simulator from another are the few validation studies that were catalogued in the Results section. These validation studies themselves must be looked at critically. In the face validation study for RoSS, over half of the test subjects responded that, among the five categories evaluated, RoSS was either “somewhat close” or “not close” to the actual da Vinci surgical system.⁷ In addition, in the Conclusions section of the one validation study for SEP Robot, the authors state that improvements must be made in that system before implementation in a training curriculum.¹³ According to Sweet et al,²⁴ the term validity itself has become a catch-all term that has led to justification for utilization in medical education, as long as a device proves a facade of face, content, and construct validity. It is akin to evaluating the results of a clinical study; although the statistics may show significance, the results may be, nonetheless, irrelevant. In addition, the one validation strategy believed to provide a clinically meaningful assessment, predictive validity, has yet to be addressed in the evaluations of VR robotic simulators.²⁵ Finally, it should be paramount that VR simulation not be appreciated as a set of valid technical skills alone, but as part of a comprehensive curriculum aimed at specific educational goals and objectives. Validation of such a curriculum would carry more meaningful results.

An additional potential of VR simulation is as an assessment tool. One of the inherent problems with credentialing for robotic surgery is that, as opposed to laparoscopic surgery and FLS, there is no standardized tool to determine proficiency, and privileging is instead institution dependent, allowing for significant variability. This problem is currently being addressed by multiple governing bodies, including Society of Urologic Surgeons (SURS), who are in the process of developing a centralized robotic surgery curriculum. The validated skill sets and performance metrics inherent in the available VR simulators are attractive as a possible adjunct to these efforts.

All available VR robotic simulators have a cache of skill sets. These exercises certainly have merit with regard to technical training, and have also been proven to augment performance during an actual operation when used by a surgeon as a preoperative warm-up.²⁶ Still, one goal of VR robotic simulation is procedural, where a trainee can perform an operation to proficiency before touching an actual patient. The immense amount of research and development time and money that is involved in producing this technology is the barrier that has prevented its widespread availability. Currently, the only such application that is offered in VR robotic simulation is HOST, which is an application for RoSS that takes the trainee through a virtual procedure while they sit engaged at the console. Although still in prototype phase, the future applications of this and other related programs is enticing, with the virtual mentoring changing to approximate the skill level of the trainee. A novice can sit through a virtual operation that he or she is seeing for the first time; a skilled trainee (chief resident, fellow, or young attending surgeon) can draw upon a library of surgical maneuvers that are performed by experts in the field; and a seasoned surgeon can import anatomic imaging and other characteristics of a particular patient to create a virtual roadmap for a specific operation.

In regard to our survey, it was sent out as an e-mail blast to all members of the Endourological Society. The purpose of the blast e-mail was to accumulate as many responses as possible (65 total), with the downside being a low response rate (<10%). Still, in evaluating the demographic details of the respondents, a fairly representative cross section of practicing urologists was obtained. Taking into account that the Endourological Society represents a body that readily accepts and adapts to new technology, it comes as no surprise in our survey that over 90% of respondents believed that VR simulators have an expanding or advanced role in surgical training, although only 30% had access to one. Interestingly, the majority of respondents who claim to own a VR simulator report that it is the da Vinci Skills Simulator, although it has just become available on the market at the time of writing this article. One advantage of this simulator is that it utilizes the actual Si console as its platform, circumventing any hardware issues that may be present with other VR simulators. Still, the da Vinci Skills Simulator has no direct validation studies in the literature, as with RoSS and dV-Trainer, nor does it have any procedural components at this time. More recently, there has been independent face, content, and construct validation of the da Vinci Skills Simulator, with a publication pending.²⁷

Another significant matter that was raised from our survey is cost. Although the concept of VR simulation and surgical training is attractive to a representative cohort of the Endourological Society, 80% do not consider the current pricing for the available VR simulators to be reasonable. The only exception is the SEP Robot, which is about half the price of the others, but which also is the only to have failed a published validation study. Cost concerns are magnified when considering that all of the simulators have an annual service maintenance fee and that the primary source of funding for most survey respondents is departmental, as opposed to grant-awarded or philanthropic. At the recent SURS meeting at the 2010 World Congress of Endourology in Chicago, most of the resistance to acceptance of VR simulation was cost-related, with several members stating that current training curricula were much less expensive and just as effective. This is another major hurdle to widespread utilization of VR simulators.

The purpose of our study is not to favor one simulator over another, but to give an objective overview of the VR simulators that are currently available. We admit several limitations to the study. First, there is a relative paucity of published reports on robotic VR simulation and simulators, and although we sought to report on all, the evidence is, nonetheless, lacking. Additionally, there have been no head-to-head comparisons of one simulator to another; such an evaluation would undoubtedly provide a more objective assessment. Finally, VR simulation is yet an unproven adjunct to surgical simulation, and although there seems to be much enthusiasm surrounding its potential, its place in surgical training has yet to be determined.

Conclusions

The evolving field of minimally invasive surgery and external pressures for patient safety coupled with technological advances has opened the door for simulation-based training, namely, VR simulation, for robotic surgery. The commercially available VR simulators have potential, but several questions need to be addressed before widespread acceptance: (1) although all simulators contain basic skill sets with self-assessment software, procedure-based modules must be developed with realistic tissue-behavioral characteristics; (2) comprehensive and multi-institutional validation studies need to be performed and published, including head-to-head comparisons of different VR simulators; (3) the cost of the simulators needs to be set at a level that it is thought to be fiscally commensurate to resident, fellow, and staff training so that it is not considered a luxury but a viable learning and assessment tool.

Footnotes

Disclosure Statement

J.W. Davis is Principal Investigator for studies funded by Genprobe and Janssen Pharmaceuticals. C.D. Lallas is a consultant for Ethicon Endosurgical, and an investor/consultant for Intuitive Surgical.

Abbreviations Used

Appendix I. Robotic Surgery Simulation Survey

Appendix II. Results of Robotic Surgery Simulation Survey

References

American Council on Graduate Medical Education (ACGME). Program Requirements for Graduate Medical Education in Surgery: Common Program Requirement, Effective: January 1, 2008, Section II D (2) 10. 2009. www.acgme.org/acWebsite/downloads/RRC_progReq/440_general_surgery_01012008_u08102008.pdf. 2010 November 1.

American Board of Surgery (ABS). Booklet of information for certifying exam. 12. 2009. http://home.absurgery.org/xfer/BookletofInfo-Surgery.pdf. 2010 November 1.

Derossis

, Fried

, Abrahamowicz

et al. Development of a model for training and evaluation of laparoscopic skills. Am J Surg, 1998; 175:482–487.

Sweet

, McDougall

. Simulation and computer-animated devices: The new minimally invasive skills training paradigm. Urol Clin North Am, 2008; 35:519–531.

Seymour

, Gallagher

, Roman

et al. Virtual reality training improves operating room performance: Results of a randomized, double-blinded study. Ann Surg, 2002; 236:458–463.

Carter

, Schijven

, Aggarwal

et al. Consensus guidelines for validation of virtual reality surgical simulators. Surg Endosc, 2005; 19:1523–1532.

Seixas-Mikelus

, Kesavadas

, Srimathveeravalli

et al. Face validation of a novel robotic surgical simulator. Urology, 2010; 76:357–360.

Seixas-Mikelus

, Stegemann

, Kesavadas

et al. Content validation of a novel robotic surgical simulator. BJU Int, 2011; 107:1130–1135.

Kenney

, Wszolek

, Gould

et al. Face, content, and construct validity of dV-trainer, a novel virtual reality simulator for robotic surgery. Urology, 2009; 73:1288–1292.

10.

Lendvay

, Casale

, Sweet

et al. Initial validation of a virtual-reality robotic simulator. J Robotic Surg, 2008; 2:145–149.

11.

Lerner

, Ayalew

, Peine

et al.

Does training on a virtual reality robotic simulator improve performance on the da Vinci surgical system?

J Endourol, 2010; 24:467–472.

12.

Sethi

, Peine

, Mohammadi

et al. Validation of a novel virtual reality robotic simulator. J Endourol, 2009; 23:503–508.

13.

van der Meijden

, Broeders

, Schijven

. The SEP “Robot”: A valid virtual reality robotic simulator for the Da Vinci surgical system? Surg Technol Int, 2010; 19:51–58.

14.

Jones

. Video trainers, simulation and virtual reality: A new paradigm for surgical training. Asian J Surg, 2007; 30:6–12.

15.

Sachdeva

, Pellegrini

, Johnson

. Support for simulation-based surgical education through American College of Surgeons—accredited education institutes. World J Surg, 2008; 32:196–207.

16.

Scott

, Dunnington

. The new ACS/APDS Skills Curriculum: Moving the learning curve out of the operating room. J Gastrointest Surg, 2008; 12:213–221.

17.

Knudsen

, Matsumoto

, Chew

et al. A randomized, controlled, prospective study validating the acquisition of percutaneous renal collecting system access skills using a computer based hybrid virtual reality surgical simulator: Phase I. J Urol, 2006; 176:2173–2178.

18.

Manyak

, Santangelo

, Hahn

et al. Virtual reality surgical simulation for lower urinary tract endourologic surgery. Adv Exp Med Biol, 2003; 539:841–852.

19.

Matsumoto

, Pace

, D'A Honey

. Virtual reality ureteroscopy simulator as a valid tool for assessing endourological skills. Int J Urol, 2006; 13:896–901.

20.

Sweet

, Kowalewski

, Oppenheimer

et al. Face, content and construct validity of the University of Washington virtual reality transurethral prostate resection trainer. J Urol, 2004; 172:1953–1957.

21.

Watterson

, Denstedt

. Ureteroscopy and cystoscopy simulation in urology. J Endourol, 2007; 21:263–269.

22.

Pandey

, Wolfe

, Black

et al. Self-assessment of technical skill in surgery: The need for expert feedback. Ann R Coll Surg Engl, 2008; 90:286–290.

23.

Zhou

, Zhang

, Sha

et al. A discrete mechanics framework for real time virtual surgical simulations with application to virtual laparoscopic nephrectomy. Stud Health Technol Inform, 2009; 142:459–464.

24.

Sweet

, Hananel

, Lawrenz

. A unified approach to validation, reliability, and education study design for surgical technical skills training. Arch Surg, 2010; 145:197–201.

25.

Gallagher

, Ritter

, Satava

. Fundamental principles of validation, and reliability: Rigorous science for the assessment of surgical education and training. Surg Endosc, 2003; 17:1525–1529.

26.

Kahol

, Satava

, Ferrara

et al. Effect of short-term pretrial practice on surgical proficiency in simulated environments: A randomized trial of the “preoperative warm-up” effect. J Am Coll Surg, 2009; 208:255–268.

27.

Hung

, Zehnder

, Patil

et al. Face, content, and construct validity of a novel da Vinci Surgical Simulator. Poster #1484. American Uroloigical Association Annual Meeting, Washington, D.C., 2011.