The CREST Simulation Development Process: Training the Next Generation

Background

Inherent challenges for training in endourology include overcoming the loss of direct palpation, managing hemostasis, the fulcrum effect, multiple degrees of freedom of instrumentation, indirect three-dimensional (3D) to two-dimensional (2D) translation of images, tissue extraction, disorientation of adding a “camera,” flexible instrumentation, and learning principles of “parallax” in the case of percutaneous access to the kidney.

The early development of simulation training systems was more art than science and was typically guided by an inspired clinician educator paired with an engineer. Anatomic, physiologic, and tissue likeness were sometimes replicated based on that one clinician's input and not necessarily guided by a true understanding of the learning objectives.

With a mission of “advancing performance in healthcare through catalyzing interdisciplinary research, development, utilization, and commercialization of simulation solutions,” the Center for Research in Education and Simulation Technologies was created and has had a strong focus on the development and validation of endourologic applications.

Over the past decade, CREST has developed a process that has streamlined the development of curricula and models to best meet the needs of the endourologic community. We describe that process in detail for nine endourologic training systems.

Methods

Wiggins's and McTighe's “backwards design principle” as applied to healthcare applications was the foundation. First, we defined a desired learning outcome around a specific training problem(s) and learning group(s). ^1,2 Next, key learning objectives were elicited. In the worst-case scenario, these were determined by a series of expert educator endourologists. In the best-case scenario, it was performed by a society-appointed panel.

Performance of technical and nontechnical skills is directly related to Cognitive (decision-making), Psychomotor, Communication, and Affective domains. ³ Task deconstruction was performed for each learning objective to determine what learning domain(s) predominate.

After defining the acceptable ways of completing a task, cognitive task analysis was performed. Critical anatomic structures, important decisions, anatomic “cues,” innate tissue behaviors, as well as human–human, tissue–tool, and tissue–tissue interactions were defined. Important physiologic considerations, medical devices, and “tips and tricks” were also documented. Discrete and measurable elements (metrics) and critical errors were then identified. A weight for each step/metric and error was determined relative to the outcome. In three of the nine systems described below, [GreenLight Sim (GL Sim), beating artery models for basic laparoscopic urologic surgery (BLUS), and Pyeloplasty], a panel of expert endourologists was used to accomplish this step. ^{4 –6} In these cases, a modified Delphi method of weighing importance/risk was performed. Each panelist privately rated relative importance/risk. The results were collated, outliers were openly discussed, and a second final private rating was performed and documented. The relative importance of anatomic, physiologic, and tissue fidelity was assessed during this step.

In three cases [Vesicourethral anastomosis (VU trainer), the C-arm percutaneous access trainer (CAT), and vascular control models], this process was done through private interviews in series with several individual expert endourologists without group discussion/consensus. ⁷

Reference anatomy was obtained from Dicom images that were translated into a deformable MESH in Maya (Autodesk, Inc., San Rafael, CA) by a digital artist (Dan Burke). For physical models [kidney ureter (KU), kidney ureter bladder (KUB), BLUS arteries, NLM, Pyeloplasty, VU model, vascular control models, and the CAT], the Maya model was translated into Solid Works (Dassault Systemes, Concord, MA) as a computer-assisted model (CAM) for 3D printing/sculpting as either a negative space to mold materials around or a “casted” anatomic structure. ^{5 –9} In the case of a dynamic virtual model, such as the GL Sim, the deformable meshes of anatomy were textured to match important visual cues and directly imported into the virtual scene. ³

For pyeloplasty, CAT, KU/KUB, and vascular control models, where tissue–tool interactions were important, studies were performed on, or referenced from, fresh cadaveric tissue.

A unique process for simulating vaporization of prostate tissue was applied in the case of GL Sim. In this case, a finite element analysis was performed with bull prostate data provided by American Medical Systems and novel visualization methods and mass spring deformation algorithms were then created to match the tissue behavior of the FEM. ^{10 –12}

Wherever possible, the CREST process embedded objective means of assessing the critical metrics and errors defined during the CTA. In virtual systems, these data are automatically generated as any element of the simulation can be defined, tracked, and logged as digital “events.”

A learning management system (LMS) was created to impact learning, provide assessment, and organize and present performance data in real time (formative) and at the end of an exercise (summative). CREST has developed a procedure-specific LMS that was applied to GL Sim. ⁴ Currently, the endourologic physical models are not connected to an LMS.

Verification, acceptability, usability, and early validation studies were performed with early prototypes and not with “final” versions of the models. These studies were easy to perform and guided the refinement of the model. External or multi-institutional studies of more advanced prototypes were completed in the case for all of the models presented, with exception of the VU and NLM models. ^{5 –9,13 –18} Table 1 provides a summary of models with published validity evidence.

Results

Physical models

CREST created a beating artery model for assessing laparoscopic clip applying for the Basic Laparoscopic Urologic Surgery curriculum (Fig. 1). Using the CREST process, the American Urological Association (AUA) BLUS working group created objective metrics for clip applying accuracy and effectiveness. An “animated” physical sealed organosilicate tube was filled with blood at an average mean arterial pressure. Clipping on the black line targets, cross clips, and leakage were easily observable by a proctor. Good validity evidence for use as an assessment tool of BLUS was collected through external and multi-institutional trials. The BLUS arteries are officially a part of the AUA BLUS psychomotor skills program. ^5,14,15

OPEN IN VIEWER

FIG. 1.

CREST beating artery for AUA BLUS project. AUA = American Urological Association; BLUS = basic laparoscopic skills; CREST = Center for Research in Education and Simulation Technologies.

Due to usability issues, the AUA hands-on course directors for laparoscopic pyeloplasty wanted to replace the animal kidney model. Leakage, patency, and degree of twisting were the important objective measurable metrics. An organosilicate model was created with an open funnel in the renal pelvis measuring relative leakage and patency (Fig. 2A). Black Light Assessment of Surgical Technique measured degree of twisting (Fig. 2B). These lines were invisible to the naked eye and only seen under black light by the proctors. These were used effectively for residency training courses, AUA, and Society of Laparoendoscopic Surgeons hands-on pyeloplasty courses to train practicing surgeons. Over 2 years, good validity evidence was established for training and assessing practicing surgeons externally at the AUA hands-on courses. ⁶

OPEN IN VIEWER

FIG. 2.

(a) CREST pyeloplasty model and (b) pyeloplasty model seen under black light.

VU Model: Guided by the AUA LRNST committee's recommendations, a realistic-behaving vesicourethral anastomosis model was conceived and created (Fig. 3a). Objective ability to measure “leak” was obtained by using an open funnel bladder that fluid could be poured into. The relative amount of “leak” could therefore be calculated when poured at a defined pressure. BLAST could be used potentially for assessment as well (Fig. 3b). Individual expert providers were used to early verification and thought it to be extremely realistic, but no validity trials have been performed with this model to date. The prototype is not in public use at this time.

OPEN IN VIEWER

FIG. 3.

(a) and (b) CREST vesicourethral model.

KU and KUB models were made out of organosilicates to train and assess Cystoscopic and Ureteroscopic access and manipulations (Fig. 4) These “nondestructive” models served as an anatomic conduit. They were one of the first models in healthcare to be generated using 3D printing (2007). Single-site validation as part of a ureteroscopic OSATS led to minor modifications and a subsequent multi-institutional study accumulated excellent validity evidence for assessing endoscopic skills. ^8,13 Hundreds of the Version 1.0 models are being used today. Based on feedback on Version 1.0, Version 2.0 prototype has been created out of polyurethane that more accurately simulates ureteral friction forces. ¹⁹

OPEN IN VIEWER

FIG. 4.

CREST KUB external (a) and internal (b) views. KUB = kidney ureter bladder.

Training parallax during fluoroscopically guided percutaneous renal access was determined to be a skill gap. Models to date allowed for direct visualization or used animal models/cadavers with actual fluoroscopy. Version 1.0 of the fluoroless C-arm Trainer (CAT) was created using the 3D printed cast from the KU models as a negative space in clear silicone. A mini-c-arm equipped with two cameras and fake skin, using a black and white filter, provided the motions and correct visual fidelity of fluoroscopy and the needle. The upper camera captured the needle and the lower camera captured the collecting system, and the images were fused using green screen technology (Fig. 5a). This underwent a preliminary validation study. ⁷ Feedback about poor registration of the two cameras of Version 1.0 at certain angles drove a change to a single camera and a shield (Version 2.0), which was simpler and provided a clearer image (Fig. 5b). Version 2.0 along with a Version 2.1 that allows for endoscopic-assisted entry (EASE) through painting the calices has been actively used in hands-on courses for the AUA to train over 150 practicing urologists over the past 2 years. A version 3.0 prototype has been developed to better simulate the needle insertion forces of the layers of the human flank by measuring them in a fresh cadaver and emulating the distinct layers of different durometer (hardness of material) polyurethanes and silicone. ²⁰

OPEN IN VIEWER

FIG. 5.

(a) Fused Image from Version 1.0 of the CAT (b) Version 2.0 with a single camera and a shield at a hands-on AUA course at the Smith Institute for Urology 2015.

A second model created based on requirements for the AUA Hands-on percutaneous course was for nephrolithotomy (aka “NaiL ‘eM”) (Fig. 6a, b). The course directors honed in on and collectively provided the desired outcomes, learning objectives and important metrics around flexible nephroscopy, basketing as well as rigid ultrasonic and pneumatic lithotripsy through an upper, inter-, and/or lower pole access. The silicone model flank was built around the inner renal pelvis/ureteral layers of the KUB model described above. To date, formal validity testing has not been performed for this model, but it, like the KUB models are “non destructive,” serves as an anatomic conduit and has been used to effectively train over 150 practicing urologists at the AUA hands-on course over the past 2 years.

OPEN IN VIEWER

FIG. 6.

(a) NaiL eM external. (b) Internal views. NaiL eM = nephrolithotomy model.

CREST built a vascular control partial task trainer (Fig. 7). This reposable trainer allowed the trainee to repair an injury to an actively bleeding vena cava or renal artery laparoscopically and/or robotically. An anatomic replica of the two organosilicate vascular structures was created roughly referencing data from the CREST cadaveric tissue database. Blood loss was measured. An IV bag of artificial blood was hung at a height to simulate central venous pressure in the case of the IVC and mean arterial pressure in the case of the renal artery. Pulsatile flow was simulated through dynamically rolling the IV clamp. Internal validity evidence was gathered, and external validation is underway at several sites. ⁹ The IVC model has been used to train hundreds of practicing urologists, gynecologists, and general surgeons at the Society for Laparoscopic Surgeons Hands-on courses over the past 3 years.

OPEN IN VIEWER

FIG. 7.

CREST bleeding IVC model.

Virtual models

In 2007, CREST was contracted by American Medical Systems (now Boston Scientific, Marlborough, MA) to design and create the GreenLight Sim (Fig. 8). This project followed all the phases of the CREST process. A panel of BPH experts generated outcomes, learning objectives, metrics, and errors. Task deconstruction was performed, leading to “exercises” focused on sweep speed, laser tissue distance, hemostasis, anatomic orientation, and power settings. Six different full anatomic cases were created based on common and difficult cases. Data from laser tissue studies fed a finite element model. ^10,11 These behaviors were replicated using faster mass-spring methods. ^4,12 A LMS managed learning metrics within and across sites. Over 150 U have been deployed worldwide, training hundreds of practicing urologists and residents. The simulator was integrated into training pathways by the company and represents one of the first full VR procedural trainers on the market. It dropped the average training cost per physician when compared with other training modalities and contributed significantly to PVP adoption and safety. It has accumulated external validation evidence for both training and assessment by several groups. ^{16 –18}

OPEN IN VIEWER

FIG. 8.

AMS GL simulator designed and created by CREST.

Discussion

Backwards design vis-à-vis the CREST process has led to effective training and assessment solutions for endourology. This process has been an evolution of over 17 years of experience in a burgeoning field.

Both physical and virtual systems have their educational strengths and weaknesses. Simulations of tissue behavior, subtleties in the difference in textures of tissue planes, and poor representation of haptics have been the downfall of interactive virtual reality models for surgical applications. However, the ability to demonstrate complex anatomic relationships and the ability to collect an enormous amount of objective data have been strengths. Physical systems historically have been better at representing tissue behavior and haptics, although silicone has not behaved like human tissue and few physical systems on the market can automatically collect objective data. Applications of BLAST for plastics/reconstructive operations in endourology represent a step in this direction.

Tissue behavior is an underappreciated and poorly understood aspect for soft tissue simulation of endourologic procedures. Whether it is representation of tissue in the virtual or physical environment, understanding and defining these behaviors with data are important if we are to improve the tissue fidelity of our simulation systems. CREST has been steadily collecting mechanic, electric, thermal, and optical properties of fresh human cadaveric tissue to build a database to serve as a foundational benchmark for such development.

There are several other future technologies that promise to improve the fidelity and usefulness of endourology training and assessment systems. Conformable sensor technologies have led to new “smart tissue,” allowing physical trainers to measure interactions directly. This has been implemented in some nonurologic applications. 3D printing of soft materials with embedded electronics/sensors is the next frontier that will facilitate rapid manufacturing and patient-specific rehearsal. ²¹

One of the biggest developments that promise to impact the future of simulator development across healthcare is the advanced modular Manikin (AMM) project. This Department of Defense Joint Program Committee-1 funded initiative contracted CREST to develop an open source platform, reference anatomy, and set of standards by which modules (including endourologic) can be made by any company and functionally connect and communicate with other modules. Such a platform will tie partial task trainers with physiology and other organ systems. It will also unify virtual and physical systems so learners can benefit from the advantages of both. We have already demonstrated the ability to integrate physiology, virtual patients, and physical anatomic forms through our platform so as to create the ability to have the models interact with other organs and tissues and affect the status of an entire patient.

Funding sources for simulation applications in endourology have been sparse, as Federal simulator development funding has tended to focus on broader applications in healthcare. These projects, however, have been conceived and executed through a combination of healthy partnerships between academia, societies, and industry with a shared common goal of patient safety and education. As these training systems continue to evolve, the development of standardized programs administered by our Societies and/or Boards needs to rapidly coevolve to integrate competency-based credentialing of endourologic skill. This is being done in many other disciplines already, such as colorectal surgery, general surgery, and orthopedics. ²²

Conclusion

Luminaries like Dr. Arthur Smith have pioneered minimally invasive techniques for Urology. The CREST development process has and will allow our group to develop systems to train and assess endourologic skills to the next generation. New technological capabilities promise to rapidly expand the fidelity, usability, manufacturability, functionality, and number of applications while diminishing the cost.