Simulation for Percutaneous Renal Access: Where Are We?

Abstract

Objectives:

Percutaneous renal access (PCA) is a challenging step during percutaneous nephrolithotomy. The aim of this study is to review the literature for different types of simulators described for PCA.

Methods:

Databases of Medline, Embase, Cochrane Library, OvidSP, and Google Scholar were systematically searched until May 2016. The studies were analyzed regarding the type of simulator (nonbiologic, biologic, live animal, and virtual reality [VR]), type of validity (face, content, construct, and predictive), cost-effectiveness, and whether these simulators have been used for training and/or assessment of PCA. In addition, the study looked at the educational impact of these simulators in terms of the transfer of PCA skills to the operating room.

Results:

Several bench, animal, and VR simulators for training in PCA were identified. Only few studies were found on assessment of PCA skills. Biological bench models used porcine or bovine kidneys wrapped within foam, silicone, chicken carcass, or full-thickness skin flap alone. Other biological models used additional subcutaneous fascia, muscle, or ribs. Nonbiological models used prototypes, including 3D printing. Only one study reported the use of anesthetized live pig for training. The PERC Mentor™ was the only VR simulator, which has been validated for training and assessment of PCA skills. However, none of these studies assessed the educational impact of PCA simulators. Furthermore, most of the studies did not address the validity and the cost of the simulator.

Conclusions:

While several biological and nonbiological PCA models exist, there is paucity of literature regarding the validity and educational impact of these simulators. The PERC Mentor simulator is the sole validated simulator for training and assessment of PCA skills. However, it is expensive and there is little evidence of its educational impact. Therefore, more research is needed to validate the available simulators and assess their educational impact for urology trainees.

Introduction

There is no doubt that percutaneous renal access (PCA) is a challenging step during percutaneous renal surgery.^1,2 An appropriate PCA is considered an integral preliminary step during percutaneous nephrolithotomy (PCNL). This is due to the fact that the kidney is surrounded by important structures such as the colon, the liver (on the right side), the spleen (on the left side), the pleura, and the lungs (on both sides). In addition, the renal vasculature and renal pelvicaliceal orientation should be appropriately approached to minimize hemorrhagic complications.¹ Therefore, inaccurate placement of the puncture needle can lead to injuries either to the kidney or to adjacent structures.³ While previous studies reported that percutaneous procedures are performed by 69.6% of practicing urologists,⁴ only 11% of them were found to obtain PCA themselves.³ Lack of training during urology training was cited as a cause. Other possible causes included personal preference, shortage of operating room (OR) time, financial considerations, or availability of better equipment in the radiology suite.⁵ Nevertheless, the need for independence regarding the timing and location of obtaining PCA pushed an increasing percentage of urologists to obtain the PCA themselves.^1,6 In a retrospective single-center study, the outcomes of PCNL procedures were better whenever the PCA was obtained by the urologist rather than the interventional radiologist.³ However, achieving competency in PCA is quite difficult due to the steep learning curve and the fact that PCA is usually performed only once per PCNL procedure.^7,8 There are increasing concerns about patients' safety in light of decreased exposure to the OR and restricted training hours during the week. In addition, fluoroscopy-guided PCA is the step that is accompanied with the most radiation exposure during PCNL.⁹ Therefore, simulators are needed for practicing an unlimited number of this critical step in a stress-free and radiation-free environment outside the OR, with flexibility regarding the time and place of practice.^10,11 Different bench (nonbiological and biological), live animal, and virtual reality (VR) simulators have been introduced for training in PCA. Another important function of these VR simulators is that they give immediate formative feedback, which could be used for training and assessment of trainees in PCA skills. The aim of this study was to review the literature for the currently available simulators for training and assessment of PCA and address their cost-effectiveness and educational impact.

Methods

Databases of PubMed, Medline through OvidSP, Embase, Google scholar, and Cochrane library were systematically searched for articles written in the English language using the medical subject heading (MeSH) keywords: percutaneous nephrolithotomy, outcome and process assessment, clinical skills, and computer simulation. In addition, the following non-MeSH keywords were also searched: percutaneous access, skills, assessment, training, and virtual reality. This online search was conducted on June 15, 2016, without specifying the type of study and the year of publication. The search was broadened to include references from retrieved articles. All studies that described the development and/or validation of a simulator for training and assessment of PCA skills were included. To be consistent, all articles were reviewed by a single author (Y.A.N.). Review articles were excluded. The studies were analyzed regarding the type of simulator (bench, live animal, and VR), type of validity (face, content, construct, and predictive), cost-effectiveness, and whether the simulator has been used for training and/or assessment of PCA. In addition, the current review article examined the educational impact of simulators in terms of transfer of PCA skills from the simulator to the OR.

Results

Twenty studies on training and assessment of PCA were identified. Five review articles were excluded. Table 1 lists 20 studies in a chronological order of publication. It also summarizes their methodologies and conclusions. Table 2 summarizes the list of published simulators in addition to their type (biological, nonbiological, VR), cost, validity, training, assessment, and educational impact in terms of transferring PCA skills to the OR.

Table 1.

Studies on Simulators for Percutaneous Renal Access in Chronological Order

Study	Year	Country	No. of participants	Aim and methodology	Conclusions
Earp¹⁴	2003	Brazil	—	To assess the feasibility of performing fluoroscopic-guided PCA using the ordinary PCNL instruments on a model comprising a porcine kidney with catheterized ureter within foam layers closed with a wide scotch tape.	The model was feasible for training in PCA
Hammond et al.¹⁷	2004	United States	—	To assess the feasibility of performing fluoroscopic-guided PCA using the ordinary PCNL instruments on a porcine kidney within intact chicken carcass.	The model was feasible for training in PCA and was widely accepted by all trainees
Strohmaier and Giese¹⁶	2005	Germany	—	To assess the feasibility of performing fluoroscopic- and ultrasound-guided PCA using ordinary PCNL instruments on a model consisting of a porcine kidney and ureters embedded in silicone.	The model was useful in training in PCA
Knudsen et al.²⁸	2006	Canada, United States	63 trainees (31 medical students, 31 PGTs, and 1 fellow)	To assess the efficacy of training in PCA on the PERC Mentor™ simulator. Trainees were categorized into two groups (training and nontraining) following obtaining pretest scores. After training, both groups were tested using the objective parameters from simulator and PCNL-GRS.	The training group showed significantly better performance in terms of the objective and subjective parameters
Häker et al.¹⁸	2007	Germany	PGTs (unknown number)	To assess the feasibility of performing fluoroscopic- and ultrasound-guided PCA using ordinary PCNL instruments on a model comprising a porcine kidney and ureter surrounded by gel placed in an eviscerated chicken carcass.	The model is simple and cheap and could be used for training in PCA
Zhang et al.²¹	2008	China	33 urologists for hands-on training and 42 for face validity	To assess the feasibility of using a model comprising a porcine kidney and ureter wrapped in a full-thickness skin flap with subcutaneous fascia and muscles for learning and training on fluoroscopic- and ultrasound-guided PCA using ordinary PCNL instruments. Furthermore, to assess the face validity of this model.	The study confirmed the feasibility of using this model for training and proved the face validity
Bruyère et al.²³	2008	France	2 (1 attending and 1 trainee) trained for 6 times on the model and performed one real PCNL on a patient	To assess the efficacy of training on a model created by rapid prototyping, based on abdominal CT images of a patient with lower pole stone, scheduled to undergo PCNL. Furthermore, to assess whether training on this model affects the outcome inside the OR.	The model was useful. The PCNL technique on the patient, from whom the abdominal CT images were obtained, was successful with uneventful outcome
Mishra et al.²⁹	2010	India	24 (15 novices, 9 experts)	To assess the face, content, construct, convergent, and predictive validities of the PERC Mentor simulator. All participants performed a specific task on the simulator and the performance was compared between novices and experts. In addition, a questionnaire was used to assess the face and content validity. Following training, a subset of 5 novices were asked to perform PCA on pigs to assess the predictive validity.	The study confirmed the face, content, construct, convergent, and predictive validities of the PERC Mentor simulator
Imkamp et al.¹⁹	2011	Austria	36 urologists	To assess the feasibility of using a model comprising a porcine kidney and ureter wrapped in a full-thickness skin flap for training in fluoroscopic- and ultrasound-guided PCA. Furthermore, the face validity was assessed.	The study confirmed the feasibility of using this model for training in PCA and proved the face validity
Qiu et al.²²	2011	China	126 urologists for face validity and 122 for hands-on training on the model	To assess the feasibility of using a model comprising a porcine kidney and ureter wrapped with a full-thickness thoracic wall flap with skin, subcutaneous fascia, muscle, and two ribs for training in fluoroscopic- and ultrasound-guided PCA.	The study confirmed the feasibility of using this model for training in PCA and proved the face validity
Papatsoris et al.³¹	2012	United Kingdom	36 PGTs	To assess the efficacy of training in PCA on the PERC Mentor simulator. Baseline PCA skills were assessed. Then, some of the trainees were allowed to practice for 2 hours and the performance of all trainees was assessed afterward.	The performance of the training group was significantly better than the nontraining group
Abdallah et al.¹⁵	2013	Egypt	10 (5 PGTs and 5 interns)	To compare training in PCA using the bull's eye and triangulation techniques using a model comprising a bovine kidney and ureter with the renal artery and vein covered with a sponge and placed inside a human-like mannequin.	The model was easily constructed and feasible for training. The learning curve of both techniques was comparable
Zhang et al.³⁰	2013	China	21 urologists	To assess the efficacy of training in PCA on the PERC Mentor simulator. Baseline PCA skills of all trainees were tested, then all participants were left to practice 2 hours daily for two successive days before undergoing post-test evaluation. The performance was assessed using the objective parameters from simulator and PCNL-GRS.	Following the 2-day training, all trainees were able to independently complete the procedure
Turney²⁵	2014	United Kingdom	—	To assess the feasibility of using a nonbiological model (from rapid prototyping and 3D printing) for training on obtaining the fluoroscopic-guided PCA.	The model was feasible for training in PCA
Zhang et al.²⁴	2014	China	39 (9 experts and 30 novices)	To assess the feasibility of training in PCA using a nonbiological model comprising a kidney with dilated pelvicaliceal system, a ureteral stamp, and nontransparent perirenal tissue.	The model was feasible for training in PCA with proven construct, content, and face validity
Jutzi et al.²⁰	2014	Germany	11 urologists	To assess the feasibility of using a model comprising 2 full-thickness porcine skin flaps in a laparoscopy trainer. To assess the face validity.	The model was feasible for training in PCA with proven face validity
Kallidonis et al.²⁷	2015	Greece, Austria, Germany	3 (2 PGTs and 1 expert)	To assess the feasibility of training in PCA on live anesthetized pigs and the effect of this training on the learning curve of PCNL during clinical practice. The PCNL-GRS was used by a PCNL expert to assess trainees' performance inside the OR.	Training in PCA on live anesthetized pig was feasible and established a clinical performance comparable with a PCNL expert after 30 PCNL procedures
Veneziano et al.²⁶	2015	United States	14 participants	To assess the feasibility and efficacy of using a low-cost physical fluoro-less CAT for training in PCA.	The study proved the efficacy of using the CAT for training in PCA
Noureldin et al.¹²	2015	Canada	13 PGTs	The PERC Mentor simulator was used to assess the PCA skills of urology PGTs during OSCE. The objective parameters from the simulator were used.	PGTs with prior PCA experience performed better in terms of FT and complications
Noureldin et al.¹³	2016	Canada	26 PGTs	The PERC Mentor simulator was used to assess competency of PGTs in PCA during OSCE. PGTs were assessed using the PCNL-GRS and the objective parameters from the simulator. The overall PCNL-GRS score was used to set passing score based on the norm-referenced method.	Competent PGTs had significantly better performance in terms of FT, PCNL-GRS score, and complications

CAT = C-arm trainer; FT = fluoroscopy time; GRS = global rating scale; OR = operating room; OSCE = objective structured clinical examination; PCA = percutaneous renal access; PCNL = percutaneous nephrolithotomy; PGTs = postgraduate trainees.

Table 2.

Available Simulators for Training and Assessment of Percutaneous Renal Access

Type of simulator	Year	Guidance	Cost (U.S. $)	Description	Validation	Training	Assessment	Transfer of PCA to the OR
Biological porcine model¹⁴	2003	Fluoro	Low cost	Ex vivo porcine kidney hidden within foam.	—	Yes	—	—
Biological porcine model¹⁷	2004	Fluoro	Low cost	Ex vivo porcine kidney hidden within a chicken carcass.	Face validity	Yes	—	—
Biological porcine model¹⁶	2005	Flouro+US	Low cost	Ex vivo porcine kidney and ureter placed between two layers of silicone.	—	—	—	—
Virtual reality model²⁸	2006	Flouro	100,000	The PERC Mentor simulates the loin of a patient lying in prone position. It gives tactile feedback during the puncture. In addition, there is a virtual C-arm controlled by a foot pedal and virtual ureteral catheter for retrograde contrast injection. This simulator gives real-time feedback and produces an objective report following completion of the task.	Construct, face, predictive, and content validity	Yes	Yes	—
Biological porcine model¹⁸	2007	Flouro+US	Low cost	It is a modification of Hammond model using ex vivo porcine kidney hidden within a chicken carcass. Gel was placed around the kidney and within the carcass.	Face validity	Yes	—	—
Biological porcine model²¹	2008	Flouro+US	25	The porcine kidneys were wrapped in a full-thickness skin flap with subcutaneous fascia and muscle and fixed to a wooden board with 2 long steel nails. The ureters were catheterized to inject radiologic contrast medium or physiologic saline.	Face validity	Yes		—
Nonbiological (rapid prototype) model²³	2008	Fluoro	3690	Rapid prototyping was used to create a virtual kidney and abdominal cavity of a patient with lower pole renal stone. A balloon was placed between the kidney and the abdomen and connected to a respiratory machine (60 breaths/min and 300 mL tidal volume) to simulate the renal movements during respiration	—	Yes	—	(only one PCNL on a patient)
Biological porcine model¹⁹	2011	Flouro+US	Low cost	Ex vivo porcine kidney covered with a full-thickness porcine skin flap. The ureter was catheterized to wash and inject contrast.	Face validity	Yes	—	—
Biological porcine model²²	2011	Flouro+US	25	Ex vivo porcine kidney with >3 cm ureteral stump and thoracic wall flap with the skin, subcutaneous fascia, muscle, and two ribs. All were fixed to a wooden board with two long steel nails.	Face validity	Yes	—	—
Biological bovine model¹⁵	2013	Fluoro	Low cost	The kidney, ureter, renal artery, and vein of freshly slaughtered cow were placed in the loin of a commercial model (mannequin). Two sheets of sponge were used to cover the kidney and a window was created in the mannequin where the needle puncture should start. Part of an aubergine (eggplant, Solanum melongena) was used to mimic the human cuticle.	—	Yes	—	—
Biological porcine model²⁰	2014	Flouro+US	30	Ex vivo porcine kidney covered with 2 full-thickness porcine skin flaps in an existing laparoscopy trainer (SITUS box).	—	Yes	—	—
Nonbiological model²⁴	2014	Flouro+US	550	It consisted of three parts; a kidney with dilated pelvicaliceal system, a ureteral stamp, and nontransparent perirenal tissue. Each model has 13 calices at different directions.	Construct, face, and content validity	Yes	—	—
Nonbiological (silicone) model²⁵	2014	Fluoro	12,919	Routine CT urograms were used to extract and reformat collecting systems using specialized software. CT images printed in a water-soluble plastic on 3D printer to create biomodels. These models were embedded in silicone and were then dissolved in water to leave a hollow collecting system within a silicone model. These PCNL models were filled with contrast medium and sealed. Then, a layer of dense foam was placed to act as a spacer to replicate the tissues between skin and kidney.	—	Yes	—	—
Nonbiological model²⁶	2015			The SimPORTAL fluoro-less CAT was introduced to mimic the C-arm during PCNL. It was designed using rapid prototyping 3D printer and scaled to fit on a standard desk and accommodates two webcams connected to an Apple Macbook Pro. Images obtained are filtered, and fused using the blue screen technique to give a final on-screen simulated image.	—	No	—	—
Porcine model²⁷	2015	Fluoro	Low cost	Anesthetized live pig (used for Module 1 of the Modular Training Scheme).	—	Yes	Yes	Yes

US = ultrasound.

Several biological (porcine or bovine) and nonbiological, including VR, simulators for training in PCA were identified (Table 2). Most of the publications addressed validation of these simulators. Few studies were found on assessment of PCA skills.^12,13 The biological bench models used porcine or bovine kidneys wrapped within foam,¹⁴ foam in a mannequin,¹⁵ silicone,¹⁶ chicken carcass,^17,18 full-thickness skin flap,^19,20 or additional subcutaneous fascia, muscle,²¹ and ribs.²² Other nonbiological bench models used prototyping or 3D printing.^23

–26 Only one study reported the use of anesthetized live pigs as module 1 of the Modular Training Scheme, which included renal puncture, tract dilatation using the Amplatz dilators, and orientation with the nephroscope in a live pig.²⁷ The PERC Mentor™ (Simbionix, Cleveland, OH) was the only VR simulator that has been validated for training and assessment of PCA skills.^{12,13,28

–31} However, none of these studies assessed the educational impact of PCA simulators in terms of transferring PCA skills from the benchtop to the OR. Furthermore, most of these studies did not address the validity and the cost of these simulators.

Training in PCA skills

Training programs in Canada, Australia, Scotland, and United States have established competency as the target of training to meet the criteria of their certifying bodies.³² Competency is defined as “an observable ability of a health professional, integrating multiple components such as knowledge, skills, values, and attitudes.”³² Recently, competency-based medical education (CBME) defined as “an outcome-based approach to the design, implementation, assessment, and evaluation of medical education programs using an organizing framework of competencies” has become the cornerstone of training, assessing, and promoting postgraduate trainees (PGTs).³² Therefore, different competency-based frameworks, such as the Outcome Project of the United States Accreditation Council for Graduate Medical Education, CanMEDS, and the Scottish Doctor, were described as the basis for CBME.^33
–35

Despite being the first-line option for managing large renal calculi, the indications for PCNL are expanding to include medium-sized and small renal calculi with recent modifications of the PCNL technique, such as mini-PCNL, ultramini-PCNL, and micro-PCNL.^6,36,37 However, PCA is still an integral step in all of these modifications and obtaining an appropriate PCA is mandatory for the PCNL procedure.^1,3 In a study by de la Rosette and colleagues,³⁸ learning PCA was responsible for the long learning curve of PCNL. Furthermore, a recent study by Noureldin et al.⁹ found that obtaining fluoroscopy-guided PCA is responsible for the higher radiation exposure during PCNL when compared with PCNLs performed in patients with preformed tracts. Thus, it is not surprising that the learning curve of PCNL is long, requiring 36–60 cases to achieve competency and >100 cases to achieve excellence.^7,8,38 The First European Urolithiasis Society meeting during September 2011 confirmed the steep learning curve of PCA and identified two main etiologies that make training urology PGTs in PCA difficult: First, the complexity of obtaining a safe access to the kidney for lithotripsy. Second, fluoroscopic-guided access is being performed most of the time, thus the potential for injuring structures from the skin to the renal capsule.³⁹ Therefore, it is mandatory for urology PGTs to receive appropriate PCNL training during their residency period to be familiar with this complex endourologic procedure. A recent report from the United Kingdom examined the exposure of urology PGTs to various urological procedures and noted that on average PGTs performed or assisted in only 19 PCNLs during their residency training period, while a wide variability among PGTs was noted with exposures to PCNL ranging from 0 to 125 procedures.⁴⁰ Another survey from the United States showed that PGTs are comfortable with PCA after an average of 21.2 ± 4.5 procedures.⁵ Compared with urologists who were uncomfortable in performing PCA, urologists who were comfortable in performing PCA had performed significantly higher number of PCA procedures during their residency (24.4 ± 5.6 vs 10.6 ± 3.1; p = 0.046).⁵ Therefore, more than 24 PCA procedures were recommended to obtain the necessary PCA skills during the residency period.³⁸ Thus, there should be a curriculum for training in PCNL during the urology residency programs. This curriculum should clearly describe the cornerstones for training in PCNL, define the learning curve, and determine surrogates in evaluating technical competency. In their proposal for minimal requirements of PCNL curriculum, de la Rosette and co-workers.³⁸ defined the medical expertise in PCNL as theoretical and practical knowledge of the PCNL technique and its indications, and defined the technical expertise in PCNL as an appropriate access and lithotripsy in simple complicated and medium complicated PCNL cases under strict supervision. Previously, the stone-free rate and complications have been used as clinical surrogates for PCNL outcomes. Other parameters such as operative time, fluoroscopy time (FT), and estimated blood loss are also considered.^41,42 However, recent reductions in residency work hours resulted in reduced access of PGTs to the OR. In addition, the fact that PCA is almost always performed only once per PCNL procedure necessitated the introduction of simulators for training PGTs outside the OR.

In this study, 14 simulators were identified for training in PCA; 12 dry laboratory bench models, 1 wet laboratory animal model, and 1 VR simulator (Table 2). Ex vivo animal models (biological bench models) have been used for PCNL training. These include porcine or bovine kidneys, with and without a part of the ureter, and with and without other parts of the body wall. The stones are introduced into the renal pelvis through a pyelotomy, which is then closed in a watertight manner, and the distal end of the ureter is tied over a ureteral catheter to give the opportunity to inject physiologic saline or contrast medium.¹⁶ Thus, PGTs can then train in PCA skills using either fluoroscopic or ultrasound guidance. These models offer tactile sensation that may be comparable with real life. However, preparation of these models consumes time and money. In addition, the PCA procedure can only be performed a limited number of times per model. Finally, there are ethical considerations in using animal models. All of these factors limit their availability for routine training.

VR simulators are software-dependent simulators creating an environment similar to the real clinical situation. They are user-friendly and enjoy several advantages: (1) the procedure could be repeated unlimited times in a stress-free and radiation-free environment, (2) they have multiple varieties of preset clinical scenarios with different levels of difficulty, and (3) they generate a performance report following completion of the task. Thus, objective assessment and monitoring of trainees' progress could be achieved. However, these simulators are expensive and may malfunction during practice since they are software dependent.^43,44 The PERC Mentor (Simbionix) was the only VR simulator for PCNL training in normal and obese models using fluoroscopic guidance.⁴⁵ It gives the opportunity to practice PCA with various procedural steps such as (1) puncturing the pelvicaliceal system (PCS) using a 15-cm needle for normal patients and 20-cm needle for obese patients, (2) aspiration from the puncture needle using a virtual syringe producing a yellow color of urine, whenever the needle was introduced into the PCS and a red color of blood, whenever the needle hits a blood vessel causing vascular injury, (3) injecting a contrast medium through a virtual ureteral catheter, (4) monitoring the anatomy of the PCS, planning the puncture, and following the progress of the puncture needle into the PCS while the kidney is moving up and down with respiration, using biplanar VR C-arm that could be controlled using a foot pedal, and (5) providing feedback on performance parameters such as procedure time, FT, number of attempts to puncture the PCS, number of vascular injuries, number of PCS perforations, number of rib collisions, and the amount of contrast medium used during antegrade and/or retrograde urography.¹ In a study by Knudsen and colleagues,²⁸ the PERC Mentor simulator was validated for training in PCA.²⁹ However, it seems like VR simulators are underutilized in training of PGTs. This might be due to the lack of wide availability of these simulators due to their high cost. In a recent survey of 91 urology PGTs and 172 urologists from the United Kingdom, there was consensus on the lack of sufficient simulation-based urology training for common urological procedures such as cystoscopy (69%–74%), ureteroscopy (URS) (47%–59%), nephrectomy (62%), transurethral resection of the prostate (56%–65%), and percutaneous renal surgery (73%–76%).⁴⁶

Assessment of PCA skills

Clinical competence is the habitual and judicious use of communication, knowledge, technical skills, clinical reasoning, emotions, values, and reflection in daily practice for the benefit of the individual and community being served.⁴⁷ Assessment of technical competency is an integral part of the whole assessment of clinical competency. This will not only give information about the fitness to practice, but also it provides constructive feedback for further learning and improvement. This is important as it has been confirmed that there is a significant negative correlation between the level of competency and patients' morbidity and mortality rates.⁴⁸

Evaluation of competency involves assessment of skills, knowledge, beliefs, and attitudes in a quantitative and qualitative manner.⁴⁹ Competency has been assessed using various assessment tools. These tools include generic and procedure-specific checklists, in-training evaluation reports, logbooks, direct observation of procedural skills, case-based discussion, outcome data on morbidity and mortality rates, peer assessment, multisource feedback, and evidence of participation in different clinical and nonclinical update courses.^45,50
–52 However, the lack of objectivity of these tools favors VR simulators, which enjoy objectivity in assessing technical skills. Objective assessment from VR simulators could be combined with validated subjective assessment tools, such as the global rating scale (GRS) and objective structured assessment of technical skills, to assess transfer of these skills to the OR.^13,53 The GRS has been validated for assessing competency of trainees and specialists in real and simulated environments.^53
–55

Simulators used for assessment of PCA skills include human cadavers, live animals, and VR simulators to simulate human patients.^56,57 In a recent study by Noureldin et al.,^12,13 the PERC Mentor simulator was incorporated into urology objective structured clinical examinations to assess PCA skills of PGTs. PGTs with previous PCA experience had performed the PCA with significantly shorter FT (5.1 ± 0.7 vs 10 ± 1.5; p = 0.04), fewer infundibular injuries (2 ± 0.4 vs 7.4 ± 1.5; p = 0.004), fewer PCS perforations (4.5 ± 1.2 vs 11 ± 1.7; p = 0.01), and fewer attempts to puncture the PCS (13 ± 1.8 vs 21 ± 2.3; p = 0.02) when compared with PGTs without previous PCA experience, respectively.¹² Since the PERC Mentor simulator does not give a global score, the authors were not able to set a passing score for competency. Therefore, they performed another study to assess competency in PCA using task 4 on the PERC Mentor simulator, where each participant was asked to correctly access and pop seven balloons in seven renal calices. The 26 participants included PGTs from all four training programs in the province of Quebec from PGY-3 to 5. All participants were assessed both objectively using the PERC Mentor simulator and subjectively using the PCNL-GRS, and the competency passing score was calculated from the PCNL-GRS score using the norm-referenced method. Competent PGTs performed task 4 with significantly higher percentage of successful attempts to pop the balloons (p < 0.001), shorter FT (9.8 vs 6.5 minutes; p = 0.01), lower complications (p = 0.01), and higher PCNL-GRS scores (p < 0.001).¹³

Transfer of PCA skills from VR to OR

Most of the literature on VR simulation is related to laparoscopic surgery and gastrointestinal endoscopy.^3,58 The analysis of clinical trials in a Cochrane review study reported the efficacy of training on VR simulators for acquisition of laparoscopic skills,⁵⁹ However, there is paucity of data on the efficacy of training on VR simulators for endourologic skills, including PCNL. While three studies showed the potential improvement of performance following training on the PERC Mentor simulator,^28,29,31 there is still lack of literature on the educational impact of training on simulators in terms of improved performance in PCA skills inside the OR. In a level 1 evidence-based study by Chou and co-workers,⁶⁰ the early clinical practice of rigid URS improved following training on a VR simulator. Another study by Aloosh et al. recruited urology PGTs for training on the URO Mentor™ simulator (Simbionix). Trainees were assessed on the simulator and inside the OR using the validated URS-GRS. A significant positive correlation between URS-GRS scores obtained on the simulator and in the OR (r = 0.9, p = 0.03) was found. Therefore, they concluded that the flexible URS skills obtained from training on the URO Mentor™ simulator could be transferred to the OR.⁶¹ However, there are no data on the transfer of PCA skills from training on VR simulators to the OR. Therefore, future prospective studies need to examine transfer of PCA skills from training on VR simulators to performance in the OR in terms of FT, OR time, and any complications relating to PCA.

Conclusions

There is paucity of literature regarding the current simulators for training and assessing PCA skills. The PERC Mentor simulator is currently the only validated VR simulator used for training and assessing PCA skills. However, it is expensive and there is no evidence regarding the transfer of PCA skills from the PERC Mentor to the OR. Therefore, more research is needed to validate these simulators and assess their educational impact for urology trainees and attending urologists.

Footnotes

Acknowledgments

This work was partially sponsored by Fonds de la Recherche Santé du Québec (FRSQ) Chercheur Boursier grant to Dr. Sero Andonian and by a grant from the Urology Care Foundation Research Scholars Program and the Boston Scientific Corporation, The Endourological Society, and the “Friends of Joe” to Dr. Yasser Noureldin and the CUASF-SIU International Scholarship grant to Dr. Yasser Noureldin.

Author Disclosure Statement

No competing financial interests exist.

Abbreviations Used

References

Stern

, Zeltser

, Pearle

. Percutaneous renal access simulators. J Endourol, 2007; 21:270–273.

. Training in percutaneous nephrolithotomy: The learning curve and options. Arab J Urol, 2014; 12:54–57.

Watterson

, Soon

, Jana

. Access related complications during percutaneous nephrolithotomy: Urology versus radiology at a single academic institution. J Urol, 2006; 176:142–145.

Kauer

, Laguna

, Alivizatos

, et al.; on behalf of the members of ESUT. Present practice and treatment strategies in endourological stone management: Results of a survey of the European Society of Uro-technology (ESUT). Eur Urol, 2005; 48:182–188.

Lee

, Anderson

, Monga

. Residency training in percutaneous renal access: Does it affect urological practice?. J Urol, 2004; 171:592–595.

Ghani

, Andonian

, Bultitude

, Desai

, Giusti

, Okhunov

, Preminger

, de la Rosette

. Percutaneous nephrolithotomy: Update, trends, and future directions. Eur Urol, 2016; 70:382–396.

Ziaee

SAM

, Sichani

, Kashi

, et al. Evaluation of the learning curve for percutaneous nephrolithotomy. Urol J, 2010; 7:226–231.

Jang

, Choi

, Yang

, et al. The learning curve for flank percutaneous nephrolithotomy for kidney calculi: A single surgeon's experience. Korean J Urol, 2011; 52:284–288.

Noureldin

, Elkoushy

, Andonian

. Does the presence of a percutaneous renal access influence fluoroscopy time during percutaneous nephrolithotomy?. Asian J Urol, 2015; 2:220–223.

10.

Matsumoto

, Hamstra

, Radomski

, Cusimano

. A novel approach to endourological training: Training at the Surgical Skills Center. J Urol, 2001; 166:1261–1266.

11.

Mishra

, Jagtap

, Sabnis

, Desai

. Training in percutaneous nephrolithotomy. Curr Opin Urol, 2013; 23:147–151.

12.

Noureldin

, Elkoushy

, Andonian

. Assessment of percutaneous renal access skills during urology objective structured clinical examinations. Can Urol Assoc J, 2015; 9:104–108.

13.

Noureldin

, Fahmy

, Anidjar

, Andonian

. Is there a place for virtual reality simulators in assessment of competency in percutaneous renal access?. World J Urol, 2016; 34:733–739.

14.

Earp

. Percutaneous renal surgery: New model for learning and training. Int Braz J Urol, 2003; 29:151–154.

15.

Abdallah

, Salem

, Badreldin

, Gamaleldin

. The use of a biological model for comparing two techniques of fluoroscopy-guided percutaneous puncture: A randomised cross-over study. Arab J Urol, 2013; 11:79–84.

16.

Strohmaier

, Giese

. Ex vivo training model for percutaneous renal surgery. Urol Res, 2005; 33:191–193.

17.

Hammond

, Ketchum

, Schwartz

. A new approach to urology training: A laboratory model for percutaneous nephrolithotomy. J Urol, 2004; 172(5 Pt 1):1950–1952.

18.

Häcker

, Wendt-Nordahl

, Honeck

, Michel

, Alken

, Knoll

. A biological model to teach percutaneous nephrolithotomy technique with ultrasound- and fluoroscopy-guided access. J Endourol, 2007; 21:545–550.

19.

Imkamp

, von Klot

, Nagele

, Herrmann

. New ex-vivo organ model for percutaneous renal surgery. Int Braz J Urol, 2011; 37:388–394.

20.

Jutzi

, Imkamp

, Kuczyk

, Walcher

, Nagele

, Herrmann

. New ex vivo organ model for percutaneous renal surgery using a laparoendoscopic training box: The sandwich model. World J Urol, 2014; 32:783–789.

21.

Zhang

, Ou

, Jia

, Gao

, Cui

, Wu

, Wang

. Novel biologic model for percutaneous renal surgery learning and training in the laboratory. Urology, 2008; 72:513–516.

22.

Qiu

, Yang

, Zhang

, Sun

. Modified biological training model for percutaneous renal surgery with ultrasound and fluroscopy guidance. Chin Med J (Engl), 2011; 124:1286–1289.

23.

Bruyère

, Leroux

, Brunereau

, Lermusiaux

. Rapid prototyping model for percutaneous nephrolithotomy training. J Endourol, 2008; 22:91–96.

24.

Zhang

, Yu

, Jin

, Li

, Na

. Validation of a novel non-biological bench model for the training of percutaneous renal access. Int Braz J Urol, 2014; 40:87–92.

25.

Turney

. A new model with an anatomically accurate human renal collecting system for training in fluoroscopy-guided percutaneous nephrolithotomy access. J Endourol, 2014; 28:360–363.

26.

Veneziano

, Smith

, Reihsen

, Speich

, Sweet

. The SimPORTAL fluoro-less C-arm trainer: An innovative device for percutaneous kidney access. J Endourol, 2015; 29:240–245.

27.

Kallidonis

, Kyriazis

, Vasilas

, et al. Modular training for percutaneous nephrolithotripsy: The safe way to go. Arab J Urol, 2015; 13:270–276.

28.

Knudsen

, Matsumoto

, Chew

, et al. A randomized, controlled, prospective study validating the acquisition of percutaneous renal collecting system access skills using a computer based hybrid virtual reality surgical simulator: Phase I. J Urol, 2006; 176:2173–2178.

29.

Mishra

, Kurien

, Patel

, Patil

, Ganpule

, Muthu

, Sabnis

, Desai

. Validation of virtual reality simulation for percutaneous renal access training. J Endourol, 2010; 24:635–640.

30.

Zhang

, Yu

, Liu

, Wang

, Zhu

, Na

. Training for percutaneous renal access on a virtual reality simulator. Chin Med J (Engl), 2013; 126:1528–1531.

31.

Papatsoris

, Shaikh

, Patel

, Bourdoumis

, Bach

, Buchholz

, Masood

, Junaid

. Use of a virtual reality simulator to improve percutaneous renal access skills: A prospective study in urology trainees. Urol Int, 2012; 89:185–190.

32.

Frank

, Snell

, Cate

, Holmboe

, Carraccio

, Swing

. Competency-based medical education: Theory to practice. Med Teach, 2010; 32:638–645.

33.

Simpson

, Furnace

, Crosby

, et al. The Scottish doctor–learning outcomes for the medical undergraduate in Scotland: A foundation for competent and reflective practitioners. Med Teach, 2002; 24:136–143.

34.

Frank

, Snell

, Sherbino

, eds. Draft CanMEDS 2015 Physician Competency Framework–Series III. Ottawa: The Royal College of Physicians and Surgeons of Canada; 2014 September. Available at: www.royalcollege.ca/portal/page/portal/rc/common/documents/canmeds/framework/canmeds2015_framework_series_III_e.pdf (Last accessed on June 18, 2016 ).

35.

Frank

, Danoff

. The CanMEDS initiative: Implementing and outcomes-based framework of physician competencies. Med Teach, 2007; 29:642–647.

36.

Armagan

, Tepeler

, Silay

, Ersoz

, Akcay

, Akman

, et al. Micropercutaneous nephrolithotomy in the treatment of moderate-size renal calculi. J Endourol, 2013; 27:177–181.

37.

Ganpule

, Bhattu

, Desai

. PCNL in the twenty-first century: Role of microperc, miniperc, and ultraminiperc. World J Urol, 2015; 33:235–240.

38.

de la Rosette

, Laguna

, Rassweiler

, Conort

. Training in percutaneous nephrolithotomy—A critical review. Eur Urol, 2008; 54:994–1001.

39.

Ather

, Ng

, Pourmand

, Osther

. Training the resident in percutaneous nephrolithotomy. Arab J Urol, 2014; 12:49–53.

40.

Gill

, Stewart

, George

, Eardley

. Operative experience of urological trainees in the UK. BJU Int, 2012; 109:1296–1301.

41.

Allen

, O'Brien

, Tiptaft

, Glass

. Defining the learning curve for percutaneous nephrolithotomy. J Endourol, 2005; 19:279–282.

42.

Tanriverdi

, Boylu

, Kendirci

, Kadihasanoglu

, Horasanli

, Miroglu

. The learning curve in the training of percutaneous nephrolithotomy. Eur Urol, 2007; 52:206–212.

43.

Noureldin

, Elkoushy

, Andonian

. Assessment of photoselective vaporization of the prostate skills during urology objective structured clinical examinations (OSCE). Can Urol Assoc J, 2015; 9:e61–e66.

44.

Noureldin

, Aloosh

, Andonian

. How to use virtual-reality simulators to assess competency in basic endourologic and robotic skills?. Videourology January, 2016; DOI: 10.1089/vid.2016.0014.

45.

Ahmed

, Jawad

, Dasgupta

, Darzi

, Athanasiou

, Khan

. Assessment and maintenance of competence in urology. Nat Rev Urol, 2010; 7:403–413.

46.

Aydin

, Ahmed

, Shafi

, Khan

, Dasgupta

. The role of simulation in urological training—A quantitative study of practice and opinions. Surgeon, 2015; pii: S1479-666X(15)00059-1.

47.

Epstein

, Hundert

. Defining and assessing professional competence. JAMA, 2002; 287:226–235.

48.

Prystowsky

, Bordage

, Feinglass

. Patient outcomes for segmental colon resection according to surgeon's training, certification, and experience. Surgery, 2002; 132:663–670; discussion 670–672.

49.

Ahmed

, Jawad

, Abboudi

, et al. Effectiveness of procedural simulation in urology: A systematic review. J Urol, 2011; 186:26–34.

50.

Feldman

, Hagarty

, Ghitulescu

, et al. Relationship between objective assessment of technical skills and subjective in-training evaluations in surgical residents. J Am Coll Surg, 2004; 198:105–110.

51.

Shah

, Darzi

. Surgical skills assessment: An ongoing debate. BJU Int, 2001; 88:655–660.

52.

Shaharan

, Neary

. Evaluation of surgical training in the era of simulation. World J Gastrointest Endosc, 2014; 6:436–447.

53.

Martin

, Regehr

, Reznick

, MacRae

, Murnaghan

, Hutchison

, Brown

. Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg, 1997; 84:273–278.

54.

Aggarwal

, Grantcharov

, Moorthy

, Milland

, Darzi

. Toward feasible, valid, and reliable video-based assessments of technical surgical skills in the operating room. Ann Surg, 2008; 247:372–379.

55.

Beard

, Choksy

, Khan

. Assessment of operative competence during carotid endarterectomy. Br J Surg, 2007; 94:726–730.

56.

Moorthy

, Munz

, Sarker

, Darzi

. Objective assessment of technical skills in surgery. BMJ, 2003; 327:1032–1037.

57.

Reznick

, MacRae

. Teaching surgical skills changes in the wind. N Engl J Med, 2006; 355:2664–2669.

58.

Felsher

, Olesevich

, Farres

, et al. Validation of a flexible endoscopy simulator. Am J Surg, 2005; 189:497–500.

59.

Gurusamy

, Aggarwal

, Palanivelu

, et al. Virtual reality training for surgical trainees in laparoscopic surgery. Cochrane Database Syst Rev, 2009; 1:CD006575.

60.

Chou

, Abdelshehid

, Clayman

, McDougall

. Comparison of results of virtual-reality simulator and training model for basic ureteroscopy training. J Endourol, 2006; 20:266–271.

61.

Aloosh

, Noureldin

, Andonian

. Transfer of flexible ureteroscopic stone-extraction skill from a virtual reality simulator to the operating theatre: A pilot study. J Endourol 2016; In press. PMID: 27532227; DOI:10.1089/end.2016.0365