Current Educational Interventions for Improving Technical Skills of Urology Trainees in Endourological Procedures: A Systematic Review

Abstract

Objective:

Endourology continues to grow with the introduction of new technologies into clinical practice. Simulators and training models have been developed to improve comfort and proficiency in endoscopic procedures. The purpose of this systematic review was to examine the current educational interventions utilized to improve the performance of endourology trainees and to critically appraise the strengths and limitations of each.

Methods:

A search of the Ovid MEDLINE, EMBASE, PsycINFO, and the Cochrane Library databases was performed to identify literature focused on current educational interventions for improving technical skills of trainees in endourologic procedures. The Medical Education Research Study Quality Instrument (MERSQI) was used to evaluate the methodological quality of the abstracted articles.

Results:

Of the 2236 articles identified, 22 met the inclusion criteria. The types of educational interventions included: bench/wet lab models, virtual reality simulators, and instructional courses. Metrics used to quantify the impact of these interventions include global rating scales, Objective Structured Assessment of Technical Skills (OSATS) scores, and task-specific checklists. The setting of these evaluations comprises both virtual reality simulators and live surgery.

Conclusions:

In the surgical education literature, simulation-based training and assessment continues to play a prominent role in urologic training. The educational interventions highlighted in this review address various aspects of endourology, from stone management to transurethral resection. Additional work is needed to correlate technical performance in clinical and nonclinical settings with patient outcomes and develop a focused approach to nontechnical skill training.

Introduction

Urologic education is evolving from the Halstedian apprenticeship model of graduated autonomy in surgery toward a more proactive approach, including simulation and competency-based training. In the current climate of reduced work hours for residents and financial constraints, educators and training programs are facing increasing pressures to improve efficiency in training without compromising patient safety. These limitations have driven investment in ex vivo education in surgery, with formal curricula incorporating this modality to better prepare trainees for the operating room experience.¹

Endourology, in particular, has faced unique training challenges due to the rapid implementation of newer more sophisticated technologies.² For example, the number of approaches for benign prostatic hyperplasia (BPH) surgery has exploded in the last decade with the adoption of procedures such as Holmium Laser Enucleation of the Prostate (HoLEP), GreenLight™ Laser Photoselective Vaporization of the Prostate (PVP), and Aquablation^® into clinical guidelines.²

Stone management and transurethral resection remain key components of a urologist's surgical repertoire. As such, endoscopic skills are relevant in both community/private and academic practices and must be mastered by graduating residents. In recent years, many groups have developed simulators and training models to improve comfort and proficiency in endoscopic procedures. The purpose of this systematic review is to examine current educational interventions to improve endourologic performance of trainees and to critically appraise the strengths and limitations of each intervention. In addition, a contemporary framework was used to evaluate the quality of the evidence presented in the included studies.

Methods

This systematic review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol.³

Eligibility criteria

Articles exploring educational interventions to improve the endoscopic procedural skills of urology trainees at all levels (medical students, residents, fellows, and staff) were included. Endoscopic procedures were broadly categorized into cystoscopy, upper tract stone management, transurethral resection, and others. Studies that did not include urology participants were excluded. Studies which did not assess participants pre- and poststudy were excluded. Articles were eligible for inclusion regardless of publication status and study type. Reviews, editorials, opinion letters, and case reports were excluded.

Information sources

A search was conducted in Ovid MEDLINE, Embase Classic, PsycINFO, and the Cochrane Library on January 18, 2019.

Search

The search strategy was designed to capture educational interventions impacting the complete surgical repertoire of a urologist, including open surgical, laparoscopic, endoscopic, and robot-assisted surgical skills. Medical subject headings (MeSH) terms included “clinical competence,” “cognitive,” “computer simulation,” “curriculum,” “education, medical, graduate,” “endoscopes,” “laparoscopy,” “non-technical,” “performance,” “robotic surgical procedure,” “simulation training,” “skill,” “surgery,” “surgical training,” “technical,” “urologic surgical procedures,” “urologists,” and “urology.”

Study selection

Titles and abstracts were independently screened by the authors for full-text review. Articles referenced in the included studies and previous review articles were eligible. Disagreements were resolved by consensus.

Data collection

The following data were extracted from the included articles: type of endoscopic procedure, setting and type of intervention, sample size, participant type, measures of improvement, limitations of the intervention, and information relevant to the assessment of study quality.

Evaluation of study quality

The Medical Education Research Study Quality Instrument (MERSQI) is a standardized tool used to assess the methodological quality of medical education research.⁴ The MERSQI tool was used to evaluate the following domains in the included articles: study design, sampling, type of data, validity of the evaluation instrument, appropriateness and sophistication of data analysis, and outcomes. A comprehensive explanation of the components of MERSQI is provided in Appendix A1.⁴

Results

The search terms yielded 2236 articles, of which 22 were selected for full text review (Fig. 1). These articles are summarized in Table 1 with the corresponding MERSQI scores listed. As mentioned, studies were organized into the following categories: cystoscopy, upper tract stone management, transurethral resection, and others. Of the studies included, six were rated as high quality, defined as a MERSQI score ≥14.

FIG. 1.

Preferred Reporting Items for Systematic Reviews and Meta-Analyses flow chart.

Table 1.

Studies included, MERSQI Scores, and Limitations

Procedure	Type of intervention	No. of trainees	Measures of improvement (corresponding MERSQI score in parentheses)	Stated limitations
Evaluation
Flexible cystoscopy	Simulator	Novice (n = 39)	Shah⁸ (11.5)	Cost of virtual reality simulators Flags and digital markers are easily visible compared to bladder pathology Absence of haptic feedback Some simulators were unable to simulate bleeding
		Novice (n = 39)	Task completion times improved after 10 sessions
		13 MS	Number of flags identified increased
		3 R	Zhang⁷ (10.0)
		2 F	Total operating time lower after 8 sessions
		21 S	Frequency of injury decreased
		Expert (n = 7)	Number of digital markers observed increased
		Expert (n = 7)	GRS scores improved
		7 S	Bach⁹ (13.5)
		7 S	Participants performed the task of flexible cystoscopy in approximately half (50.96%) of the original time taken by the end of the study.
	Bench model	Novice (n = 36)	Hu⁵ (13.5)	Limited availability and cost
		Novice (n = 36)	OSATS score on Uro-Scopic Trainer improved following training on transparent model compared to nontransparent model
		36 MS
		Expert (n = 10)
		10 S
Stone management
Semirigid ureteroscopy	Bench model	Novice (n = 66)	Brehmer¹⁰ (12.0)	Lacked real surgical instruments Limited by the number of patients with mid ureteral stones in a limited time frame, as well as the ethical issue of allowing a medical student to perform ureteroscopy unassisted for study purposes. Other factors, such as patient anatomy, stone size, and location, made intraoperative assessment difficult.
		Novice (n = 66)	Task-specific checklist for ureteroscopy improved
		40 MS	Global score improved
		26 R	Total score improved
			Matsumoto¹¹ (13.5)
			There was a significant effect of hands-on training on endourologic performance
			For bench model fidelity, the low fidelity group did significantly better than the didactic group
Flexible ureteroscopy	Wet lab model	Novice (n = 56)	Hu⁵ (13.5)	No improvement in mucosal trauma and perforation rates Does not train preoperative preparation and setup Brightness settings light in ureteroscopy not representative of actual procedure
		Novice (n = 56)	Operative time improved
		36 R and MS (distribution unspecified) 20 S	Global rating scores improved
			Pass rates improved
			Learning curve stable after six simulations
			Ganesamoni¹² (14.5)
			Global rating scores improved in both groups with a mentor and without a mentor
			Trainees receiving active mentoring outperformed those without mentoring in terms of global rating scores and task completion time

	Simulator	Novice (n = 5)	Knoll¹³ (13.5)	Lacks haptic feedbackHigh prices limit the availability of computer simulators
		Novice (n = 5)	Ureteroscopy-naive residents who trained on the URO Mentor had significantly shorter operating times in their first four live ureteroscopy cases compared to the nontraining group
		5 R
		Expert (n = 20)
		20 S
	Course	Novice (n = 105)	Huri¹⁴ (10.0)	No bleeding, absence of respirations, lack ureteral toneLack initial application of ureteral access sheathOnly incorporated one feedback session; therefore, the impact of additional early or late feedback sessions is unknownInability to control for any additional training that participants might have had in fURS during the SBT period.
		Novice (n = 105)	Post-training procedural times were significantly shorter than pretraining
		72 S
		12 R	Soria¹⁵ (13.5)
			At the end of Module 1, all trainees passed the theoretical test.
		21 MS	Trainees found an increase in skill for GRS and RIRS checklist.
		Expert (n = 15)	On assessing the individual skill increase among trainees, it was found that 73.4% of trainees increase their endourologic skills by over 40%
		15 S
		15 S	Lee¹⁶ (14.5)
			Time to completion and performance score improved
			Earlier expert feedback resulted in shorter task completion times and better performance scores compared to later feedback for medical students
			Kim¹⁷ (15.0)
			Time to completion and performance score improved
			Earlier expert feedback resulted in shorter task completion times and better performance scores compared to later feedback for residents
Percutaneous nephrolithotomy	Simulator	Novice (n = 84)	Karagozlu¹⁸ (13.5)	Does not simulate respirationsOnly simulates percutaneous renal accessHigh price limits the availability of the percutaneous nephrolithotomy simulator
		Novice (n = 84)	Procedure time improved
		21 S	Fluoroscopy time improved
		1 F	Puncture attempts improved
		67 R	Knudsen¹⁹ (15.5)
		31 MS	Global rating scores, procedure time, fluoroscopy time, number of needle punctures, and number of infundibular punctures improved
		Novice (n = 120)
		Expert (n = 5)
		5 S	Papatsoris²⁰ (15.5)
			Procedure time, radiation exposure time, radiation dose, correct caliceal puncture, and time to correct caliceal access improved
Transurethral Resection
Photoselective vaporization of the prostate (PVP)	Simulator	Novice (n = 51)	Aydin²¹ (13.5)	Novices included were not representative of a true resident population and had no previous endoscopic experience
		Novice (n = 51)	A significant improvement in performance was seen in all modules; Anatomy identification, sweep speed, tissue-fiber distance, bleeding coagulation, and power settings.
		45 MS
		6 S
		6 S	A significant reduction in operative time and operative error was observed
		Expert (n = 7) 7 S
			Angulo²² (11.5)
			For 55 g prostate, learning curve stable after 4 procedures
			For 70 g prostate, learning curve stable after 10 procedures
Transurethral resection of the prostate	Simulator	Novice (n = 46)	He²³ (12.0)	Some models did not simulate bleeding
		Novice (n = 46)	Global rating scores of trainees increased
		11 MS	Rate of capsule resection decreased
		35 S	Blood loss decreased
		Expert (n = 12) 12 S	External sphincter injury decreased
		Expert (n = 12) 12 S	Kallstrom²⁴ (13.5)
			Task completion time decreased
			Resected volume increased
			Completeness of resection improved
			Blood loss decreased
			Irrigation fluid used decreased
			Task checklist score improved, max 20
			Inactive time reduced
			High pressure time reduced
Transurethral resection of bladder tumor	Simulator	Novice (n = 15)	Kruck²⁵ (14.5)	Cost (∼$75,000 EUR)No significant improvement in hemostasis score
		Novice (n = 15)	Novice, after five scenarios:
		15 R	Bladder inspection rates improved
		Expert (n = 15) 15 S	Tumor resection rate improved
		Expert (n = 15) 15 S	Expert, after five scenarios with photodynamic diagnostics view:
			Bladder inspection rates improved
			Tumor resection rate improved
Other
Endoscopic correction of vesicoureteral reflux	Porcine model	Novice (n = 11)	Soltani²⁶ (13.0)	Anatomic differences between porcine and human model. Ureteral orifices are located medially and distally in the bladder neck, which make injection more challengingNo standardized assessment tool to evaluate this procedure
		Novice (n = 11)	Trainee scores improved
		11 R and F (distribution unspecified)	Specific areas showing significant improvement included: identification of ureteral orifices, ureteral orifice hydrodistention, first and second injection, and location, size, and depth of mound after injection

EUR = Euros; F = fellow; fURS = flexible ureteroscopy; GRS = Global rating scale; MERSQI, Medical Education Research Study Quality Instrument; MS = medical student; OSATS = Objective Structured Assessment of Technical Skills; R = resident; RIRS = retrograde intrarenal surgery; S = staff; SBT = simulation-based training.

Cystoscopy

Flexible cystoscopy is a fundamental diagnostic skill in urology and is often taught to junior trainees. To date, three studies have explored educational interventions to improve cystoscopic proficiency, of which two were simulation based and one was a bench model. Hu et al. found that transparent anatomical models were more effective than nontransparent models in cystoscopy-naive trainees using the Uro-Scopic Trainer, with a mean Objective Structured Assessment of Technical Skills (OSATS) score of 21.83 ± 3.64 and 18.50 ± 4.03 (p = 0.022), respectively.⁵ The benefits of the transparent model include allowing trainees to appreciate the internal anatomy and receive visual feedback during the procedure, while enabling educators to provide immediate and relevant feedback. The UroMentor by Simbionix is a virtual reality simulator widely used to practice endoscopic procedures.⁶ Following eight sessions on the UroMentor, trainees were observed to have faster procedure times (511 ± 67 seconds vs 111 ± 10 seconds, p < 0.001), fewer iatrogenic injuries (12 ± 2–5 ± 1 times, p < 0.001), improved identification of digital markers (9 ± 0–10 ± 1, p = 0.005), and better overall performance on global rating scales (1.3 ± 0.2–3.9 ± 0.2 points, p < 0.001).⁷ Shah and Darzi reported similar improvements in trainees following 10 trials on the UroMentor, with shorter task completion times (4.89 minutes vs 2.33 minutes, p = 0.005) and improved lesion identification (8.0 vs 9.2, p = 0.05).⁸ Bach and colleagues developed a “homemade” transurethral resection of the prostate (TURP) simulator composed of 7 cm of a 30F garden hose, a suprapubic tube, a 1500 cc Tupperware container, three Foley catheter plugs, and a small tube of silicone gel, with an equipment cost totalling $40 U.S.⁹ Investigators observed that the participants performed the task of flexible cystoscopy in approximately half (50.96%) of the original time taken by the end of the study.⁹

Upper tract stone management

Semirigid ureteroscopy

Brehmer and Swartz evaluated the effectiveness of bench model teaching in hands-on courses on cystoscopy and semirigid ureteroscopy.¹⁰ Following course completion, procedural confidence (0.46/2–1.88/2), global scores (2.6/9–7.8/9), total scores (7.7/19–17.2/19), and task-specific checklist for ureteroscopy (5.1/10–9.2/10) all improved. Matsumoto et al. demonstrated that training on low-fidelity models is a cost-effective alternative to learning basic ureteroscopy skills, with comparable global rating scores (p = 0.08), checklist scores (p = 0.17), and task completion times (p = 0.80) as high-fidelity models.¹¹ Both training models outperformed didactic teaching.

Flexible ureteroscopy

Several modalities have been explored to teach flexible ureteroscopy in trainees composed of staff, residents, and medical students. For bench models, Hu and colleagues developed a wet lab model using isolated kidneys and ureters in vitro.⁵ Participants composed of 20 staff had to perform ureteroscopy and look for a foreign body randomly placed in the kidneys and identify the exact renal calix. Researchers observed that operative time (18 ± 3.4–11 ± 1.2 minutes; p < 0.05), global rating scores (9.3 ± 0.8–28.7 ± 1.1; p < 0.05), and pass rates (30%–95%; p < 0.05) improved by the study's completion.⁵ In addition, Ganesamoni et al. demonstrated that training on the Endo-Urologie-Modell (Karl Storz) resulted in significant improvements in global rating scores (19.1 ± 3.8–25.2 ± 6.1, p = 0.002) of both medical students and residents.¹² They show the value of receiving active mentoring during training, with better global rating scores (29.5 ± 3.5 vs 25.2 ± 6.1, p < 0.02) and task completion times (213 ± 55 seconds vs 342 ± 135 seconds, p < 0.001) than those without mentoring.

For virtual reality simulators, Knoll et al. conducted a study on 20 staff urologists and 5 junior residents to monitor their improvement in flexible ureteroscopy training on the UroMentor (Simbionix). Participants were evaluated based on total operation time, guidewire insertion time, time of progression from orifice to the stone, stone contact time, number of perforations, bleeding events, laser misfiring, scope damage, and treatment success. The staff with greater experience in flexible ureteroscopy (>40 ureteroscopies) were faster than those with less experience (<40 ureteroscopies). The junior residents who received training on the URO Mentor were faster than the untrained residents. This study demonstrated that ureteroscopy-naive trainees had shorter operating times (p < 0.05) in their first four live ureteroscopy cases in the operating room following 10 training sessions on the URO Mentor (Simbionix).¹³

Regarding curricular design, Huri et al. developed a program incorporating didactic lectures, video presentations, anatomy dissection, and hands-on training using cadavers.¹⁴ After course completion, trainees with no prior flexible ureteroscopy experience demonstrated shorter procedure times on cadavers (3.56 ± 2.0–1.76 ± 1.54 minutes, p = 0.008). Soria et al. developed a retrograde intrarenal surgery (RIRS) training course to train a group of urologists with no prior experience in RIRS.¹⁵ The course was composed of three modules, including theoretical knowledge, bench model practice, and training on live porcine models. At the end of Module 1, all trainees passed the theoretical test. Participants observed an increase in skill by 43.89% for global rating scores and 29.66% for RIRS checklist.¹⁵ Furthermore, it was measured that 73.4% of trainees were able to increase their endourologic skills by over 40%. A 2016 study by Lee et al. developed a simulation-based training curriculum for medical students composed of didactic lectures, hands-on session of ureteroscopic instrumentation and skills, and practice sessions using dry-lab models.¹⁶ Time to completion (23.9 ± 3.7–20.3 ± 3.4 minutes, p < 0.001), as well as performance score, improved (7.97 ± 1.44–11.81 ± 2.68, p = 0.001). Furthermore, they found that earlier expert feedback before individual practice sessions resulted in shorter task completion times (19.2 minutes vs 21.5 minutes, p < 0.01) and better performance scores (13.1 vs 10.5, p < 0.01) compared to delayed feedback.¹⁶ They found similar benefits for task completion times (15.2 minutes vs 9.1 minutes, p < 0.0001) of early expert feedback for residents, with significantly higher mean postcourse performance scores (25.7 vs 22.5, p = 0.05).¹⁷

Percutaneous nephrolithotomy

A study by Karagozlu et al. in 2016 used a simulator to teach a group of urologists and pediatric surgeons without percutaneous nephrolithotomy (PCNL) experience.¹⁸ They used a box trainer with two open sides and radiopaque pelvicaliceal system tools to train participants in percutaneous renal puncture. They noted that procedure time (183–45 seconds; p < 0.001), fluoroscopy time (77–15 seconds; p < 0.001), and puncture attempts (4–1; p < 0.001) improved. Knudsen et al.¹⁹ found that training on a PCNL simulator, consisting of percutaneous renal puncture followed by introduction of a guidewire into the collecting system, resulted in better global rating scores, total procedure time, fluoroscopic time, and attempted fewer needle punctures and caused fewer blood vessel injuries than the untrained group (p < 0.001). For lower pole access and ureteral guidewire insertion, Papatsoris et al.²⁰ found that a 2-hour simulator training session resulted in improvements in procedural time (15.8 ± 7.8–6.49 ± 3.17 minutes, p < 0.01), duration of radiation exposure (3.97 ± 1.86–2.25 ± 1.62 minutes, p < 0.01), radiation dose (4.13 ± 2.05–2.33 ± 1.64 mSV, p < 0.01), correct caliceal puncture (77.7%–91.6%, p < 0.01), and time to correct caliceal access (10.82 ± 9.16–4.48 ± 5.55 minutes, p < 0.05).

Transurethral resection

Photoselective vaporization of the prostate (PVP)

Two studies used simulation to train groups of residents and medical students in PVP. Aydin et al. used a GreenLight simulator composed of part-task training modules (short two five-minute videos on anatomy identification, sweep speed, tissue-fiber distance, power settings, and coagulation) and case modules consisting of six clinical cases of BPH.²¹ They found significant improvements in performance in all modules: anatomy identification (p < 0.001), sweep speed (p < 0.001), bleeding coagulation (p < 0.001), and power settings (p = 0.007). A reduction in operative time (p < 0.001) and operative error (p = 0.017) was observed.¹⁷ A 2014 study by Angulo used a virtual reality simulator for training PVP.²¹ In this study, participants were able to practice prostate vaporization with diode laser. They were trained in all aspects of the procedure, including anatomical margin identification, equipment operation, coagulation, and carrying out safe examination at the conclusion of the procedure.²² The investigators observed that for a 55 g prostate, learning curve was stable after 4 procedures; for a 70 g prostate, learning curve was stable after 10 procedures. Overall, the simulators were able to demonstrate good functional task alignment, with realistic vaporization and instrumentation.²² The graphics and haptics of the virtual reality simulator were considered to be realistic and helpful for participants gaining comfort with the procedure.

Transurethral resection of the prostate

A study by He et al. used the TURPSim simulator for training a group of urologists.²³ Participants were evaluated objectively by the simulator based on their performance during the TURP. They observed an increase in global rating scores (18.0 ± 4.0 vs 12.4 ± 4.2, p < 0.001), decrease in the rate of capsule resection (26.3% ± 0.6% vs 21.2% ± 0.4%, p < 0.001), decrease in blood loss (125.8 ± 86.3 vs 83.7 ± 41.6 mL, p < 0.001), and decrease in external sphincter injury (3.6 ± 2.9 vs 2.0 ± 2.0, p < 0.001).²³ A 2010 study by Kallstrom and coworkers evaluated the effectiveness of the PelvicVision TURP simulation model.²⁴ They noted that there was an improvement for students overall, and improvement was observed for the staff with respect to total operative time, resection speed, completeness, irrigation fluid used, and checklist sources. Students noted that the procedure became easier over time.²⁴

Transurethral resection of the bladder tumor (TURBT)

Kruck et al. evaluated the effectiveness of a virtual reality simulator to teach transurethral resection of the bladder tumor (TURBT), as well as the effectiveness of photodynamic diagnostic (PDD), in a group of residents and staff.²⁵ Participants were introduced to the theory underlying the simulator and the principles of TURBT using instruction. Afterward, they were exposed to hands-on training on the simulator involving detection, resection of bladder tumors, and management of bleeding. The novice group completed five standardized 5-minute resection scenarios, while the expert group of TURBT-trained urologists was evaluated using the virtual reality PDD-TURBT program using white light visualization with and without PDD assistance. In the novice group, after five practice scenarios: bladder inspection rates (36.8% ± 12.9%–54.3% ± 7.3%; p = 0.0008) and tumor resection rate improved (26.5% ± 9.6%–52.0% ± 6.0%; p < 0.0001).²⁵ In the expert group, after five practice scenarios with PDD view: bladder inspection rates (52.2% ± 3.4%–62.7% ± 4.3%; p = 0.003) and tumor resection rate (43.8% ± 3.3%–57.1% ± 5.3%; p = 0.002) improved. Overall, participants found that this simulator improved their understanding of the anatomy involved.²⁵

Other

Endoscopic correction of vesicoureteral reflux

Soltani and colleagues studied the utility of a porcine bladder model in training a group of residents and fellows to perform endoscopic correction of vesicoureteral reflux.²⁶ The simulator consisted of a dissected ex vivo porcine bladder with distal ureters and urethra secured. First, participants were exposed to a short video outlining the steps of the procedure. They were then presented with a demonstration, followed by a 2-hour hands-on training of the simulator. This study evaluated the technique of administering two injections on the simulation bladder without assistance.²⁶ Trainee scores improved from 56% to 92% (p = 0.008). Specific areas showing significant improvement included: identification of ureteral orifices (Δ = 33%, p = 0.025), ureteral orifice hydrodistention (Δ = 33%, p = 0.010), first and second injection administration technique (Δ = 46%, p = 0.006; Δ = 46%, p = 0.006), as well as location (Δ = 50%, p = 0.041), size (Δ = 50%, p = 0.004), and depth of the mound (Δ = 58%, p = 0.013) after injection.²⁶

Discussion

In the modern era of surgical education, there continues to be push for skill acquisition outside the immediate clinical space. This has distinct advantages by increasing the volume of exposure obtained by surgical trainee while limiting patient exposure to novices. Training programs and trainees must adapt to increased competition for limited learning opportunities due to reduced work hours, emphasis on reducing medical errors, hospital regulations, and limited funding. This is all in an era where medical technology is advancing faster than it ever has, necessitating practicing surgeons to learn these novel procedures. The resultant trickle-down effect on trainees even further accentuates their reduced exposure to learning opportunities in the clinical space. It is also important to note that of 2236 articles identified, only 22 qualified for this systematic review. The low inclusion rate is a reflection of the reality that we do not have sufficient evidence across all of these procedure types. This article highlights the fact that we need to study not just the development, face validation, and practicality of various study methods. It is imperative to study the impact that these interventions have in a real-world clinical setting.

Educational interventions highlighted in this review aim to address this gap in endourology and are categorized into virtual reality-based simulators, bench models, and training courses. However, each type of intervention carries limitations as outlined in Table 1. Physical models often lack intrinsic quantitative methods of evaluating performance; they are often single- or limited use and lack human physiology such as bleeding and respiration. In contrast, while virtual models address some of the shortcomings of physical models with detailed performance reports and simulated scenarios, they lack haptic feedback and latency. Course-based interventions are logistically challenging, resource intense, and thus not easily accessible.

A notable limitation in the current literature is the paucity of evidence correlating technical performance with clinical performance and outcomes. Only one of the included studies demonstrates true technical improvements in the operating room.¹³ This issue is prevalent in other aspects of urologic surgical education, including robot-assisted surgery.²⁷ For educational interventions to be worth widespread implementation, they should be associated with clinically meaningful improvements in performance. In general surgery, Williams et al. demonstrated that general surgery trainees achieve similar adenoma detection rates as gastroenterologists following an endoscopy simulation curriculum.²⁸

As we continue to shift surgical education outside of the operating room, we must also set standardized benchmarks for simulation training. Von Websky et al. suggested that peer-group derived metrics should form a fundamental part of basic skill training before advancing to more sophisticated simulators.²⁹ Therefore, achieving these metrics will help trainees reach milestones and aid with surgical skill acquisition. Rashid et al. identified different TURP resection strategies between novices, residents, and experts based on amount of prostate resected, blood loss, irrigation volume used, foot pedal use, and time spent with orientation, cutting, or coagulation.³⁰ An evolution in strategy and predefined metrics, such as adhering to a systematic approach, less orientation time, adequate hemostasis, and judicious use of irrigation fluid may therefore serve as surrogate markers for attaining TURP proficiency. For accreditation, the Japanese Urologic Association and Japanese Society of Endourology and Extracorporeal Shock Wave Lithotripsy developed the Endoscopic Surgical Skill Qualification System to evaluate competence in urologic laparoscopic skills.³¹ Using video assessments of laparoscopic nephrectomies and adrenalectomies, applicants were certified based on specific intraoperative errors, with a 66% pass rate.

Equally important is the necessity of implementing curricula or simulation to achieve proficiency in nontechnical skills (NTS) in endourologic procedures. Brewin and colleagues created a full immersion TURP simulation to develop technical and NTS using the Bristol TURP simulator, scrub nurse, and anesthetist.³² This demonstrated multiple elements of assessment validity, but the impact on technical performance was not studied. In robot-assisted surgery, the NASA Task Load Index is the most widely studied NTS assessment tool, but validity evidence supporting other more clinically relevant tools is limited.³³ NTS training can also be extended into endourologic crises, such as bladder perforation, and TUR syndrome. Such simulation of high stress scenarios can improve team dynamics and educational competencies.³⁴ Goldenberg et al. developed a simulation-based exercise to access technical and NTS of urology residents during a vasovagal response to pneumoperitoneum and inferior vena cava injury during laparoscopic surgery.³⁵ The authors found that level of training correlated with technical and nontechnical performance.

With a plethora of educational interventions available, the next step is to determine how these modalities can be effectively integrated into the urology residency program. Key questions that need to be addressed include which trainees are best suited for each intervention, what educational interventions should be used, what competencies should be addressed, and when trainees should be approved to perform a clinical activity. The feasibility of a centralized curriculum for simulation has been demonstrated by Khan et al. with excellent validity and participant satisfaction.³⁶ It would be valuable to study how an integrated endourologic curriculum composed of simulation and models highlighted from this systematic review would impact trainee performance and clinical outcomes.

Conclusion

Simulation-based exercises continue to play a prominent role in urologic training. The educational interventions highlighted in this review address various aspects of endourology, from stone management to transurethral resection. Additional work is needed to correlate technical performance in nonclinical settings with patient outcomes and developing an effective approach to NTS training.

Footnotes

Authors' Contributions

Conception and design: I.A. and M.G.G. Collection and assembly of data: I.A. and J.C.C.K. Data Analysis and interpretation: I.A. and J.C.C.K. Article writing: I.A., J.C.C.K., T.C., J.Y.L., and M.G.G. Final Approval of article: I.A., J.C.C.K., T.C., J.Y.L., and M.G.G.

Author Disclosure Statement

No competing financial interests exist.

Funding Information

No funding was received for this article.

Abbreviations Used

Appendix A1.

Explanation of the Components of the Medical Education Research Study Quality Instrument

MERSQI domain	Criteria and scores
Type of study	Single cohort survey or postintervention: 1
	Single cohort pre- and postintervention: 1.5
	Nonrandomized, 2 cohorts: 2
	Randomized controlled trial: 3
Sampling: number of institutions of participants	1 institution: 0.5
	2 institutions: 1
	≥3 institutions: 1.5
Sampling: response rate of participants	Not applicable
	<50% or not reported: 0.5
	50%–74%: 1
	≥75%: 1.5
Type of data	Subjective, assessed by participant: 1
Type of data	Objective, assessed by observer: 3
Validity evidence of evaluation instrument scores	Not applicable
	Content (using theory, guidelines, experts, existing tools): 1
	Internal structure (reliability, factor analysis): 1
	Relationship (expert vs novice, correlation with other variables): 1
Data analysis: sophistication	Descriptive (i.e., frequency, mean, median): 1
Data analysis: sophistication	Beyond descriptive (any statistical inference): 2
Data analysis: appropriate	Analysis appropriate for study and type of data: 1
Outcome	Satisfaction, attitudes: 1
	Knowledge, skills: 1.5
	Behavior in clinical setting: 2
	Patient outcome: 3

Adapted from Cook and Reed.⁴

References

Reznick

, MacRae

. Teaching surgical skills—Changes in the wind. N Engl J Med, 2006; 355:2664–2669.

Nickel

, Aaron

, Barkin

, Elterman

, Nachabé M, Zorn KC. Canadian Urological Association guideline on male lower urinary tract symptoms/benign prostatic hyperplasia (MLUTS/BPH): 2018 update. Can Urol Assoc J, 2018; 12:303.

Moher

, Liberati

, Tetzlaff

, Altman

. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. Ann Intern Med, 2009; 151:264–269.

Cook

, Reed

. Appraising the quality of medical education research methods: The medical education research study quality instrument and the Newcastle–Ottawa scale-education. Acad Med, 2015; 90:1067–1076.

, Feng

, Wang

, Song

, Xu

, Zhou

, Huang

. Ureteroscopy and cystoscopy training: Comparison between transparent and non-transparent simulators. BMC Med Educ, 2015; 15:93.

Michel

, Knoll

, Köhrmann

, Alken

. The URO Mentor: Development and evaluation of a new computer-based interactive training system for virtual life-like simulation of diagnostic and therapeutic endourological procedures. BJU Int, 2002; 89:174–177.

Zhang

, Liu

, Gang WA

CF, He

, NA

. Effectiveness of the UroMentor virtual reality simulator in the skill acquisition of flexible cystoscopy. Chin Med J, 2013; 126:2079–2082.

Shah

, Darzi

. Virtual reality flexible cystoscopy: A validation study. BJU Int, 2002; 90:828–832.

Bach

, Geavlete

, Herrmann

, Gross

. “Homemade” TUR-Simulator for Less Than $40 US? The “TupperTM” Experience. J Endourol, 2009; 23:509–513.

10.

Brehmer

, Swartz

. Training on bench models improves dexterity in ureteroscopy. Eur Urol, 2005; 48:458–463.

11.

Matsumoto

, Hamstra

, Radomski

, Cusimano

. The effect of bench model fidelity on endourological skills: A randomized controlled study. J Urol, 2002; 167:1243–1247.

12.

Ganesamoni

, Mishra

, Kumar

, Ganpule

, Vyas

, Ganatra

, Sabnis

, Desai

. Role of active mentoring during flexible ureteroscopy training. J Endourol, 2012; 26:1346–1349.

13.

Knoll

, Trojan

, Häcker

, Alken

, Michel

. Validation of computer-based training in ureterorenoscopy. BJU Int, 2005; 95:1276–1279.

14.

Huri

, Skolarikos

, Tatar İ, et al. Simulation of RIRS in soft cadavers: A novel training model by the Cadaveric Research On Endourology Training (CRET) Study Group. World J Urol, 2016; 34:741–746.

15.

Soria

, Morcillo

, Serrano

, et al. Development and validation of a novel skills training model for retrograde intrarenal surgery. J Endourol, 2015; 29:1276–1281.

16.

Lee

, McDougall

, Lineberry

, Tekian

. Optimizing the timing of expert feedback during simulation-based spaced practice of endourologic skills. Simul Healthc, 2016; 11:257–263.

17.

Kim

, Blankstein

, Ordon

, Pace

, Honey

, Lee

, Lantz Powers

. Evaluation of optimal timing of expert feedback in a simulated flexible ureteroscopy course. J Endourol, 2019; 33:463–467.

18.

Karagozlu

, Unal

, Demirbas

, Oner

, Ucar

, Akgul

, Guzelsoy

. A simple, non-biological model for percutaneous renal access training. Eur Urol Suppl, 2016; 15:e1272–e1275.

19.

Knudsen

, Matsumoto

, Chew

, et al. A randomized, controlled, prospective study validating the acquisition of percutaneous renal collecting system access skills using a computer based hybrid virtual reality surgical simulator: Phase I. J Urol, 2006; 176:2173–2178.

20.

Papatsoris

, Shaikh

, Patel

, Bourdoumis

, Bach

, Buchholz

, Masood

, Junaid

. Use of a virtual reality simulator to improve percutaneous renal access skills: A prospective study in urology trainees. Urol Int, 2012; 89:185–190.

21.

Aydin

, Muir

, Graziano

, Khan

, Dasgupta

, Ahmed

. Validation of the Green Light™ Simulator and development of a training curriculum for photoselective vaporisation of the prostate. BJU Int, 2015; 115:994–1003.

22.

Angulo

, Arance

, García-Tello

, Las Heras

, Andrés

, Gimbernat

, Lista

, de Fata

. Virtual reality simulator for training on photoselective vaporization of the prostate with 980 nm diode laser and learning curve of the technique. Actas Urol Esp (English Edition), 2014; 38:451–458.

23.

, Zhang

, Liu

, Gang

, Yu

, Na

. Virtual reality simulator for training urologists on transurethral prostatectomy. Chin Med J, 2013; 126:1220–1223.

24.

Källström

, Hjertberg

, Svanvik

. Construct validity of a full procedure, virtual reality, real-time, simulation model for training in transurethral resection of the prostate. J Endourol, 2010; 24:109–115.

25.

Kruck

, Bedke

, Hennenlotter

, Amend

, Merseburger

, Stenzl

, Sievert

. Virtual bladder tumor transurethral resection: An objective evaluation tool to overcome learning curves with and without photodynamic diagnostics. Urol Int, 2011; 87:138–142.

26.

Soltani

, Hidas

, Kelly

, et al. Endoscopic correction of vesicoureteral reflux simulator curriculum as an effective teaching tool: Pilot study. J Pediatr Urol, 2016; 12:45-e1.

27.

Goldenberg

, Goldenberg

, Grantcharov

. Surgeon performance predicts early continence after robot-assisted radical prostatectomy. J Endourol, 2017; 31:858–863.

28.

Williams

, Crossett

, Cleveland

, Smoot

, Aluka

, Coviello

, Davis

. Equivalence in colonoscopy results between gastroenterologists and general surgery residents following an endoscopy simulation curriculum. J Surg Educ, 2015; 72:654–657.

29.

von Websky

, Raptis

, Vitz

, Rosenthal

, Clavien

, Hahnloser

. Access to a simulator is not enough: The benefits of virtual reality training based on peer-group-derived benchmarks—A randomized controlled trial. World J Surg, 2013; 37:2534–2541.

30.

Rashid

, Kowalewski

, Oppenheimer

, Ooms

, Krieger

, Sweet

. The virtual reality transurethral prostatic resection trainer: Evaluation of discriminate validity. J Urol, 2007; 177:2283–2286.

31.

Matsuda

, Ono

, Terachi

, Naito

, Baba

, Miki

, Hirao

, Okuyama

. The endoscopic surgical skill qualification system in urological laparoscopy: A novel system in Japan. J Urol, 2006; 176:2168–2172.

32.

Brewin

, Tang

, Dasgupta

, Khan

, Ahmed

, Bello

, Kneebone

, Jaye

. Full immersion simulation: Validation of a distributed simulation environment for technical and non-technical skills training in Urology. BJU Int, 2015; 116:156–162.

33.

Kwong

, Lee

, Goldenberg

. Understanding and assessing nontechnical skills in robotic urological surgery: A systematic review and synthesis of the validity evidence. J Surg Educ, 2019; 76:193–200.

34.

Marr

, Hemmert

, Nguyen

, et al. Team play in surgical education: A simulation-based study. J Surg Educ, 2012; 69:63–69.

35.

Goldenberg

, Fok

, Ordon

, Pace

, Lee

. Simulation-based laparoscopic surgery crisis resource management training—Predicting technical and nontechnical skills. J Surg Educ, 2018; 75:1113–1119.

36.

Khan

, Ahmed

, Gavazzi

, et al. Development and implementation of centralized simulation training: Evaluation of feasibility, acceptability and construct validity. BJU Int, 2013; 111:518–523.