Validation of a Full-Immersion Simulation Platform for Percutaneous Nephrolithotomy Using Three-Dimensional Printing Technology

Abstract

Introduction and Objectives:

The restriction of resident hours with an increasing focus on patient safety and a reduced caseload has impacted surgical training. A complex and complication prone procedure such as percutaneous nephrolithotomy (PCNL) with a steep learning curve may create an unsafe environment for hands-on resident training. In this study, we validate a high fidelity, inanimate PCNL model within a full-immersion simulation environment.

Methods:

Anatomically correct models of the human pelvicaliceal system, kidney, and relevant adjacent structures were created using polyvinyl alcohol hydrogels and three-dimensional-printed injection molds. All steps of a PCNL were simulated including percutaneous renal access, nephroscopy, and lithotripsy. Five experts (>100 caseload) and 10 novices (<20 caseload) from both urology (full procedure) and interventional radiology (access only) departments completed the simulation. Face and content validity were calculated using model ratings for similarity to the real procedure and usefulness as a training tool. Differences in performance among groups with various levels of experience using clinically relevant procedural metrics were used to calculate construct validity.

Results:

The model was determined to have an excellent face and content validity with an average score of 4.5/5.0 and 4.6/5.0, respectively. There were significant differences between novice and expert operative metrics including mean fluoroscopy time, the number of percutaneous access attempts, and number of times the needle was repositioned. Experts achieved better stone clearance with fewer procedural complications.

Conclusions:

We demonstrated the face, content, and construct validity of an inanimate, full task trainer for PCNL. Construct validity between experts and novices was demonstrated using incorporated procedural metrics, which permitted the accurate assessment of performance. While hands-on training under supervision remains an integral part of any residency, this full-immersion simulation provides a comprehensive tool for surgical skills development and evaluation before hands-on exposure.

Introduction

Percutaneous Nephrolithotomy (PCNL) is the standard procedure for management of large renal calculi and is considered one of the most complex endourology procedures with a significant learning curve.^1

–4 It is proposed that residents at a minimum should perform close to 24 PCNL procedures to obtain proficiency during their training period.¹ This steep learning curve is the result of a complication-prone procedure, especially when performed with limited hands-on training. Poor needle placement during percutaneous renal access (PRA) results in procedure compromise and more so an increased risk of hemorrhagic complications.^5
–7 Notably, it is only after completing 45 PCNL cases that complication rates are significantly reduced.³ Given the importance of PCNL within the endourological armamentarium, it is important to ensure adequate training for this procedure.

Assuming the caseload of urology residents over their training period, an academic center with one new resident/year should have a throughput of ∼500 cases of renal stone disease per year to ensure adequate training, a number beyond the reach of most centers.³ Recent changes to educational surgical curricula imposed by increasing standards for patient safety, limited trainee work hours, and increasing budgetary constraints has lead to a gap between actual case load and that required for adequate training, resulting in only a fraction urology residents graduating competent in performing PCNL.⁸

To improve resident training, multiple PCNL, simulators have been explored including virtual reality (VR) systems, animal models, and benchtop nonbiological simulators. However, to effectively bridge the training gap, simulation-based training must evolve from simply practicing basic tasks into an accurate re-creation of the operating room experience especially for complex procedures.⁹ We present a high fidelity, realistic, nonbiological, single use, full procedure, benchtop PCNL simulator in which face, content, and construct validity are demonstrated.

Materials and Methods

Model creation

This Simulated Inanimate Model (SIM) is composed mainly of a hydrogel polymer (polyvinyl alcohol [PVA]). Using multiple patients' imaging as a reference, an idealized pelvicaliceal system (PCS) and staghorn calculus were created using Meshmixer (Version 3.0; Autodesk, San Rafael, CA). Virtual representations were then converted into injection molds, three-dimensional printed, and filled with PVA to create individual organs. The model included a kidney harboring a functioning PCS with a staghorn stone, stented ureter, thoracolumbar spine (with 11th and 12th ribs), surrounding adipose tissue, and all layers of the posterior abdominal wall assembled in anatomical fashion (Fig. 1). The different mechanical properties of each component were modeled to mimic human tissue by altering polymer concentrations and degree of cross-linking, based on expert feedback after field-testing, redesign, and retesting of several prototypes.

FIG. 1.

SIM-PCNL model. (A) Left—External view of the model with an access sheath in place following a simulation, Right—Skin retracted showing bony landmarks. (B) Functional pelvicaliceal system harboring a staghorn stone. (C) Exposure of the fascia layer with access sheath traversing the lower pole of the kidney. (D) Removal of residual stones through pyelotomy following the completion of a simulation. PCNL = percutaneous nephrolithotomy; SIM = Simulated Inanimate Model.

Post hoc biomechanical testing of percutaneous needle insertion forces, modeled after the work by Poniatowski and colleagues¹⁰ was performed to demonstrate tissue mechanical properties during PRA. The model was placed on a horizontal platform below an 18-gauge, diamond-tip needle (Cook Urological, Spencer, IN), secured to an Instron 8841 servohydraulic materials testing machine (Instron Corp., Norwood, MA). Data acquisition and testing protocol were handled by Bluehill software (Instron Corp). Five trials were completed with the tip of the needle moving perpendicular to the surface of the model in the known trajectory of the PCS. The resulting force data were compared to human cadaveric tissue samples from existing literature.¹⁰ Our model falls within one standard deviation of cadaveric tissue and accurately recreates the forces experienced by the surgeon when passing a needle though the abdominal wall during PRA (Table 1).

Table 1.

Biomechanical Model Testing

	Fresh cadaver # 1 (0.16 cm/sec)	Fresh cadaver # 2 (0.48 cm/sec)	Simulated inanimate model (0.05 cm/sec)
Average maximum force through abdominal layers	4.34	1	2.6
Average magnitude of the force decrease after achieving the maximum force	10.9	3	9

Key force measurements of needle insertion force vs needle tip displacement for the first 5 cm of motion toward the PCS. Cadaver data are included from literature.¹⁰

PCS = pelvicaliceal system.

Model validation

After approval by the Institutional Review Board at the University of Rochester, 5 experts (100+ PCNL cases) and 10 novices (< 20 PCNL cases) from Urology (full procedural simulation) and interventional radiology (IR) departments (access and dilatation only) were recruited. All participants were requested to treat the simulation as a live case after reviewing procedural steps, evaluation metrics, and possible complications. Objective metrics of performance, including fluoroscopy time, attempts for percutaneous access, needle repositioning, intraoperative complications, and stone clearance (Urology only) were recorded. Attempts for percutaneous access were when the needle was completely withdrawn and access reattempted, needle repositioning was when the needed was readjusted within the same tract. Stone clearance was evaluated by measuring residual fragments (mm), extracted from the model postsimulation (Fig. 1D). Complications were any deviation from the normal course of the operative procedure. Complications that necessitated premature conclusion of the simulation were classified as major, while those that did not preclude completion of the simulation were considered minor.

To enhance the educational benefit a debrief session using the nonjudgmental approach to debriefing¹¹ discussing errors, potential causes, and remediation plans was completed after each simulation by an endourology trained urologist. Two questionnaires with questions rated on a 5-point Likert scale were completed to evaluate face validity (realism) and content validity (educational effectiveness) of the simulator on completion. For construct validity a more objective approach was utilized, where the above-mentioned metrics of performance were used to establish the ability of the model to differentiate between varying levels of expertise. All data were collected and stored in a de-identified manor using Excel (2011, Microsoft, Redmond, WA). Groups were compared using univariate chi-squared and Student's t-tests using SPSS (Version 19.0; IBM, Armonk, NY). Exact Wilcoxon rank-sum test (for numerical variables) and Fisher's exact (for categorical outcomes) was used was used for statistical comparison of the groups.

Results

A total of 15 participants, including 5 experts (1 IR, 4 urology) and 10 novices (2 IR, 8 urology) completed the a full-immersion simulation replicating all salient steps of the procedure, using standard equipment within a hybrid operating room (Fig. 2) (a video demonstrating this technique is available online at www.liebertpub.com/end). Face (realism) and content (educational effectiveness) validity of SIM were both deemed to be excellent, with an average score of 4.25/5 and 4.75/5, respectively (Figs. 3 and 4). Experts rated relevant anatomical landmarks (4.85) and fluoroscopic realism (4.71) of the model to be the most realistic. Procedural steps that replicated the live procedure with the greatest realism and educational impact included percutaneous access and dilatation, lithotripsy, and nephroscopy respectively (4.57, 4.33, and 4.25). Radiological appearance via ultrasound was rated the least for realism (3.0). The greatest education impact of the model was its use as a tool for teaching and refining technical skills (4.71) and evaluation of performance (4.57). Expert participants strongly agreed that skilled surgeons should perform well on the model (4.57) while novices unanimously agreed that practicing using this model should precede live surgery (5.0) (Figs. 3 and 4).

FIG. 2.

Full-immersion PCNL simulation. Fluoroscopic imaging of PCNL model (A—KUB and B—retrograde study). (C) Side by side image of external and fluoroscopic views during percutaneous access. (D) Endoscopic access following tract dilatation, the stone can be visualized at the tip of the access sheath. KUB = kidney, ureter, and bladder radiograph.

FIG. 3.

Face Validity Assessment, ratings of realism on 5-Point Likert Scale from postsimulation survey (n = 10).

FIG. 4.

Content Validity Assessment, ratings of educational effectiveness on 5-Point Likert Scale from postsimulation survey (n = 10).

Analysis for construct validity demonstrated that experts outperformed novice with less mean fluoroscopy time (p = 0.0007), fewer attempts for percutaneous access (p = 0.006), fewer times of needle repositioning (p = 0.0003), less complications (0.24), and better stone clearance for urologists only (p = 0.58) (Table 2, Fig. 5). Median expert mean fluoroscopy time of 5 minutes (300 seconds) was almost half of that used by the novice group averaging at 10.66 minutes (640 seconds). All experts were able to access the desired calyx in their first attempt with a median of three needle repositioning maneuvers, compared to novices that required a median of three percutaneous attempts and nine needle repositioning maneuvers. During PRA, two PCS perforations and a renal vein injury (only major complication) were reported in novices, with only a single lost access (no safety wire) in the expert group. During nephroscopy and lithotripsy an evident PCS tear during stone extraction and infundibula injury during sheath manipulation were recorded in novices, while a single irretrievable migrated ureteric stone was reported in the expert group.

FIG. 5.

Comparison of expert and novice simulation procedure metrics, chi-squared test (all metrics demonstrated statistical significance p < 0.05).

Table 2.

Objective Measures of Performance Used to Establish Construct Validity

	Mean fluoroscopy time (seconds)	Number of attempts for PC access	Number of times needle repositioned	Residual stones (mm) (urologists only)	Complications
Expert	300	1	3	10	2
Novice	640	3	9	13	5
p	0.0007	0.006	0.0003	0.58	0.24

p-Values were statistically significant.

Discussion

Various PCNL simulations have been described, which for the purpose of this article will be grouped into three categories: VR simulators, animal models, and nonbiological benchtop models.¹² The PERC Mentor (Simbionix, Ltd, Israel) is a VR simulator designed to train PRA under fluoroscopy, including X-ray orientation, puncture, and guidewire manipulations. A great advantage of VR simulators is that, apart from various scenarios of different levels for repetitive practice, its implementation is relatively uncomplicated without the need for fluoroscopic facilities or ethical concerns but still remains a less appealing alternative to hands-on operative training due to fidelity differences and limited realism.^13,14 As all VR simulators, a high purchase cost ($60,000–70,000), difficulties in transportation, and software malfunction^15,16 preclude widespread use.

Despite limited dissemination, the PERC Mentor remains the most validated access only PCNL simulator¹⁷ but without the ability to practice an entire PCNL procedure. Sixty-eight novices completed test-retest reliability on the PERC Mentor to establish face, content, and construct validity,² where participants training on a PRA scenario demonstrated statistically significant improvement on repeat testing using objective (simulator metrics) and subjective measures (modified endourology Global Rating Scale [GRS]) compared to a no training group. Although no experts were recruited in the study face and content validity were established through the ability of subjects to successfully in establish PRA after a short training session. Finally, convergent validity (a subtype of construct validity) was established through the correlation of subjective (GRS) and objective performance data. Other noteworthy series demonstrated the transfer of skills acquired on the PERC Mentor to a successful PRA in a porcine model within a nonclinical training setting.¹⁸

Several bovine and porcine animal models for practicing PCNL have also been described in the literature. Before this hydrogel model, animals were the only platform for full procedural training despite absence of stone pathology.¹⁹ While, animal models provide a high fidelity platform, their use for regular training is hampered by enduring ethical concerns, maintenance, and disposal costs.^1,2 No specific reports exist validating animal models for acquisition of PCNL related skills. However, Mishra and colleagues²⁰ compared a live porcine model and the PERC Mentor during PRA in 24 experts who rated both models for realism and educational usefulness. The porcine model showed significant superiority to the PERC Mentor in overall realism, tactile feedback, fluoroscopic realism, and complications encountered but inferior to the PERC Mentor in flank orientation and ability for repetitive performance. In terms of usefulness ratings, the overall usefulness and utilization as a training tool were similar; however, the porcine model was rated as a better assessment tool (p < 0.001). This study affirmed the realism and usefulness of VR and animal models as an adjunct for PRA training, but discounted the PERC Mentor for overall realism and tactile feedback.

Nonbiological benchtop models obviate the financial, ethical, and organizational concerns encountered with VR or animal models. All PCNL simulators in this category remain limited to PRA training. The model described in Zhang and colleagues,²¹ is primarily manufactured from silicone and represents a generic dilated PCS. It demonstrated excellent face and content validity despite lacking important anatomical landmarks (spine or ribs). A significant performance difference among 9 experts and 30 novices was established with experts demonstrating a shorter time, reduced X-ray exposure, and fewer attempts for puncture. Furthermore, the usefulness of this model for acquisition of PRA skills in novices was confirmed by a significant reduction in procedural metrics on repeating the same scenario after 24 hours. Another benchtop nonbiological model described by Veneziano and colleagues²² consists of two-parts; a functional C-arm and replaceable silicone flank model (a transparent silicone cast of the ureter, calyces, and renal pelvis with an overlay of the ribs, limited to 30 PRA repetitions). The greatest benefit this model offers is its built-in fluoroscopy that is devoid of radiation hazards. Currently, no clinical validation data exist for this model and remains an access-only PCNL simulation. A multilayer synthetic modification of the original model known as the SimPORTAL fluoro-less C-arm trainer was mechanically tested for needle insertion forces used during PRA. The overall slope for all force vs displacement of the needle in the multilayered model resembled testing in two fresh human cadaveric flanks.¹⁰ It remains the only published PCNL trainer with biomechanical tissue validation.

SIM-PCNL is a high fidelity, full procedure, simulation platform that demonstrates excellent face, content, and construct validity with representative biomechanical properties and radiological appearance on ultrasound and fluoroscopy. It offers a full immersion platform similar to live procedures and permits practice of all salient aspect of the procedure including PRA, complete tract dilatation, nephroscope navigation, and lithotripsy. In addition, construct validity was established using clinically relevant procedural metrics including stone clearance, which is necessarily not the best metric for assessing PCNL learning curve but remains one of the most clinically relevant metrics for patients based on a study by Tanriverdi and colleagues.²³

However, despite demonstrating construct validity through significant differences using clinically relevant metrics, the number of participants was limited by our single institution design for this study. In addition, the nonbiological hydrogel utilized in this model allows for a full procedural simulation but limits its durability to a single use due to tract dilatation and lithotripsy. However, if used as a PRA model, similar to its other counterparts,^21,22 it can be utilized for several training sessions. In this study the extent to which the performance on the simulator can translate into actual surgical proficiency (predictive validity) was not established. Predicative validity remains scarce, as demonstrated in a recent simulation training literature review for stone surgery.¹⁶ Likewise, only two models described have been validated beyond face validity: the PERC-Mentor,¹⁸ and model developed by Zhang and colleagues,²¹ both of which are limited to PRA.

Furthermore, to address the forthcomings of single use and lack of predictive validity we have transitioned to a patient-specific simulation platform. Utilizing the same technique virtual representations are based on individualized patient CT scans (DICOM data), thus creating patient-specific models. When used during rehearsal before live surgery on complex PCNL cases, downstream improvements in patient outcomes was demonstrated²⁴ proving that practice on the simulator is transferrable to a live setting with enhanced performance and less errors (i.e., predictive validity) compared to patients without a preoperative rehearsal. This was also the first study where both urologists and interventional radiologists participated in validation of a PCNL simulator. IR participation has been neglected despite their major role in obtaining percutaneous access in the United States, even though evidence demonstrates significant reductions in stone-free rates in urology-accessed cases.²⁵

Conclusions

This study established the face, content, and construct validity of an inanimate, polymer based, full task PCNL trainer that extends beyond PRA to include nephroscopy and lithotripsy simulations. At times where regulations in work schedules directed at health and safety considerations for both doctors and patients have resulted in reduced operating theater attendance with a concomitant reduction in exposure to clinical situations and in the requisite number of operative procedures. Full-immersion simulations that can accurately re-create an operating room experience could effectively bridge the gap in resident caseload and serve as an addition platform for surgical skills development and evaluation in PCNL before hands-on exposure.

Footnotes

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This article was supported by funding from NIH grant number UL1TR002001.

Abbreviations Used

References

Allen

, O'Brien

, Tiptaft

, Glass

. Defining the learning curve for percutaneous nephrolithotomy. J Endourol, 2005; 19:279–282.

de la Rosette

, Laguna

, Rassweiler

, Conort

. Training in percutaneous nephrolithotomy—A critical review. Eur Urol, 2008; 54:994–1001.

Jang

, Choi

, Yang

, Han

. The learning curve for flank percutaneous nephrolithotomy for kidney calculi: A single surgeon's experience. Korean J Urol, 2011; 52:284–288.

Kallidonis

, Kyriazis

, Vasilas

, et al. Modular training for percutaneous nephrolithotripsy: The safe way to go. Arab J Urol, 2015; 13:270–276.

, Stoianovici

, Jarrett

, et al. Robotic percutaneous access to the kidney: Comparison with standard manual access. J Endourol, 2002; 16:471–475.

Watterson

, Soon

, Jana

. Access related complications during percutaneous nephrolithotomy: Urology versus radiology at a single academic institution. J Urol, 2006; 176:142–145.

Stern

, Zeltser

, Pearle

. Percutaneous renal access simulators. J Endourol, 2007; 21:270–273.

Christian

Training in percutaneous nephrolithotomy—A critical review. Eur Urol, 2008; 54:994–1001.

Gallagher

, Henn

Simulation fidelity: More than experience and mere repetition?. Medicine Meets Virtual Reality 21 Westwood

J.D.

10.

Poniatowski

, Somani

, Veneziano

, McAdams

, Sweet

. Characterizing and simulating needle insertion forces for percutaneous renal access. J Endourol, 2016; 30:1049–1055.

11.

Rudolph

, Simon

, Dufresne

, Raemer

. There's no such thing as “nonjudgmental” debriefing: A theory and method for debriefing with good judgment. Simul Healthc, 2006; 1:49–55.

12.

Noureldin

, Elkoushy

, Fahmy

, Carrier

, Elhilali

, Andonian

. Assessment of photoselective vaporization of prostate skills during urology objective structured clinical examinations (OSCE). Can Urol Assoc J, 2015; 9:e61–e66.

13.

Leblanc

, Senagore

, Ellis

, et al. Hand-assisted laparoscopic sigmoid colectomy skills acquisition: Augmented reality simulator versus human cadaver training models. J Surg Educ, 2010; 67:200–204.

14.

Knudsen

, Matsumoto

, Chew

, Johnson

, Margulis

, Cadeddu

, et al. A randomized, controlled, prospective study validating the acquisition of percutaneous renal collecting system accessskills using a computer based hybrid virtual reality surgical simulator: Phase I. J Urol, 2006; 176:2173–2178.

15.

Noureldin

, Aloosh

, Andonian

How to use virtual- reality simulators to assess competency in basic en-dourologic and robotic skills?. J Endourol, 2017; 31:S10–S19.

16.

Sarmah

, Voss

, Ho

, Veneziano

, Somani

. Low vs. high fidelity: The importance of ‘realism’ in the simulation of a stone treatment procedure. Curr Opin Urol, 2017; 27:316–322.

17.

Mishra

, Jagtap

, Sabnis

, Desai

. Training in percutaneous nephrolithotomy. Curr Opin Urol, 2013; 23:147–151.

18.

Mishra

, Kurien

, Patel

, Patil

, Ganpule

, Muthu

, Sabnis

, Desai

. Validation of virtual reality simulation for percutaneous renal access training. J Endourol, 2010; 24:635–640.

19.

Knoll

Training in percutaneous nephrolithotomy—A critical review.. Eur Urol, 2008; 54:994–1001.

20.

Mishra

, Kurien

, Ganpule

, Muthu

, Sabnis

, Desai

. Percutaneous renal access training: Content validation comparison between a live porcine and a virtual reality (VR) simulation model. BJU Int, 2010; 106:1753–1756.

21.

Zhang

, Yu

, Jin

, Li

, Na

. Validation of a novel non-biological bench model for the training of percutaneous renal access. Int Braz J Urol, 2014; 40:87–92.

22.

Veneziano

, Smith

, Reihsen

, Speich

, Sweet

. The SimPORTAL fluoro-less C-arm trainer: An innovative device for percutaneous kidney access. J Endourol, 2015; 29:240–245.

23.

Tanriverdi

, Boylu

, Kendirci

, Kadihasanoglu

, Horasanli

, Miroglu

. The learning curve in the training of percutaneous nephrolithotomy. Eur Urol, 2007; 52:206–211.

24.

Stone

, Bonamico

, Erturk

, Ghazi

Improvement of surgical outcomes after preoperative rehearsal using 3D printed patient specific simulation for percutaneous nephrolithotomy. Presented at the AUA, 2017, Boston, MA.

25.

Tomaszewski

, Ortiz

, Gayed

, Smaldone

, Jackman

, Averch

. Renal access by urologist or radiologists during percutaneous nephrolithotomy. J Endourol, 2010; 24:1733–1737.