Initial Content Validation Results of a New Simulation Model for Flexible Ureteroscopy: The Key-Box

Abstract

Purpose:

We sought to test the content validity of a new training model for flexible ureteroscopy: the Key-Box.

Material and Methods:

Sixteen medical students were randomized to undergo a 10-day training consisting of performing 10 different exercises aimed at learning specific movements with the flexible ureteroscope, and how to catch and release stones with a nitinol basket using the Key-Box (n = 8 students in the training group, n = 8 students in the nontraining control group). Subsequently, an expert endourologist (O.T.) blindly assessed skills acquired by the whole cohort of students through two exercises on ureteroscope manipulation and one exercise on stone capture selected among those used for the training. A performance scale (1–5) assessing different steps of the procedure was used to evaluate each student. Time to complete the exercises was measured. Mann–Whitney Rank Sum test was used for comparisons between the two groups.

Results:

Mean scores obtained by trained students were significantly higher compared with those obtained by nontrained students (all p < 0.001). All trained students were able to complete the two exercises on ureteroscope manipulation within 3 minutes, whereas two students (25%) were not able to finish the exercise on stone capture. Conversely, four (50%) and six (75%) nontrained students were not able to finish one out of the two exercises on ureteroscope manipulation and the exercise on stone capture, respectively. The mean time to complete the three exercises was 76.3, 69.9, and 107 and 172.5, 137.9, and 168 seconds in the trained and nontrained groups, respectively (all p < 0.001).

Conclusions:

The K-Box^® seems to be a valid easy-to-use training model for initiating novel endoscopists to flexible ureteroscopy.

Introduction

The trend in surgical management of upper tract urinary calculi over the past two decades has been dramatically shifting toward ureteroscopy with a consensual reduction of extra-corporeal shock wave lithotripsy.¹

The need for offering a high-level healthcare service, whatever the level of expertise of the surgeon as well as for reducing operating room costs and time, waiting lists, and medico-legal issues due to medical error,^2,3 has led to the development of surgical training models as convenient tools with high educational impact for acquiring technical skills before undertaking endoscopic procedures on patients.⁴

A great variety of training simulators has been described for ureteroscopy, ranging from computer-assisted virtual reality to the home-made bench model, each of them with different peculiarities, prices, and proven efficacy.⁵

For instance, the past-generation virtual-reality high-fidelity ureteroscopy simulator seems to improve surgical performance and may help shorten the learning curve early in training of novice endoscopists, although high prices may limit the distribution and widespread introduction of such a device into a training curriculum of an academic center.^6,7 Moreover, there is no clear evidence regarding the most cost-effective training strategy.^8,9

In this context, we sought to evaluate the content validity¹⁰ of a new low-fidelity bench model, the Key-Box (K-Box^®), by conducting a randomized trial on medical students with no previous experience on flexible ureteroscopy.

Specifically, we assessed technical skills acquisition after a didactic teaching with instructional video and an intensive 10-day specific training with the K-Box or didactic teaching alone to judge the appropriateness of this simulator as an effective training modality.

Materials and Methods

The Key-Box

The Key-Box (K-Box; Porgès-Coloplast, France) is a new-generation portable bench-training training model for flexible ureteroscopy composed of four different independent boxes¹¹ (Fig. 1). Each box has three entries and three exits on each side, allowing the access of the ureteroscope to three different spaces (Fig. 2A). Each box is provided with flaps, which remain closed, forcing the user to watch what he or she is doing on the screen without direct visual inspection. However, these flaps can be opened if assistance is required, allowing the user to roughly simulate real-life radiological control (Fig. 2B). Each space of the K-Box does not reproduce the exact anatomy of the renal collecting system but creates an environment that forces the user handling the flexible ureteroscope to make the specific movements required when performing a flexible ureteroscopy in humans (Fig. 3).

FIG. 1.

The four different independent boxes composing the K-Box.

FIG. 2.

Axial view of the K-Box, showing the three different entries that allow the access to three different spaces (A) and the fourth box with the left flap opened, allowing to have a look at the cavities inside (B).

FIG. 3.

Main movements required to explore the environment with a flexible ureteroscope: upward or downward deflection of the tip of the ureteroscope to explore the vertical axis (A), pronation and supination of the wrist handling the ureteroscope to explore along the horizontal axis (B), and pushing and pulling the shaft of the ureteroscope while keeping straight the tip of the ureteroscope to explore the space forward or backward (C).

Training curriculum

Sixteen medical students (n = 9 women and n = 7 men, aged 22–23 years) were identified on a volunteer basis to receive didactic teaching with a theoretical lesson and a 10-minute video on practical demonstration on the manipulation of a flexible ureteroscope (Flex-XC; Karl Storz) and a nitinol basket (1.9F ZeroTip; Boston Scientific) for stone retrieval by using the K-Box trainer.

At baseline, all students completed the specific questionnaire aimed at assessing the basic knowledge of endourological procedures and endoscopic instruments (Fig. 4).

FIG. 4.

Baseline questionnaire administered before students' randomization.

Subsequently, they were randomly divided into two groups: The first group (group A, n = 8 students) underwent a 10-day training consisting of personally repeating different exercises aimed at learning specific movements with the flexible ureteroscope (n = 5 days, 10 minutes per exercise per day) and how to catch and release stones with a nitinol basket (n = 5 days, 10 minutes per exercise per day) by using the K-Box trainer. An experienced endourologist followed students during each training session, instructing them on how to acquire specific skills required to perform various exercises. The second group (group B, n = 8 students) served as a nontrained control group and received only didactic teaching.

Students' assessment

After a wash-out period of 2 weeks (at the end of the training), a further video demonstration was offered to all the students to review the manipulation of the flexible ureteroscope and the nitinol baskets inside the K-Box. Finally, an expert endourologist (O.T.) blindly assessed the whole cohort of students through a final examination consisting of performing two exercises on ureteroscope manipulation and one exercise on stone capture using the K-Box selected by the examiner among those used for the training. A performance scale was created by relying on the global rating scale described by Matsumoto and colleagues aimed at assessing the hands-on training with the high-fidelity genitourinary bench model (Uro-Scopic Trainer)¹² and used to evaluate each student in terms of respect for surrounding environment, hand movements, handling of endoscope, finesse in the procedure, knowledge of the procedure, use of assistants, and basket handling (Table 1).

Table 1.

Performance Scale Used to Evaluate Each Student on the Basis of Specific Parameters

	Score
Parameters	1	2	3	4	5
Respect for surrounding environment	Scope frequently pushed into the artificial channel wall		Scope occasionally pushed into the artificial channel wall		Careful handling of the scope with no contact with the artificial channel wall
Hand movements	Many unnecessary moves		Some unnecessary moves		No unnecessary moves
Handling of endoscope	Frequently had scope pointing away from the center. Scope poorly aligned during procedure		Had scope centered for the most part. Better use of scope angle during procedure		Scope always centered and set at a good angle throughout procedure.
Finesse in the procedure	Frequently stopped or need advice or assistance from examiner		Ability to think forward with relatively steady progression of procedure		Obviously planned procedure from beginning to end with fluid motion
Knowledge of the procedure	Deficient knowledge, needed specific instructions at most operative steps		Knew all important aspects of operation		Demonstrated familiarity with all aspects of operation
Use of assistants	Failed to have assistants help with stone basketing		Appropriate use of assistants most of the time		Strategically used assistants to the best advantage at all times
Basket handling	Needed to repeatedly attempt basketing of the stone		Able to basket stone within first few trials. Occasional awkward maneuver.		Able to basket with fluid motion and no awkwardness.

A specific score ranging from 1 to 5 was assigned to every student for each specific domain. Time to complete the exercise was also measured. The success rate was defined as the ability of completing these exercises within 3 minutes. Descriptive statistics were used to detail the results of the two groups. Wilcoxon–Mann–Whitney Rank Sum test was used for comparisons between the two groups, with a significance level set at p < 0.05.

Results

The results of the questionnaire aimed at providing a basic evaluation of the whole cohort showed that six and five students in the trained and nontrained groups, respectively, had already heard about endourology but none of the students had ever assisted an endourological procedure nor had any practical skills in this field.

After specific training, all students (n = 8) were able to complete (within 3 minutes) the two exercises on ureteroscope manipulation, whereas two students (25%) were not able to finish the exercise aimed at capturing the stone with a nitinol basket. Conversely, four (50%) and six (75%) nontrained students were not able to complete (within 3 minutes) one out of the two exercises on ureteroscope manipulation and the exercise on stone capture, respectively.

At final examination, mean scores (±standard error of the mean) were significantly higher in the trained group compared with the nontrained group in terms of respect for surrounding environment, hand movements, handling of endoscope, finesse in the procedure, knowledge of the procedure, use of assistants, and basket handling (Table 2, all p < 0.001). Indeed, trained students learnt properly how to handle the ureteroscope to explore the K-Box cavities with no unnecessary hands movements (Supplementary Video S1; Supplementary Data are available online at www.liebertpub.com/end) and how to use the assistant for basket handling with only a few awkward maneuvers (Supplementary Video S2).

Table 2.

Mean Score (±SEM) at Final Examination of Students in the Trained and Nontrained Groups

Parameters used for students' assessment	Group A (trained students)	Group B (nontrained students)	p
Ureteroscope manipulation	Mean score ± SEM	Mean score ± SEM	<0.001
Respect for surrounding environment	4.44 ± 0.24	1.87 ± 0.28	<0.001
Hand movements	4.44 ± 0.24	2.00 ± 0.25	<0.001
Handling of endoscope	4.55 ± 0.17	1.87 ± 0.21	<0.001
Finesse in the procedure	4.22 ± 0.15	1.87 ± 0.12	<0.001
Knowledge of the procedure	4.66 ± 0.17	1.75 ± 0.15	<0.001
Use of assistants	4.44 ± 0.19	1.83 ± 0.21	<0.001
Basket handling	4.33 ± 0.16	1.62 ± 0.25	<0.001

SEM = standard error of the mean.

The mean time (±standard error of the mean) to complete each exercise was 76.3 (±11.6), 69.9 (±9.9), and 107 (±22.5) seconds in the trained group compared with 172.5 (±6.3), 137.9 (±9.4), and 168 (±17.8) seconds in the nontrained group (all p < 0.001).

Discussion

Halstedian concepts of surgical apprenticeship introduced in the early 1950s, based on the principle of “see one, do one, teach one” and adopted for a long time in the past decades for the urologic endoscopic training of surgical residents, has been largely overcome by the development and introduction of simulators and training models that allow young residents and inexperienced surgeons to become confident with surgical instruments and to acquire technical skills before performing endoscopic procedures on live patients in the operating theatre.^13,14

In this context, the K-Box has been developed as a bench model made of four different boxes that can be assembled in different configurations to let the trainees familiarize with flexible ureteroscope handling as long as basic movements and steps are required during endoscopic procedures of the upper urinary tract.

The main aim of the specific 10-day training used in the current study was to enable students to explore the surrounding space in all the three axes. To get this target, didactic teaching and an instructional video offered to the whole cohort of students before their randomization depicted the movement of upward or downward deflection of the tip of the ureteroscope (to see any object that is placed in a vertical axis compared to the position of the ureterscope, Fig. 3A), pronation and supination of the wrist handling the ureteroscope (to see any object that is placed on a horizontal axis compared with the position of the ureteroscope, Fig. 3B), and pushing and pulling the shaft of ureteroscope while keeping straight the tip of the ureteroscope (to see the space forward or backward, Fig. 3C).

The results of the current study showed that students underwent a 10-day specific training and obtained higher scores at final evaluation in all the domains assessing the maneuverability of the ureteroscope compared with nontrained students (all p < 0.001, Table 2). Moreover, grasping and releasing the stone with the nitinol basket was significantly better in the trained group compared with their nontrained counterparts.

The benefit of approaching real-life endoscopic cases with a certain degree of expertise rather than being novice endoscopists has been demonstrated by several authors, demonstrating the role of the learning curve in affecting the success rate, the risk of complication, and endoscope breakage when the instruments were used improperly or by inexperienced users.^15
–17 Moreover, better knowledge of instrument handling and a fluent procedure may not only increase the likelihood of completing the surgery successfully, thus limiting medico-legal concerns, but also can reduce the operative time, with positive implications on waiting lists and cost containment.

In this context, the results of the current study demonstrated that a specific training increases the success rates (100% and 75% of students in the trained group vs 50% and 25% in the control group completed within 3 minutes the two exercises on ureteroscope manipulation and the exercise aimed at using the nitinol basket, respectively) and reduces the time to complete the procedure successfully [76.3 (±11.6), 69.9 (±9.9) and 107 (±22.5) seconds in the trained group compared with 172.5 (±6.3), 137.9 (±9.4), and 168 (±17.8) seconds in the nontrained group, all p < 0.001)] (Supplementary Videos S1 and S2).

Although the K-Box has to be considered as a low-fidelity portable bench training model, since it has not been developed to reproduce accurately the anatomy of the upper urinary tract, but rather to create an environment with different tracks and cavities where the trainees can learn how to manipulate endoscopic instruments and to explore the surrounding space, our findings demonstrate that it achieves the goal for which it was designed, that is, to teach what it is expected to teach.

Generally, it may be argued that surgical training with a high-fidelity or in vivo animal models is the preferred option of the trainees and may be better in terms of face and content validity and have a higher educational value than relying on low-fidelity bench models.^18,19 However, this intuitive belief is not supported by any study comparing the two different training modalities.

Conversely, a randomized trial has demonstrated that hands-on training with a low-fidelity model (consisting of penrose drain, inverted cup, molded latex in portable plastic case, and two embedded straws ∼8 mm in diameter as substitutes for urethra, bladder dome, bladder base, and bilateral ureters, respectively) leads to achieve equal performance as compared with a high-fidelity one (Uro-Scopic Trainer, by Limbs and Things, Bristol, United Kingdom), with no difference in terms of global rating scale, checklist, pass rating, and time needed to complete the task (i.e., to remove a left or right mid ureteral stone),⁷ although the cost was definitely in favor of the low-fidelity model (Canadian $20 vs Canadian $3,700 to produce the low- and to purchase the high-fidelity model, respectively).¹²

Various high-fidelity models (Uro-Scopic Trainer, Scope Trainer, Adult Ureteroscopy Trainer, UroMentor) have been developed and evaluated in terms of face, content, construct, educational impact, and predictive validity,^{8,12,20

–25} thus being at present available to be incorporated into the educational program of academic centers. However, the hypothetical advantage of training on more realistic models or on living animals with the possibility for the trainees to face with respiratory movements, bleeding, and haptic feelings must be balanced either with ethical issues and availability of adequate labs for the in vivo animal models and with the higher costs of purchase and maintenance of certain simulators as compared with the low-fidelity bench models.¹⁴

The K-Box has been designed to let the trainees acquire technical skills while handling endoscopic instruments. Although step-by-step placement of guidewire and access sheath is possible, the K-Box does not guarantee simulation feelings comparable to those offered by the more advanced bench model or virtual-reality trainers. However, this disadvantage has to be weighed against the cost of other more expensive trainers and K-Box seems to be a good alternative to them.

Although our preliminary findings showed the effectiveness of the K-Box for acquiring basic endourological skills, further studies are necessary to evaluate construct, concurrent, and predictive validity of this innovative training model according to specific criteria established in literature^10,26 to demonstrate with an adequate level of evidence its reliability and appropriateness when incorporated into training curricula of novice endoscopists.

Conclusions

Students trained on the K-Box for 10-days achieved higher performance in terms of respect for surrounding environment, hand movements, handling of endoscope, finesse in the procedure, knowledge of the procedure, use of assistants, and basket handling compared with students who received didactic teaching only. The success rate and the time to complete the exercises were in favor of trained students. Further studies are necessary to confirm the reliability and validity of the K-Box as a valid teaching modality for technical skills acquisition of novel endoscopists.

Footnotes

Acknowledgments

The authors want to thank Paola Garavelli and Jeremie Berg for their unconditioned support. Moreover, they acknowledge the Scientific Program of 33rd World Congress of Endourology and SWL held in London, October 2015, where the abstract of the current article first appeared.

Author Disclosure Statement

O.T. is consultant for Olympus, Coloplast, Boston Scientific, Rocamed. Regarding the other authors, no competing financial interests exist.

Abbreviations Used

References

Oberlin

, Flum

, Bachrach

, Matulewicz

, Flury

. Contemporary surgical trends in the management of upper tract calculi. J Urol, 2015; 193:880–884.

Viberg

, Forsberg

, Borowitz

, Molin

. International comparisons of waiting times in health care—Limitations and prospects. Health Policy, 2013; 112:53–61.

Tevlin

, Doherty

, Traynor

. Improving disclosure and management of medical error—An opportunity to transform the surgeons of tomorrow. Surgeon, 2013; 11:338–343.

Schout

, Hendrikx

, Scherpbier

, Bemelmans

. Update on training models in endourology: A qualitative systematic review of the literature between January 1980 and April 2008. Eur Urol, 2008; 54:1247–1261.

Brunckhorst

, Aydin

, Abboudi

, et al. Simulation-based ureteroscopy training: A systematic review. J Surg Educ, 2015; 72:135–143.

Ogan

, Jacomides

, Shulman

, Roehrborn

, Cadeddu

, Pearle

. Virtual ureteroscopy predicts ureteroscopic proficiency of medical students on a cadaver. J Urol, 2004; 172:667–671.

Knoll

, Trojan

, Haecker

, Alken

, Michel

. Validation of computer-based training in ureterorenoscopy. BJU Int, 2005; 95:1276–1279.

Chou

, Abdelshehid

, Clayman

, McDougall

. Comparison of results of virtual-reality simulator and training model for basic ureteroscopy training. J Endourol, 2006; 20:266–271.

Matsumoto

, Hamstra

, Radomski

, Cusimano

. The effect of bench model fidelity on endourological skills: A randomized controlled study. J Urol, 2002; 167:1243–1247.

10.

McDougall

. Validation of surgical simulators. J Endourol, 2007; 21:244–247.

11.

Villa

, Somani

, Sener

, et al. Comprehensive flexible ureteroscopy (FURS) simulator for training in endourology: The K-box model. Cent European J Urol, 2016; 69:118–120.

12.

Matsumoto

, Hamstra

, Radomski

, Cusimano

. A novel approach to endourological training: Training at the Surgical Skills Center. J Urol, 2001; 166:1261–1266.

13.

Carter

. The fruition of Halsted's concept of surgical training. Surgery, 1952; 32:518–527.

14.

Cloutier

, Traxer

. Do high-fidelity training models translate into better skill acquisition for an endourologist?. Curr Opin Urol, 2015; 25:143–152.

15.

Afane

, Olweny

, Bercowsky

, et al. Flexible ureteroscopes: A single center evaluation of the durability and function of the new endoscopes smaller than 9Fr. J Urol, 2000; 164:1164–1168.

16.

Ravintharan

, Lim

, Chng

. Ureterorenoscopy: Factors influencing success. Singapore Med J, 1991; 32:151–153.

17.

Daniels

Jr. , Garnett

, Carter

. Ureteroscopic results and complications: Experience with 130 cases. J Urol, 1988; 139:710–713.

18.

Soria

, Morcillo

, Sanz

, Budia

, Serrano

, Sanchez-Margallo

. Description and validation of realistic and structured endourology training model. Am J Clin Exp Urol, 2014; 2:258–265.

19.

Grober

, Hamstra

, Wanzel

, Reznick

, Matsumoto

, Sidhu

, Jarvi

. The educational impact of bench model fidelity on the acquisition of technical skill: The use of clinically relevant outcome measures. Ann Surg, 2004; 240:374–381.

20.

Brehmer

, Tolley

. Validation of a bench model for endoscopic surgery in the upper urinary tract. Eur Urol, 2002; 42:175–179.

21.

Brehmer

, Swartz

. Training on bench models improves dexterity in ureteroscopy. Eur Urol, 2005; 48:458–463.

22.

White

, Dehaan

, Stephens

, Maes

, Maatman

. Validation of a high fidelity adult ureteroscopy and renoscopy simulator. J Urol, 2010; 183:673–677.

23.

Watterson

, Beiko

, Kuan

, Denstedt

. Randomized prospective blinded study validating acquistion of ureteroscopy skills using computer based virtual reality endourological simulator. J Urol, 2002; 168:1928–1932.

24.

Wilhelm

, Ogan

, Roehrborn

, Cadeddu

, Pearle

. Assessment of basic endoscopic performance using a virtual reality simulator. J Am Coll Surg, 2002; 195:675–681.

25.

Jacomides

, Ogan

, Cadeddu

, Pearle

. Use of a virtual reality simulator for ureteroscopy training. J Urol, 2004; 171:320–323.

26.

Van Nortwick

, Lendvay

, Jensen

, Wright

, Horvath

, Kim

. Methodologies for establishing validity in surgical simulation studies. Surgery, 2010; 147:622–630.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

66.79 MB

77.40 MB