Abstract
Introduction and Objectives:
Recently, different competency-based frameworks have formed the foundations of surgical training and assessment. 1,2 Competency assessment process has been traditionally performed using subjective in-training evaluation reports, filled out by the program directors. 3 However, there is lack of validity and reliability of these subjective reports necessitating the invention of alternative objective assessment tools. 3 Recently, several virtual-reality simulators were introduced, validated, and used for training and assessment of endourologic and robotic skills. 4 –11 The objective of this video was to show the incorporation of virtual-reality simulators into objective structured clinical examinations (OSCEs) to assess competency of urology postgraduate trainees (PGTs) in basic endourologic and robotic skills.
Materials and Methods:
Between December 2012 and December 2015, PGTs from all four urology training programs in Quebec were recruited for assessment of their basic endourologic or robotic skills during five OSCEs using three validated virtual-reality simulators. The GreenLight™ simulator, invented by Dr Robert Sweet (American Medical Systems, Guelph, ON), was used to assess their photo-selective vaporization of the prostate (PVP) skills by vaporizing a 30 g normal prostate during two OSCEs. The PERC Mentor simulator from Simbionix (Cleveland, OH) was used for assessment of their percutaneous access skills by performing task 4 during two OSCEs. Finally, the da Vinci surgical skills simulator (dVSSS) from Intuitive Surgical (Sunnyvale, CA) was used for assessment of their basic robotic skills by performing Pick and Place task and Energy Dissection level 1 task during an OSCE. Assessment was performed during a 20-minute station during each OSCE and competency pass scores were calculated using the norm-referenced method by three experts. 6,10 –12
Results:
For the GreenLight simulator, 37 PGTs participated in the two OSCEs. There were significantly more competent PGTs among those who had previously practiced on the simulator (32.7% vs 10.2%; p = 0.009). When comparing global scores from the first and second OSCEs, there was significant improvement in the number of grams vaporized (2.9 g vs 4.3 g; p = 0.003) and global score (100 vs 165; p = 0.03). Furthermore, there was good correlation between the number of previously performed PVPs and global scores (r = 0.4, p = 0.04). 6 For the PERC Mentor simulator, 26 PGTs were recruited. When compared with the 21 PGTs without practice, all 5 PGTs who had practiced on the simulator were competent (p = 0.03). Competent PGTs performed the task with significantly higher percentage of successful attempts to access renal calices (p < 0.001), shorter fluoroscopy time (9.8 minutes vs 6.5 minutes; p = 0.01), higher PCNL-GRS scores (p < 0.001), and lower complications (p = 0.01). 10 For the dVSSS, nine PGTs were recruited. There was significant difference among PGY levels in terms of competency for the basic robotic skills tested (p = 0.01) since all three (33%) competent PGTs were from the final PGY-5 level. 11
Conclusion:
High-fidelity virtual-reality simulators could be effectively incorporated into OSCEs to assess competency of urology PGTs in basic endourologic and robotic skills. Future international studies should include more complex exercises with larger sample sizes.
Acknowledgments:
This work was supported, in part, by the Fond de Recherche en Santé du Québec (FRSQ) Research-Scholar Grant to Dr. Sero Andonian and by a grant from the Urology Care Foundation Research Scholars Program and the Boston Scientific Corporation, The Endourological Society, and the “Friends of Joe” to Dr. Yasser A. Noureldin and the Canadian Urological Association Scholarship Foundation-Sociéte Internationale d'Urologie (CUASF-SIU) International Scholarship grant to Dr. Yasser A. Noureldin.
No competing financial interests exist
.
Runtime of video: 9 mins 50 secs
