Preliminary Study of Video-Based Pediatric Endoscopic Surgical Skill Assessment Using a Neonatal Esophageal Atresia/Tracheoesophageal Fistula Model

Abstract

Purpose:

Thoracoscopic esophageal atresia/tracheoesophageal fistula (EA/TEF) repair in neonates is technically difficult because of the small working space and fragility of tissues. This study aimed to show the construct validity of the neonatal EA/TEF simulator by video-based assessment of endoscopic suturing skill.

Materials and Methods:

A rapid-prototyped neonatal chest model with an artificial esophagus model similar to the actual neonatal esophagus was developed. Forty pediatric surgeons performed an endoscopic intracorporeal suturing task and a knot-tying task using the model, and a questionnaire survey was administered. Each task was video recorded and assessed using two skill assessment methods (the 29-point checklist method and the error assessment sheet method). The task completion time and the number of manipulations were measured.

Results:

With regard to experience in performing thoracoscopic TEF repair, the experienced surgeons (ESs, ≥3 EA/TEF repair experiences, n = 6) were significantly superior to the inexperienced surgeons (ISs, <3 experiences, n = 34) in all metrics. Upon comparison by the pediatric Endoscopic Surgical Skill Qualification (ESSQ) status, there were no significant differences in all metrics between the ESSQ-qualified (n = 15) and nonqualified (n = 25) surgeons. The qualified ESs (n = 6) were significantly superior to the qualified ISs (n = 9) in all metrics.

Conclusion:

Video-based endoscopic surgical skill assessment using the neonatal EA/TEF simulator could differentiate ESs from ISs. The construct validity of the simulator was demonstrated, and the simulator would be useful especially for practicing thoracoscopic EA/TEF procedures.

Introduction

Since the early 1990s, pediatric endoscopic surgery has become widespread, but some advanced endoscopic surgeries have not become standardized yet.¹ Recently, many advanced endoscopic procedures, such as treatments for choledochal cyst and biliary atresia, have been performed,^2,3 and these advanced procedures require highly advanced surgical skills. In particular, thoracoscopic esophageal atresia/tracheoesophageal fistula (EA/TEF) repair in neonates is technically difficult because of the remarkable difficulties associated with the very small working space and fragility of tissues. Moreover, pediatric surgeons have limited opportunities to perform these endoscopic operations compared with general surgeons due to the small number of pediatric patients.^4,5

To deal with the aforementioned problems, some pediatric-specific endoscopic training tools have been developed. Animal training models such as a porcine or rabbit model are useful, but have ethical problems.^6,7 Azzie et al.⁸ reported the usefulness of a small box trainer that had been designed based on a child's body size. Recently, realistic models simulating pediatric diseases have been reported. Barsness et al.^9–12 developed rapid-prototyped simulators for tasks in EA/TEF repair, infant lobectomy, duodenal atresia repair, and others, incorporating animal tissues and artificial models. Obata et al.¹³ developed a simulator of congenital diaphragmatic hernia repair and validated their model for the assessment of endoscopic surgical skills. Jimbo et al.¹⁴ developed a laparoscopic fundoplication simulator and showed its effectiveness in short-term endoscopic surgical skill training for young pediatric surgeons.

Our group previously developed a rapid-prototyped pediatric chest model of a one-year-old patient and performed a video-based skill assessment of endoscopic suturing in the model, demonstrating that the model was useful to distinguish experienced pediatric surgeons.^15,16 Based on this achievement, we developed a neonatal EA/TEF simulator. The aim of this study was to show the construct validity of the neonatal EA/TEF model by video-based assessment of endoscopic suturing skill.

Materials and Methods

The protocol of this study was approved by the Ethics Committee of the Graduate School of Medicine and Faculty of Medicine, The University of Tokyo (No. 10333). Examinees were well briefed on the experiment, and written informed consent was obtained from all examinees.

Neonatal EA/TEF model and experimental setups

A three-dimensional, rapid-prototyped neonatal ribcage was developed using a set of computed tomography data of a neonatal patient. The neonatal ribcage was arranged in a near-prone position with the right side slightly elevated and covered with a 10-mm synthetic sheet made of elastomer (Fig. 1a). An artificial esophagus model (WetLab Co.), which was made of polyvinyl alcohol and was similar in size and texture to the actual neonatal esophagus, was placed in the ribcage (Fig. 1b). The esophagus model was fixed in the ribcage, and a cut was made at the level of the third thoracic vertebra to simulate esophageal atresia. A 5-mm port for a thoracoscope was inserted in the fifth intercostal space on the caudal side of the angulus inferior scapulae, and a 5-mm 30-degree endoscope was inserted into the port and fixed using a scope holder. Two 3-mm ports were inserted in the third intercostal space near the axillary region and the fifth intercostal space of the dorsal area for insertion of a needle driver and forceps, respectively (Fig. 1a).

FIG. 1.

Neonatal EA/TEF model setup (a) A three-dimensional, rapid-prototyped neonatal ribcage was covered with a 10-mm synthetic sheet and three trocars were inserted into the thoracic cavity. (b) An artificial esophagus model, which was made of polyvinyl alcohol, with the aorta and heart model was placed in the ribcage. A semicircular cut was made in the tube representing the esophagus at the level of the third thoracic vertebra to simulate esophageal atresia. EA/TEF, esophageal atresia/tracheoesophageal fistula.

Experiment

Pediatric surgeons who are able to perform endoscopic intracorporeal suturing and knot tying in a commercial box trainer of an adult size were recruited at a national pediatric conference, at an official training seminar of pediatric endoscopic surgery organized by the Japan Society for Endoscopic Surgery (JSES), and at some children's hospitals.

At the beginning of the experiment, the endoscope, forceps, and needle driver were already inserted into the respective ports in the neonatal EA/TEF simulator. The thread was located inside the ribcage model of the simulator. Using the instruments in the simulator, the surgeon was asked to grab the thread and close the open cut of the esophagus model endoscopically, performing a single suture, followed by three times of intracorporeal knot tying. The examinees were instructed to perform the task accurately, safely, and fast, using the intracorporeal slip knot technique if possible. A 5-0 VICRYL suture with a 13-mm 1/2 circle needle (J433H; Ethicon Endo-Surgery) whose thread was cut at 60 mm was used. The artificial esophagus is disposable and was replaced with a new one for each trial. When a subject failed to finish the task (e.g., because a thread snapped or the esophagus model was torn), the data were invalidated, and the examinee was instructed to restart the task from the beginning. Each task was video recorded, and the task completion time and the number of manipulations were measured.

All examinees were categorized using the criteria below:

A. Experienced Surgeons (ESs) or Inexperienced Surgeons (ISs): ESs had experience of performing three or more thoracoscopic TEF repairs, while ISs had experience of performing less than three or no thoracoscopic TEF repair.

B. Endoscopic Surgical Skill Qualification (ESSQ)-qualified surgeons or nonqualified surgeons: The pediatric ESSQ system was developed by the JSES in 2008 to guarantee the ability of surgeons to complete common pediatric laparoscopic procedures.⁴ Eligible applicants are required to submit a video showing an endoscopic view of an entire pediatric laparoscopic fundoplication to obtain this qualification, and each video is evaluated by two expert pediatric endoscopic surgeons. No experience in advanced neonatal surgeries is required.

Skills assessment

Two blinded pediatric surgeons assessed the recorded videos using two evaluation methods reported previously: the 29-point checklist method and the suturing error score sheet method.^16–20 The 29-point checklist method was first reported by Moorthy et al.¹⁷ and the checklist consisted of 29 items in six categories. Each item was scored as 1 or 0, and the total score and subtotal scores of each category were calculated for each examinee. The suturing error score sheet method was first reported by Sickle et al.²⁰ We revised the definitions of the errors in the suturing error score sheet in our previous article.¹⁶ Each of the nine subtasks (e.g., First bite, Second bite, First throw, and Second throw) was evaluated using ten criteria, including Missed grasp, Needle out of view, and Missed loop. For example, Missed grasp was defined as the jaws of the instrument being opened and closed without retaining the desired target (suture/tissue/needle), and Needle out of view was defined as a grasped needle being completely out of view, but a grasped suture with a hanging needle did not count. Multiple errors in a subtask were counted as one error. The total error count and the subtotal error counts in each category were measured for each examinee. The task completion time and the number of needle manipulations were measured from the videos by one reviewer.

Questionnaire survey

All examinees were asked about their experience in thoracoscopic EA/TEF repair and their qualification status in the ESSQ system. Thereafter, the examinees were classified as ESs or ISs and ESSQ qualified or nonqualified. For ESs, an additional questionnaire survey was conducted, which asked them to rate the simulator. The additional questionnaire consisted of 19 items on a 5-point scale in six categories, whose original version was proposed by Barsness et al.⁹

Statistical analyses

All data are expressed as median (interquartile range) values. The results were compared by the Wilcoxon rank sum test between the groups in each setup. To determine the inter-rater reliability of the checklist score method and the error score method, Cronbach's alpha coefficient was used. After confirmation of the inter-rater reliability of the methods, the averages of the two observers' scores were used for assessment. All analyses were performed using the JMP statistical software (SAS Institute, Inc.), and P values of <.05 were considered to be statistically significant.

Results

Skills assessment

Forty pediatric surgeons were enrolled in this study and performed the task in the neonatal EA/TEF model. Two of the 40 participants without experience in thoracoscopic EA/TEF repair (one was an ESSQ-qualified surgeon, and the other was not) failed to complete the task because the esophagus model became torn when excessive force was applied.

Six pediatric surgeons, all of whom were attending surgeons, were classified as ESs. The other 34 pediatric surgeons were classified as ISs; 24 of them were surgical residents or fellows; and the remaining examinees were attending surgeons in general pediatric surgery, but had no experience in performing thoracoscopic EA/TEF repair. The ES group was significantly superior to the IS group in all four metrics (the 29-point checklist method, the suturing error score sheet method, the task completion time, and the number of needle manipulations) (Table 1). The subtotal error scores of Missed grasp and Needle out of view for the ES group were significantly lower than those for the IS group.

Table 1.

Comparison of Suturing Performance Between ESs and ISs

	ESs (n = 6)	ISs (n = 34)	P value
29-Point checklist, points (median [IQR])	24.3 (23.8–24.6)	20.8 (18.0–23.0)	.012
Error score, points (median [IQR])	7.5 (6.9–10.6)	12.3 (8.4–15.0)	.045
Missed grasp, points (median [IQR])	1.0 (0.4–2.5)	2.5 (1.5–3.5)	.028
Needle out of view, points (median [IQR])	0 (0–0)	1.0 (0.4–1.5)	.003
Number of manipulations (median [IQR])	18.5 (16.5–19.3)	28.5 (23.3–40.5)	.001
Task completion time, seconds (median [IQR])	228.0 (193.3–258.5)	368.5 (262.8–475.0)	.015
Clinical experience^a, years (median [IQR])	27.5 (20.5–32.8)	11.5 (7.0–21.0)	.011

ESs: pediatric surgeons with experience in performing thoracoscopic EA/TEF repair with a caseload of three or more.

ISs: pediatric surgeons with either no experience in performing thoracoscopic EA/TEF repair or with experience in performing the procedure with a caseload of less than three.

Clinical experience, number of years of experience as a pediatric surgeon and not as a pediatric endoscopic surgeon.

EA/TEF, esophageal atresia/tracheoesophageal fistula; ESs, experienced surgeons; IQR, interquartile range; ISs inexperienced surgeons.

Fifteen pediatric surgeons were classified as ESSQ-qualified surgeons, and the other 25 pediatric surgeons were classified as ESSQ-nonqualified surgeons. Among the ESSQ-qualified surgeons, all but one examinee were attending pediatric surgeons. There were no significant differences between the two groups in all four metrics and subtotal scores of error items (Table 2). Among the 15 ESSQ-qualified surgeons, 6 pediatric surgeons were classified as ESSQ-qualified ESs and the other 9 pediatric surgeons were classified as ESSQ-qualified ISs. The ESSQ-qualified ESs were significantly superior to the ESSQ-qualified ISs in all four metrics. The subtotal scores of Missed grasp, Excessive manipulation, and Needle out of view of the ESSQ-qualified ESs were significantly lower than those of the ESSQ-qualified ISs.

Table 2.

Comparison of Suturing Performance Between the ESSQ-Qualified Surgeons and ESSQ-Nonqualified Surgeons

	ESSQ-qualified surgeons (n = 15)	ESSQ-nonqualified surgeons (n = 25)	P value
29-Point checklist, points (median [IQR])	21.5 (21.5–24.5)	20.5 (17.3–23.3)	.077
Error score, points (median [IQR])	11.0 (7.5–15.0)	12 (7.8–14.5)	.978
Missed grasp, points (median [IQR])	2.5 (1.0–3.0)	2.4 (1.8–3.8)	.290
Needle out of view, points (median [IQR])	0.5 (0–1.5)	0.5 (0–1.5)	.730
Number of manipulations (median [IQR])	21.0 (18.0–34.5)	28.5 (24.0–40.5)	.051
Task completion time, seconds (median [IQR])	272.0 (205.0–471.0)	371.0 (248.0–466.5)	.386
Clinical experience^a, years (median [IQR])	28.0 (21.0–33.0)	9.0 (6.5–12.0)	<.0001

Clinical experience, number of years of experience as a pediatric surgeon and not as a pediatric endoscopic surgeon.

ESSQ, Endoscopic Surgical Skill Qualification; IQR, interquartile range.

The inter-rater reliabilities of the 29-point checklist score method and the suturing error score method were 0.90 and 0.89, respectively. Both values were considered to be sufficiently high.

Questionnaire survey

Six pediatric surgeons with experience in performing thoracoscopic EA/TEF repair answered the additional questionnaire. The average scores are shown in Table 3. The items rated as 4.5 or higher were Outer circumference of the cavity and Depth of the cavity in the category of Realness of size and shape; Position of the incision in the category of Similarity to the clinical situation; and For training and For skill assessment in the category of Usefulness of the model. On the other hand, Gap between the rib bones and The mediastinum model in the category of Realness of size and shape and realness of the skin sheet, rib bones, and the esophagus in the category of Realness of materials were rated lower than 3.9.

Table 3.

Results of the Questionnaire Survey That Was Administered to ESs

Category	Item	Mean score
Realness of size and shape	Outer circumference of the cavity	4.8
	Depth of the cavity	4.7
	Gap between the rib bones	3.8
	Diameter of the esophagus model	4.3
	Thickness of the esophagus model	4.0
	The mediastinum (heart and aorta) model	3.8
	Thickness of the skin model	4.3
Realness of materials	Overall impression	4.2
	Skin sheet	3.8
	Rib bones	3.8
	The esophagus	3.8
	The mediastinum (heart and aorta) model	3.6
Similarity to the clinical situation	Positions of instruments	4.2
	Position of the incision	4.6
	Overall similarity	4.0
Usefulness of the model	For training	5.0
Usefulness of the model	For skills assessment	4.5
Relevance of this simulator to your practice		4.3
Difficulty of the task		3.3

The ESs were asked to rate each item on a 5-point scale, whose original version was proposed by Barsness et al.⁹

ESs, experienced surgeons.

Discussion

The purpose of this study was to show the construct validity of the neonatal EA/TEF simulator by video-based endoscopic suturing skill assessment of pediatric surgeons. ESs were significantly better than ISs in all four metrics. Moreover, the subtotal scores of the error items, Missed grasp and Needle out of view, in the ES group were significantly lower than those in the IS group. This suggests that the video-based skill assessment of the suturing task in the neonatal TE/TEF simulator can distinguish ESs. In contrast, there were no significant differences in all four metrics between the ESSQ-qualified surgeons and the ESSQ-nonqualified surgeons. On the other hand, among the ESSQ-qualified surgeons, ESSQ-qualified ESs were significantly superior to the ESSQ-qualified ISs in all four metrics.

One of the provisions of the ESSQ application is that the applicant should have performed 20 or more advanced endoscopic surgeries such as fundoplication, splenectomy, anorectoplasty, and pulmonary lobectomy,⁴ but does not necessarily have experience in advanced neonatal endoscopic surgeries such as EA/TEF repair and duodenal atresia repair. Our results showed significant differences in metrics between the qualified ES and the qualified IS, and this suggests that the suturing task in the neonatal EA/TEF model is challenging even for ESSQ-qualified surgeons who have the ability to complete common types of advanced laparoscopic surgeries. Obata et al.¹³ reported that pediatric surgeons are required to have wide-ranging endoscopic surgical skills because of the wide variety of patients' size and fewer clinical opportunities compared with general surgeons. In particular, neonatal endoscopic surgery, including thoracoscopic EA/TEF repair, requires pediatric-specific expert skills. Therefore, pediatric-specific pathological models are useful for pediatric surgical training and skills assessment. In other words, training using the neonatal EA/TEF model may be useful not only for pediatric surgeons in training but also for pediatric surgeons preparing for advanced endoscopic surgery.

In the present study, we used four video-based skill assessment methods. The 29-point checklist method was designed for quality assessment of surgical manipulation, such as precision and efficiency, and the suturing error score sheet method was designed for safety assessment.¹⁶ The items Missed grasp in the error score sheet and the Number of needle manipulations are also related to task efficiency.²⁰ The item Needle out of view, which is a measure of the ability to keep the needle in view at all times regardless of the very small workspace, is related to safety. Our results showed that ESs were superior to ISs in these metrics. Therefore, ESs seem to be superior to ISs in terms of quality, safety, and efficiency.

The results of the questionnaire survey showed that our neonatal EA/TEF simulator well replicated the neonatal cavity and the clinical situation of esophageal suturing. The small workspace is a feature of neonatal endoscopic surgery, and pediatric surgeons require high skills for manipulation in a small workspace.¹⁰ On the other hand, the ESs evaluated that the materials of the mediastinum and rib bones were not sufficiently realistic. However, the usefulness of the model as a skill assessment or training model was rated as high.

One limitation of the present study is that the sample size of ESs was small as thoracoscopic EA/TEF repair is not a standard procedure in Japan. However, this procedure was covered by insurance starting in April 2016, and thus our simulator would play a more important role from now on. A second limitation is that examinees were asked to perform only one suturing and knotting task for skills assessment, and the full procedure of thoracoscopic EA/TEF repair was not replicated. The full procedure is time-consuming and complicated, which might make examinees less motivated and make the analysis difficult. We are planning to develop a full procedure model, including closure of the fistula, dissection of the upper pouch, and esophageal anastomosis for skills assessment and training.

In conclusion, the model that we developed successfully reproduced the neonatal cavity and the clinical situation of esophageal suturing. In addition, this study revealed that pediatric surgeons with thoracoscopic TEF repair experience showed significantly better endoscopic surgical skills using the neonatal EA/TEF simulator than those without such experience. Therefore, the construct validity of the neonatal EA/TEF simulator was demonstrated by video-based endoscopic suturing skills assessment. More importantly, this skill difference was also demonstrated upon comparison between ESs and ISs among ESSQ-qualified surgeons. Therefore, the simulator would be useful especially for pediatric endoscopic surgeons who are learning advanced endoscopic surgeries such as thoracoscopic EA/TEF procedures.

Footnotes

Acknowledgments

This study was partially supported by a Grant-in-Aid for Scientific Research (B) (No. 26293378), Grant-in-Aid for Scientific Research (S) (No. 23226006) from the Ministry of Education, Culture, Sports, Science and Technology (MEXT), and the project, Assessment methodology for innovative minimally invasive therapeutic devices, materials, and nano-biodiagnostic devices, from the Accelerating Regulatory Science Initiative, Ministry of Health, Labour and Welfare (MHLW), Japan.

The authors thank Prof. Osamu Segawa of Tokyo Women's Medical University Hospital, Dr. Joji Yoshizawa of Jikei University Hospital, and Dr. Hiroshi Kawashima of Saitama Children's Medical Center for kindly providing the opportunity for conducting the experiments. The authors also thank Hideyuki Sato and Takashi Yamashita of School of Engineering, The University of Tokyo, for their help during the experiments.

Disclosure Statement

The authors have received no financial support and do not have any potential conflicts of interest to report.

References

Iwanaka

, Yamataka

, Uemura

, Okuyama

, Segawa

, Nio

, Yoshizawa

, Yagi

, Ieiri

, Uchida

, Koga

, Sato

, Soh

, Take

, Hirose

, Fukuzawa

, Mizuno

, Watanabe

. Pediatric Surgery. Asian J Endosc Surg, 2015; 8:390–407.

Diao

, Li

, Cheng

. Role of laparoscopy in treatment of choledochal cysts in children. Pediatr Surg Int, 2013; 29:317–326.

Esteves

, Clemente Neto

, Ottaiano Neto

, Devanir

Jr. , Esteves Pereira

. Laparoscopic Kasai portoenterostomy for biliary atresia. Pediatr Surg Int, 2002; 18:737–740.

Iwanaka

, Morikawa

, Yamataka

, Nio

, Segawa

, Kawashima

, Sato

, Terakura

, Take

, Hirose

, Yagi

. Skill qualifications in pediatric minimally invasive surgery. Pediatr Surg Int, 2011; 27:727–731.

Ieiri

, Ishii

, Souzaki

, Uemura

, Tomikawa

, Matsuoka

, Takanishi

, Hashizume

, Taguchi

. Development of an objective endoscopic surgical skill assessment system for pediatric surgeons: Suture ligature model of the crura of the diaphragm in infant fundoplication. Pediatr Surg Int, 2013; 29:501–504.

Narayanan

, Cohen

, Shun

. Technical tips and advancements in pediatric minimally invasive surgical training on porcine based simulations. Pediatr Surg Int, 2014; 30:655–661.

Esposito

, Escolino

, Draghici

, Cerulo

, Farina

, De Pascale

, Cozzolino

, Settimi

. Training Models in Pediatric Minimally Invasive Surgery: Rabbit Model Versus Porcine Model: A Comparative Study. J Laparoendosc Adv Surg Tech A, 2016; 26:79–84.

Azzie

, Gerstle

, Nasr

, Lasko

, Green

, Henao

, Farcas

, Okrainec

. Development and validation of a pediatric laparoscopic surgery simulator. J Pediatr Surg, 2011; 46:897–903.

Barsness

, Rooney

, Davis

, O'Brien

. Evaluation of Three Sources of Validity Evidence for a Synthetic Thoracoscopic Esophageal Atresia/Tracheoesophageal Fistula Repair Simulator. J Laparoendosc Adv Surg Tech A, 2015; 25:599–604.

10.

Barsness

, Rooney

, Davis

. Collaboration in simulation: The development and initial validation of a novel thoracoscopic neonatal simulator. J Pediatr Surg, 2013; 48:1232–1238.

11.

Barsness

, Rooney

, Davis

, O'Brien

. Preliminary evaluation of a novel thoracoscopic infant lobectomy simulator. J Laparoendosc Adv Surg Tech A. 2015; 25:429–434.

12.

Barsness

, Rooney

, Davis

, O'Brien

. Evaluation of three sources of validity evidence for a laparoscopic duodenal atresia repair simulator. J Laparoendosc Adv Surg Tech A, 2015; 25:256–260.

13.

Obata

, Ieiri

, Uemura

, Jimbo

, Souzaki

, Matsuoka

, Katayama

, Hashizume

, Taguchi

. An endoscopic surgical skill validation system for pediatric surgeons using a model of congenital diaphragmatic hernia repair. J Laparoendosc Adv Surg Tech A, 2015; 25:775–781.

14.

Jimbo

, Ieiri

, Obata

, Uemura

, Souzaki

, Matsuoka

, Katayama

, Masumoto

, Hashizume

, Taguchi

. Effectiveness of short-term endoscopic surgical skill training for young pediatric surgeons: A validation study using the laparoscopic fundoplication simulator. Pediatr Surg Int, 2015; 31:963–969.

15.

Harada

, Takazawa

, Tsukuda

, Ishimaru

, Sugita

, Iwanaka

, Mitsuishi

. Quantitative pediatric surgical skill assessment using a rapid-prototyped chest model. Minim Invasive Ther Allied Technol, 2015; 24:226–232.

16.

Takazawa

, Ishimaru

, Harada

, Tsukuda

, Sugita

, Mitsuishi

, Iwanaka

. Video-based skill assessment of endoscopic suturing in a pediatric chest model and a box trainer. J Laparoendosc Adv Surg Tech A, 2015; 25:445–453.

17.

Moorthy

, Munz

, Dosis

, Bello

, Chang

, Darzi

. Bimodal assessment of laparoscopic suturing skills: Construct and concurrent validity. Surg Endosc, 2004; 18:1608–1612.

18.

Aggarwal

, Hance

, Undre

, Ratnasothy

, Moorthy

, Chang

, Darzi

. Training junior operative residents in laparoscopic suturing skills is feasible and efficacious. Surgery, 2006; 139:729–734.

19.

Kroeze

, Mayer

, Chopra

, Aggarwal

, Darzi

, Patel

. Assessment of laparoscopic suturing skills of urology residents: A pan-European study. Eur Urol, 2009; 56:865–872.

20.

Van Sickle

, Baghai

, Huang

, Goldenberg

, Smith

, Ritter

. Construct validity of an objective assessment method for laparoscopic intracorporeal suturing and knot tying. Am J Surg, 2008; 196:74–80.