Development of a Skill Evaluation System for the Camera Assistant Using an Infant-Sized Laparoscopic Box Trainer

Abstract

Aims:

Our aims were to develop a training system for camera assistants (CA), and evaluate participants' performance as CA.

Methods:

A questionnaire on essential requirements to be a good CA was administered to experts in pediatric endoscopic surgery. An infant-sized box trainer with several markers and lines inside was developed. Participants performed marker capturing and line-tracing tasks using a 5-mm 30° scope. A postexperimental questionnaire on the developed system was administered. The task completion time was measured.

Results:

The 5-point evaluation scale was used for each item in the questionnaire survey of experts. The abilities to maintain a horizontal line (mean score: 4.5) and to center the target in a specified rectangle on the monitor (4.5) as well as having a full understanding of the operative procedure (4.3) were ranked as highly important. Fifty-two participants, including 5 surgical residents, were enrolled in the evaluation experiment. The completion time of capturing the markers was significantly longer in the resident group than in the nonresident group (244 versus 124 seconds, P = .04), but that of tracing the lines was not significantly different between the groups. The postexperimental questionnaire showed that the participants felt that the line-tracing tasks (3.7) were more difficult than marker-capturing tasks (2.9).

Conclusions:

Being proficient in manipulating a camera and having adequate knowledge of operative procedures are essential requirements to be a good CA. The ability was different between the resident and nonresident groups even in a simple task such as marker capturing.

Introduction

Laparoscopic surgical simulations have been developed, but most of them were developed for skill assessment and training of operators. Surgical residents are often assigned the role of camera assistant (CA) during endoscopic surgeries without any advance instructions or practice. However, operators rely on laparoscopic images to perform surgical tasks, and the skills of the CA influence the performance of surgery. There may be skills unique to camera navigation especially in pediatric endoscopic surgery, where the small working space makes intraoperative collisions between the surgeon's instruments and the camera more likely, but studies on the necessary skills to be a good CA are scarce. Several research groups have developed systems to evaluate angled telescope skills and showed that simulator-based training, including a virtual reality (VR) simulator and physical simulator is effective for acquisition of camera navigation skills.^1–8 However, these studies were performed in the adult surgical setting, and there are no studies regarding systems of skill evaluation and training for CA in pediatric endoscopic surgery.

Therefore, before developing a training system for CA, we administered a questionnaire survey to experts of pediatric endoscopic surgery on essential requirements to be a good CA in pediatric endoscopic surgery. Then, we developed an infant-sized laparoscopic box trainer and evaluated participants' performance as CA.

Materials and Methods

Questionnaire survey

A questionnaire was sent in July 2014 by mail to 14 Japanese experts in pediatric endoscopic surgery, all of whom were members of the Pediatric Endoscopic Surgical Skill Qualification (ESSQ) Board in The Japan Society for Endoscopic Surgery.⁹ The ESSQ system was developed by the Japan Society for Endoscopic Surgery in 2008 to guarantee the ability of surgeons to complete common pediatric laparoscopic procedures. Surgeons who apply for ESSQ certification have completed at least a 7-year general and pediatric surgical training program and performed more than 50 pediatric minimally invasive surgery procedures, including 20 advanced procedures, and are required to submit an unedited video showing an endoscopic view of an entire pediatric laparoscopic fundoplication. The ESSQ certification is given after evaluation of the video and approved by 2 expert pediatric endoscopic surgeons.

The questionnaire included the following items: (1) the respondent was asked to rate the importance of 13 skills needed to be a good camera operator by a 5-point Likert scale; (2) the respondent was asked which procedures require a good camera operator in an open-ended question; (3) whether the respondent designates a specific person to serve as a camera operator during endoscopic surgeries; (4) the respondent was asked who is more important, the camera operator or the assistant; and (5) the respondent was asked whether a training system for camera operators would be useful (Tables 1 and 2).

Table 1.

Importance of 13 Skills That Are Needed to Be a Good Camera Operator as Rated by Experts of Pediatric Endoscopic Surgery

Rank	Skill	Points
1	Not moving the scope or changing the view on the monitor frequently	4.8 (0.4)
2	Maintaining a horizontal line on the monitor	4.5 (0.5)
3	Centering the target	4.5 (0.5)
4	Having a full understanding of the operative procedure	4.3 (0.9)
5	Moving the scope smoothly	4.2 (0.8)
6	Moving the scope at the appropriate speed	4.1 (0.8)
7	Avoiding intracorporeal collision	3.9 (0.9)
8	Having a good knowledge of the surgeon's preference	3.7 (1.1)
9	Pointing out surgeon's mistakes	3.6 (1.1)
10	Avoiding extracorporeal collision	3.5 (0.9)
11	Having much clinical experience as a camera assistant	2.9 (0.7)
12	Having more clinical experience as an endoscopic surgeon than as an assistant	1.4 (0.6)
13	Senior to a surgeon	1.1 (0.3)

The 13 experts of pediatric endoscopic surgery rated the importance of each skill on a scale from 1 point (not important) to 5 points (very important). All data are shown as mean (standard deviation).

Table 2.

Questionnaire Regarding the Camera Operator in Pediatric Endoscopic Surgery: Responses of 13 Experts of Pediatric Endoscopic Surgery

	Question	Answer
Q1	Are there any procedures where a skilled camera operator is needed?	Yes: 11 (85%)/No: 2
(To respondents who answered “Yes” in Q1)
Q1-a	Which procedure? (free description)	Fundoplication 6/anorectoplasty 4/TEF repair, CBD 3/Lobectomy, BA, splenectomy 2
Q2	Do you designate a specific person to serve as a camera operator?	No: 9 (64%)/depends on the procedure: 4
(To respondents who answered “Depends on the procedure” in Q2)
Q2-1	Which procedure? (free description)	TEF repair, CBD: 3/lobectomy, BA: 1
Q3	Who is more important, the camera operator or the assistant?	Camera operator: 9 (64%)/assistant: 2/does not care: 1/depends on the procedure: 1
Q4	Would a training system for camera operators be useful?	Yes: 2/rather yes: 7/hard to say: 2/rather no: 1/no: 1

BA, biliary atresia; CBD, congenital biliary dilatation; TEF, tracheoesophageal fistula.

Development of an infant-sized box trainer

An infant-sized box trainer mimicking the size constraints of infantile abdominal laparoscopic surgery was developed based on measurements using computed tomography (CT) images of a 6-month-old boy with abdominal distention due to intestinal obstruction, because performing CT on a patient under the condition of pneumoperitoneum was ethically and technically difficult at our institute (Fig. 1a). A camera port (which is labeled as the “Scope insertion position” in Fig. 1a) was placed at the center of the superior aspect of the box to imitate the position of the umbilicus, and two markers were placed on the top inner wall of the box at the locations of the working ports (Fig. 1b). Three sets of three side-by-side markers (i.e., three green dots, three blue dots, and three red dots) on a trapezoid were placed in front of the posterior wall to mimic the rectal wall in the pelvic floor. Finally, long transverse lines were placed on the left side, posterior, and right side walls, respectively (Fig. 1b). An image processing program was developed to automatically detect the capturing of marker(s) and line in a specified space on the monitor. The system can also automatically measure the marker size and line angle in the specified space as well as the captured time.

FIG. 1.

Infant-sized box trainer. (a) External appearance. A camera port has been placed at the center of the superior aspect of the box. (b) Internal appearance. Two markers have been placed on the top inner side. Three sets of three side-by-side markers on a trapezoid have been placed in front of the posterior wall. Long transverse lines have been placed on the left-side, posterior, and right-side walls, respectively.

Experiment to evaluate angled telescope skills

Participants were recruited at a domestic annual congress of pediatric surgery. The participants performed the following tasks as a camera operator, which were designed based on infantile pelvic surgery such as definitive surgery for Hirschsprung's disease or anorectal malformation, using a 5-mm 30° scope. At the beginning of the experiment, the scope was already inserted into the camera port. Assigned tasks were as follows: (1) to capture each marker on the top inner side, show it in the rectangle with a specified size on the monitor and hold the image of the marker in the rectangle for 5 seconds; (2) to capture each marker unit that had been placed on the bottom surface, show it within the other displayed rectangle, and hold the image of the marker in the rectangle for 5 seconds; and (3) to trace each line from the left edge to right edge, keeping the line horizontal on the monitor. In tasks 1 and 2, once the system recognized the target as properly captured in the rectangle, the length of time was automatically measured. However, when one or more markers deviated from the specified rectangle or appeared smaller than the specified size after the 5-second count was started, the timer was reset and the task had to be performed from the beginning. In task 3, once the line appearing on the monitor tilted over 10° from the horizontal line, the timer was reset, and the task had to be restarted from the beginning. Representative images on the monitor during each task are shown in Figure 2. The task completion time for each task was measured. After the experiment, a written questionnaire on the realness and usefulness of this system and difficulty of the tasks on a 5-point scale was administered to the participants.

FIG. 2.

Images on the monitor during each task. (a) Capturing a marker on the top inner side within the rectangle on the monitor for 5 seconds. The automatic capturing-time counter measures the amount of time that the marker is held within the rectangle. If the marker deviates from the rectangle, the timer is reset to 0 and the task has to be performed from the beginning. (b) Capturing a marker unit placed on the bottom surface within the rectangle on the monitor for 5 seconds. (c) Tracing a line from the left to right edge keeping the line horizontal.

Statistical analysis and ethical concerns

Statistical analysis was performed by the Wilcoxon rank sum test using commercially available software (JMP^® Pro 11.0.0; SAS Institute Japan Ltd., Tokyo, Japan). P values of <.05 were considered to be statistically significant. All data are shown as mean values (standard deviation).

This skill evaluation study was approved by the Research Ethics Committee of the Graduate School of Medicine and Faculty of Medicine, The University of Tokyo (protocol No. 11226).

Results

Questionnaire survey

Answer sheets were collected from 13 (93%) of the 14 experts. In the 5-point evaluation, skills in movement or maintaining proper position of the scope such as the abilities to maintain a horizontal line (mean score [standard deviation]: 4.5 [0.4]), to center the target (4.5 [0.5]), and to move the scope smoothly (4.2 [0.8]) were ranked as highly important on the list. Moreover, they rated that having a full understanding of the operative procedure was also required to be a good camera operator (4.3 [0.9]). The mean scores of the ability to avoid intracorporeal or extracorporeal collision were not as high (3.9 [0.9], 3.5 [0.9], respectively) (Table 1). They indicated that a skilled camera operator was needed in fundoplication, anorectoplasty, and repair of esophageal atresia or congenital biliary dilatation. Nine (69%) of the 13 board-certified surgeons answered that the camera operator was more important than the assistant and that a training system for camera operators would be useful (Table 2).

Experiment to evaluate angled telescope skills

Fifty-two participants were enrolled in this study, including 5 surgical residents with <2 years' clinical experience, and 7 skilled pediatric endoscopic surgeons certified by the ESSQ system. The remaining 40 participants were pediatric endoscopic surgeons.

The results of the task completion time are shown in Figure 3. The completion time of the whole task was significantly longer in the resident group than in the nonresident group (500 [203] versus 303 [149] seconds, respectively, P = .04). The completion time of capturing the markers was also significantly longer in the resident group than in the nonresident group (240 [103] versus 136 [64] seconds, respectively, P = .04), but that of tracing the lines was not significantly different between the 2 groups. There were no significant differences in the task completion times between the ESSQ-qualified surgeons and the others.

FIG. 3.

Comparison of the task completion times in the angled telescope skill evaluation experiment between the Nonresident and Resident group. The completion times of the total task and marker-capturing task were significantly longer in the resident group than in the nonresident group.

The results of the postexperimental questionnaire are shown in Table 3. Realness of the box size, insertion position of the scope, and assigned tasks were scored as 4.0 (0.9), 3.9 (0.8), and 3.6 (0.7), respectively. The usefulness of this system for skill evaluation of the CA, training of novices, and training for the participant himself/herself were scored as 4.1 (0.6), 4.3 (0.6), and 4.0 (0.7), respectively. Line-tracing tasks (3.7 [1.0]) were considered to be more difficult than marker-capturing tasks (2.9 [0.9]).

Table 3.

Results of a Questionnaire After the Angled Telescope Skill Evaluation Experiment: Responses of the Participants

Realness (5: very similar—1: dissimilar)	Size of the dry box			4.0 (0.9)
	Insertion position of the scope			3.9 (0.8)
	Assigned tasks			3.6 (0.7)
Usefulness of this system (5: very useful—1: no effect)	For skill evaluation of the camera assistant			4.1 (0.6)
	For training of novices			4.3 (0.6)
	For training of the participant himself/herself			4.0 (0.7)
Difficulty of task (5: very hard—1: very easy)	Marker-capturing tasks	2.9 (0.9)	Ceiling markers	2.8 (0.9)
			Marker unit placed in front	2.6 (0.9)
			Marker units placed obliquely	3.4 (0.8)
	Line-tracing tasks	3.7 (1.0)	Left side wall	3.9 (0.9)
			Posterior wall	2.9 (1.0)
			Right side wall	4.2 (0.6)

The 52 participants were asked to rate each item on a 5-point Likert scale. Data are shown as mean (standard deviation).

Discussion

This study showed results of a questionnaire survey on essential requirements to be a good CA and the results of an experiment that evaluated participants' angled telescope skills in an infant-sized laparoscopic box trainer. This is the first study to focus on the skills of the CA in pediatric endoscopic surgery.

The questionnaire survey revealed that experts of pediatric endoscopic surgery thought that being proficient in manipulating a camera and having adequate knowledge of operative procedures are essential requirements to being a good camera operator, and that its role is more crucial in a procedure that requires sophisticated techniques in a small working space such as fundoplication, anorectoplasty, and repair of esophageal atresia or common bile duct dilatation. Among the skills of movement or maintaining the position of the scope, the abilities to maintain a horizontal line on the monitor and to center the target were ranked as highly important on the list. Based on these results, we developed a skill evaluation system for a CA using an infant-sized laparoscopic box trainer.

In the questionnaire administered after the evaluation experiments, the developed system was evaluated as useful for skill evaluation and training of CA. As for realness of the developed simulator, the size was good, but there is room for improvement in the scope insertion position and the task settings. Among the tasks, capturing the marker units that had been placed obliquely and tracing the line on the left or right wall tended to be difficult for the participants. The developed box was a cuboid and was made of acrylic plate. It is different from an infant's body whose abdominal wall is flexible and whose abdominal cavity is ellipsoidal. These discrepancies between the box and the human body may have caused a sense of participants' dissatisfaction; the hardness of the box may have made scope manipulation difficult and the difference in shape might have affected spatial perception in the tasks mentioned above, because these tasks required the participants to move the scope widely. Also, the tasks of capturing the marker units that had been placed obliquely and tracing the line on the left or right wall were more complicated than capturing ceiling markers or marker units placed in front, and tracing the line on the posterior wall, respectively.

The results of the task completion time showed that the ability as a CA differed between the resident and nonresident groups. The residents needed significantly more time to complete the total task and the marker-capturing task than nonresidents. However, there was no significant difference between the 2 groups in the line-tracing task, and participants answered that line tracing was more difficult than marker capturing. This suggests that the line-tracing task was quite difficult for all participants to complete and required a long period of time, or the difference in ability of angled telescope skill could be seen even in a simple task such as marker capturing. This knowledge might be beneficial when a CA training program for novices is developed in the future. Yee et al. reported that after a simple and brief training session, novices were able to effectively use the 30° laparoscope.⁵ Novices might be able to acquire enough skills by receiving training in a simple task before clinical practice. Surgical residents often use an angled telescope for the first time and play the role of CA in the operating room without any preoperative instruction or practice. In contrast, most experts answered that they need a skilled CA in some specific procedures and that the CA was is more important than the assistant in the current questionnaire survey. The quality of the images captured by the CA affects surgical performance, and simulation-based training improved the technical skills required for camera navigation in randomized trials.^1,3 Two types of simulated training have been developed: VR systems and videotrainer (VT) systems using a box trainer or physical models. Both VR and VT simulators are useful training devices and should be widely used to improve laparoscopic camera navigation skills.⁶ The currently available VR simulators are costly to install, and they were developed for adult cases. To the best of our knowledge, there is no commercially available VR or VT simulator designed for pediatric cases. Therefore, low-cost VT simulators are needed especially for pediatric cases in the future.

This study has several limitations that merit consideration. The number of respondents in the questionnaire survey and the sample size in the evaluation experiment were not large. In addition, there is room for improvement in the scope insertion position in the developed box and the task settings. These issues may have resulted in bias in our data and findings.

Despite these limitations, the current study revealed that experts in pediatric endoscopic surgery thought that being proficient in manipulating a camera, especially maintaining a horizontal line and centering the target on the monitor, and having adequate knowledge of operative procedures are essential requirements to being a good CA. Moreover, the results of the task completion time in evaluation experiments showed that the ability as a CA differed between the resident and nonresident groups even in a simple task such as marker capturing. This knowledge would be beneficial in developing a training system for novices to acquire angled telescope skills before clinical experience in the operating room.

Footnotes

Acknowledgment

This study was supported by Grant-in-Aid for Scientific Research^© (No. 15K10020) from the Ministry of Education, Culture, Sports, Science and Technology of Japan.

Disclosure Statement

No competing financial interests exist.

References

Nilsson

, Sorensen

, Konge

, et al. Simulation-based camera navigation training in laparoscopy-a randomized trial. Surg Endosc, 2017; 31:2131–2139.

Daniel

, Tapia

, Arturo

, et al. Adapting to the 30-degree visual perspective by emulating the angled laparoscope: A simple and low-cost solution for basic surgical training. Simul Healthc, 2014; 9:384–391.

Franzeck

, Rosenthal

, Muller

, et al. Prospective randomized controlled trial of simulator-based versus traditional in-surgery laparoscopic camera navigation training. Surg Endosc, 2012; 26:235–241.

Bennett

, Birch

, Menzes

, et al. Assessment of medical student laparoscopic camera skills and the impact of formal camera training. Am J Surg, 2011; 201:655–659.

Yee

, Karmali

, Sherman

. Validation of a simple camera navigation trainer. J Am Coll Surg, 2009; 209:753–757.

Stefanidis

, Haluck

, Pham

, et al. Construct and face validity and task workload for laparoscopic camera navigation: Virtual reality versus videotrainer systems at the SAGES Learning Center. Surg Endosc, 2007; 21:1158–1164.

Ganai

, Donroe

, St Louis

, et al. Virtual-reality training improves angled telescope skills in novice laparoscopists. Am J Surg, 2007; 193:260–265.

Korndorffer

Jr ., Hayes

, Dunne

, et al. Development and transferability of a cost-effective laparoscopic camera navigation simulator. Surg Endosc, 2005; 19:161–167.

Iwanaka

, Morikawa

, Yamataka

, et al. Skill qualifications in pediatric minimally invasive surgery. Pediatr Surg Int, 2011; 27:727–731.