ProMIS ™ Can Serve as a da Vinci ® Simulator

Abstract

Purpose:

The purpose of this study was to investigate if the ProMIS™ simulator could serve as a training platform for the da Vinci^® surgical system and if this constellation could prove construct validity.

Materials and Methods:

The da Vinci system was connected to the ProMIS simulator, which registered objective data concerning how the surgeon performed in the box environment related to time, path, and smoothness. Five experienced robotic surgeons passed four different surgical tasks with progressive difficulty. A novice group—constituted of 13 consultants and 6 residents, none of them with any previous experience in the da Vinci system—passed the same tasks and the data were compared with the results from the expert group.

Results:

A statistically significant difference between experts and novices was demonstrated in all tasks concerning time and smoothness. For the parameter path, significant difference was only noted in the more complex tasks.

Conclusions:

Our study showed that ProMis could differentiate between experienced robotic surgeons and novices, thereby proving construct validity. Smoothness appeared to be the most sensitive objective parameter in our study. Tasks with high complexity are recommended when designing the program for robotic training.

Introduction

R obot-assisted laparoscopic technology is today an adopted method for minimal invasive surgery. It has been suggested that the advantage of the robotic system allows the surgeon to perform more complex tasks, restoring surgical dexterity, hand–eye coordination, and depth perception.¹ When the da Vinci^® surgical system (Intuitive Surgical, Mountain View, CA) was released a decade ago, it was first used in the field of thoracic surgery.² Binder and Kramer³ performed the first robot-assisted laparoscopic prostatectomy in May 2000. The technique of robot-assisted laparoscopic prostatectomy is now a widespread procedure and the use of robot-assisted laparoscopic technology is a rapidly expanding phenomenon with applicability in numerous disciplines of surgery. A demanding mission today is how we can facilitate the progress of educating the next generation of robotic surgeons.

Seymour et al⁴ proved that virtual reality (VR) training transfers technical skills to the operating room. Later, Ahlberg et al⁵ showed that training in a VR simulator to a level of proficiency significantly improved intraoperative performance during a resident's first 10 laparoscopic cholecystectomies.⁵ In 2008, Gurusamy et al⁶ investigated 23 trials in which VR training was compared with conventional laparoscopic training in surgical trainees. They observed that VR training resulted in a greater reduction concerning operating time, number of errors, and unnecessary movements than standard laparoscopic training.⁶

To our knowledge, there is no validated simulator designed for specific training in the da Vinci surgical system. However, several studies have observed construct validity for ProMIS™ (Haptica, Dublin, Ireland)—a modular VR simulator of hybrid type constructed for laparoscopic training—when differentiating between experienced and novice laparoscopic surgeons.^7,8 We find it important to investigate if an already existing simulator can find a position in the training of robot-assisted surgery.

The aim of this study was to investigate if the ProMIS simulator could serve as a training platform for the da Vinci surgical system and if this constellation can prove construct validity.

Materials and Methods

Subjects

Twenty-four surgeons at the Karolinska University Hospital were included in this study. Five experienced robotic surgeons from the urology department constituted the expert group (EG). Their mean age was 49 (range: 36–64) and they were all men. Mean postgraduate years were 18 (range: 5–35). The novice group (NG) was constituted of 13 consultants (12 males and 1 female) and 6 residents (3 men and 3 women) from the gynecology, urology, and general surgery departments—none of them with any previous experience in the da Vinci system. Mean age in the NG was 43 (range: 30–58). Mean postgraduate years were 13 (range: 1–27). One participant in the study was left handed and one was ambidextrous—both belonging to the NG. Demographic data concerning experiences in robotic, laparoscopic, and open surgeries are shown in Table 1.

Table 1.

Demographic Data from the Novice and Expert Groups Concerning the Number of Robotic, Laparoscopic, and Open Surgeries Performed

Number of operations performed		0	1–20	21–50	51–100	>100
	Robotic operation	100% (19)	0%	0%	0%	0%
N	Laparoscopic operation	26% (5)	11% (2)	11% (2)	21% (4)	32% (6)
	Open surgery	0%	5% (1)	11% (2)	0%	84% (16)
	Robotic operation	0%	0%	20% (1)	20% (1)	60% (3)
E	Laparoscopic operation	0%	40% (2)	20% (1)	20% (1)	20% (1)
	Open surgery	0%	0%	20% (1)	20% (1)	60% (3)

N = novice; E = expert.

Apparatus

ProMIS is a hybrid simulator that enables virtual and physical reality to be used together; however, in this study we do not use the incorporated VR technology. The simulation training, using physical reality, emerges from a laparoscopic interface that consists of a torso-shaped mannequin with a neoprene cover, connected to a portable computer. Different trays, containing different training modules such as suturing pads or knot-tying tasks, can be placed in the mannequin. Three separate camera tracking systems are placed inside the mannequin, arranged to record the three-dimensional position of the tip of the instruments 30 times a second. A broad range of instrument types can be used though the distal ends of the laparoscopic instruments shaft are covered with two pieces of yellow electrical tape to serve as a reference point for the camera tracking system. Instrument movement is recorded and stored in distinct sections, based on the time the tips of the instrument are detected until they are removed from the mannequin. The ProMIS simulator analyzes three objective parameters: time, path, and smoothness. The time can be recorded automatically or manually by using a pedal. The system calculates the path as the sum of the distances from each point-to-point and the path is measured for the right and left arm, respectively. The smoothness is analyzed for the right and left arm as a result of the paths taken by the instruments and scores it for smoothness, such as sudden change in direction and velocity. The data from the simulator are processed using the portable computer.

The da Vinci robot system is a master-slave telemanipulator arrangement with true three-dimensional vision, tremor filter, 10 times magnification, motion scaling, and the patented Endowrist technology allowing 6 degrees of freedom. The da Vinci system consists of one surgeon's console (master) and one cart (slave), with three or four arms depending on the model. One arm holds the camera and the others are dedicated for the instruments. The instruments used in this study were two large needle holders and one pair of Potts scissors.

Simulation

The da Vinci system was connected to the ProMIS simulator, which registered objective data concerning how the surgeon performed in the box environment related to time, path, and smoothness (Fig. 1). The subjects passed four different surgical tasks with progressive difficulty. The data (time, path, and smoothness) from the EG was collected and compared with the data collected from the NG. Then, a single observer subjectively measures the physical outcome for the more complex task number 3 and 4 and the scoring was compared for the two groups.

FIG. 1.

The da Vinci^® Surgical System connected to the ProMIS simulator.

All participants received both text and one-to-one uniform verbal information about the da Vinci robot and about the four different tasks. The joysticks at the surgeon's console, as well as the two robot instruments and the camera, were in the same position for all participants in the beginning of each task. We used a pedal to start and stop the recorded time and the participants began on command.

Task 1: Pull and loosen elastic bands

The task consisted of five small poles with a height of 25 mm—one in the center and four around at about 12, 3, 6, and 9 o'clock positions (Fig. 2A). The distance from the center pole to the surrounding poles was 37 mm. With the right and left hand in an alternate order, the participant should pull an elastic band with a diameter of 25 mm, connected round the center pole, to one of the surrounding four poles. When four elastic bands have been attached to the four surrounding poles, the participant should loosen the elastic bands in the opposite succession in such a manner that the elastic bands again only surrounded the center pole.

FIG. 2.

(A) Task 1: Pull and loosen elastic bands. (B) Task 2: Cutting a circle. (C) Task 3: Suture and tying drill. (D) Task 4: Vesicourethral anastomosis drill.

Task 2: Cutting a circle

This task consisted of a rubber glove strapped onto a tight frame of a basin (Fig. 2B). Air was pumped into the basin so the glove bulged and the palmer and dorsal rubber membrane of the glove were pressed together. The participant should use the left hand to lift the outer membrane, while the right hand cut clockwise along a marked circle with a diameter of 33 mm—starting and stopping at 6 o'clock position.

Task 3: Suture and tying drill

This task imitates wound closure with two interrupted sutures (Fig. 2C). A silicone plate, 3 mm thick, was nailed to the module's plastic foundation. A longitudinal incision of 40 mm was made in the silicone plate. Four dots, with a distance of 2 cm apart, were marked 5 mm lateral to the incision on both sides. With the needle holder, the participant should grasp a suture needle (3-0 Biosyn, CV-25) and try to hit the lower dot on the right side of the incision. Then, the needle should be pulled through the silicone plate so the tip of the needle comes out between the edges of the incision. Again the participant should grasp the needle and then try to hit the lower dot on the left side from below. The participant should then close the wound by tying a knot (two by one by one). The same procedure was performed at the upper part of the wound.

The test leader marked the specimen for postprocedure assessment. The scoring was based on the following fundamental principle: one point if the needle hit the two dots, one point if the silicone plate was not injured, and one point if the edges were not overlapping. The entire task could bring maximum six points—three points for each suture.

Task 4: Vesicourethral anastomosis drill

This final task imitates the vesicourethral anastomosis during a radical prostatectomy (Fig. 2D). In the middle of a concave bowl, imitating the pelvic floor, a small hole was made, in which we placed a silicone tube with a size similar to the urethra. A smaller convex silicone bowl with a hole in the center, mimicking the bladder with a dissected bladder neck, was placed close to the artificial pelvic floor. To avoid slipping of the artificial bladder, we put the first two running sutures through the bladder and urethra at 5 and 7 o'clock positions, using a 20-cm double-armed suture (3-0 Biosyn, CV 25). The participant then completed the rest of the anastomosis by three running sutures on the right and left side, respectively, and finally tied the sutures end at 12 o'clock position. To simplify the tying, both needles were cut off using a pair of laparoscopic scissors.

The test leader marked the specimen for postprocedure assessment in a similar manner compared with task 3: two points were given if there was complete adaptation between the artificial urethra and the bladder. Further two points were added if the anastomosis was wide open and not closed by wrongly addressed sutures. Two points were given if the urethra was not injured—if there was one incision in the urethra the subject got one point, but if two or more incisions were seen the subject got zero points. In the same way, two points were added if the bladder neck was not injured. Finally, one point was added if the sutures were tight, adding up to a maximum of nine points for the tasks.

Statistical analysis

The Mann–Whitney U-test and t-test were used to compare the expert and NG concerning subjective and objective scoring, respectively. The objective scores were log-transformed to be able to use t-test in an appropriate way. All scores have been transformed back.

Results

Objective data from tasks 1, 2, 3, and 4 concerning mean time (seconds), mean path (mm), and mean smoothness for the right and left hand, respectively, are shown in Table 2. A significant difference between the NG and EG concerning time and smoothness was demonstrated for all tasks. The left path showed a significant difference between the two groups in tasks 2, 3, and 4. The right path showed a significant difference between the NG and EG only in tasks 3 and 4. Consequently, no significant difference between the two groups was shown for the left path in task 1 and for the right path in tasks 1 and 2 (Table 2). If the ambidextrous and the left-handed participants were excluded, no change concerning the significant differences between the two groups was observed.

Table 2.

Data Including Difference in Mean and p-Values for Tasks 1, 2, 3, and 4 Concerning Mean Time (Seconds), Mean Path (mm), and Mean Smoothness for the Novice and Expert Groups

Parameter	Task	Novice group		Expert group		Difference in mean	95% Confidence interval		p-Value (t-test)
Time (seconds)	1	92	SD 27	48	SD 15	−44.2	−82.1	−17.3	0.001
	2	204	SD 65	100	SD 38	−104.1	−203.9	−37.1	0.002
	3	433	SD 110	168	SD 36	−264.9	−405.5	−158.8	<0.001
	4	1034	SD 256	351	SD 73	−683.3	−1006.8	−436.9	<0.001
Left path (mm)	1	205	SD 64	145	SD 46	−59.8	−151.6	3.6	0.082
	2	588	SD 323	247	SD 119	−340.3	−1013.0	−26.6	0.037
	3	748	SD 231	361	SD 84	−387.1	−698.2	−167.4	<0.001
	4	3249	SD 1625	1282	SD 371	−1966.5	−3723.9	−826.0	<0.001
Left smoothness	1	350	SD 116	157	SD 70	−193.2	−384.5	−69.5	0.002
	2	993	SD 356	442	SD 188	−551.4	−1129.5	−186.1	0.002
	3	1514	SD 391	572	SD 145	−942.1	−1457.2	−557.7	<0.001
	4	4661	SD 1367	1583	SD 436	−3077.8	−4950.1	−1742.0	<0.001
Right path (mm)	1	168	SD 61	126	SD 39	−41.6	−132.1	17.2	0.209
	2	203	SD 111	110	SD 70	−92.5	−353.1	21.5	0.163
	3	735	SD 279	368	SD 81	−366.6	−771.2	−105.7	0.005
	4	1789	SD 538	1211	SD 377	−578.6	−1342.3	−43.3	0.042
Right smoothness	1	293	SD 104	133	SD 61	−159.4	−332.9	−50.4	0.003
	2	745	SD 264	347	SD 160	−398.3	−843.9	−119.5	0.004
	3	1359	SD 394	482	SD 154	−876.8	−1436.5	−480.5	<0.001
	4	3861	SD 1068	1393	SD 412	−2468.5	−3946.2	−1399.8	<0.001

Standard deviations (SD) are shown in italics. No significant difference between the two groups is demonstrated for the left path in task 1 and for the right path in tasks 1 and 2.

The difference in mean between the two groups concerning time, path, and smoothness is illustrated in Figure 3 and the corresponding data including the p-values are shown in Table 2. In every task, the left smoothness and the left path demonstrated a larger difference in mean compared with the right smoothness and the right path, respectively. As the tasks became more complex, the difference in mean increased for each parameter—task 4 presented the largest difference in mean.

FIG. 3.

Point estimation (black square) with 95% confidence interval for difference in mean between the EG and NG. Estimations are shown for each parameter and each task. The unit for time is seconds and for path is millimeters. There is no unit for smoothness. Confidence interval below zero indicates that the EG performed better than the NG. EG, expert group; NG, novice group.

The postprocedure assessment for task 3 demonstrated a mean value of 3.4 points (standard deviation [SD] = 1.1) for the NG and 4.8 points (SD = 1.0) for the EG. The difference was significant (p-value = 0.024; Fig. 4). The postprocedure assessment for task 4 demonstrated a significant difference between the two groups, with a mean value of 5.3 points (SD = 2.1) for the NG and 8.2 points (SD = 1.0) for the EG (p-value = 0.011; Fig. 4).

FIG. 4.

The postprocedure assessment from task 3 demonstrated a mean value of 4.8 (SD = 1.0) for the EG and 3.4 (SD = 1.1) for the NG (p-value = 0.024). Task 4 demonstrated a mean value of 8.2 (SD = 1.0) for the EG and 5.3 (SD = 2.1) for the NG (p-value = 0.011).

Discussion

Traditionally, surgical skills are achieved in a master–apprentice relation in the operating room.⁹ However, in the way the da Vinci system is currently designed, with a “co-pilot” function only if you have two surgeons' consoles connected to each other, the master–apprentice model is rarely applicable. According to these circumstances, the use of simulators may be important in the teaching design concerning robot-assisted laparoscopy, both for training skills and to assess objective parameters concerning performance.^10,11

An advantage with simulation technology is the possibility to create proficiency-based training models, in which experts' performance set the criterion level.¹² Previous studies concerning laparoscopic training have proven that motion analysis is a useful parameter for skills assessment and our results suggest that this is also true for assessing robotic skills using ProMIS.¹³ Feifer et al¹⁴ observed that the smoothness metric alone demonstrated construct validity by differentiating between experienced and novice surgeons in a laparoscopic environment. In accordance with those results, we also proved construct validity when using training modules as described above for the da Vinci–ProMis platform.

In addition, we observed a significant difference between the NG and EG concerning the postprocedure assessment scores for tasks 3 and 4. This analysis was important, as it shows that the EG performed the different tasks carefully, and not in a slovenly manner in an attempt to receive proper results (short time and path) from the motion analysis calculated by the ProMIS. Concisely, the EG received superior results from the motion analysis at the same time as they performed sutures and anastomoses of superior quality compared with the NG.

Concerning smoothness and time, a statistically significant difference between experts and novices was demonstrated in all tasks, including task 1, which was considered as the simplest to perform. Relating to smoothness, the difference is seen for both the right and left hands in a very similar pattern (Fig. 4). Smoothness is also the parameter that constitutes the largest difference between the participants in the same group, irrespective of studying experienced robotic surgeons or novices. Therefore, it seems as if smoothness is the most sensitive parameter in evaluating performance in the da Vinci–ProMis platform. Even though movement of the instrument tip during actual surgery is more complicated than in the simulated environment, the subjects require precise control of the instrument tip for effective performance of each task.¹⁵

For the parameter path, significant difference was only noted in the more complex tasks, which might be a problem when defining construct validity for this metric. The fact that ProMIS could not detect any difference concerning the right path between the two groups in tasks 1 and 2 is probably an indication of how intuitive the da Vinci robotic system is, even for a novice. To mimic the multifaceted action of the da Vinci robotic system in actual surgery is a challenge for simulators. Therefore, suitable tasks with high complexity are recommended when designing the curricula for robotic training.¹⁶

The results of this study propose that ProMis can distinguish between novices and experts if the simulation task is complicated enough. Validation of a simulated environment is a crucial and mandatory step before inclusion in any clinical training curricula.^17,18 An advantage in using a simulator of hybrid type is the possibility to create tasks with high level of reality for any procedural training. In addition, the system allows for objective assessment with the introduction of any laparoscopic or robotic instrument.

Conclusions

Our study provides objective evidence that ProMis can differentiate between experienced robotic surgeons and novices when using training models as described in this article, thereby proving construct validity. Smoothness seems to be the most sensitive objective parameter distinguishing between experts and novices, and not surprisingly, a more technical demanding task gives a more obvious difference.

Footnotes

Disclosure Statement

No competing financial interests exist.

Abbreviations Used

References

Falk

, Mintz

, Grunenfelder

et al. Influence of three-dimensional vision on surgical telemanipulator performance. Surg Endosc, 2001; 15:1282–1288.

Loulmet

, Carpentier

, d'Attellis

et al. Endoscopic coronary artery bypass grafting with the aid of robotic assisted instruments. J Thorac Cardiovasc Surg, 1999; 118:4–10.

Binder

, Kramer

. Robotically-assisted laparoscopic radical prostatectomy. BJU Int, 2001; 87:408–410.

Seymour

, Gallagher

, Roman

et al. Virtual reality training improves operating room performance: Results of a randomized, double-blinded study. Ann Surg, 2002; 236:458–463discussion 463–454.

Ahlberg

, Enochsson

, Gallagher

et al. Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg, 2007; 193:797–804.

Gurusamy

, Aggarwal

, Palanivelu

et al. Systematic review of randomized controlled trials on the effectiveness of virtual reality training for laparoscopic surgery. Br J Surg, 2008; 95:1088–1097.

Pellen

, Horgan

, Barton

et al. Construct validity of the ProMIS laparoscopic simulator. Surg Endosc, 2009; 23:130–139.

Neary

, Boyle

, Delaney

et al. Construct validation of a novel hybrid virtual-reality simulator for training and assessing laparoscopic colectomy; results from the first course for experienced senior laparoscopic surgeons. Surg Endosc, 2008; 22:2301–2309.

Satava

, Gallagher

, Pellegrini

. Surgical competence and surgical proficiency: Definitions, taxonomy, and metrics. J Am Coll Surg, 2003; 196:933–937.

10.

Narazaki

, Oleynikov

, Stergiou

. Objective assessment of proficiency with bimanual inanimate tasks in robotic laparoscopy. J Laparoendosc Adv Surg Tech A, 2007; 17:47–52.

11.

Haluck

, Krummel

. Computers and virtual reality for surgical education in the 21st century. Arch Surg, 2000; 135:786–792.

12.

Aggarwal

, Grantcharov

, Moorthy

et al. A competency-based virtual reality training curriculum for the acquisition of laparoscopic psychomotor skill. Am J Surg, 2006; 191:128–133.

13.

Gallagher

, Ritter

, Champion

et al. Virtual reality simulation for the operating room: Proficiency-based training as a paradigm shift in surgical skills training. Ann Surg, 2005; 241:364–372.

14.

Feifer

, Delisle

, Anidjar

. Hybrid augmented reality simulator: Preliminary construct validation of laparoscopic smoothness in a urology residency program. J Urol, 2008; 180:1455–1459.

15.

Fried

, Feldman

, Vassiliou

et al. Proving the value of simulation in laparoscopic surgery. Ann Surg, 2004; 240:518–525discussion 525–518.

16.

Chitwood

Jr. , Nifong

, Chapman

et al. Robotic surgical training in an academic institution. Ann Surg, 2001; 234:475–484discussion 484–476.

17.

Paisley

, Baldwin

, Paterson-Brown

. Validity of surgical simulation for the assessment of operative skill. Br J Surg, 2001; 88:1525–1532.

18.

Schijven

, Jakimowicz

. Construct validity: Experts and novices performing on the Xitact LS500 laparoscopy simulator. Surg Endosc, 2003; 17:803–810.

ProMIS ™ Can Serve as a da Vinci ® Simulator—A Construct Validity Study

Abstract

Purpose:

Materials and Methods:

Results:

Conclusions:

Introduction

Materials and Methods

Subjects

Apparatus

Simulation

Task 1: Pull and loosen elastic bands

Task 2: Cutting a circle

Task 3: Suture and tying drill

Task 4: Vesicourethral anastomosis drill

Statistical analysis

Results

Discussion

Conclusions

Footnotes

Disclosure Statement

Abbreviations Used

References