Comparison of Adult and Pediatric Surgeons: Insight into Simulation-Based Tools That May Improve Expertise Among Experts

Abstract

Background:

Laparoscopic models are increasingly recognized as important tools in surgical training. The purpose of this study was to compare pediatric and adult laparoscopic surgical skills, and gain insight into the upskilling in both groups.

Materials and Methods:

Adult- and pediatric-sized laparoscopic simulators were fitted with custom-built motion tracking hardware and software. Participants were recruited at the Education Booth of the 2012 combined SAGES/IPEG meeting. They each performed 1 adult and 1 pediatric intracorporeal suturing task. Velocity, acceleration, and range were studied in all degrees of freedom available during laparoscopic surgery (pitch, yaw, roll, and surge). Participants were stratified by expertise based on the traditional metrics of self-reported caseloads.

Results:

A total of 57 participants (15 novices, 7 intermediates, and 35 experts) were recruited. Experts had significantly higher extreme events in three of the four degrees of freedom when using the pediatric simulator than when using the adult simulator. Few significant differences were seen when comparing novice and intermediate performances on the adult versus pediatric simulator. Linear regression showed no difference between adult and pediatric experts tested on the adult or pediatric simulator.

Conclusions:

Experts were more challenged with the pediatric than with the adult suturing task. No difference was noted for overall averaged performance metrics comparing adult and pediatric experts suturing in adult versus pediatric simulators. As a participant's level of expertise improves, a model progressing from larger to smaller domains in the performance of defined laparoscopic tasks may, by virtue of its greater challenge, encourage psychomotor development.

Introduction

Laparoscopic models are increasingly recognized as important tools in surgical training.^1–7 Psychomotor expertise has not been compared between participants from different surgical specialties. This study was undertaken to look at whether laparoscopic skills are equivalent or not between experts in pediatric minimal access surgery (MAS) and experts in adult MAS. The secondary objective of this study was to compare the performance of an adult versus a pediatric laparoscopic suturing task among participants of different levels of expertise (novices, intermediates, and experts). The task consisted of performing a defined intracorporeal suturing task in both pediatric and adult box simulators. Although many parameters could be measured and assessed, we chose to focus on one summative parameter (time for task completion) and one formative parameter (analysis of motion). Time for task completion and motion analysis metrics were recorded. Analysis of motion has previously been validated for the formative assessment of this defined intracorporeal suturing task in both adult and pediatric laparoscopic box trainers.^8,9

As laparoscopic training advances, identification of tasks to help improve expertise, even among experts, may be increasingly valuable for purposes of ongoing self-directed learning. Identification of such tasks may also provide insight into a model for the development of advanced laparoscopic expertise.

Materials and Methods

The Pediatric Laparoscopic Surgery (PLS) simulator, a pediatric-sized trainer measuring 18 × 14 × 11 cm (Fig. 1), and the adult box trainer measuring 50 × 37 × 18.5 cm (Fig. 2), each consist of a polyethylene box with a high definition webcam. Tracking sensors are present on the inner surface of trocar ports through which the laparoscopic instruments are inserted. Motion was assessed in the four degrees of freedom available in traditional laparoscopic surgery: pitch, which corresponds to moving the instrument up or down around a fixed point; yaw, which corresponds to moving the instrument side-to-side around a fixed point; roll, which corresponds to pronation and supination, thus rotating the instrument; and surge, which corresponds to pushing the instrument in and pulling the instrument out. Pitch, yaw, and roll are rotational degrees of freedom; surge is a translational degree of freedom.

FIG. 1.

Left: Pediatric Laparoscopic Surgery (PLS) pediatric box trainer with laparoscopic webcam view; right: PLS box trainer posterior view, for size comparison with the adult box trainer (Fig. 2).

FIG. 2.

Adult box trainer.

Participants were recruited at the Education Booth of the 2012 combined SAGES/IPEG meeting, and were stratified into novice (<10 laparoscopic cases performed per year), intermediate (10–50 laparoscopic cases per year), and expert (>50 laparoscopic cases per year) groups based on self-reported caseloads. Adult laparoscopic experts were defined as participants who performed >50 adult laparoscopic cases per year and ≤50 pediatric laparoscopic cases per year. Pediatric laparoscopic experts were defined as participants who performed >50 pediatric laparoscopic cases per year and ≤50 adult laparoscopic cases per year. Any laparoscopic expert who performed >50 adult and >50 pediatric laparoscopic cases per year was excluded from analysis.

Participants performed a standard intracorporeal suturing task in the adult and pediatric box trainers. The pediatric intracorporeal suturing task involved using smaller instruments (3 mm versus 5 mm instruments for the adult task) and a smaller penrose drain, in a smaller domain (the PLS box trainer is one-twelfth the overall volume of the adult box trainer).

Three motion analysis parameters (MAPs) were recorded: velocity, acceleration, and 90% range (see following paragraphs in this section). Time to task completion was also recorded.

To assess performance using motion analysis, three MAPs were defined on the basis of velocity, acceleration, and range of motion.⁸ The metrics for velocity and acceleration are the number of events measured that fall outside 2 standard deviations (SDs) of a participant's overall velocity and acceleration data, referred to as “the number of extreme velocity and acceleration motion events” (Fig. 3).

FIG. 3.

Model of extreme motion events based on acceleration or velocity output from a motion-tracking sensor during an intracorporeal suturing task. The solid central line represents the participant's hypothetical mean. The dashed lines hypothetically delineate all data lying within 2 standard deviations. The arrows indicate each hypothetical “event” where the participant goes beyond 2 SDs of their mean: we refer to these events as “extreme motion events.” SD, standard deviation.

Velocity and acceleration extreme motion events have had construct validity established for this defined intracorporeal suturing task in an adult as well as a pediatric laparoscopic box trainer.^8,9 They allow discrimination based on “jerkiness” of movement: every time an operator makes a rough, uncontrolled movement, this is represented by a sudden increase in the amplitude measurement of the motion sensor. The number of these extreme velocity and acceleration events is higher in the group of less experienced participants, and decreases with increasing level of expertise. For the purpose of this study, the analysis of motion was carried out by the participants' dominant hand.

Range of motion (90%) represents the range of movement made by each participant in the four degrees of freedom available. We used the 90% interval as a measure of the space used by each participant while completing the intracorporeal suturing task. The rationale was that by removing the extreme 10% of the range of motion, we would have an overall understanding of each participant's use of the space within the box trainer. Range of motion (90%) was not discriminatory between levels of expertise in previous studies.^9,10

The MAPs described earlier were measured for all candidates, and cross-referenced to their level of expertise.

Statistical analysis

After the tracking data were compiled, we calculated the number of extreme velocity events, the number of extreme acceleration events, and the 90% range of motion in each of the four degrees of freedom available. Importantly, each SD analysis was based on the individual participant's own pattern of motion and mean. Time to task completion was also recorded. Statistical analyses were performed using STATA software (StataCorp. 2013. Stata Statistical Software: Release 13; StataCorp LP, College Station, TX). Statistical significance was considered as a P value <.05.

Paired two-tailed t-test analysis for the number of extreme velocity and acceleration events and the 90% range of motion in all four degrees of freedom were employed to compare novices', intermediates' and experts' performances on the laparoscopic adult versus pediatric box trainers. Time to task completion comparing expertise levels' performance on the adult and pediatric box was also calculated using a paired two-tailed t-test analysis. Linear regression determined whether adult versus pediatric MAS expertise predicted performance metrics in the adult versus pediatric box trainers.

Results

A total of 57 participants (15 novices, 7 intermediates, and 35 experts) took part in the study. Novices and intermediates were not subclassified as adult or pediatric. Experts were subclassified as either adult experts or pediatric experts. Within each cohort of participants (novices, intermediates, and experts), the time required to complete the defined intracorporeal suturing task and the number of extreme motion events were greater in the smaller simulator than in the larger simulator (Table 1). This trend attained significance among experts (P < .05) for all performance metrics, except the MAPs in the roll degree of freedom (Fig. 4). Simply stated, experts took longer and had more extreme motion events (both acceleration and velocity) in three of the four degrees of freedom when completing the defined intracorporeal suturing task within the pediatric simulator than within the adult simulator.

FIG. 4.

Comparison of expert (n = 35) performance metrics for the adult suturing versus pediatric suturing tasks.

Table 1.

Comparison of Novice, Intermediate, and Expert Performance Metrics for the Adult Versus Pediatric Intracorporeal Suturing Task

	Novice (n = 15)			Intermediate (n = 7)			Expert (n = 35)
	Adult task (mean ± SD)	Pediatric task (mean ± SD)	P	Adult task (mean ± SD)	Pediatric task (mean ± SD)	P	P value adult task (mean ± SD)	Pediatric task (mean ± SD)	P
Time (seconds)	466.3 ± 278.7	658.5 ± 295.7	.0241	360.4 ± 238.3	414.6 ± 132.8	.4479	216.7 ± 108.8	297 ± 190.1	.0155
Pitch
Extreme velocity events	76.3 ± 61.2	97.5 ± 74.8	.2101	44.6 ± 34.6	64.6 ± 22.9	.0963	23 ± 16.3	36.7 ± 30.8	.0179
Extreme acceleration events	128.3 ± 84.2	150.1 ± 117.1	.4024	77.7 ± 62.7	104.3 ± 35.2	.1448	42 ± 25	62.3 ± 45.8	.0084
Range 90%	10.3 ± 1.6	10.5 ± 2.4	.7569	8.9 ± 1.6	10.7 ± 2.3	.1306	10.6 ± 2.8	10.6 ± 2.8	.9391
Yaw
Extreme velocity events	83.5 ± 52.6	102.1 ± 74.5	.2876	54.3 ± 46.4	69.7 ± 24.3	.2266	31.5 ± 19.9	51.6 ± 36.8	.0020
Extreme acceleration events	139.9 ± 84.4	158.2 ± 107.8	.4460	97.9 ± 77.2	123.9 ± 52.3	.1874	57.5 ± 32.4	86.1 ± 52	.0006
Range 90%	18.1 ± 3.9	11.5 ± 2.1	.0001	16.7 ± 5.4	10.4 ± 2.6	.0162	18.3 ± 3.9	10.5 ± 2.6	<.0001
Roll
Extreme velocity events	24.7 ± 20.8	24 ± 21.9	.8958	15.3 ± 12.3	19.7 ± 16.9	.5266	6.9 ± 8.3	5.7 ± 9.6	.5300
Extreme acceleration events	46.8 ± 37.5	41.3 ± 47.3	.6868	24.1 ± 20.1	32.3 ± 30	.4892	11.8 ± 15	11.6 ± 19	.9546
Range 90%	1533.8 ± 756.5	3024.4 ± 3625.8	.1087	1674.4 ± 1467.3	1602.1 ± 573.8	.9215	1972.4 ± 1242^a	1893.2 ± 1131.6	.7624
Surge
Extreme velocity events	41.6 ± 25.9	63.7 ± 38.7	.0272	32.1 ± 34.1	45.9 ± 21	.2899	14.5 ± 13.4	25.1 ± 18.2	.0003
Extreme acceleration events	66.6 ± 46.5	99.1 ± 63.1	.0682	59 ± 57.7	75.3 ± 28	.4326	20.5 ± 21.1	37.3 ± 26.1	.0002
Range 90%	6471.5 ± 2001.6	5701.1 ± 1827.1	.1708	5808.2 ± 842.9	7595.4 ± 3607.2	.2289	5639.7 ± 1786.8^a	10269.5 ± 12903.5	.0473

Data not available for 2 participants (n = 33).

All P values calculated using two-tailed paired t-test analysis.

Bold values indicate p < 0.05.

SD, standard deviation.

Among novices, only the time to task completion, surge extreme velocity events, and yaw range 90% were significantly different when comparing intracorporeal suturing in the adult versus pediatric simulator (Table 1 and Fig. 5). Among intermediates, the only significant difference noted in the MAPs was yaw range 90% (Table 1 and Fig. 6).

FIG. 5.

Comparison of novice (n = 15) performance metrics for the adult suturing versus pediatric suturing tasks.

FIG. 6.

Comparison of intermediate (n = 7) performance metrics for the adult suturing versus pediatric suturing tasks.

Interestingly, the average number of extreme events and average time to task completion decrease in a stepwise manner as follows: novice group performing the pediatric task, novice group performing the adult task, intermediate group performing the pediatric task, intermediate group performing the adult task, expert group performing the pediatric task, and expert group performing the adult task (Fig. 7). The only overlap seen between expertise levels' average number of extreme motion events was the surge metrics between intermediates performing the pediatric task and novices performing the adult task. In other words, the suturing task within the pediatric simulator was more difficult as assessed by formative (analysis of motion) and summative (time for task completion) means.

FIG. 7.

Schematic illustrating decreasing difficulty completing tasks as expertise increases, with the pediatric intracorporeal suturing task being more difficult than the adult intracorporeal suturing task for all levels of expertise (based on results section).

Overall, linear regression showed no difference between adult and pediatric experts tested on the adult and pediatric simulators (Table 2): linear regression analysis using adult MAS experts' and pediatric MAS experts' performances showed no difference in the predictive value of a model differentiating between adult and pediatric MAS experts based on this data set. With the exception of the yaw range 90% comparison, there were no significant P values (<.05), and all R² values were small (not approaching 1), indicating the model has no predictive ability. Given that the R² value for the yaw range 90% comparison is only 0.112, the importance of the difference between adult and pediatric expert performances is minimal. For the yaw range 90% data, the pediatric MAS experts used on average slightly smaller 90% ranges than the adult MAS experts (9.7 versus 11.1, respectively).

Table 2.

Linear Regression Comparing Adult and Pediatric Experts' Performances of the Adult and Pediatric Laparoscopic Suturing Tasks

	Linear regression values for expert comparison (adult expert n = 16, pediatric expert n = 19)
	Adult task		Pediatric task
	R²	P	R²	P
Time (seconds)	0.069	.128	0.026	.357
Pitch
Extreme velocity events	0.013	.507	0.097	.068
Extreme acceleration events	0.011	.549	0.036	.275
Range 90%	<0.001	.871	0.080	.099
Yaw
Extreme velocity events	0.017	.454	0.046	.214
Extreme acceleration events	0.019	.426	0.011	.551
Range 90%	0.015	.490	0.112	.049
Roll
Extreme velocity events	0.104	.058	0.013	.507
Extreme acceleration events	0.104	.058	0.002	.801
Range 90%	0.107^a	.064^a	<0.001	.943
Surge
Extreme velocity events	<0.001	.927	0.022	.397
Extreme acceleration events	<0.001	.884	0.015	.478
Range 90%	0.002^a	.789^a	<0.001	.897

Data not available for 2 participants (adult expert n = 14).

Bold values indicate p < 0.05.

Discussion

Experts struggled more with the pediatric intracorporeal suturing task than with the adult suturing task. This was demonstrated by higher number of extreme motion events and increased time to complete the task in the pediatric simulator than in the adult simulator. Because the difference between the two simulators is essentially related to size (smaller domain, smaller penrose drain, smaller instruments, and smaller sutures), our interpretation of the results is that miniaturizing increases the challenge of the task.

Given these findings, as a participant's level of expertise improves, a model progressing from larger to smaller domains in the performance of defined laparoscopic tasks may, by virtue of its greater challenge, encourage psychomotor development. One implementation of this finding would be to create laparoscopic suturing educational training programs that, once a participant masters a skill in a larger domain, then challenge the participant to further their skills by mastering a similar skill in a smaller domain, with smaller instruments and targets. All experts may benefit from deliberate practice on more challenging pediatric simulators. Previous educational models have focused on training novices and intermediates,^11–16 without considering whether the models could be extended to address improving surgical technique among experts.

Our results did not demonstrate significant differences when comparing novices and intermediates performing the adult suturing versus the pediatric suturing tasks. There are two possible explanations. Given that the trend is still toward higher number of extreme motion events and longer time to task completion for the pediatric versus the adult task, it is possible that the sample sizes were too small to demonstrate significant differences. The other plausible explanation is that novice and intermediate participants struggled equally with both the adult and the pediatric tasks, and struggled significantly^8,9 more with both tasks than the experts struggled with either suturing task. The implication, from an educational standpoint, is that nonexperts should master a task in an easier environment (adult simulator) before moving to a more challenging environment (smaller domain). A similar finding was ascertained for an advanced suturing task,¹⁰ wherein it was determined that novices and intermediates may benefit more from first mastering a standard intracorporeal suturing technique, whereas laparoscopic experts may benefit from practicing a more ergonomically challenging laparoscopic suturing task. We are in the process of exploring collaborative opportunities to implement and further test such a program.

No difference was noted for overall averaged performance metrics when comparing adult and pediatric experts suturing in adult versus pediatric simulation environments. The implication is that for this defined intracorporeal suturing task, there may be significant crossover of psychomotor skills acquired by adult and pediatric experts. What is more, this may highlight the significance of adaptive expertise. True adaptive experts can apply elements of their expertise to circumstances outside their comfort zone. In this study, adult experts performed as well as pediatric experts in the pediatric simulator, and pediatric experts performed as well as adult experts in the adult simulator. As mentioned earlier, both struggled more on the pediatric simulator than on the adult simulator, possibly indicating that the task became more challenging in a smaller domain.

Limitations

We continue to explore our evolving understanding and definition of expertise. The limits of self-reported caseloads as a benchmark for expertise are possibly reached with studies of the kind presented here. This concern, regarding self-reported caseloads as oversimplified and unreliable, has also been raised by colleagues.¹³ The performances recorded and analyzed for this study demonstrate a wide variation of technique and skill within the novice, intermediate, and expert groups. We are currently exploring alternative means of assessing pure psychomotor expertise.

Footnotes

Acknowledgments

We thank all surgeons, residents, and students who contributed their time to this study, and for their valuable input and expertise.

Disclosure Statement

No competing financial interests exist.

References

Fried

, Feldman

, Vassiliou

, et al. Proving the value of simulation in laparoscopic surgery. Ann Surg, 2004; 240:518–528.

Dhariwal

, Prabhu

, Dalvi

, Supe

. Effectiveness of box trainers in laparoscopic training. J Minim Access Surg, 2007; 3:57–63.

Palter

, Grantcharov

. Simulation in surgical education. Can Med Assoc J, 2010; 182:1191–1196.

Scott

, Bergen

, Rege

, et al. Laparoscopic training on bench models: Better and more cost effective than operating room experience?. J Am Coll Surg, 2000; 191:272–283.

Gurusamy

, Nagendran

, Toon

, Davidson

. Laparoscopic surgical box model training for surgical trainees with limited prior laparoscopic experience. The Cochrane Library. Cochrane Database Syst Rev, 2014:CD010478. DOI: 10.1002/14651858.CD010478.pub2.

Beyer

, Troyer

, Mancini

, Bladou

, Berdah

, Karsenty

. Impact of laparoscopy simulator training on the technical skills of future surgeons in the operating room: A prospective study. Am J Surg, 2011; 202:265–272.

Chang

, King

, Modest

, Hur

. Developing an objective structured assessment of technical skills for laparoscopic suturing and intracorporeal knot tying. J Surg Educ, 2016; 73:258–263.

Nasr

, Carrillo

, Gerstle

, Azzie

. Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: Validation and potential use in teaching and assessing surgical skills. J Pediatr Surg, 2014; 49:791–794.

Trudeau

, Nasr

, Carrillo

, Gerstle

, Azzie

. Construct validity and educational role for motion analysis in a laparoscopic trainer. Surg Endosc, 2015; 29:2491–2495.

10.

Trudeau

, Carrillo

, Nasr

, Gerstle

, Azzie

. Educational role for an advanced suturing task in the pediatric laparoscopic surgery simulator. J Laparoendosc Adv Surg Tech A, 2017; 27:441–446.

11.

Oropesa

, Sanchez-Gonzalez

, Lamata

, et al. Methods and tools for objective assessment of psychomotor skills in laparoscopic surgery. J Surg Res, 2011; 171:e81–e95.

12.

Luursema

, Rovers

, Groenier

, van Goor

. Performance variables and professional experience in simulated laparoscopy: A two-group learning curve study. J Surg Educ, 2014; 71:568–573.

13.

Retrosi

, Cundy

, Haddad

, Clarke

. Motion analysis-based skills training and assessment in pediatric laparoscopy: Construct, concurrent, and content validity for the eoSim simulator. J Laparoendosc Adv Surg Tech A, 2015; 25:944–950.

14.

Hiemstra

, Chmarra

, Dankelman

, Jansen

. Intracorporeal suturing: Economy of instrument movements using a box trainer model. J Minim Invasive Gynecol, 2011; 18:494–499.

15.

Oropesa

, Chmarra

, Sanchez-Gonzalez

, et al. Relevance of motion-related assessment metrics in laparoscopic surgery. Surg Innov, 2013; 20:299–312.

16.

van Dongen

, Ahlberg

, Bonavina

, et al. European consensus on a competency-based virtual reality training program for basic endoscopic surgical psychomotor skills. Surg Endosc, 2011; 25:166–171.