Abstract
Background:
To increase complex minimally invasive skills (MIS), frequent training outside the clinical setting is of uttermost importance. This study compares two low-cost pediatric MIS simulators, which can easily be used preclinically.
Materials and Methods:
The LaparoscopyBoxx is a portable simulator without a tracking system, with costs ranging from €90 to €315. The EoSim simulator has a built-in camera and tracking system and costs range from €780 to €1800. During several pediatric surgical conferences and workshops (January 2017–December 2018), participants were asked to use both simulators. Afterward, they completed a questionnaire regarding their opinion on realism and didactic value, scored on a five-point Likert scale.
Results:
A total of 50 participants (24 experts and 25 target group, one unknown) evaluated one or both simulators. Both simulators scored well on the questionnaire. The LaparoscopyBoxx scored significantly better regarding the “on screen representation of the instrument actions” (mean 4.2 versus 3.5, P = .001), “training tool for pediatric surgery” (mean 4.4 versus 3.9, P = .005), and “appealing take-home simulator” (mean 4.6 versus 4.0, P = .002).
Conclusion:
The simulators tested in this study were both regarded an appealing take-home simulator. The LaparoscopyBoxx scored significantly better than the EoSim, even though this is a low budget simulator without tracking capabilities.
Introduction
In pediatric surgery, there is an increasing tendency to use minimally invasive surgery (MIS) for the correction of congenital malformations, such as duodenal atresia, congenital diaphragmatic hernia, or esophageal atresia. Because the incidence of congenital malformations is relatively low, the specific surgical procedures to correct these malformations are not often performed.1–4 Furthermore, the skills required to perform these procedures correctly and safely are very specific and advanced. To master these complex procedures, pediatric surgeons need to be trained regularly also have to keep their skills on a high level after they have overcome their learning curve. This can be challenging in the clinical setting. Training during a workshop or course is good to learn to know the procedure and to obtain the first skills and explanation. However, continuous training or warming up before a procedure enhances the performance and reduces operation time more effectively.5–7 Unsupervised home-based training on training models or simulators can be beneficial to gain and retain MIS skills and could lead to an individualized schedule, based on the trainees need. 8 Training on a take-home box simulator was proved to be associated with an improvement in basic laparoscopic skills and enabled access to simulation training. 9 To enable this continuous training, it is important that the simulator is portable and not fixed in a hospital or skills center to ensure that at-home training is possible. However, most simulators are not suitable for this purpose. They are not portable due to large housings and the need for a laparoscopy tower or computer-based screen for visualization of the performance. Another important factor is the affordability of the simulator. Most simulators are too expensive to use at home, especially for a surgical resident or surgeons from low- and middle-income countries.8,10–13
There are not many MIS simulators available to learn pediatric surgical skills, let alone portable and affordable simulators. Therefore, the aim of this study was to compare two affordable take-home simulators suited as pediatric MIS training tools and assess whether they would be sufficient for home-based training purposes.
Materials and Methods
Simulators
The simulators used in this study were the LaparoscopyBoxx (Fig. 1) and the EoSim (Fig. 2) laparoscopic simulators.

The LaparoscopyBoxx simulator (pediatric version) with a tablet.

The EoSim minimally invasive skills simulator connected to a laptop.
LaparoscopyBoxx
The LaparoscopyBoxx is a self-assemble training box, consisting of multiple wooden parts. It is a low-cost MIS training simulator, which is lightweight and easy to transport. There are two versions available, and the advanced, pediatric version is used in this study (Fig. 1). The cost of a basic version is €90, whereas the advanced/pediatric version costs €100. Laparoscopic instruments can be purchased with it as well (€295 including three 5 mm instruments and €315 including three 3 mm instruments). 14
The distances and height of the simulator can be adjusted, to resemble the pediatric size. There are five trocar openings that can be used in this simulator to choose the optimal trocar distance. There is an opening in the center of the top panel, which is designed for the camera of a tablet or even a smartphone. However, it is also possible to use the simulator with a laparoscopic camera. In this study, the tablet option was used. The different task boards were fixed in the simulator by clicking them in the posterior pillars in the proper position.
EoSim laparoscopic simulator
The EoSim MIS is an augmented reality laparoscopic simulator by Eosurgical Ltd. (Edinburgh, Scotland, United Kingdom). It consists of a validated inanimate box trainer setup with a laptop and supplied software, to guide the trainee and track the instrument tips15–18 (Fig. 2). This simulator is available in three options, based on the tasks, one would like to train. The most basic version is suitable for basic skills and includes three 5 mm instruments for €780. For the pediatric version, including five 3 mm instruments, the costs are €1800. The simulator base is a suitcase, which can hold the instruments and task trays, which makes it easy to transport.
There is the opportunity to use a tablet as a screen, without the tracking option, but in this study, a laptop, connected to the simulator, was used as screen for optimal performance and user experience. The supplied SurgTrac software allowed for instrument tracking by colored markings at the tip of the instruments (blue for the left hand and red for the right hand). In addition, the software contains an online mentor function, where the performance video and parameters can be analyzed by a self-appointed mentor to provide feedback. The parameters tracked by the software were not collected for this study because there was no useful tracking option available for the specific pediatric surgical tasks used in this study.
Participants
Participants with a pediatric MIS background were recruited to participate in this study. For this evaluation, the aim was to include both “experts” and “target group” participants. Experts had performed at least >50 general MIS procedures and >50 basic pediatric MIS procedures or >10 advanced pediatric/neonatal MIS procedures. The target group had basic laparoscopic experience and at least some basic pediatric MIS experience. They were included during several national pediatric surgical courses in the Netherlands from January 2017 until October 2018, at the Pediatric Minimally Invasive Symposium, September 2018, Utrecht, the Netherlands and at the 11th European Pediatric Colorectal congress, December 6–8, 2018, Nijmegen, the Netherlands. Because there was no medical intervention and participation was voluntary, there was no Ethics Board approval required.
Protocol
Participants were asked to perform a MIS pediatric/neonatal task (Table 1) on the LaparoscopyBoxx and/or EoSim simulator, while using 3 mm instruments (needle holder, dissector, scissors). There was a choice of instruments, lengths, and manufacturers (LaproSurg, Aesculap [B. Braun], Medtronic, Karl Storz, and instruments without a brand). Next, a questionnaire was completed by all participants. The aim was for all participants to evaluate both simulators for the optimal comparison. However, all opinions are included in this study, regardless whether only one or both simulators were evaluated.
Tasks that Could Be Practiced in Either of the Simulators
Questionnaire
The questionnaire consisted of two parts. The first part of the questionnaire was on the demographics and clinical MIS experience. The MIS experience was defined as number of basic laparoscopic procedures (cholecystectomy and appendectomy), basic pediatric laparoscopic procedures (pyloromyotomy and appendectomy), and advanced pediatric laparoscopic procedures (including intracorporeal suturing). It concluded with an informed consent, which had to be signed by all participants. The second part of the questionnaire consisted of items regarding the technical aspects and the value of the simulator as a training tool and as take-home simulator, scored on a five-point Likert scale (1 = very unrealistic, 2 = unrealistic, 3 = neutral, 4 = quite realistic, and 5 = very realistic). A mean of >3.5 was considered a significantly better opinion than neutral and a mean of >4.0 was considered as a potent training tool. At the end of the questionnaire a comments option was included for participants to leave remarks about each simulator. The questionaire can be found in the Supplementary Data.
Statistics
Statistical analysis was performed using IBM SPSS Statistics 25. All values were represented as mean with standard deviation or as percentage. An independent samples t-test was used to determine significant differences in opinion between the simulators and significant differences in opinion between the expert and target group. A P value of <.05 was considered to be statistically significant.
Results
Demographics
A total of 50 participants were included in this study. Due to time restrictions and the availability of the simulators, not all participants were able to evaluate both simulators. The participants were divided into two groups: the expert group (24 participants: 20 pediatric surgeons, 1 pediatric urologist, 2 fellows pediatric surgery, 1 surgical resident) and the target group (25 participants: 4 pediatric surgeons, 4 general surgeons, 1 fellow pediatric surgery, 16 surgical residents), 1 participant did not complete the questions on the clinical experience. Twenty-six participants completed questionnaires on both simulators. The EoSim was evaluated by 32 participants (13 experts, 18 target group, 1 unknown). The LaparoscopyBoxx was evaluated by 44 participants (22 experts, 21 target group, 1 unknown), as shown in Table 2. There were no statistically significant differences between the group that evaluated the EoSim and the group that evaluated the LaparoscopyBoxx, and experts and target group participants were equally divided for both simulators.
Demographic Properties of the Participants
All values are stated in percentage (number), only the age is stated in median with range.
Four missing values in total group and three missing in both subgroups.
Three missing values in total and LaparoscopyBoxx group and two missing values in EoSim group.
One missing value in all groups.
EoSim
The EoSim was evaluated with means ranging from 3.3 to 4.0 in the target group and 3.6 to 4.5 in the expert group. A significantly lower score was given for “training tool for pediatric MIS training” by the target group compared to the expert group (3.6 versus 4.3, P = .048), and for “appealing take home simulator” (3.6 versus 4.5, P = .011), as shown in Table 3. When evaluating the total group, the overall mean scores were the highest for “Possibility to use own instruments” (4.1) and “Appealing take home simulator (4.0),” followed by “Trocar opening position” (3.9), Simulator interface (3.9), and “Training tool for pediatric MIS training” (3.9). “Ergonomics” (3.8) and “on screen representation of instrument actions” (3.5) were scored the lowest (Table 4).
Opinion on EoSim
Values are stated in mean (standard deviation). Statistically significant differences between the groups were calculated with the independent t-test, a P value of <.05 was considered a significant difference indicated by italics.
MIS, minimally invasive skills.
Opinion of the Total Groups on the Take-Home Pediatric Minimally Invasive Skills Simulators
All values are stated in mean (standard deviation). Statistically significant differences between the two simulators were calculated with the independent t-test, in which a P value of <.05 was considered a significant difference indicated by italics.
MIS, minimally invasive skills.
LaparoscopyBoxx
The LaparoscopyBoxx was evaluated with means ranging from 3.9 to 4.5 in the target group and 4.1 to 4.6 in the expert group. There were no significant differences in the opinion on the LaparoscopyBoxx between the expert and the target group (Table 5).
Opinion on LaparoscopyBoxx
Values are stated in mean (standard deviation). Statistically significant differences between the groups were calculated with the independent t-test, a P value of <.05 was considered a significant difference.
MIS, minimally invasive skills.
The overall mean scores were highest for “Appealing take-home simulator” (4.6), “Possibility to use own instruments” (4.4), and “Training tool for pediatric MIS training” (4.4), followed by “Simulator interface” (4.3) and “On screen representation of instrument actions” (4.2). “Trocar opening position” (4.1) and “Ergonomics” (4.0) were given the lowest score (Table 5).
Comparing the simulators
The LaparoscopyBoxx scored significantly better than the EoSim for the items “on screen representation of instruments in action” (4.2 versus 3.5, P = .001), “simulator interface and user friendliness” (4.3 versus 3.9, P = .005), and “possibility to use own instruments” (4.4. versus 4.1, P = .02) (Table 5). The features which make a simulator a potent training tool were scored significantly better for the LaparoscopyBoxx as well: “Training tool for pediatric MIS training” (4.4 versus 3.9, P = .005) and “appealing take-home simulator” (4.6 versus 4.0, P = .002). When analyzing only the Target group, the differences in the opinions between the simulators as training tools were even larger. As take-home simulator, the EoSim was rated a mean of 3.6 and the LaparoscopyBoxx a mean of 4.5 (P < .001). For the training of pediatric MIS skills, the EoSim was rated a mean of 3.6 by the target group as well, whereas the LaparoscopyBoxx was rated a mean of 4.3 (P < .001) (Tables 3 and 4).
The most often stated comment was that the webcam of the EoSim often lost focus, which made the visualization on the screen blurry.
Discussion
The EoSim and the LaparoscopyBoxx were both regarded an appealing pediatric MIS take-home simulator. It was noticeable that the LaparoscopyBoxx scored significantly better than the EoSim, for example, for the on-screen representation of the instruments. This could be based on the loss of focus on the screen, due to tracking system of the EoSim. The tracking system relies on the marked instrument tips and focuses the camera on these, however, surrounding colors or the use of other instruments could interfere with this feature. This might explain why the use of instruments scored lower for the EoSim as well despite the fact that all instruments can be used in both simulators. Another possible explanation is that the EoSim was used, as it is advertised, with a webcam and attached to a laptop instead of a tablet. The LaparoscopyBoxx was used, as it is designed with a tablet, making the simulator easy to use in any setting and keeping the costs low. Previous studies on the use of a tablet as screen showed good or even superior results for basic tasks with regard to on screen response and visualization, compared to a regular box trainer.19–22 This study shows that for more advance tasks, the results are regarded superior as well.
The EoSim, with more features such as a tracking system, seemed more promising at first sight. Yet, the more affordable LaparoscopyBoxx scored significantly better. The LaparoscopyBoxx (including 3 mm instruments) is one-sixth of the costs of the EoSim, making it suitable for affordable simulation-based training. In general, little is known about the costs of simulation-based training. A systematic review showed that only 1.6% of studies that focus on simulation in medical education provide cost comparison with another instruction approach. 23 Based on the limited studies available, they conclude that the cost-effectiveness varies depending on the specific implementation of the simulation-based training. 24 Studies evaluating the impact of simulation-based pretraining regimen before an extensive course concluded that pretraining decreased training time and expenses during the course.25,26 Furthermore, laparoscopic training using either high or low fidelity trainers can improve laparoscopic performance.9,24,27 This enhances the notion that there is no need for expensive high-fidelity trainers and that affordable simulators can be used to train the most important steps of difficult procedures. Both simulators used in this study are low fidelity simulators, which are affordable and easily transported to enhance home-based training. Practicing at home reduces the need for a skills laboratory in a hospital setting and can be done after working hours, both factors that are likely to make it more cost-effective. For basic tasks, at-home training has previously been proved to be significantly better than skills center-based virtual reality training of the same tasks.9,28 At-home training allows for deliberate practice and has the possibility to focus on specific component tasks, in contrast to the clinical setting.29,30 Focusing on specific tasks will improve the transfer of the acquired skills to the clinical setting.22,31, 32 Therefore, it is of uttermost importance that a simulator specifically designed for pediatric surgery is used to focus on the specific and advanced skills needed. In the currently studied simulators, it is possible to adjust the simulator to mimic a procedure in a larger child, infant, and neonate. This could enhance the value of home-based training for pediatric surgery and encourage the surgical curricula to invest in home-based training. To adequately assess the additive value of at-home training on the clinical outcome, studies on skill improvement and transferability of skills to the clinical setting are needed. These studies should focus on whether simulation-based training on these simulators results in skill improvement and on the transferability of the newly acquired skills to the clinical setting. Furthermore, research is needed focusing on whether this improvement in skills results in an improved clinical outcome on the short term or long term.
Limitations
Because the EoSim was developed with a webcam-based tracking system, this version has been used in this study. The focus was not on the type of visualization on the screen, however, the results show that this could be a factor in the opinion of the participants. Possibly using the EoSim with a tablet would have affected the outcome of that feature in the questionnaire. Another limitation results from the fact that not all participants were able to try both simulators. This resulted in less participants giving their opinion on the EoSim. Overall, the experts scored the EoSim higher than the target group, therefore, the inclusion of more experts could shift the ratings of the EoSim slightly, although experts and target group participants were equally divided for both simulators.
Conclusion
Both take-home simulators scored well in this study and they were both considered a potent training tool for pediatric surgical home-based training. However, the more affordable LaparoscopyBoxx was considered the best take-home pediatric MIS simulator.
Footnotes
Disclosure Statement
Drs. G.M.J.B. and Dr. S.M.B.I.B. are co-owners of PediatrickBoxx, which is a small non-for-profit cooperation that has developed the pediatric version of the LaparoscopyBoxx used in this study. This cooperation is founded to develop and sell the pediatric training models for low budget prices, to make sure all pediatric surgeons over the world can have access to the models. The minimal profit on selling the models is used in the development and production of new models. Drs. M.J., Drs. E.L., Dr. B.H.V., and Dr. I.d.B. have no conflicts of interest or financial ties to disclosure. Informed consent was obtained from all individual participants included in the study.
Funding Information
No funding was received for this article.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
