Effect of Playing Video Games on Laparoscopic Skills Performance: A Systematic Review

Abstract

Background and Purpose:

The advances in both video games and minimally invasive surgery have allowed many to consider the potential positive relationship between the two. This review aims to evaluate outcomes of studies that investigated the correlation between video game skills and performance in laparoscopic surgery.

Methods:

A systematic search was conducted on PubMed/Medline and EMBASE databases for the MeSH terms and keywords including “video games and laparoscopy,” “computer games and laparoscopy,” “Xbox and laparoscopy,” “Nintendo Wii and laparoscopy,” and “PlayStation and laparoscopy.” Cohort, case reports, letters, editorials, bulletins, and reviews were excluded. Studies in English, with task performance as primary outcome, were included. The search period for this review was 1950 to December 2014.

Results:

There were 57 abstracts identified: 4 of these were found to be duplicates; 32 were found to be nonrelevant to the research question. Overall, 21 full texts were assessed; 15 were excluded according to the Medical Education Research Study Quality Instrument quality assessment criteria. The five studies included in this review were randomized controlled trials. Playing video games was found to reduce error in two studies (P 0.002 and P 0.045). For the same studies, however, several other metrics assessed were not significantly different between the control and intervention group. One study showed a decrease in the time for the group that played video games (P 0.037) for one of two laparoscopic tasks performed. In the same study, however, when the groups were reversed (initial control group became intervention and vice versa), a difference was not demonstrated (P for peg transfer 1 – 0.465, P for cobra robe – 0.185). Finally, two further studies found no statistical difference between the game playing group and the control group's performance.

Conclusion:

There is a very limited amount of evidence to support that the use of video games enhances surgical simulation performance.

Introduction

Laparoscopic surgery needs a specific set of skills because of the fulcrum effect, hand-eye coordination, and the transformation of a three-dimensional working space into a two-dimensional monitor. Playing video games has become a ubiquitous activity in today's society, and games that necessitate interaction with virtual reality situations can lead to the acquisition of complex skills. Video games affect the visuospatial skills of a person, which can have a significant impact on laparoscopic skills performance. In addition, spatial tests have been shown to predict the potential to learn open and laparoscopic surgical skills.¹

Video game-based methods are a popular medium for training in laparoscopy. Previous research demonstrates that video game attributes, such as level of difficulty, realism, and interactivity, affect learning outcomes in game-based learning environments.² There are a variety of arguments for the adoption of video game-based training tools. Among these is the potential to capitalize on the motivational draw of game play. Knowledge and skills acquired during game play have been demonstrated to have been transferred to real world tasks.³ The research on video game-based training remains limited, however. Video game-based training may be accepted in medical education programs, but only if it is proved to be effective in improving patient outcomes.

This systematic review aims to evaluate outcomes of studies that investigated the correlation between video game skills and performance in laparoscopic surgery.

Methods

The systematic review was conducted according to the PRISMA (Preferred Reporting Items for Systematic reviews and Meta-Analyses) guidelines.⁴ A systematic search of the literature was performed using the PubMed and EmBase databases. The MeSH terms and keywords “video games and laparoscopy,” “computer games and laparoscopy,” “Xbox and laparoscopy,” “Nintendo Wii and laparoscopy,” and “PlayStation and laparoscopy” were used for all journal articles prior to December 2014. Reference lists were also assessed to identify additional studies.

Two authors conducted the primary search, simultaneously and independently. The identified studies were assessed for relevance by reading the abstracts of the studies. In the case of disagreement between the two authors, a third author was asked to advise. Studies were included only if the consensus of all three authors involved in this process was achieved. Reviews, cohort, case reports, letters, editorials, and bulletins were excluded. Studies in English, with task performance as the primary outcome, were included. The full text was retrieved for all remaining studies, and two authors assessed their relevance independently. The same process, as described above, was followed in case of disagreement.

Quality assessment

To ensure that only high-level evidence was identified for review, each article was assessed for quality before inclusion in the study. The Medical Education Research Study Quality Instrument (MERSQI score) was used to assess study design, sampling, type of data (subjective or objective), validity, data analysis, and outcomes.⁵ Articles were analyzed and MERSQI items were scored on ordinal scales and summed to determine a total score. The maximum score for each domain was 3, producing a maximum possible MERSQI score of 18.⁵ MERSQI scores identify areas of methodologic strengths and weaknesses in articles.⁶ All studies were individually assessed to be 14 or over.

Search outcome

The search period for this review was 1950 to 2014. There were 57 abstracts identified (Fig. 1). Four of these were found to be duplicates; 32 were found to be nonrelevant to the research question. Overall, 21 full texts were assessed. Of them, 15 were excluded according to the MERSQI quality assessment criteria. Five randomized controlled trials were therefore reviewed.

FIG. 1.

Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) flow diagram showing the total number of hits followed by short-listing of studies for inclusion in the analysis. CINAHL = Cumulative Index to Nursing and Allied Health Literature; MERSQI = Medical Education Research Study Quality Instrument.

Results

The five studies included in this review were analyzed in chronologic order (Table 1). Plerhoples and associates⁷ performed a prospective, randomized controlled trial assessing surgical performance on a simulator after the use of a video game. The 40 surgical trainees, characterized as laparoscopy novices, were randomized to an intervention or control group. Subjects in the first group played two games on the Apple iPhone (Cupertino, CA) before performing two laparoscopic tasks on a virtual reality (VR) simulator. The control group performed the two laparoscopic tasks without any game play.

Table 1.

Summary from Results from the Studies Included

	Paricipants	Results				Findings
Rosenberg et al, 2005	11 medical students	No statistically significant difference between pre- and posttraining.				Practicing video games for 2 weeks did not improve laparoscopic skills.
Schilickum et al, 2009	40 medical students		Half Life game	Chess Master game	Control	Video game training improved surgical performance on the laparoscopic simulator.
		Total scoreMist –VR p	0.035	0.008	0.13
		Efficiency of screening on GI mentor P	0.009	0.67	0.07
		Surface examined on GI Mentor p	0.039	0.20	0.79
Boyle et al 2011	22 medical students		Intervention	Control	P value	Results reached a statistical significance only for bead dropping task.
		Glove Cutting, Time	406	435	NS
		IPL	11,698	13,200	NS
		IS	2793	3198	NS
		Virtual task time	371	385	NS
		IPL	7270	7330	NS
		IS	1135	1075	NS
		Bead dropped	1	0.66
Plerhoples et al, 2011	40 surgical residents (postgraduate years 1–4)		Intervention	Control	P value	Warm-up using a mobile device resulted in significant reduction in errors.
		Object positioning task, errors	0.2	0.7	0.01
		Tissue manipulation task, errors	0.15	0.55	0.05
		Total score, errors	0.35	1.25	0.002
Jalink et al 2014	29 laparoscopic experts (surgeons, residents, gynecologists performed >30 laparoscopic procedures in last 3 years		Intervention	Control	P value
		Peg transfer score (day 1)	122.77 seconds	143.31 seconds	0.037
		Cobra rope score (day 2)	19.61% increased score	0.77% decreased score	0.002	Warm-up using a video game significantly improved score in peg transfer on day 1 between control and intervention and on cobra rope on day 2 between control and intervention.

IPL = instrument path length; IS = instrument smoothness.

The metrics that were tested and compared between the two groups included time, path length, smoothness, hand dominance (hand usage), and errors. A statistically significant difference was only established for reduction of errors in the intervention group (task 1—object positioning, P value 0.01; task 2—tissue manipulation task, P value 0.05; and overall for both tasks, P value 0.002). The statistical significance is at 0.05, and it is marginal for the tissue manipulation result.

Boyle and colleagues⁸ conducted a randomized study, recruiting 22 medical students with no laparoscopic experience. Subjects were randomized into two groups. Both groups were asked to perform two physical tasks (bead transfer and glove cutting) and a virtual laparoscopic task (on ProMIS, Haptica, Boston, MA) at baseline. Only group 2 then proceeded to play four games for 15 minutes on the Nintendo Wii™. Both groups repeated the same physical and virtual tasks 7 days later. Improvement in performance was compared between the two groups. Improvement in instrument path and smoothness was not statistically significant. Bead dropping occurred less within the intervention group; however, the result is marginally significant with a P value of 0.045.

Schlickum and coworkers⁹ also performed a prospective trial assessing whether systematic training on video games improves performance on virtual reality endoscopic simulators. Thirty medical students were stratified and consequently randomized into two groups. These were matched to a control group. All three groups underwent baseline testing of their abilities on a laparoscopic and an endoscopic simulator (Mist-VR^® Ethiskill, Cincinnati, OH). Consequently, the two intervention groups practiced on a different video game each, Chessmaster (CM) and Half Life (HL), respectively, for 5 weeks.

The two intervention groups and the control group were reassessed on the same baseline procedures after the completion of the 5 months “video game playing period.” The authors then compared the performance on the second test to the performance at baseline for each individual group. The two intervention groups found to have a significantly better performance during the second testing on the VR simulator MIST-VR (Ethiskill, Cincinnati, OH) (P value 0.035 and 0.008, HL and CM, respectively). HL also had a statistically significant improvement for both tasks performed on the endoscopy simulator GI Mentor II (Simbionix, Cleveland, OH) (P value 0.009 efficiency of screening, P value 0.039 surface examined). No statistically significant improvement was established for the control group or for the CM group on the endoscopy simulator.

Rosenberg and colleagues¹⁰ performed a randomized controlled trial asking whether video games could be used to predict or improve laparoscopic skills. Eleven medical students were recruited. They were asked to play three video games for 30 minutes and consequently perform four laparoscopic tasks on anesthetised pigs. The participants were then randomized to two groups—the control (n = 6) and the training (n = 5) group. The latter played video games for 2 weeks. All participants were then called back to perform the same four tasks. On both occasions, a consultant urologist or trainee assessed the performance of both groups. The laparoscopic task measurement that best correlated with videogame performance was the time taken to complete the objectives (P ≤ 0.05). There was no statistical difference in improvement of performance between the training and the control group.

Jalink and associates¹¹ conducted a prospective randomized study to look at whether a video game “warm-up” can improve one's basic laparoscopic skills. There were 29 laparoscopic experts recruited. Participants were randomized into two groups. The study was conducted in 2 days. On day 1, group 1 acted as a control group and group 2 as an intervention group; on day 2, the roles were reversed. On both days, the control group performed two laparoscopic tasks (peg transfer and copra rope drill). The intervention group performed a 15-minute warm-up before performing the same laparoscopic tasks as the control group. On day 1, the intervention group performed peg transfer significantly faster than the control group (P 0.037), but there was no significant difference in the time taken to perform the cobra rope drill on day 1. Similarly, there was no difference in the time taken to perform either of the laparoscopic tasks on day 2 (P 0.465, P 0.185).

Discussion

Five randomized controlled trials, assessing the effect of video game use on laparoscopic surgical skills, are included in this review.^7

–11 Two studies showed a significant reduction in error after the use of video games.^7,8 Several other variables (time, path length of instruments, smoothness), however, were assessed in these two studies without yielding any significant difference between control and intervention groups.^7,8 One study¹¹ showed a reduction in time to perform three of four laparoscopic tasks after video game preparation. Two studies showed no difference between control (no video game warm-up) and intervention group (video game warm-up).^9,10

Plerhoples and coworkers⁷ made a considerable effort to eliminate confounders through stratification and randomization. There were, however, 15% more experienced trainees (>41 laparoscopic procedures performed) in the intervention group compared with the control group. One can also argue that trainees who performed more than 41 laparoscopic procedures are not novices, as proclaimed by the authors. Furthermore, no power calculations were performed, and inclusion and exclusion criteria are not clear. Conversely, the assessment tool, ProMIS laparoscopic simulator, has been previously validated for evaluation purposes and provides objective “built in” assessment metrics,^12
–14 and the sample size of the study is acceptable.

Boyle and associates⁸ used blocked randomization.⁸ This method can lead to predictability of allocation, thus introducing selection bias.¹⁵ The limited sample size enhances this possibility. Stratification before randomization may have been appropriate. Furthermore, the significant dropout rate and the lack of sample size calculations compromises further the reliability of results. Conversely, the simulator used for evaluation has been previously validated for that purpose,^12
–14 and the methods used were described adequately to allow retest reliability.

The design of the study by Schlickum and colleagues⁹ is somewhat flawed. The control group was not included in the randomization process, and the comparisons made were largely intra- and not intergroup (performance on the MIST-VR, on GI Mentor)—i.e., the subjects in the HL, CM, and control group acted as their own controls. This limits severely the reliability of the findings, because the sample size for each comparison is restricted to 15, 15, and 10 participants, respectively.

Therefore, the lack of statistical significance for the control group may be attributed to the extremely limited sample size (n = 10). Furthermore, the noninclusion of the control group in the randomization process raises concerns about the similarity of the groups at baseline, an issue that may extend beyond the increased visuospatial ability of the control group, as identified by the authors. Conversely, the authors used clear inclusion and exclusion criteria; they conducted a blinded study and used a virtual reality simulator that has been previously validated for assessment purposes.¹⁶

The study by Rosenberg and coworkers¹⁰ has a very limited sample size, which was not based on sample size calculations. This raises concerns about the equality of groups at baseline. For instance, the sex ratio is 2:9 M:F. Despite the fact that a questionnaire concerning the baseline characteristics of the participants was filled in, these do not seem to have been taken into consideration during the analysis of the results. Furthermore, the tasks and the video game performance were assessed by more than one assessor of variable experience; however, there is no evaluation of the inter-rater variance.

In their study, Jalink and associates¹¹ recruited laparoscopic experts. This differs from the other studies analyzed that used novices. While the groups were randomized using an online randomizer, the process is not fully explained. Baseline demographics are listed but not described for each group, and therefore one cannot identify whether one group may be more experienced in laparoscopy or previous game experience compared with the other group. The authors did explicitly describe inclusion and exclusion criteria and performed a power calculation to achieve a desired sample size.

The two assessors were blinded to the study participants with intraclass correlation testing, but a third reviewer, who only took part in the study if there were wide interobserver reliability, was not blinded. This could raise a possibility of introducing bias by the third reviewer. The study did ensure that all participants did not introduce confounding factors—e.g., avoidance of laparoscopic procedures and video gaming during the 2 days of the study. Statistical tests were appropriately used and all results between the two groups were described whether or not they were significant.

The authors concluded appropriately that one of the validated laparoscopic tasks did show significant improvement after video game playing, but the other task did not show any significant change.¹¹ This study did acknowledge that while concluding that preoperative warm-up with a video game can improve operative time, the authors did not analyze laparoscopic performance in the operating theater and suggested this as a future study.

Questions

What is the question asked?

The question in four of the five studies analyzed here was whether video games improve surgical skills. None of the postgame playing assessments, however, were performed in a theater environment. The question that should be posed according to the methodology (as correctly asked by Schlickum and coworkers⁹) is whether playing video games improves the performance on laparoscopic surgical simulators.

Does playing video games improve the performance on laparoscopic surgical simulators?

Three of the five studies analyzed, with the exception of error reduction, failed to show a statistically important improvement in performance after the use of video games. Two studies showed improvement after the use of a specific video game on laparoscopic simulator performance. One of these studies partially failed to show this with a second video game. Nevertheless, the study design was flawed and the sample size was very limited.⁹

What does it all mean?

It is important to consider the implications of video games improving performance on a surgical simulator. Considering that the studies which showed a correlation between video games and simulator performance involved VR simulators and box trainers (unlike the one study on an animal model that showed no correlation), the meaning of these findings could be one of the following two: (a) practicing on VR simulators has been shown to improve real time theater skills,¹⁷ therefore video game playing may do so as well; (b) VR simulators are similar to video games, using the same principles, therefore video game players perform better on VR simulators because of familiarity with the VR concept. The latter is supported by a recent study by Willis and colleagues.¹⁸ Further research is necessary to establish which of these two statements is true.

Why video games?

The rationale behind assessing the link of video game use and laparoscopic skills, is to potentially replace something expensive (VR simulators) with a product that is cost-effective but has the same didactic value.¹⁹ Such a training tool, however, already exists and has been proven to be equivalent to VR simulators for skills acquisition.²⁰ Laparoscopic box trainers are cost-effective, portable, and efficient. Therefore, until a study demonstrates superiority of video games versus box trainers, one would question why we are seeking an association between video games and laparoscopic skills.

Strengths and limitations of this review

This review is limited by the search being performed electronically only, without a manual search of relevant studies. The full article for one of the studies identified during the search process was not recovered. A significant number of studies were identified and appropriately assessed, however, according to their quality before inclusion in this review. This ensures that only high-level evidenced randomized controlled trials are presented in this study.

Recommendations for future work

Ideally, an appropriately designed training study should include a group that controls for a wide variety of potential confounds inherent to training studies and an experimental group that is trained on the potentially enhancing protocol. These potential confounds incorporate the effect of test–retest (i.e., how much improvement can be expected simply from taking the test a second time) and the psychological and motivational effects (Hawthorne effect)²¹ in which persons who have an active interest taken in their training and performance tend to, all other things being equal, outperform persons where no such interest is taken.

Furthermore, a fight-or-flight response has been known to occur with video games because of stimulation of the autonomic system²²; therefore, it is important to assess performance and skill levels at least a full day after the cessation of training to ensure that the observations are not contaminated by purely transient effects (e.g., level of circulating hormones, activity of neuromodulatory systems, etc.).

Because such transient effects (Mozart effect) are known to exist,²³ it is impossible to interpret studies in which the posttest is administered immediately following the cessation of the training exercise as a permanent change in performance or behavior. We have observed that the majority of the literature on video games and laparoscopy have tested participants shortly after playing video games rather than several days later, and thus they may reflect transient effects rather than the type of long-lasting changes.

Finally, the common denominator in all the studies included in this review is that the assessment of laparoscopic skills takes place on a simulator. Hence, even if a relationship between performance and video game playing were established, it would only refer to simulated performance. The authors suggest a future randomized controlled trial assessing the impact of video game playing on laparoscopic skills in an operating room. It would be more ethically sound for the control group to undergo a form of training as well. For reasons explained above, it is suggested that the control group uses box trainers to practice while the intervention group uses video games to practice.

Conclusion

From this systematic review, it is apparent that there is a limited amount of evidence to support that the use of video games enhances surgical simulation performance. The evidence that video games enhance laparoscopic surgery performance is nonexistent.

Author Disclosure Statement

No competing financial interests exist.

Footnotes

Abbreviations Used

References

Keehner

, Lippa

, Montello

, et al. Learning a spatial skill for surgery: How the contributions of abilities change with practice. Appl Cognit Psychol, 2006; 20:487–503.

Perrotta

, Featherstone

, Aston

, Houghton

. Game-based Learning: Latest Evidence and Future Directions (NFER Research Programme: Innovation in Education). Slough: National Foundation for Educational Research. 2013.

Connors

, Chrastil

, Sánchez

, Merabet

. Action video game play and transfer of navigation and spatial cognition skills in adolescents who are blind. Front Hum Neurosci, 2014; 11; 8:133.

Moher

, Liberati

, Tetzlaff

, Altman

; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. J Clin Epidemiol, 2009; 62:1006–1012.

Reed

, Cook

, Beckman

, et al. Association between funding and quality of published medical education research. JAMA, 2007; 298:1002–1009.

Reed

, Beckman

, Wright

, et al. Predictive validity evidence for medical education research study quality instrument scores: Quality of submissions to JGIM's Medical Education Special Issue. J Gen Intern Med, 2008; 23:903–907.

Plerhoples

, Zak

, Hernandez-Boussard

, Lau

. Another use of the mobile device: Warm-up for laparoscopic surgery. J Surg Res, 2011; 170:185–188.

Boyle

, Kennedy

, Traynor

, Hill

. Training surgical skills using nonsurgical tasks—can Nintendo Wii^™ improve surgical performance?. J Surg Educ, 2011; 68:148–154.

Schlickum

, Hedman

, Enochsson

, et al. Systematic video game training in surgical novices improves performance in virtual reality endoscopic surgical simulators: A prospective randomized study. World J Surg, 2009; 33:2360–2367.

10.

Rosenberg

, Landsittel

, Averch

. Can video games be used to predict or improve laparoscopic skills?. J Endourol, 2005; 19:372–376.

11.

Jalink

, Heineman

, Pierie

, Ten Cate Hoedemaker

. The effect of a preoperative warm-up with a custom-made Nintendo video game on the performance of laparoscopic surgeons. Surg Endosc, 2015; 29:2284–2290.

12.

Stefanidis

, Scott

, Korndorffer

Jr.

Do metrics matter? Time versus motion tracking for performance assessment of proficiency-based laparoscopic skills training. Simul Health, 2009; 4:104–108.

13.

Feifer

, Delisle

, Anidjar

. Hybrid augmented reality simulator: Preliminary construct validation of laparoscopic smoothness in a urology residency program. J Urol, 2008; 180:1455–1459.

14.

Ritter

, Kindelan

, Michael

, et al. Concurrent validity of augmented reality metrics applied to the fundamentals of laparoscopic surgery (FLS). Surg Endosc, 2007; 21:1441–1445.

15.

Efird

. Blocked randomization with randomly selected block sizes. Int J Environ Res Public Health, 2011; 8:15–20.

16.

Taffinder

, Sutton

, Fishwick

, et al. Validation of virtual reality to teach and assess psychomotor skills in laparoscopic surgery: Results from randomised controlled studies using the MIST VR laparoscopic simulator. Stud Health Technol Inform, 1998; 50:124–130.

17.

Franzeck

, Rosenthal

, Muller

, et al. Prospective randomized controlled trial of simulator-based versus traditional in-surgery laparoscopic camera navigation training. Surg Endosc, 2012; 26:235–241.

18.

Willis

, Gomez

, Ivatury

, et al. Virtual reality simulators: Valuable surgical skills trainers or video games?. J Surg Educ, 2014; 71:426–433.

19.

Adams

, Margaron

, Kaplan

. Comparing video games and laparoscopic simulators in the development of laparoscopic skills in surgical residents. J Surg Educ, 2012; 69:714–717.

20.

Diesen

, Erhunmwunsee

, Bennett

, et al. Effectiveness of laparoscopic computer simulator versus usage of box trainer for endoscopic surgery training of novices. J Surg Educ, 2011; 68:282–289.

21.

Lied

, Karzandjian

. A Hawthorne strategy: Implications for performance measurement and improvement. Clin Perform Qual Health-Care, 1998; 6:201–204.

22.

Hebert

, Beland

, Dionne-Fournelle

, et al. Physiological stress response to video-game playing: The contribution of built-in music. Life Sci, 2005; 76:2371–2380.

23.

Rauscher

, Shaw

, Ky

. Music and spatial task performance. Nature, 1993; 365: 611.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.11 MB