Do Video Games Predict an Early Advanced Capacity to Learn Interventional Radiology Skills?

Introduction

Once perceived as a niche past-time, video gaming has transformed through rapid online expansion into a pervasive force in American society. In 2018, consumer spending in the video game (VG) industry topped $43 billion in the United States. Currently 65% of American adults play VGs, which represents over 164 million people, ¹ tens of millions who engage in the global online community.^2,3

VG play has long been a curiosity of the surgical community, and its relationship with surgical skills has been well-investigated. ⁴ In this respect VG play has been established to correlate positively with fundamental surgical skills.^5-8 Furthermore, playing VGs have been shown to be an effective mode of improving laparoscopic skills in surgical novices. ⁹ Given that interventional radiology (IR) and minimally invasive surgery both require a singular combination of dexterity and perception of 3-D environments in 2-D, VGs may also be relevant to the field of IR. Previous literature has gone further to demonstrate that VGs increase visuospatial and attention skills,^10,11 and that such visuospatial ability (VSA) correlates not only with surgical performance but also rate of surgical skill acquisition.^12-15 Therefore, VGs and VSA may similarly be interlinked with IR skill acquisition.

Radiologists have also been historically interested in VGs, particularly in their application as pedagogical tools.^16,17 One recent study demonstrated that implementing gamification elements into electronic learning can at once significantly improve diagnostic confidence, reduce error rates, and offer fun interaction to learners. ¹⁸ Thus, the intersection of VGs and skill acquisition in the realm of IR may be of similar potential in future training paradigms. The current study aims to determine if VG play and VSA are related to fundamental IR technical skills.

Materials and Methods

This prospective study recruited 20 medical students attending the skills workshop at our 2018 Medical Center Interventional Radiology Symposium. Procedures used in this study adhered to the tenets of the declaration of Helsinki and were approved by the institutional review board at our institution. Signed informed consents were obtained from each participant.

The study design was modeled on previous literature, ⁵ and centered on the trial of skills: a sequential battery of tests assessing IR skill, VG skill, and VSA. Each participant began by completing a survey inquiring of demographics, VG experience, and prior IR experience and then participated in the trial of skills.

Part 1: IR Skill

First, IR skill was evaluated on the previously validated VIST G5® Simulator (Mentice AB, Gothenburg, Sweden) (Figure 1).^19-25 The VIST G5® is a high-fidelity endovascular simulator which supports real clinical devices, table controls, and advanced haptic feedback. Tools are inserted into a control box to interact with a number of different computer software-generated clinical scenarios. In this study participants worked in the transarterial chemoembolization procedure environment. The task was determined to be selection of the right gastric artery via right common femoral access. Since procedure time has been the metric most consis-tently reported as a statistically significant correlate of clinical experience,^19-25 IR skill was measured by the time taken to select the right gastric artery. This particular route was chosen to test the finesse required to maneuver a catheter and wire into arteries branching at disagreeable angles. Students were assumed to have minimal experience or to be unfamiliar with IR tools, and thus were limited to a Cobra 1 (Cook, Bloomington, Indiana) and microcatheter for simplicity, eliminating any requirement for catheter exchange mid-examination. Prior to beginning the examination a standardized explanation was given to each participant, including relevant anatomy and operation of the catheter wire system and fluoroscopy pedal. They were instructed to select the celiac artery with the Cobra 1, then deploy the microcatheter coaxially to select the common hepatic artery and finally the right gastric artery. Students were then given 1 minute to familiarize themselves with the tools, and tie began when catheter or wire was moved from the right common femoral artery. Time stopped once the microcatheter was 2 cm out into the right gastric artery.

OPEN IN VIEWER

Figure 1.

Mentice VIST G5 is a high-fidelity endovascular simulator which supports the use of real clinical devices and provides advanced haptic feedback (https://www.mentice.com/vist-g5). The transarterial chemoembolization environment was utilized in the current study (https://www.mentice.com/transarterial-chemoembolization).

Part 2: Video Game Skill

Video game skill was subsequently evaluated by performance on 3 VGs. Games were selected to examine interactions that may parallel aspects of practicing IR, validation in previous investigations,^5,26 and by having a single measurable variable representative of performance—either time to completion or number of targets hit.

Video game 1 was Super Monkey Ball (SMB) (Sega of America Inc, San Francisco, California) for Nintendo Wii (Nintendo Co Ltd, Tokyo, Japan), wherein players pilot a spherical ball through a course by tilting a handheld device to guide a character’s speed and direction. SMB was selected primarily to challenge the participant’s fine manipulation of a 3-D object represented in 2-D space. Areas along the course without support railing offer a challenge for players to keep the spherical ball along its course; inadvertent falls resulted in a reset to the starting line without a reset of the timer. Performance was measured by total time to complete the course.

Video game 2 was Link’s Crossbow Training (LCT) for Nintendo Wii (Nintendo Co Ltd, Tokyo, Japan). In this game players use motion controls to shoot red targets by pointing the Wii Remote at the screen to aim and fire a virtual crossbow. This game was selected to test the tandem actions of visually perceiving a target and then navigating towards it. An automated course guides the player through several different scenarios of targets for 60 seconds. Performance was measured by total number of red targets hit.

Video game 3 was Hit the Dot (HTD) (Kijug Software, Version 1.3, June 2016) for iPhone 6, in which players tap as many dots as possible in 10 seconds as they appear randomly across the screen. ²⁶ HTD was selected to evaluate reaction time. Performance was measured by the cumulative number of dots tapped over 3 attempts.

Raw scores were standardized by subtracting the mean and dividing by the standard deviation. A composite VG score was then calculated to reflect overall VG performance by the following formula: # targets hit + # dots hit—SMB completion time. Greater number of targets hit and number of dots hit indicated higher skill, while shorter time to complete the SMB course indicated higher skill; thus, higher composite score indicated higher VG skill.

Part 3: Visuospatial Aptitude

Finally, VSA was assessed via the Cube Comparison Test, a previously validated visual-spatial exam developed by the Educational Testing Service. ²⁷ This purpose of this test is to evaluate a participant’s ability to mentally rotate 3-D cubes on a 2-D plane. Standardized instruction was provided to each participant and the examination was administered. The VSA score represented the number of questions answered correctly.

Statistical Analysis

Statistical analysis was performed by the Department of Biostatistics at our institution using SAS 9.4. Pearson’s r was used for correlation analysis to assess the strength of relationships between continuous variables. Spearman’s rho was employed to assess the strength of the relationship between ordinal and continuous variables. In order to evaluate why some participants performed better than their peers on endovascular simulation, participants were grouped by IR skill into top and bottom halves, which represented skilled and less skilled groups, and characteristics of the 2 groups were compared.

Results

Overall 14 males and 6 females with a mean age of 23.9 (range = 21-29, SD = 2.09,) took part in the study. All players indicated either “action” as their top VG genre or listed an action VG as their top most played game. Further survey responses are presented in the “Comparison between Groups Stratified by IR” section.

Mean IR simulation completion time was 163 seconds and ranged from 38 to 300 seconds (SD = 70.5). A total of 2 participants were unable to complete the task by 300 seconds and were therefore assigned a time of 300 seconds. VG scores ranged from −1.55 to 1.39 with a mean of 0 (SD = 0.79). VSA scores ranged from 21 to 40, mean 28.4. (SD = 5.7).

Correlation Analysis

Composite VG score did not correlate with IR simulation time (r = −0.22, P = .35) (Figure 2A), nor did any of the raw VG scores correlate with IR simulation time (HTD r = −0.13, P = .59; SMB r = 0.17, P = .48; LCT r = −0.22, P = .35). Greater number of years playing VGs correlated with lower IR simulation time (Spearman’s rho = −0.45, P < .05) (Figure 3). There was no correlation between visuospatial ability and IR simulation time (r = 0.14, P = .57) (Figure 2B).

OPEN IN VIEWER

Figure 2.

Scatter plots displaying composite VG score compared to IR simulation time (A) and visuospatial ability compared to IR simulation time (B). There was no statistically significant correlation for either comparison (r = –0.22, P = .35; and r = 0.14, P = .57).

OPEN IN VIEWER

Figure 3.

Relationship between number of years VG experience and IR skill. Left box representing those with less than 5 years of video game play and associated time needed to complete simulation. Right box representing those with greater than 15 years of play and associated time needed to complete simulation (Spearman’s rho = –0.45, P < .05).

Comparison between Groups Stratified by IR Skill

Table 1 compares participant characteristics and past VG experience between skilled and less skilled groups. Overall, the skilled group more commonly reported being male, playing VGs for a longer total duration, spending more time playing VGs, and having a higher self-reported VG skill level as compared to the less skilled group. The skilled group also achieved higher composite VG scores, although this did not reach statistical significance (mean, 0.42 vs 0.10; P = .4042). Of the raw video game scores, participants in the skilled group performed better on SMB (mean, 41.4s vs 78.8s; P = .061) and tended to hit more targets on average in LCT than the less skilled group (50.6 vs 46.8) (P = .3292), although neither trend reached statistical significance. There was no difference in HTD scores between the 2 groups (mean, 61 vs 61 dots) (P = 1). Visuospatial aptitude scores were not significantly different between the 2 groups (mean, 27.4 vs 29.2; P = .69).

OPEN IN VIEWER

Table 1.

Comparison of survey responses between skilled and less skilled participants by IR skill.

Gender
Skilled	Male (90%)	Female (10%)
Less skilled	Male (50%)	Female (50%)
Total number of years playing VGs
Skilled	<5 (10%)	5 to 10 (0%)	>15 (90%)
Less skilled	<5 (60%)	5 to 10 (10%)	>15 (30%)
Hours of VGs played per week at height of gaming
Skilled	<5 (10%)	5 to 10 (20%)	10 to 15 (10%)	15 to 20 (20%)	>20 (40%)
Less skilled	<5 (60%)	5 to 10 (0%)	10 to 15 (30%)	15 to 20 (10%)	>20 (0%)
Perceived VG skill level
Skilled	Not skilled at all (10%)	Not very skilled (0%)	Average (30%)	Skilled (50%)	Highly skilled (10%)
Less skilled	Not skilled at all (40%)	Not very skilled (20%)	Average (20%)	Skilled (20%)	Highly skilled (0%)

IR as Specialty of Choice

Five of the total participants reported IR as their specialty of choice. Of these medical students, the majority had extensive prior VG experience: they had played for >15 years (n = 4, 80%), for at least 10 hours/week at the height of their gaming (n = 3, 60%), and reported being “skilled” or “highly skilled” at VGs (n = 3, 60%).

Discussion

Previous literature questions what exactly allures medical students to the specialty of IR. One may speculate the adrenaline thrill associated with the precise handling of tools to produce an immediate effect that is insatiable—the deployment of an embolization coil to stop a life-threatening hemorrhage, etc. One previously entertains: “It is like playing video games with human stakes.” ²⁸

Herein we demonstrate a statistically significant, albeit relatively weak, positive correlation between past VG experience and IR skill that is consistent with analogous studies in the surgical literature.^5-8 Several cognitive learning theories describing skill acquisition have been promulgated to better understand simulation training in healthcare. ²⁹ Transfer of learning details the effective extent to which past experiences in 1 context affect learning and performance in a new, but similar context. ³⁰ This concept, today a common machine learning problem, may also be applied to explain why in the current study prior VG play (past experience) correlated with IR skill (new, but similar context). Participants’ prior VG experience with controller and monitor may have furnished intuition for learning the precise mechanics of catheter-wire manipulation and the perception of 3-D anatomy and tools via a 2-D display. On the other hand, this finding could simply represent the natural inclination of individuals to activities at which they tend to excel.

Perhaps the most striking result of this investigation is the lack of correlation between IR skills and VG skills, which is at odds with prior literature comparing VG to laparoscopic skills.^5,6 In general, it was thought that study power rather than the revelation of contradictory truths accounted for the discrepancy between current and past investigational findings, further discussed in limitations. The observation that VG skill was not directly applicable to IR skill may highlight the importance of acquiring transferable skills through situations highly specific to the task, such as is achievable through medical simulation. Exciting new technology such as virtual reality, artificial intelligence and similar medical simulation software may provide for additional ability to stratify these relationships at a hands-on level.

Simulation based training is a rapidly spreading and attractive notion in medical education that shows promise to influence the future of IR education.^19,31,32 Its virtue is that it offers learners a forgiving environment wherein they are allowed to make mistakes and learn without harming patients. While studies have shown users to improve on local metrics, there is a paucity of evidence corroborating simulation in IR to improve clinical outcomes.^19,31,32 This calls into question the aptness of the metrics used in IR simulation’s status quo and should motivate the search for discretely learnable skills that are measurable and transferable outside of simulation. Ultimately, widespread validation of simulation based learning tools and substantiation of skill transferability will be paramount if they are to be successfully implemented in IR.

Adjacent to the concept of simulation for medical education is that of VGs for the same object. “Serious games” are digital games designed for a purpose other than mere entertainment, at which point they diverge from conventional games. ³³ Such games are being applied broadly as tools to train medical professionals. ³⁴ Formally, a serious game is defined as an “interactive computer application, with or without a significant hardware component, that has a challenging goal, is fun to play and engaging, incorporates some scoring mechanism, and supplies the user with skills, knowledge or attitudes useful in reality.” ³⁵ By this definition it appears that some instances of IR simulation may be indiscernible from serious gaming.

Indeed, the veritable power switch of gamification, viz. the application of gaming principles to non-game contexts, in radiology education has been thrown. Five principles whereby pedagogical games should abide have recently been proposed: interactivity with formal feedback, meaningful goals, experience of growth, feeling of safety, and engagement of the senses. ¹⁶ The application of these principles to IR training may prove to have great potential for augmenting trainee learning and skill acquisition in the context of simulation and beyond. Gamification may be key to unlocking IR training that imparts the mode of learning, knowledge, and skills that translate to improved clinical outcomes for patients. Examples of this are already taking place at the gaming level, which includes virtual reality, artificial intelligence, and software simulations. The utilization of these assets in the medical education curriculum could provide for unlimited education and invaluable experience when pursuing a future specialty in medicine. Further research should urgently be undertaken to explore this domain.

There were several limitations of our study. First, our study had low sample size and a single metric proxy for endovascular performance which may have simply obscured a statistically a significant correlation between IR skill and VG skill and VSA. Second, previous studies exceeded ours in the extent to which the technical skills of participants were tested (examination of multiple simulations over time or of several tasks contemporaneously and complexity of task) and evaluated (utilization of both subjective observational assessments and multiple objective metrics such as also fluoroscopy time and contrast media usage). ¹⁹ These studies reported statistically significant differences in endovascular skill relative to level of clinical or simulated experience,^20-25 and positive correlations between VG play and laparoscopic or endovascular surgical skill.^5-8 Furthermore, our sample size was below average compared to the previous literature. Thus, it is likely that we were relatively limited by the data extracted in our study to accurately reflect IR skill and hence reveal all statistically significant trends.

In summary, in our study, though limited by power, number of years playing VGs correlated positively with simulated IR skills in medical students. Prior VG experience may predict an early advanced capacity to learn IR skills. Future research is needed to explore the potential of gamification in IR training.