Abstract
An important aspect of embodied approaches to cognition is the idea that human cognition does not occur simply in the brain, but is influenced by a complex bi-directional interplay between the brain, body, and external environment. Though embodied cognition is often studied in a controlled laboratory setting, by its very nature it can arise spontaneously in everyday life (e.g., gesturing). A recent paper by Chisholm et al. suggested that leaning while playing a video game may be another instance of a natural spontaneous expression of embodied cognition that can be studied to gain insight into a person’s ongoing covert cognition. Consistent with this proposal, Chisholm et al. found that, like gestures, leaning increases when cognitive demand is increased. However, in Chisholm et al., immersion also increased with cognitive demand. We argue that their test to exclude it as a contributing factor—by holding cognitive demand constant while manipulating immersion—was limited. Despite their test, it remains possible and plausible that cognitive demand has an effect on leaning only when immersion increases. To address this issue, the present study systematically varied demand and immersion. We replicate Chisholm et al.’s finding that leaning increases with cognitive load. We also show that the effect of load is not influenced by a robust and reliable change in immersion. Collectively our results provide new and converging evidence that spontaneous overt embodiment of an individual’s intention is modulated by cognitive demand, and emphasises the utility of using natural behaviours to understand the embodiment of cognition.
In 1975, in what is now considered one of the greatest baseball games ever, the Boston Red Sox and Cincinnati Reds were tied 3-3 after the standard 9 innings of play in Game 6 of the World Series Championship. In the 12th inning, Carlton Fisk hit a ball high and far down the left field line. If the ball remained fair, then it would be a home run and the Red Sox would win the game. If it flew foul, then the Reds would still have a chance to win the game. In an iconic moment, as the ball flew through the air, Fisk could be seen waving his arms as he tried to coax the ball to remain fair. (It did stay fair and the Sox won the game!) In far less dramatic settings, one can routinely see people in a bowling alley wiggling their hips and hands as they try to will their bowling ball to stay out of the gutter or knock over a critical pin. Indeed, the item in question does not even need to be real to witness this sort of body language. One can see video game players leaning hard to the left or right as they try to manoeuvre their character or vehicle out of danger. What do these, and many other comparable situations, share in common? They all relate to something that researchers call “teleoperation,” a term that refers to the overt act of trying to control an object that exists in a space, real or virtual, despite it being physically disconnected from the user.
Though the class of behaviour discussed above obviously has no effect on an object’s actual movement (e.g., Fisk’s waving did not keep the baseball fair), the very fact that it represents a spontaneous and ineffectual form of teleoperation—what we call incidental teleoperation—makes it all the more remarkable. But what makes incidental teleoperation particularly interesting from a researcher’s perspective, is that it can be understood as a natural real-world visual embodiment of a user’s intention. Or, to put it more precisely, it is an overt expression of a person’s internal cognitive processes. Indeed, like gesturing, incidental teleoperation can be situated within the “common coding theory” of embodied cognition (Hommel et al., 2001; Jeannerod, 1994; Prinz, 1997), which states that the same brain regions that are involved in the execution of an action are also involved in the simulation or imagination of that action. For this reason, for example, one might see a person speaking on a phone and gesturing with their hands as they convey some spatial information to the listener, despite the fact that the other person cannot see those actions. Those co-speech gestures reflect a speaker’s thoughts, just as incidental teleoperation may reflect a person’s intention. However, while the evidence is extensive and compelling that gestures reflect overt actions of internal thoughts (Wagner et al., 2004) the same has not yet been established for incidental teleoperation. The aim of the present study was to further examine this issue.
The present study is not the first to take on this topic. Chisholm, Risko and Kingstone (2014) tackled the question of embodied cognition and incidental teleoperation (what they called “remote goal-derived movement” or rGDM). However, as we note below, their study left the issue unresolved. In a series of three experiments Chisholm et al. investigated why people spontaneously lean (an act of incidental teleoperation or rGDM) to the left or right while playing a race car video game. Their first study sought to establish whether leaning was associated with either the movement of the video controller (called a local “first-order” mediated action) or the car on the screen (a remote “second-order” mediated action). Note that this was an outstanding question because usually the action one executes on the controller and the action that is produced by the car, are one and the same. That is, one moves a joystick left or right to move the car left or right. Thus, it is unclear whether leaning is associated with the movement of the controller or the car (or both). To tease this matter apart, half the participants were instructed to hold the controller upside down, so that, if one pushed the joystick to the right the car moved to the left, and vice versa. Participants in the standard condition used the controller in the normal upright manner. The results were clear-cut. Participants leaned left when the car went left, and right when the car went right, regardless of the controller’s orientation. The incidental behaviour of leaning therefore reflects a second-order mediated action.
Chisholm et al.’s second experiment sought to establish if spontaneous acts of leaning will become more prevalent when the cognitive demand is increased. According to gestures as simulated actions (GSA) theory (born from the more general common coding theory), intended actions overlap with the neural regions involved in explicit actions (Hostetter & Alibali, 2008, 2019). According to GSA theory an overt gesture occurs when the associated activation of the simulated action/perceptual state exceeds a gesture threshold—the greater the activation or the lower the threshold the more likely the gesture is to occur. Chisholm et al. (2014) thus hypothesised that this relationship between cognitive demand and overt actions that is observed for gestures may also apply to incidental teleoperation. So, in their second experiment, task difficulty was manipulated while participants played the race car game. Participants were instructed to either drive their car slowly for the low demand condition or as fast as they could for the high demand condition. The results showed that the frequency of leaning was significantly greater in the high demand condition. Therefore, consistent with the GSA theory, more overt actions were elicited under a higher cognitive demand.
However, participants in Experiment 2 were also given a state immersion questionnaire, and it was found that they experienced significantly greater immersion in the fast (high demand) condition than the slow condition. This is important because previous studies have found that when people watch videos, they are more likely to make overt behaviours (e.g., smiling or grimacing) when they feel more immersed (Freeman et al., 2000; Hoshino et al., 1997). In order to rule out an immersion-based explanation of the cognitive demand manipulation, Chisholm et al.’s final study manipulated participants’ levels of immersion. They did this by having the participants drive the car at their own pace with either the video sound on or off, expecting that immersion would be higher when the sound was on than when it was off. Results confirmed this expectation with greater immersion for the sound on condition, but leaning behaviour was unaffected by a change in immersion. Based on this finding, Chisholm et al. (2014) concluded that the increase in leaning observed in Experiment 2 reflected an increase in cognitive demand and not immersion. In sum, based on their three studies, Chisholm et al. (2014) concluded that spontaneous overt leaning behaviour, an act of incidental teleoperation, reflects second-order mediated actions that are sensitive to cognitive demand, consistent with the explanations based on GSA and the common coding theory of embodied cognition.
Though this conclusion on the face of it seems reasonable, there is a crucial limitation to Chisholm et al.’s reasoning. The concern is as follows. In Experiment 2, when task demand was manipulated there was a change in immersion, with greater demand leading to greater immersion, and more visible acts of spontaneous leaning. And though Experiment 3 demonstrates that immersion alone is not sufficient to produce a change in leaning behaviours, it need not follow that cognitive demand in Experiment 2 was solely responsible for the change in leaning. It is possible that a change in cognitive demand without a change in immersion would fail to produce a change in leaning behaviours, just as a change in immersion without a change in cognitive demand was not sufficient to produce a change in leaning. In other words, to affect a change in spontaneous leaning, it may be that demand and immersion both need to change (i.e., both need to increase or decrease). In short, the proposal is that, contrary to the conclusion of Chisholm et al. (2014), a change in cognitive demand alone may not be not sufficient to affect a change in leaning. Rather, a change in demand and immersion may be necessary for a change in leaning.
To resolve this issue, we conducted a study that was closely modelled on the Chisholm et al. (2014) investigation, but the present study systematically manipulated and fully crossed changes in cognitive demand and immersion. Specifically, in a within-subject design, participants were tested in four conditions, that crossed cognitive demand (low/high) and immersion (low/high). Moreover, to explore if the results of Chisholm et al. are robust, the present study altered the manner in which both cognitive demand and immersion were manipulated. Chisholm et al. manipulated load by varying the speed that the video game race car was to be driven (slow = low load; fast = high load); and they manipulated immersion by the presence/absence of sound (immersive/non immersive, respectively). A limitation of manipulating load by instructing participants to drive quickly or slowly, is that it leaves the actual driving speed—and hence any change in cognitive demand—up to the participant. In the current study we directly manipulate cognitive demand by varying the difficulty of controlling the car (easy = low load; hard = high load); and we manipulated immersion by varying the perspective of the viewer (third-person perspective = low immersion; first-person perspective = high immersion). It is important to note that manipulating control difficulty and viewer perspective are both validated methods for manipulating demand (Strayer, 2015) and immersion (Cummings & Bailenson, 2016). Most crucially, these variables are separable constructs, such that a change in control is not experienced with a change in perspective (Denisova & Cairns, 2015). As such they are ideal factors to examine how separate contributions combine to impact leaning behaviour.
In addition, using a potentially more effective manipulation of immersion (i.e., first or third person vs. sound on or off) provides an opportunity to re-examine its potential influence on incidental teleoperation. That is, the lack of an effect of immersion on incidental teleoperation, at least on its face, seems counterintuitive. For example, from a GSA perspective (Hostetter & Alibali, 2008, 2019), one might consider the extent to which one is “into” a game should positively correlate with the strength of the underlying activation of the simulation; and thus an increase in immersion should lead to an increase in spontaneous incidental teleoperation. Indeed, in the GSA framework there could be independent effects of both immersion (i.e., on activation level) and cognitive demand (i.e., on threshold). That there is putatively no effect of immersion thus constrains how one explains (or whether one explains) the behaviour in that framework. The lack of an effect of immersion was also important in Chisholm et al. (2014) because it argued against the idea that the leaning behaviour reflected a habitual tendency to counteract the centrifugal force experienced during turns. That is, such a behaviour seems as though it should be sensitive to the extent to which the individual feels immersed in a driving game. Thus, the perspective manipulation used here provides another opportunity to examine the influence of immersion on incidental teleoperation.
In sum, if cognitive demand alone is sufficient to affect a change in leaning behaviour, then regardless of the immersion state more leaning should be observed when demand is high. If, however, a change in demand and immersion are necessary, then leaning should increase only when demand and immersion increase. Furthermore, if we replicate the lack of an effect of immersion, this would provide further evidence against a “counteracting force” account of the behaviour and constrain the form of the GSA based account.
Methods
Participants
Seventeen participants from the University of British Columbia participated in the study. Note that one participant was excluded due to not having normal, or corrected to normal, vision. Thus, the final sample was composed of sixteen participants (75% female, Mage = 21.39, SDage = 4.10). All participants provided written informed consent, and were given course credit for participation.
Materials
A 40-inch high-definition LCD Samsung television connected to an Xbox 360 (Microsoft) video game console was used to play the circuit-racing video game Forza 3 (Microsoft Game Studios). Note, that this game is different from the one selected by Chisholm et al. (2014), which was Motorstorm (Sony Computer Entertainment). While the two games are comparable in that they both involve primarily left or right directional movement; Forza 3 is perhaps more effective at isolating left-right movement as there are fewer jumps and changes in elevation. The gameplay of Forza 3 is also generally more familiar to people, as it aims to simulate real driving with real cars, whereas Motorstrom embellishes driving with imagined cars and unrealistic physics.
A racing game in which car control is primarily performed through left and right inputs of the joystick was chosen in order to simplify the behavioural data. The Nissan MINE’s R32 Skyline GT-R was selected as the fast, difficult to control vehicle, and the Volkswagen Bora VR6 was selected as the slow, easy to control vehicle. The “bumper” camera setting was used as the first-person perspective, and the “chase far” camera setting was used as the third-person perspective. Participants played in a “hot-lap” mode, where they raced alone on the track. Traction control, stability control, and the antilock braking system (ABS) were turned on. Both cars were driven in the automatic transmission mode, with auto-brake and suggested-line functions deactivated.
Participants were seated approximately 150 cm away from the display, in a moderately lit room, and used a Microsoft Xbox 360 controller. The HDTV volume was at level 10, and the room door was shut to prevent external distractions. Two webcams were used to film the experiment. One webcam faced the HDTV and recorded on-screen events, while the other webcam faced the front of the participant, recording participant behaviour. Evocam 4.0 (Evological) was used to time-lock and stitch the video streams together into a single video file, in order to link on-screen events to spontaneous overt behaviours. Trait immersion (Witmer & Singer, 1998) and state immersion (Jennett et al., 2008) questionnaires were used to measure the tendency to be immersed in an activity, as an aspect of one’s personal character and task immersion for each driving session, respectively. In addition, the National Aeronautics and Space Administration (NASA) Task Load Index (TLX; Hart & Staveland, 1988) was used to measure the mental, physical, and temporal demand of the driving task, and a custom demographic questionnaire was used to record demographic information and prior video game experience.
Design
The study utilised a within-subjects design, such that each participant engaged in all four conditions: easy car and first-person perspective, easy car and third-person perspective, hard car and first-person perspective, and hard car and third-person perspective. The order of these four conditions was counterbalanced across participants to control for order effects. Participants completed the same cognitive demand condition of easy or hard two times in a row, while varying the immersion condition of first-person or third-person perspective (e.g., hard car third-person, hard car first-person, easy car first-person, easy car third-person). These constraints resulted in eight different configurations of condition types that were cycled through twice (N = 16).
Procedure
Each driving session was nine minutes long and was on the Circuit de Catalunya, Grand Prix Circuit. Following an explanation of the video game controls, participants participated in each of the four racing conditions. They were instructed to complete laps of the circuit in as little time as possible. After the four racing conditions, participants were given the state immersion questionnaire, and the NASA Task Load Index. Participants were instructed to fill out the state immersion questionnaire and the NASA Task Load Index with respect to each individual driving session (i.e., for every question, provide an answer for each driving session). Participants were reminded of the four driving conditions to ensure response accuracy. We delivered the subjective measures at the same time for three reasons. First, these measures tend to be reliable whether they are delivered during or after testing (e.g., Cummings & Bailenson, 2016; Jennett et al., 2008). Second, to ensure that participants have all had the same experiences at the time of testing, thereby minimising any order effects. Third, and most importantly, our primary goal is to acquire, for any given participant, measurements that reflect their relative experiences between conditions. Thus, by having participants score each of the conditions at the same time maximises this relative difference while controlling for the possibility that absolute measures may be scaled differently between participants (Smilek et al. 2008). After completion of the state immersion questionnaire and the NASA Task Load Index, participants completed the trait immersion and demographic questionnaires.
Results
The influence of vehicle type and perspective on task-demand, state immersion, and most critically leaning behaviour, was assessed by three 2 (car: easy/hard) x 2 (perspective: first/third) repeated measures analysis of variance (ANOVA). These measures are presented in Table 1.
A comparison of State Immersion, Task Demand and Leaning across the cognitive demand and immersion conditions.
TLX: Task Load Index.
Task-demand
For the NASA Task Load Index, a higher score represents greater mental, physical, and temporal demand. The ANOVA returned no significant main effect of perspective on task-load, F(1, 15) = .53, p = .48, ηp2 = .03, with no reliable difference between the mean task-demand for first-person (M = 65.69, SEM = 3.52) and third-person perspectives (M = 67.16, SEM = 3.89). However, a significant main effect of vehicle type was found, F(1, 15) = 5.51, p = .03, ηp2 = .27, such that the difficult vehicle (M = 70.50, SEM = 3.57) produced a higher average task-demand score than the easy vehicle (M = 62.35, SEM = 4.34). There was no significant interaction between perspective and vehicle type, F(1, 15) = .06, p = .81, ηp2 = .004. The mean TLX values for each cognitive demand and immersion condition are shown in Table 1.
State-immersion
State immersion scores were examined across each vehicle and perspective condition, in which a higher overall state immersion score represents a greater level of immersion in the task. The ANOVA returned a significant main effect of perspective, F(1, 15) = 6.69, p = .02, ηp2 = .31, such that the average immersion rating for driving in a first-person perspective (M = 48.94, SEM = 2.35) was significantly greater than the immersion rating when driving in a third-person perspective (M = 44.78, SEM = 1.76). Immersion scores were equivalent, however, for the two vehicle types, F(1, 15) = .003, p = .96, ηp2 < .00, with the difficult vehicle (M = 46.91, SEM = 1.98) producing a near identical overall score as the easy vehicle (M = 46.80, SEM = 2.17). There was also no significant interaction between perspective and vehicle type, F(1, 15) = .12 p = .73, ηp2 = .01. The mean immersion values across each cognitive demand and immersion conditions are expressed in Table 1.
Leaning
The videos were coded for acts of incidental teleoperation, or what we will simply refer to as leaning behaviour. The coder was instructed to watch the video recording of each racing session, and provide a rating between 1 and 10, accounting for overall rate and magnitude of the leaning behaviour. A rating of 1 would refer to a participant showing no leaning, while a rating of 10 would refer to a participant showing pronounced leaning connected to turning the vehicle. 1 In keeping with Chisholm et al. (2014), spontaneous overt behaviour was only coded as a lean if the behaviour had an obvious connection to the task-related intentions of the participant. For example, leaning while initiating a turn would be considered a lean, while leaning to scratch an itch would not. The coder was blind to condition and predicted outcomes. A second individual coded a pseudo-randomised set of 25% of the videos to attain an interrater reliability figure. Highly consistent ratings for leaning between the two raters were found, r(14) = .946, p = .001.
An ANOVA of the leaning scores returned a significant main effect of vehicle type, F(1, 15) = 11.17 p = .004, ηp2 = .43, such that leaning was greater when driving the difficult vehicle (M = 4.44, SEM = .65) than the easy vehicle (M = 2.81, SEM = .42). Leaning did not change with perspective, F(1, 15) = .06 p = .81, ηp2 = .004, with the leaning score for the first-person perspective (M = 3.66, SEM = .47) being nearly the same as the third-person perspective (M = 3.60, SEM = .54). And most crucially, no significant interaction between vehicle and perspective was returned, F(1, 15) = .14 p = .72, ηp2 = .01. Figure 1 shows these results graphically for each condition.
Leaning across perspective and vehicle type conditions with 95% confidence intervals for the individual means.
Performance
The videos were also coded for the driving skill of the participants (1 (poor) to 10 (excellent)). A second individual coded performance for all of the videos to attain interrater reliability. Strong consistent ratings were found between the two raters for the condition in which the vehicle type was easy and in first-person perspective, r(14) = .831, p = .0001. The raters judged some driving differently (e.g., slow driving as good or poor) in the easy vehicle type/ third-person perspective, resulting in one nonsignificant result, r(14) = .369, p = .194. Consistent ratings were found between the two raters for the conditions of hard vehicle type/first-person perspective r(14) = .57, p = .033, and hard vehicle type/ third-person perspective r(14) = .624, p = .017.
An ANOVA of performance scores showed a main effect of vehicle type F(1, 15) = 18.134, p = .001, ηp2 = .547, such that performance was better when driving the easy vehicle (M = 4.906, SEM = .426) than the difficult vehicle (M = 3.781, SEM = .326). Performance did not change with perspective F(1, 15) = .224, p = .643, ηp2 = .015, with performance for first-person perspective (M = 4.406, SEM = .385) being very similar to performance for third-person perspective (M = 4.281, SEM = .374). Finally, no significant interaction between vehicle type and perspective was found F(1, 15) = 2.483, p = .136, ηp2 = .142. Figure 2 shows these results graphically for each condition.
Performance across perspective and vehicle type conditions with 95% confidence intervals for the individual means.
Correlation between leaning and performance
While GSA theory does not predict a change in arousal alone should lead to a change in the overt expression of simulated behaviours, if arousal were playing a role in leaning behaviour and performance, then the two should be correlated. The relationship between performance and leaning measures was assessed within each of the condition types. A bivariate Pearson’s correlation revealed a non-significant relationship between performance and leaning in all conditions: easy vehicle type/ first-person perspective r(14) = -.213, p = .429; easy vehicle type/third-person perspective r(14) =-.143, p = .598; hard vehicle type/first-person perspective r(14) =-.404, p = .121; and hard vehicle type/third-person perspective r(14) =-.296, p = .266. This analysis fails to support the proposal that arousal plays a significant role in the present results.
Block number
Motivation and/or arousal might change with driving experience. To examine if they have an effect on performance and leaning, a repeated measures ANOVA was conducted with block number (1–4) as the condition. Performance was not significantly affected by block number F(3, 45) = 1, p = .402, ηp2 = .063. For the leaning analysis, the Greenhouse-Geisser estimate of the departure from sphericity was ε = .44. The adjusted values indicated that the amount of leaning was not significantly affected by block number F(1.332, 19.976) = .078, p = .851, ηp2 = .005. This analysis again fails to support the proposal that motivation and/or arousal plays a significant role in the present results.
Trait immersion and video game experience
The ITQ trait questionnaire measures the tendency to be immersed in an activity, as a personality trait. There was no significant correlation between the ITQ scores and leaning, r(14) = -.10, p = .71. In addition, self-rated prior video game experience did not significantly correlate with leaning, r(14) = -.08, p = .77.
Discussion
The present study examined if spontaneous leaning behaviour, an act of incidental teleoperation, should be viewed as a form of embodied cognition, similar to gesture in the GSA theory. We noted that this issue had been investigated recently by Chisholm et al. (2014) and their observation that leaning varied with cognitive demand supported the conclusion that leaning was a real-world instance of embodied cognition, like gesturing. However, we also noted that their test to rule out an alternative factor—immersion—which had co-varied with changes in cognitive demand, was incomplete. Specifically, they found that when cognitive demand was held constant, and immersion was manipulated by introducing sound or not (higher or lower immersion, respectively), leaning behaviour was not affected. While such a finding is inconsistent with the idea that immersion alone affects leaning, it fails to rule out a significant role for immersion (e.g., leaning may be affected by the co-occurrence of change in immersion and cognitive demand).
Our findings show that leaning is influenced by the ease of object control, such that the difficult-to-control vehicle led to greater leaning. In addition, the difficult-to-control vehicle led to higher task-demand ratings. This is consistent with the GSA framework that posits spontaneous behaviours like gestures, or in our case leaning, should become more evident when the cognitive demand is increased. This is because the underlying simulations of the action/perceptual state “leak out” when they exceed a threshold and holding that threshold higher requires more cognitive resources (Hostetter & Alibali, 2008, 2019) Thus, when resources have to be devoted elsewhere (e.g., to deal with the demands of driving) the threshold is lowered and the simulations are realised in an observable manner (e.g., leaning). These data also replicate and extend those of Chisholm et al. (2014) to a different form of manipulating cognitive demand.
The critical question then is what role did immersion have on leaning behaviour? First of all, it is vital to note that our manipulation of perspective had a significant effect on immersion, with immersion being significantly greater for a first-person perspective than a third-person perspective. However, our data also showed that the impact of cognitive load on leaning was not affected by changes in immersion. These data rule out the possibility that cognitive demand has an effect on leaning only when immersion changes. In other words, the present study, when combined with Chisholm et al. (2014), provides strong and convergent evidence that cognitive load alone is sufficient to produce a spontaneous change in remote goal directed actions during teleoperation. Immersion appears to play no role.
As noted in the introduction, the lack of an effect of immersion is, to some extent, counterintuitive. If the likelihood of a given simulation being expressed is a function of the strength of activation of that simulation and a threshold (which is cognitively costly to keep high), then why would the extent to which one feels “into” the game not modulate the former and thus the likelihood of the leaning behaviour emerging? One explanation (if somewhat unsatisfying) would be that whatever the subjective sense of immersion is measuring it is not positively correlated with activation level as conceptualised within the GSA account. In a similar but more theoretically grounded vein, the relation between activation level of the underlying simulation and immersion might be nonlinear such that the activation, in the present context, reaches a relatively high level quickly wherein the subjective experience of immersion might still vary without significant variation in the activation level of the simulation. This idea would be consistent with the claim in GSA that such activation is automatic (Hostetter & Alibali, 2008, 2019) and as such might be insensitive to other ongoing processes. Note that these results predict that leaning behaviour should not be affected by level of immersion in a virtual reality (VR) environment, although that remains to be confirmed empirically.
The lack of an effect of immersion does go some way to arguing against an account of leaning that is based on the idea that the experience of driving primes behaviours related to counteracting forces experienced when we actually turn. Such an account seems more difficult to align with a lack of an effect of immersion. That is, the more an individual is “into” the driving experience should be related to the likelihood that experience primes the associated behaviours. Chisholm et al. (2014) also presented preliminary evidence against such an account based on their observation that similar behaviours in a non-driving based game could not be explained by a counterforce type of account.
Why do players lean?
The present results have offered both new insights into incidental teleoperation and bolstered previous observations, thus strengthening the empirical footing on which an account of this behaviour can be built. As noted above, the spontaneous leaning behaviour observed during game play does not seem to be a product of enacting habitual behaviours during actual driving. That is, given the current state of evidence, individuals do not seem to lean because they are expecting to encounter forces produced by the act of turning. The results are consistent with the notion, based on GSA framework, that the behaviour might reflect a leakage of a simulation of the action/perceptual state associated with play-based cognitions. That is, the turning could be viewed as the enactment of the predicted perceptual effects of turning the car. That the behaviour increases with cognitive demand is consistent with GSA’s notion that the threshold used to inhibit such behaviours is cognitively costly to maintain at a high level, and thus under higher demands the threshold is reduced, making it more likely that simulated activation will exceed the threshold. As noted above, the lack of an immersion effect fits somewhat uncomfortably within this framework but it does not seem at present to represent a compelling reason to reject such an account.
A different account is that the behaviour observed here is not best interpreted in a GSA type framework, or in other words, it does not reflect an outgrowth of a latent action/perception simulation. Rather, the leaning behaviour observed might reflect a more concrete attempt at teleoperation. That is, the leaning might represent an actual attempt to turn the car that emerges when “all else fails.” This would, of course, reflect a kind of magical thinking (Rosengren & French, 2013) but not outside of that which humans engage in regularly. How would this account for the influence of cognitive demand and lack of an effect of immersion? With respect to cognitive demand, in the present investigation and in Chisholm et al. (2014) task difficulty was used to manipulate demand. A by-product of such a manipulation would be a potential increase in the need for last ditch efforts to control the vehicle (i.e., turn). Thus, in the high demand condition “all else fails” occurs more often and therefore so does the behaviour. On this account, the lack of an effect of immersion might reflect the fact that immersion does not alter the goal structure of the game (i.e., the need to turn) nor (apparently here) the likelihood that that goal might be threatened (i.e., immersion was not related to performance). As such, “all else fails” would presumably occur as often when one is more or less immersed. While this account is at present speculative, we find it intriguing and worthy of future investigation to further cull the theoretical landscape.
Conclusion
The present investigation utilised a natural behaviour approach to investigate the common spontaneous overt behaviour that occurs during video game playing, specifically, leaning during video game play. We replicate the past work of Chisholm et al. (2014) and extend their findings to different instances of cognitive demand and immersion. Most importantly, our study demonstrates unequivocally that incidental teleoperation (i.e., leaning) is driven by changes in cognitive demand and not immersion. These results put further constraints on understanding the basic mechanisms that underlie this behaviour and the role of embodiment in cognition more broadly, while also opening up intriguing lines of investigation for future research.
Footnotes
Declaration of conflicting interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Social Sciences and Humanities Research Council of Canada, (“Insight Grant”) Natural Sciences and Engineering Research Council of Canada, (“Discovery Grant”).
