Eye Movement Patterns Reflecting Cybersickness: Evidence from Different Experience Modes of a Virtual Reality Game

Abstract

This study investigated whether the degree of cybersickness varies depending on different virtual reality experience modes (playing vs. watching) and whether specific eye movement parameters reflect changes in cybersickness. Simulator Sickness Questionnaire results from 20 participants (10 playing and 10 watching) showed that cybersickness was much more severe in the watching mode, particularly during the second of the three total trials. Moreover, cybersickness' changing pattern was reflected in the center gaze ratio and scan-path length. These findings imply the importance of physiological measurements for a deeper understanding of cybersickness in theoretical and practical respects.

Introduction

In a virtual reality (VR) environment, presence is a subjective experience of immersion and is characterized by the user's experience.¹ Interestingly, presence does not increase in stages, but a trigger or break-in point exists.² Therefore, it is essential to maintain the user's “suspension of disbelief” to provide a sense of presence. According to numerous studies,^3–12 VR sickness or cybersickness is a significant obstacle to sustaining the feeling of presence in the VR experience.^13,14

Many studies have attempted to identify the causes of cybersickness¹⁵ and proposed several models to explain the underlying mechanisms.^16–18 However, no clear conclusions have been drawn yet. According to Chang et al., one of the pivotal reasons is the absence of verified methods to measure cybersickness objectively and accurately.¹⁵ Most of the studies conducted so far have measured the subjective level of cybersickness by employing an introspective questionnaire. However, since symptoms of cybersickness are induced in the human body unconsciously, the introspective measurement cannot detect real-time changes in cybersickness appearing during the VR environment.

Moreover, it is difficult to precisely track the factors involved at certain moments, for example, when there is a sudden increase or decrease in cybersickness simultaneously. Further, being exposed to the Simulator Sickness Questionnaire (SSQ) before experiencing the VR may have the side effect of users overestimating motion sickness in the virtual environment.¹⁶

For this reason, recently, various physio-cognitive techniques have been adopted for VR and cybersickness research.¹⁷ In particular, eye-tracking has an advantage in the study of cybersickness. Diels et al. investigated the effect of viewing conditions on motion sickness in a radial optic flow environment and found that vection magnitude and motion sickness were expected to increase when gaze position was directed away from the focus of expansion.¹⁹ Jäger et al. reported that the average saccadic length decreased, and the proportion of gaze duration at the center of the screen increased when motion sickness occurred.²⁰

This study aimed at investigating whether the degree of cybersickness varies depending on the mode of experiencing VR content and whether specific eye movement characteristics are associated with changes in cybersickness. It has been suggested that when participants are passively exposed to VR content, cybersickness is more likely to occur than playing the VR game.^21,22 However, most previous researchers measured cybersickness using only SSQ, and they did not tell us what kind of physio-cognitive changes emerged as the cybersickness increased. Therefore, we tried to determine whether experiencing VR content (watching vs. playing) affects the degree of cybersickness in a VR situation and investigate the critical eye movement parameters reflecting the degree of cybersickness.

Methods

The study was approved by the Institutional Review Board (IRB) protocols of Konkuk University (7001355-201909-HR-333).

Participants, experimental design, and procedures

Twenty participants (mean age 23.4; 12 men) were recruited for the experiment and randomly divided into 2 groups: 10 watching mode (4 men) and 10 playing mode (8 men). All participants had normal vision and did not have any mental issues. They had online gaming experience in the past 2 years and had regularly played games for at least 2 hours a week.

We used a VR game named “Collecting Ring Game,” consisting of LEGO-like blocks^a (Fig. 1). This VR content had already been confirmed to induce cybersickness in participants in Lim.²³ In the game, users moved freely on the constructed map to acquire rings while avoiding obstacles by controlling the left and right turn with arrow keys on the keyboard. The forward translation movement was automatically performed at a constant speed to prevent participants' differences in gaming speed. One trial lasted for 4 minutes, and keyboard operation was impossible after 4 minutes had elapsed since the game started.

FIG. 1.

The “Collecting Ring” game used in the experiment.

Participants in the playing mode were encouraged to collect as many rings as possible by passing through the middle of the ring. If participants did not take any action for a while in this mode, they collided with the obstacle and bounced off the opposite axis, making a sound.

Participants in the watching mode had watched a screen recording of somebody playing the game, and they were assigned to count how many coins the player collected in 4 minutes.

The gaze was recorded by using an RED 500 and IView X (SMI) with a 250 Hz sampling rate. Participants were seated in front of a 19″ display monitor. The distance between the participant's eyes and the display monitor was 70 cm (27.55″). The chin-rest was used to minimize head movements during the experiment.

Participants took part in three trials, and the resting time was given between the trials for filling out the SSQ. Before starting each trial, calibration was performed, and at the starting point of each trial, the + marker was presented for about 1,500 ms to set the same starting point. The subsequent trial was immediately started without providing a separate break. The entire experiment proceeded for ∼30 minutes.

Data analysis

Eye movements were analyzed by using BeGaze 3.0 (SMI). The fixations and saccades of the eye movement right before and after blinking were excluded.²⁴ And since the visual exploration process of observing the game environment is reflected in the period up to 10 seconds after the VR game begins, all gaze movements in this period were also excluded.^b

The eye-tracking data were analyzed by using saccadic amplitude, scan-path length, and the average gaze ratio on the center region of interest (ROI, Fig. 2), the most critical parameters related to cybersickness, according to Jäger et al.²⁰ Supplementarily, the average fixation duration reflecting the degree of attention was included as well.^c

FIG. 2.

The center ROI. ROI, region of interest.

Statistical analysis was conducted in two ways. First, a mixed-design analysis of variance (ANOVA) with the gaming mode as the between-subject factor and the experiment trial as the within-subject factor was performed. Further, a repeated-measures ANOVA with the trial as a within-subject factor in each experience mode to independently seize the changing pattern of cybersickness and eye movement regardless of the interaction in the first analysis was conducted. If the sphericity was not satisfied, it was corrected with the Greenhouse-Geisser value,^d and post hoc analysis was performed by using a paired t-test.

Results of SSQ Scores

The average SSQ score^e showed that the increasing pattern of cybersickness differs depending on the mode and the trial (Fig. 3). In the watching mode, the SSQ score in the first trial (185) increased sharply in the second trial (357). However, it rose only slightly in the third trial (378). In the playing mode, the SSQ score increased slightly from the first (109) to the second (134) and the third trial (184).

FIG. 3.

Average SSQ score per condition/trial. SSQ, Simulator Sickness Questionnaire.

The mixed-design ANOVA revealed that the main effect of the mode [watching = 307 vs. playing = 142; F(1, 18) = 9.844, p < 0.01] and the trial factor [147 (1st) vs. 246 (2nd) vs. 280 (3rd); F(2, 36) = 5.467, p < 0.01] were significant. Within the gaming mode, the trial effect in the watching mode was significant [F(2, 18) = 4.774, p < 0.05]. However, in the post hoc analysis, the difference between the first (185) and the second trial (357) was significant [t(9) = −3.301, p < 0.01], but not between the second (358) and the third trial (378).

In sum, in the watching mode, participants experienced higher levels of cybersickness than in the playing mode, and the degree of cybersickness started to increase in the second trial, particularly in the watching mode.

Results of Eye Movement Tracking

Saccadic amplitude was the largest in the second trial in the watching mode, whereas it increased abruptly in the third trial in the playing mode (Fig. 4).

FIG. 4.

Average saccadic amplitude per condition/trial.

The main effect of the trial was significant [F(2, 36) = 5.467, p < 0.05]. The post hoc analysis revealed that the difference in saccadic amplitude in the first (3.93) and second trial (4.37) was marginally significant [t(19) = 2.04, p = 0.055], but not between the second (4.37) and the third trial (4.75).

Within the gaming mode, the main effect of the trial factor in the playing mode was significant [F(2, 18) = 4.189, pg = 0.05]. Paired t-test revealed that the difference between the second and third trials was significant only in the playing mode [t(9) = −2.017, p < 0.05].

The average gaze ratio on the center ROI in the watching mode (89 percent) was higher than the playing mode (70 percent) (Fig. 5). Statistical analysis revealed that only the gaming mode effect reached significance [F(1, 18) = 9.845, p < 0.01]. Within the gaming mode, the main effect of the trial factor in the playing mode was significant [F(1, 9) = 4.020, p < 0.05], indicating that the center gaze ratio gradually decreased as the trial was repeated in the playing mode.

FIG. 5.

Average center gaze ratio per condition/trial.

Scan-path length in the playing mode was much longer than the watching mode and increased sharply in the third trial (6144 px) compared with the second trial (4971 px) (Fig. 6). The main effect of the playing mode was significant in the scan-path length [F(1, 18) = 5.202, p < 0.05].

FIG. 6.

Average scan-path length per condition/trial.

The average fixation duration was longer in the watching mode than the playing mode (Fig. 7). The main effect of the mode was marginally significant [F(1, 18) = 4.247, p = 0.054]. In addition, the main effect of the trial was significant [F(2, 36) = 4.808, pg <0.05]. Within the gaming mode, the trial effect in the watching mode was significant [F(2, 18) = 4.388, pg <0.05]. In the paired t-test, it was confirmed that the trial effect was attributed to the difference between the first and second trials [t(9) = −2.787, p < 0.05].

FIG. 7.

Average fixation duration per condition/trial.

Summary and Conclusion

The current study revealed that the degree of cybersickness occurring in VR environments varies depending on the mode of experiencing the VR contents: Passive watching of the VR game has caused relatively severe cybersickness, and active playing of the game has contributed to diminishing cybersickness. Such a contrast also enabled us to determine the changes in eye movement patterns depending on the different levels of cybersickness: In the passive watching mode, participants' eyes moved over a relatively shorter distance in total, staying fixated relatively longer at a particular position within a relatively narrow area around the center of the monitor as a whole. In other words, eye movements become more restricted as cybersickness increases. The question is: What is the relationship between the increase in cybersickness and the restriction of eye movements?

Since active eye movements were initiated for pursuing and collecting the coins only in the active playing mode, we suggest that more active eye movements reduced cybersickness and not vice versa. Thus, the restricted eye movements in the passive watching mode can be considered the cause of the higher level of cybersickness. Most importantly, our findings, together with those from previous studies,^19,20 point to a specific interrelationship between eye movement patterns and the degree of cybersickness.

Further evidence is, undoubtedly, required to obtain a more accurate picture of the relationship between eye movements and cybersickness. Above all, a much more detailed characterization of the restrictions of eye movements under a relatively higher level of cybersickness appears to be necessary.

Footnotes

Authors' Contributions

Y.N. and H.C. carried out the experiment. Y.N. (as the first author) and U.H. wrote the article with support from H.C. and S.R.N. Y.N. and H.C. conceived the original idea. U.H. supervised the project.

Notes

a. LEGO© block.

b. 2.59% of the saccade and 3.06% of the fixation data were excluded from the analysis.

c. It is well known that the longer the average fixation duration, the higher the level of attention deployment.^25–27

d. The p value was expressed as pg if we referred to the Greenhouse-Geisser value.

e. The SSQ score was used after normalization, according to Kennedy et al.²⁸

Author Disclosure Statement

No competing financial interests exist.

Funding Information

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF) (No. NRF-2018S1A5B6069474). Also, this work was supported by the KU Research Professor Program of Konkuk University for Yunju Nam.

References

Steuer

. Defining Virtual Reality: dimensions determining telepresence. Journal of Communications, 1992; 42:73–93.

Magnenat

, Kim

, Egges

, et al. (2005) Believability and interaction in virtual worlds. In International Multi-Media Modelling Conference. IEEE Computer Society Press, pp. 2–9.

Wilson

, Sasse

. (2000) Do users always know what's good for them? Utilising physiological responses to assess media quality. In Proceedings of HCI: People Computers XIV-Usability or Else!, Sunderland, UK: Springer, pp. 327–339.

Murray

, Arnold

, Thornton

. Presence accompanying induced hearing loss: implications for immersive virtual environments. Presence: Teleoperators Virtual Environments, 2000; 9:137–148.

Wilson

, Sasse

. (2000) Investigating the impact of audio degradations on users: subjective vs. objective assessment methods. In Proceedings of OZCHI2000. Sydney, pp. 135–142.

Wang

, Kunmi

, Liu

, et al. (2006) Evaluating the effect of real world distractions on user performance in immersive virtual environments. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST 2006. Limassol, Cyprus, pp. 19–26.

Barfield

, Baird

, Bjorneseth

. Presence in virtual environments as a function of type of input device and display update rate. Displays, 1998; 19:91–98.

Khanna

, Yu

, Mortensen

, et al. (2006) Presence in response to dynamic visual realism: a preliminary report of an experiment study. In Proceedings of the ACM Symposium on Virtual Reality Software and Technology, VRST 2006. Limassol, Cyprus, pp. 364–367.

Slater

, Khanna

, Mortensen

, et al. Visual realism enhances realistic response in an immersive virtual environment. IEEE Computer Graphics, 2009; 29:76–84.

10.

Darken

, Bernatovich

, Lawson

, et al. Quantitative measures of presence in virtual environments: the roles of attention and spatial comprehension. Cyberpsychology Behaviour, 1999; 2:337–347.

11.

Zimmons

, Panter

. (2003) The influence of rendering quality on presence and task performance in a virtual environment. In Proceedings of the IEEE Virtual Reality. Los Angeles, CA: IEEE, pp. 293–294.

12.

Chung

, Yoon

, Gardner

. Analysis of break in presence during game play using a linear mixed model. ETRI Journal, 2010; 32:687–694.

13.

Brooks

, Goodenough

, Crisler

, et al. Simulator sickness during driving simulation studies. Accident Analysis & Prevention, 2010; 42:788–796.

14.

Crowley

. Simulator sickness: a problem for Army Aviation. Aviation, Space, and Environmental Medicine, 1987; 58:355–357.

15.

Chang

, Seo

, Kim

, et al. An Integrated Model of Cybersickness: understanding users discomfort in virtual reality. Journal of KIISE, 2018; 45:251–279.

16.

Young

, Adelstein

, Ellis

. (2006) Demand characteristics of a questionnaire used to assess motion sickness in a virtual environment. In IEEE Virtual Reality Conference (VR 2006). Alexandria, VA: IEEE, pp. 97–102.

17.

Kim

, Kim

, et al. Characteristic changes in the physiological components of cybersickness. Psychophysiology, 2005; 42:616–625.

18.

Keshavarz

, Riecke

, Hettinger

, et al. Vection and visually induced motion sickness: how are they related?. Frontiers in Psychology, 2015; 6:1–11.

19.

Diels

, Ukai

, Howarth

. Visually induced motion sickness with radial displays: effects of gaze angle and fixation. Aviation, Space, and Environmental Medicine, 2007; 78:659–665.

20.

Jäger

, Gruber

, Müri

, et al. Manipulations to reduce simulator-related transient adverse health effects during simulated driving. Medical & Biological Engineering & Computing, 2014; 52:601–610.

21.

Rich

, Braun

. (1996) Assessing the impact of control and sensory compatibility on sickness in virtual environments. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting. Philadelphia, PA, vol. 40. pp. 1122–1125.

22.

Guo

, Tsoi

, Wong

, et al. (2013) Visually induced motion sickness during computer game playing. In Contemporary Ergonomics and Human Factors, Cambridge, vol. 51. pp. 51–58.

23.

Lim

, Jun

, Lim

, et al. Display Resolution and Human Factors for Presence and Motion Sickness in HMD Experiences. SID Symposium Digest of Technical Papers, 2019; 50:1142–1145.

24.

Henderson

, Luke

. Stable individual differences in saccadic eye movements during reading, pseudoreading, scene viewing, and scene search. Journal of Experimental Psychology: Human Perception and Performance, 2014; 40:1390–1400.

25.

Calen Walshe

, Nuthmann

. Asymmetrical control of fixation durations in scene viewing. Vision Research, 2014; 100:38–46.

26.

Nuthmann

. Fixation durations in scene viewing: modeling the effects of local image features, oculomotor parameters, and task. Psychonomic Bulletin & Review, 2017; 24:370–392.

27.

Cronin

, Hall

, Goold

, et al. Eye movements in real-world scene photographs: general characteristics and effects of viewing task. Frontiers in Psychology, 2019; 10:1–12.

28.

Kennedy

, Lane

, Berbaum

, et al. Simulator sickness questionnaire: an enhanced method for quantifying simulator sickness. The International Journal of Aviation Psychology, 1993; 3:203–220.