A Smartphone-based gaming system for vestibular rehabilitation: A usability study

The VR games for VEGAS were developed as an Android app using the Unity ^® Game Engine (Unity Technologies). The VEGAS app consists of a graphical interface (Fig. 1) for user information management, which allows a clinician to create and manage patient accounts and choose the game of interest for either training or assessment. All relevant patient-related information is stored on the phone in individual folders for each patient, with individual subfolders to store data from each game.

The current version of the VEGAS has two training games with graded levels of difficulty: (a) “Cardboard Break”, and (b) “Follow Me”. Both games were designed based on exercises used in conventional rehabilitation to encourage head movements/gaze stabilization. Head rotations were measured using the phone’s in-built inertial measurement unit to measure the pitch and yaw angles. These angles controlled the subject’s movements in the virtual game: pitch controlled the vertical (up-down) movements, while yaw controlled the horizontal (left-right) movements. During training, the game’s difficulty level is automatically chosen using an adaptive performance-based algorithm. The data from the device was sampled at 35 Hz frame-rate for graphics update on the Nexus 5X phone, running the VEGAS and Chromecast application. A demo video of the VEGAS is provided as a Supplementary Video S1.

2.2 Cardboard break

The “cardboard break” (CB) game aims to induce retinal slip by having subjects perform discrete head movements in the pitch and/or yaw planes (Fig. 1). This exercise is a modified version of the two target gaze exercise [33]. The subject plays the game through head movements that control the position of the reticle1 in the virtual environment (Fig. 1). The reticle is in the center of the visual field, and the goal of the subject is to position this reticle on a cardboard box that appears at some spatial location in the virtual environment. When the target cardboard box appears, the subject performs a head movement to move the reticle to the box; this induces a retinal slip. When the subject places the reticle on the cardboard box for 500 ms the box explodes (and the patient scores a point), and another box appears at a different location. Patients continue to move and break these boxes for 30 s, and their goal here is to break as many boxes as possible within the stipulated time of 30 s.

The CB game has twelve different difficulty levels, corresponding to different combinations of range of movements (near and far), movement direction (pitch, yaw or both), and visual field backgrounds (plain, vivid and dynamic). The details of the different game levels are provided in Table 2. The difficulty levels are arranged based on the assumption that:

The “follow me” (FM) game requires subjects to perform continuous or rhythmic movements in pitch, yaw, or both directions. In this game, the subject sees a red ball (target) which moves in the virtual environment following a specific trajectory. The objective is to track this target by placing the reticle on the moving target for the entire game duration. When the subject places the reticle within the target, continuously for 1 sec, he/she scores a point. A single trial of the game lasts for 30 s. The target trajectory was generated using windowed sinusoids in the pitch and yaw planes (Equation 1).

r ( t ) = [ x ( t ) y ( t ) 0 ] = [ A x sin ( ω x t ) A y sin ( ω y t ) 0 ] × w ( t ) ; w ( t ) = { 1 - e - t 2 0 ⩽ t < 3 1 3 ⩽ t < 27 1 - e - ( t - 30 ) 2 27 ⩽ t < 30(1) where r (t) is the target position in space at time t ∈ [0,  30] s; (A _x ,  A _y ) and (ω _x ,  ω _y ) are the amplitudes and angular rates of the sinusoids in the yaw and pitch planes, respectively; and w (t) is a window function that ensures the target starts and stops smoothly at the same location r (0) = r (30) = 0 (i.e. center of the visual field).

The FM game has six different difficulty levels corresponding to different combinations of range of movements (A _x ,  A _y ), target speed (ω _x ,  ω _y ), and visual field background (please refer to Table 3 for details). This game only used two of the backgrounds from the CB game – ‘vivid stationary’ and ‘vivid moving’ (Table 3); the ‘normal’ background was not used since we did not want to have too many difficulty levels in FM game.

The vestibulo-ocular reflex (VOR) cancellation is likely to be minimal in the FM game because of the use of complex target trajectories (combination of sinusoids) and the short duration for the game trials. These two factors will hinder patients from learning and predicting the target trajectory pattern, thus making the target trajectory appear random or pseudo-random to the patients. Thus, in order to play the game, patients would have to continually update their gaze and head movements to follow the target visually, which will induce retinal slip.

The CB and FM games used an adaptive algorithm to automatically select the game level during training. The adaptation was based on patient performance on the recently completed trials of a particular game. Performance p [n] in a given trial n was +1 if the game was successfully completed, else it was -1. Success in a given trial for the CB game was defined as 20 or more cardboard breaks and for the FM game a score of at least 15, i.e., the patient was able to track the target for at least 15 s of the stipulated 30 s trial duration. These success criteria were based on a pre-pilot study that was carried out on healthy subjects; we chose values that were roughly 70% of healthy performance. Adaptation of game level was based on continued success or failure represented by P [n], which is defined as the following,

P [ n ] = { P [ n - 1 ] + p [ n ] P [ n - 1 ] p [ n ] > 0 P [ n ] P [ n - 1 ] p [ n ] < 0 ; for n > 1 and P [ 1 ] = p [ 1 ](2)

The following adaptation rule was used to choose a game level for the (n + 1)  ^th game trial, L [ n + 1 ] = { L [ n ] + 1 P [ n ] = 3 L [ n ] - 2 ⩽ P [ n ] ⩽ 2 L [ n ] - 1 P [ n ] = - 3(3) where, L [n] is the game level for the n^th trial. Additionally, whenever the game level is changed for the (n + 1)  ^th trial, the continued performance variable P [n + 1] is reset, i.e. P [n + 1] = 0.

3.1 Study participants

The current study was approved by the institutional review board (IRB) of the Christian Medical College (CMC) Vellore (IRB registration number: 9774). The study included a convenience sample of 15 patients with unilateral dysfunction and 15 healthy subjects. This sample size was chosen based on the existing evidence that a sample size of at least 12 was required to estimate system usability using the SUS [23] reliably.

The healthy volunteers recruited in this study were between 18 to 60 years of age with no reports of middle ear pathology and had never suffered from dizziness, vertigo, and imbalance.

The patients in the study were recruited from the ‘Audio vestibular clinic’ of the Ear, Nose and Throat (ENT) department at CMC Vellore. An experienced otolaryngologist assessed all patients who were recruited for the study. Patients detected with unilateral peripheral hypo-function using an electronystagmogram via caloric testing were included in the study. The inclusion criteria for recruitment were: (a) age between 18 to 60 years with a minimum best-corrected vision of 6/6; (b) a clinical diagnosis of peripheral unilateral vestibular dysfunction (confirmed via electronystagmogram tests), and no other neurological deficit; (c) have one of the following complaints indicating incomplete vestibular compensation: disequilibrium, imbalance while walking, vertigo/dizziness, or head movement induced dizziness; and (d) score zero on the SSQ carried out before the start of the VEGAS training session, indicating that patients had not been experiencing any symptoms associated with their vestibular dysfunction or SS prior to the start of the study. This is an essential requirement to prevent excessive aggravation of vestibular/SS symptoms. Even in conventional vestibular rehabilitation, patients are advised not to train or perform activities when they are experiencing such symptoms [13]. Patients were excluded from the study if: (a) they were unwilling or unable to use the smartphone-based system. This was identified through a short orientation session where patients used the system for about 5 min, trying the CB and FM games; or (b) non-zero score on the pre-SSQ scored before the start of VEGAS training.

All potential study participants were provided with an information sheet regarding the study and were recruited upon providing informed consent. There were no patients with Meniere’s disease, and the cause of vestibular dysfunction in all patients, the duration of symptoms in patients, and baseline characteristics of the patients are provided in the Table S6 in the Supplementary file.

3.2 Protocol: Patients

After obtaining informed consent, patients trained with VEGAS for 20–30 min, playing the CB and FM game, in a seated position on a chair. We recruited patients with a pre-SSQ score of zero and patients who were able to tolerate the system after a short 5 min pre-test. Patients were given extra time to play the games if they wanted to play a little longer. However, if patients complained of uneasiness, headache or any symptom related to SS, the training was immediately stopped. For each of the two games, patients always started from the lowest level and played the game with the adaptive algorithm that automatically changed the difficulty level. Patients were given rests, whenever needed, in between the game trials.

3.3 Protocol: Healthy

Healthy subjects were tested in the current study for two primary reasons: (a) to evaluate the level of SS induced by VEGAS; and (b) to obtain normative data for the different games, which could serve as a benchmark for assessing patient performance in the different games. After a healthy subject gave informed consent, he/she underwent a one-off testing session with VEGAS. Each subject played all the 12 and 6 levels of the CB and FM games, respectively, in a seated position; the levels were presented in random order. After the completion of each level, subjects were allowed to rest for as long as they required. The testing sessions for each healthy subject lasted approximately 20 min.

3.4 Outcome measures and statistical analysis

The two primary outcome measures for the study were the SUS and the SSQ. The SUS is a simple questionnaire-based assessment tool for capturing the subjective assessment of the usability of a system [2]. The SSQ is currently the standard method used for assessing SS [18]. It is used in the current study to evaluate SS induced by the VEGAS HMD and the games, in both healthy and patient subjects. The SSQ was administered at the time of recruitment (pre-SSQ) to ensure none of the participants had any complaints related to SS before the start of the testing session. The pre-SSQ scores for all participants was zero. Immediately after playing the games with VEGAS, subjects filled out the SSQ (post-SSQ), followed by the SUS. A Mann-Whitney U test was carried out on the post-SSQ scores of patients and healthy subjects to test for differences in the two populations when using the VEGAS. The secondary outcome of the study was a set of movement quality measures estimated from the raw head movement data recorded by VEGAS during gameplay. The details of the analysis carried out on movement quality are provided in Section 1 of the Supplementary file.

4.1 Usability of VEGAS

The summary of the SUS obtained from patients and healthy subjects at the end of VEGAS training session is provided in Table 4. Overall, the system was found to be highly usable by patients (88.33±11.44) and healthy subjects (91.5±4.41), a score of 85 and above indicates excellent usability [18]. The details of the subjects’ response to the different SUS components is provided in Table S3 in the Supplementary file. Twelve of the 15 patients reported an overall score of 80 and above, with three patients P02, P06 and P08 reporting scores of 65, 77.5 and 67.5, respectively. Patient P02 found the system too complex to use, while P06 experienced neck pain and felt the smartphone screen was too bright (Table S6 in in the Supplementary file).

Immersive VR systems, in particular, HMDs supporting the stereoscopic 3D view, have been shown to induce SS [12]. The SS-induced by VEGAS was evaluated on healthy subjects and patients immediately following the completion of the testing session. Overall, the healthy subjects reported a mean total SSQ score of 5.41±12.43, and mean scores of 0.64±2.46, 3.02±6.24, and 1.76±4.64 for nausea, oculomotor, and disorientation SSQ subscales, respectively. Out of the 15 healthy subjects, only 4 (≈ 27%) reported any form of SS (Table 4). Ignoring the severity of rating for the different symptoms, if we only count the number of non-zero responses for the different symptoms (indicating its presence or absence), we observe an incidence rate of 0.95%, 5.71%, and 2.67% for nausea, oculomotor and disorientation symptoms, respectively (Table S4 in the Supplementary file).

On the other hand, the patients reported an average of (53±40.98) as their total SSQ score; a Mann-Whitney U test indicated that this score was significantly higher than that of healthy subjects (p = 2.296 × 10^-5). Patients reported mean scores of 7.63±8.98, 21.61±18.22, and 23.76±18.13 for the nausea, oculomotor and disorientation SSQ subscales. Out of the 15 patients, 14 (≈ 94%) patients had reported some form of SS following training (Table 4). The incidence rates for nausea, oculomotor, and disorientation symptoms were 10.48%, 31.43%, and 22.67% respectively (Table S5 in the Supplementary file). No differences in the SSQ were observed between males and females in the healthy and patient groups, unlike a recent study that found females to be more susceptible to SS following VR training using a HMD [25].

4.3 Adaptation of game difficulty

Only the patient group trained with the adaptive algorithm for automatic adjustment of game difficulty. A representative plot of the game level adaptation for the two games for a particular patient is shown in Fig. 2. The top row depicts the game level chosen for the next trial at the end of the current trial; for example, at the end of trial number 7 for the CB game, the level for the 8th trial is increased from 1 to 2. We can see that the game adapted to the patient’s performance, ensuring the patient’s performance score was around the threshold (grey line in the bottom row of plots), thus continually challenging the patient during training.

OPEN IN VIEWER

Fig.2

The plot of game levels (top row) and game scores (bottom row) as a function of the trial number for the CB (left column) and FM (right column) games for a patient. The game levels shown in the top row correspond to the game level for the next trial. The grey line in the bottom row indicates the score threshold that defines a successful or failed trial. Three consecutive successful trials will increment the game level by 1, while three failed trials will decrement the game level by 1.

The plot of head movement kinematics during training for a patient and a healthy subject is shown in Fig. 3; these plots show the target and actual pitch/yaw angles for level 1 of both games. For the CB game (top two rows of Fig. 3), the patient’s movements (left column) were slower, resulting in lesser number of cardboard breaks (number of back and forth movements). In the FM game (bottom two rows of Fig. 3), the graph of the patient’s actual yaw angle was jerkier than that of a healthy subject. The summary of the movement performance for healthy subjects for all levels of the CB and FM games is shown in Figure S1 and Figure S2 in the in the Supplementary file, respectively. This normative data distribution can serve as a benchmark to evaluate patient performance in the differentgames.

OPEN IN VIEWER

Fig.3

Raw movement data for the CB and FM games. The grey traces in these subplots are the target angles,while the black traces are the actual pitch and yaw angles.

A comparison of the normative data with that of the patient data is depicted in the boxplots in Fig. 4 and Fig. 5; these boxplots use healthy subjects’ data only from the first three and four levels of the CB and FM games, respectively; the patient data had several outliers not displayed in the boxplots. Overall, the movement measures were all statistically different between patients and healthy subjects, except for “root mean square error” measure for the FM game.

OPEN IN VIEWER

Fig.4

Comparison of movement quality for the CB game between healthy subjects and patients. The comparison is made only using the first three levels of the CB game as the maximum level reached by patients was level 3. (‘**’ implies p < 0.001).

OPEN IN VIEWER

Fig.5

Comparison of movement quality for the FM game between healthy subjects and patients. The comparison is made only using the first four levels of the FM game as the maximum level reached by patients was level four. (‘*’ implies p < 0.05).

5.1 Usability of the VEGAS

Overall, the results from the current study indicate that both healthy subjects and patients found the system to be highly usable, reporting an average SUS score of around 92 and 88, respectively. These results are very encouraging even though the current system is a preliminary lab prototype using: (a) a mid-range Android smartphone; (b) an affordable off-the-shelf headset; and (c) a set of fairly simple games. Given the increasing popularity of smartphone-based VR HMDs for gaming and entertainment, the coming years will see improved, highly customizable, and lightweight headsets; this will address the concerns related to the weight of the headset (P01 and P04 in Table S6 in the Supplementary file).

The current system has only two training games, which were played with an adaptive algorithm for automatic game difficulty adjustment. In general, patients did not have any complaints about the games, and some had verbally conveyed that they enjoyed the games and were interested in having the system for training at home. The game level selection algorithm adapted the game difficulty to keep the patient’s performance around the threshold for success/failure (Fig. 2), thus making the games challenging but achievable [6]. Based on the current study, one cannot draw any conclusions on whether patients will find the system engaging for long-term training (6–8 weeks). However, we strongly believe that with a more ergonomic headset, additional training games with appropriate feedback and progress tracking features, patients will find the system engaging even for longer unsupervised training.

5.2 Simulator sickness after VEGAS Training

Immersive VR environments are known to induce SS in healthy subjects [4, 17], whose after-effects can last a few hours following exposure. SS can lead to poor compliance, and the associated safety concerns can impede the acceptance of immersive VR systems for training. Thus, it was essential to evaluate the level of SS induced by VEGAS in both patient and healthy populations.

Following VEGAS training, the incidence rate for overall SS in healthy subjects was quite low (≈27%), indicating that the system and training were well tolerated by healthy subjects. On the other hand, patients reported a significantly higher incidence of SS (≈94%) following their training session. Given that the SS and vestibular dysfunction related symptoms overlap, the increased incidence rate is likely due to (a) the simulator, i.e., the nature of the VEGAS hardware and the VR environment; and (b) the head movement performed by the patients during training. The current study, however, does not allow us to identify the relative contribution of these two factors to the final SSQ score. One way to estimate the effect of the vestibular exercises on SSQ is to have the same group of patients train similar conventional exercises (without HMD) for 20 min in a single session. Changes in SSQ scores before and after conventional training would allow us to get an idea about the contribution of the exercises in inducing SS symptoms. Although this was not carried out in the current study, we plan to investigate this as part of our future work.

The VEGAS hardware, nature of the VR games, and the adaptive game algorithm, all play an important role in inducing SS or vestibular dysfunction related symptoms. Understanding the effect of all these factors can ensure the development of a safe and usable system for patients. Furthermore, understanding patients’ conditions and individualizing training can also ensure that therapy is challenging, well-tolerated, and safe.

5.3 Movement performance of healthy subjects and patients

Head kinematics was used to quantify the movement performance of healthy subjects and patients, which can help understand patients’ movement problems and monitor training. As expected, we observed that patients moved slower than healthy subjects in the CB game; their movement times were longer and had scored a lower number of cardboard breaks (third and fourth plots from the left in Fig. 4). Interestingly, patients appeared to have smoother and straighter movements than healthy subjects in the CB game; the absolute difference between the median values of movement smoothness are, however, quite small. We believe that this difference is due to a more cautious head movement strategy opted by patients, leading to slower, straighter, and smoother movements.

On the other hand, healthy subjects had focused on making fast movements to the different targets which could have slightly compromised their movement straightness and smoothness, because of overshoots and endpoint corrections. This is seen in the top row of Fig. 3, where the healthy subject appears to be making ballistic movements to the targets with overshoots. While the patient’s movements appear to have a “movement phase” towards the target, followed by a “slowing down” phase (the plateaus close to the target); the patient never overshoots the target, unlike the healthy subject. Such results have also been observed in other studies, where head movements of healthy subjects have more overshoots than that of patients for discrete movements [3, 19].

5.4 Limitations of the current system

The current version of the VEGAS is a preliminary prototype with several features that can be improved. Following are some of the major drawbacks identified for the system and the current study, which will be addressed by our ongoing work:

Graphics frame rate: The VEGAS requires a reasonably powerful smartphone capable of rendering the graphics at an acceptable frame rate that does not induce large delays between head movements and scene update. We found that some of the mid-range smartphones can provide framerates of 50–60 Hz while running the two games (Table S1 in the Supplementary file). However, we did not benchmark the VR scene update latency of VEGAS. This latency could have contributed to the SS reported by subjects. Benchmarking this latency will be carried out in our future work.