Usability and Tolerability of Virtual Reality-Based Cognitive Stimulation in Healthy Elderly Volunteers

Abstract

Background:

The aim of the current pilot study was to evaluate the usability, acceptability, and tolerability of virtual reality (VR)-based cognitive stimulation exercises (CSEs) in healthy young versus old populations before health care integration. A secondary aim was to assess the accuracy of VR games as a proxy for cognitive stimulation, specifically for attention. VR-based CSEs promise to improve attention and brain function through varied learning systems.

Methods:

This is a Phase 1 feasibility clinical trial at a single center. It involves 30 healthy volunteers randomly selected using the American Society of Anesthesiologists (ASA) physical status classification system. Participants fall into ASA 1 (age >18–35 years, n = 15) or ASA 2 (age >60 years, n = 15) categories. All participants tested the ReCognitionVR-based CSEs. Feasibility criterion: Participants in each group were monitored for completion of 20 minutes of VR-based CSEs. Acceptability criterion: Proportion of participants with system usability scale (SUS) >35 or SUS score of 87.5. Safety (tolerability) monitoring: Sessions were monitored for neurological, cardiovascular, or pulmonary adverse events (AEs). Safety criterion: No more than 10% of sessions stopped due to neurological, cardiovascular, or pulmonary AEs.

Results:

The primary outcome (feasibility) of ReCognitionVR-based CSEs was 100%. For the secondary outcome (acceptability), there was no group difference in SUS scores (ASA 1 = 88.17 ± 12.83 vs. ASA 2 = 88.39 ± 10.22, P = 0.81). For the tertiary outcome (safety), mild transient uneasiness was reported by two (13.4%) ASA 1 participants (resolved in 2 minutes), and one (6.67%) ASA 1 participant experienced a temporary >20% increase in blood pressure from baseline. No ASA 2 participants had AEs.

Conclusion:

The feasibility, acceptability, and safety of ReCognitionVR-based CSEs in healthy elderly volunteers are acceptable, indicating that the evaluation of the ReCognitionVR-based CSEs in hospitalized patients is reasonable.

Introduction

Cognitive stimulation (CS) involves engagement in a range of activities designed to stimulate and enhance cognitive functions, such as attention, memory, and executive functions.¹ Gamification has been widely used to enhance the engagement of participants in cognitive tasks and performance.^2–4 Computerized cognitive training programs have been shown to improve attention deficit disorder and mild cognitive impairment.⁵ In recent years, game-assisted psychological therapy combined with virtual reality (VR) technology has emerged as a research focus in health care.^6–8 Non-VR-based video games have been used to determine functional outcomes in frail and prefrail older adults and found to be feasible, acceptable, and safe for frail older adults.⁹ CS through VR is a new and upcoming area in technology that is rapidly expanding, but reports of its acceptability and utility are currently limited in the literature. The highly immersive and augmented reality features of VR technology can have a greater effect on the user’s attention and situational experience than nonimmersive VR and computerized cognitive training.¹⁰ Because VR engages multiple cognitive domains, such as memory, attention, language, and executive function, it may be a more effective technology or environment for cognitive training.¹¹ Accuracy in VR games refers to the ability of the user to interact with VR, accurately hit targets, and time and positioning of VR game tasks performance.¹² Attention in VR games refers to the user’s cognitive focus and engagement in VR.¹³ There is a strong connection between accuracy and attention.¹⁴ Accuracy will likely improve when users are more attentive to the game in VR and their tasks. The immersive nature of VR encourages the user to be more focused and engaged, which enhances performance accuracy.¹⁵ Despite the many benefits of VR games that make them suitable for CS, some VR and augmented reality users experience cybersickness.^16–18 Moreover, research has provided evidence of a higher incidence of cybersickness in older adults than in young individuals,¹⁹ which warrants evaluation of the safety of VR games-based psychological therapy in an older population. Our research group developed a 3D-simulated VR software platform prototype called ReCognitionVR to provide VR-based cognitive stimulation exercises (CSEs) that allow users to immerse themselves in a relaxing virtual environment. A description of deconstructing ReCognitionVR software, design, and development can be found in Supplementary Data S1. ReCognitionVR software allows users to immerse themselves in a relaxing virtual environment; reorient the user to person, time, and location; play minigames to encourage attention, focus, and movement of limbs; and record user progress and scores in minigames.

We hypothesized that the VR-based CSEs would be feasible and acceptable in elderly volunteers, like in younger volunteers, but less tolerable in elderly volunteers than in younger volunteers. We postulated that the test population would find VR-based CSEs feasible, tolerable, and acceptable for the following reasons: (1) VR games would allow the users to focus and be attentive during minigames; (2) playing minigames would encourage early rehabilitation in a relaxed environment; (3) relaxation would help users to be more involved in CSEs as an essential motivation for task completion and performance. The primary objective of this study was to evaluate feasibility, acceptability, and tolerability of ReCognitionVR software in elderly and young healthy volunteers to lay the groundwork for future studies focusing on elderly surgical patients.

Methods

A prospective Phase 1 feasibility trial was conducted at Houston Methodist Hospital from August 2022 to January 2023. This study was approved by the Houston Methodist Research Institute’s institutional review board (IRB) (ID # PRO00033195, approved July 28, 2022) and registered at ClinicalTrials.gov (NCT05583903). All protocols adhered to the ethical standards of our IRB. All study participants provided written informed consent.

The study participants consisted of 30 adults, both women and men, who were stratified based on American Society of Anesthesiologists (ASA) criteria 1 [fit, body mass index (BMI) <30 kg/m², nonsmoker with good exercise tolerance] or 2 [mild systemic disease (e.g., treated hypertension), BMI <35 kg/m², no functional limitations, frequent social drinker, currently nonsmoker]. Participants who were >18 or up to 35 years of age and met ASA 1 criteria and participants who were >60 years and met ASA 2 criteria were enrolled. We excluded participants with underlying cognitive disorders, schizophrenia, or associated phobias (e.g., claustrophobia) and participants who were deaf or blind. The intended sample size was 30 participants: 15 were >18–35 years and 15 were >60 years. The rationale for the age distribution in this study was to obtain information from people considered healthy by ASA criteria in a younger cohort and an older cohort. We also recognized that younger participants (i.e., >18–35 years) are more likely to have experienced VR software, and older participants (>60 years) are less likely to have experienced VR software. Participants were identified through public advertisements, which included flyers posted around the Texas Medical Center and mass emails to local universities’ listservs (e.g., University of Houston and Rice University).

Intervention

The study participants received a 3D ReCognitionVR-based CSEs program targeting attention, organized thinking, and motor activities. Figure 1 shows the equipment and user setup for the ReCognitionVR software platform used for the study trial. Figure 2 shows a stepwise photographic presentation of the CSEs, including a minigame played by study participants for CS. The VR platform consists of a VR headset manufactured by the Oculus Quest 2 glasses (Facebook Inc., USA) with stereoscopic vision and stereo sound to show the virtual world to participants, a motion sensor, and a flat-screen TV or laptop. After headset placement, study participants were able to follow instructions provided by virtual nurse AVATAR. The participants received low cognitive load exercises based on VR techniques, including a relaxed environment, time orientation, delivery instructions, and task completion motivation. The exercise task focused on attention, organized thinking, and motor activities. The patient had an option to choose between a “game” mode and a “relax” mode. The study participants received approximately 20 minutes of VR-based CS sessions and completed a postuse survey. All participants chose the “game” mode. In this mode, participants played multiple rounds of the game with increasing levels of difficulty based on the increasing speed of the balloons spawning. The balloons in the game were red, yellow, and blue in color. Participants popped the balloons using hand-controlled motion sensors. Accuracy was recorded for each level of game in the easy, medium, hard, and finally, survival mode (where the patient had three chances to survive as long as possible). For the easy, medium, and hard levels, study participants had 60 seconds in each round to pop as many balloons as accurately as possible. The accuracy percentage was displayed at the end of each round. For survival mode, the game and timer continued for as long as the study participant could keep 100% accuracy. The session ended immediately if the study participant incorrectly popped a balloon. The speed of the balloons flying to the sky increased with each difficulty level. The number of game attempts (total attempts to complete the entire game) and the number of user errors (total number of wrong balloons popped) were recorded. Physiological data (vital signs) were recorded at baseline (before starting the game), 10 minutes into the game, and finally, after the end of the game. After the game, the participants completed the system usability scale (SUS) questionnaire.²⁰

FIG. 1.

Equipment and set-up for ReCognitionVR software.

FIG. 2.

VR-based CSEs. User views from within the ReCognitionVR software (right) with corresponding user actions (left). CSE, cognitive stimulation exercise; VR, virtual reality.

Acceptability survey

At the end of the study, all participants were given the SUS questionnaire²⁰ (Supplementary Data S2) to complete. The validated SUS questionnaire consisted of 10 questions and was ranked from 1 to 5 based on their experience, with 1 - strongly disagree and 5 - strongly agree. If the user could not answer a particular question on the scale, they marked the scale’s center point to ensure their response remained neutral and did not skew the overall results. Once the SUS questionnaire was completed, the SUS score was calculated. To calculate the SUS score, we adjusted the scores for each item. For the odd-numbered items (1, 3, 5, 7, and 9), we subtracted 1 from the score. For the even-numbered items (2, 4, 6, 8, and 10), we subtracted the score from 5. Once all the scores were adjusted, we summed them. Finally, the summed score from the scale was multiplied by 2.5 to obtain the overall SUS score. This final number represented the system usability score as evaluated by the users. According to Lewis and Sauro,²¹ SUS scores above 68 are considered average, while an SUS score above 80 is considered above average. An open-source SUS toolkit²⁰ was used here to analyze and present our findings. Acceptability was defined as the proportion of participants with a SUS >35 or SUS score >87.5. The accuracy of VR games calculated by ReCognitionVR software automatically depends on the mode of the games. Easy, medium, and hard scores were calculated as percentages using the number of balloons popped by study participants, and survival mode scores were calculated in times (seconds) based on shooting the wrong target.

Outcome’s measurement

The primary outcome was the feasibility of ReCognitionVR-based CSEs, defined as 70% of participants in each group completed 20 minutes of ReCognitionVR-based CSEs. The secondary outcome was to determine the acceptability of ReCognitionVR-based CSEs using an SUS questionnaire.²⁰ Acceptability was defined as the proportion of participants with SUS >35 or SUS score of 87.5. The tertiary outcome was the safety (tolerability) of ReCognitionVR-based CSEs. Safety was assessed using the following endpoints: (1) mean change from preintervention to 10 minutes postexercise in heart rate variability (HRV), pulse oximetry oxygen saturation (SpO2), respiratory rate (RR), and blood pressure (BP); and (2) mean change from preintervention to the end of the VR session in HRV, SpO2, RR, and BP. HRV was measured using the iPhone 8 health application. Vital signs were measured using standard vital signs monitors.

The safety (tolerability) goal was no more than 10% of sessions aborted due to adverse events (AEs). Predefined AEs, if they occurred within 5 minutes after starting 3D-simulated VR sessions, were recorded by organ-system class. Neurological AEs¹⁴ captured were cybersickness (a subtype of motion sickness that may accompany immersion in VR). Other AEs captured were gastrointestinal (e.g., stomach awareness, nausea, vomiting), central (e.g., fainting, lightheadedness, blurred vision, disorientation, dizziness, the sensation of spinning), peripheral (e.g., sweating, feeling hot), and other symptoms (annoyance, drowsiness, tiredness, uneasiness).

Cardiovascular AEs were defined as new-onset hypotension [systolic blood pressure (SBP) <90 mmHg and diastolic blood pressure (DBP) <60 mmHg], bradycardia with HR <50 bpm, and tachycardia with HR >130 bpm. Pulmonary AEs were defined as new-onset hypoxia (SpO2 < 88%) and tachypnea (RR >20 breaths/min).

Statistical analysis

All baseline characteristics are reported as means with standard deviations, medians with interquartile ranges (IQRs), and numbers (%) by groups and overall. The outcomes feasibility, acceptability, and safety are presented as numbers with percentages. The change >20% for each vital sign from baseline to either 10 minutes after start of intervention or at the end of the VR session is also presented as number (%) by groups, respectively. Analyses were performed using data from participants who met all inclusion and exclusion criteria. Any participant who was incorrectly assigned was analyzed for safety events only. Fisher’s exact test for categorical variables and Mann–Whitney test for continuous variables were used to compare participants between ASA 1 and ASA 2. All AEs were captured and tabulated between the two groups. All analyses were performed with STATA version 16 (StataCorp. 2019. Stata Statistical Software: Release 16. College Station, TX: StataCorp LLC). Statistical significance was defined as two-tailed P < 0.05 for all tests.

Results

In this study, all 30 participants who signed the informed consent completed the study. Fifteen participants were enrolled in each group. One participant in the ASA 2 group was incorrectly enrolled (BMI >35.1 kg/m²). This participant was excluded from the feasibility and acceptability analysis but was included in the safety analysis. Six of fifteen (40%) participants in the ASA 2 group had hypertension compared with no participants in ASA 1 (P = 0.017).

There were no statistically significant differences in baseline characteristics between ASA 1 and ASA 2 other than age, which was used to define the groups ASA 1 and ASA 2 (Table 1). All study participants in both ASA 1 and 2 groups completed the ReCognitionVR-based CSEs with a feasibility of 100% (Table 2). For the minigames’ accuracy based on the scores (Table 2), the ASA 2 group had a significantly lower median (IQR) survival mode score [49 (42, 56)] and hard mode score [95 (92,98)] compared with the ASA 1 group [59 (55,63), P = 0.002] and [100 (97,100), P = 0.001], respectively. The ASA 2 group participants made more attempts to complete the game than those in the ASA 1 group (64% vs. 13% for 2 and 3 attempts, P = 0.01).

Table 1.

Baseline Characteristics

	Total	ASA 1	ASA 2
	N = 30	N = 15	N = 15	P value
Age in years, median (IQR)	47.50 (28.00–65.00)	28.00 (25.00–31.00)	65.00 (62.00–67.00)
Gender, n (%)				0.14
Female	13 (43.33)	4 (26.67)	9 (60.00)
Male	17 (56.67)	11 (73.33)	6 (40.00)
Race, n (%)				0.17
Asian	12 (40.00)	6 (40.00)	6 (40.00)
Black or African American	2 (6.67)	0 (0.00)	2 (13.33)
White	14 (46.67)	9 (60.00)	5 (33.33)
Unknown/not reported	2 (6.67)	0 (0.00)	2 (13.33)
Ethnicity, n (%)				1.00
Hispanic or Latino	2 (6.67)	1 (6.67)	1 (6.67)
Not Hispanic or Latino	28 (93.33)	14 (93.33)	14 (93.33)
BMI (kg/m²), median (IQR)	24.50 (22.30–26.50)	23.60 (21.30–25.10)	25.20 (23.80–29.30)	0.056
Nonsmoker, n (%)	30 (100.00)	15 (100.00)	15 (100.00)	1.00
Hypertension, n (%)	6 (20.00)	0 (0.00)	6 (40.00)	0.017^*
Diabetes, n (%)	2 (6.67)	0 (0.00)	2 (13.33)	0.48
Hypercholesterolemia, n (%)	3 (10.00)	0 (0.00)	3 (20.00)	0.22

Data are presented as n (%) for frequency data and median (IQR) for continuous variables. Statistical significance (P values) are reported for tests comparing ASA 1 and ASA 2; Fisher’s exact test was used for categorical variables and Mann–Whitney test was used for continuous variables.

P < 0.05.

ASA, American Society of Anesthesiology; BMI, body mass index; IQR, interquartile range.

Table 2.

Feasibility, Accuracy, and System Usability Scale Score by American Society of Anesthesiologists

	Total	ASA 1	ASA 2
	N = 29	N = 15	N = 14	P value
Feasibility
Exercise completion within 20 minutes, n (%)	29 (100)	15 (100)	14 (100)
Acceptability
SUS Score, n (%)				0.26
>35	18 (62)	11 (73)	7 (50)
SUS score, mean ± SD	88.28 ± 11.44	88.17 ± 12.83	88.39 ± 10.22	0.81
SUS score, median (IQR)	90.00 (85.00–97.50)	90.00 (85.00–97.50)	87.50 (81.88–100.00)	0.81
Accuracy
Survival mode score, median (IQR)	56 (48–59)	59 (55–63)	49 (42–56)	0.002^*
Number of attempts to complete the game				0.010^*
1	18 (62)	13 (87)	5 (36)
2	10 (34)	2 (13)	8 (57)
3	1 (3)	0 (0)	1 (7)
Easy mode score, median (IQR)	100 (100–100)	100 (100–100)	100 (100–100)	0.57
Medium mode score, median (IQR)	100 (97–100)	100 (97–100)	99 (96–100)	0.20
Hard mode score, median (IQR)	97 (95–100)	100 (97–100)	95 (92–98)	0.001^*

Data are presented as n (%) for frequency data and median (IQR) or mean ± SD for continuous variables. Statistical significance (P values) are reported for tests comparing ASA 1 and ASA 2; Fisher’s exact test was used for categorical variables and Mann–Whitney test was used for continuous variables.

P < 0.05.

SD, standard deviation; SUS, system user scale.

The mean SUS scores for ASA 1 and ASA 2 were 88.17 ± 12.83 and 88.67 ± 9.90, respectively (Table 2, Fig. 3). There were no statistically significant differences in SUS scores between ASA 1 and ASA 2 (P = 0.81; Fig. 3).

FIG. 3.

Average SUS score per item of ASA 1 and ASA 2 criteria. ASA, American Society of Anesthesiologists; SUS, system user scale.

A statistically significant mean difference in HRV between the two groups was observed at preintervention (P = 0.034) and the end of the VR session (P = 0.036) but not at 10 minutes post-exercise (P = 0.1; Table 3). A statistically significant mean difference in SBP between the two groups was observed at preintervention (P = 0.003) but not 10 minutes postexercise (P = 0.083) or the end of the VR session (P = 0.09). A statistically significant mean difference in DBP was observed between the two groups at preintervention (P = 0.012) and 10 minutes postexercise (P = 0.007) but not at the end of the VR session (P = 0.091). Likewise, the mean BP was different between ASA 1 and ASA 2 at preintervention (P = 0.003) and 10 minutes postexercise (P = 0.007) but not at the end of the VR session (P = 0.061). For all vital signs, there were no statistically significant differences in the change from preintervention to 10 minutes postexercise or from preintervention to the end of the VR session.

Table 3.

Safety Assessed as Changes in Vital Signs According to American Society of Anesthesiologists Grouping

	Total	ASA 1	ASA 2
	N = 30	N = 15	N = 15	P value
HRV
Preintervention	63.63 ± 9.91	67.47 ± 7.20	59.80 ± 10.96	0.031^*
10 minutes after start of intervention	64.47 ± 8.19	66.93 ± 7.04	62.00 ± 8.74	0.100
End of VR session	63.97 ± 8.47	67.07 ± 7.64	60.87 ± 8.35	0.043^*
Change from pre- to 10 minutes	0.83 ± 5.96	−0.53 ± 3.46	2.20 ± 7.59	0.21
Change >20%	3 (10.00)	0 (0.00)	3 (20.00)	0.22
Change from pre- to end of VR session	0.33 ± 4.98	−0.40 ± 3.81	1.07 ± 5.97	0.43
Change >20%	1 (3.33)	0 (0.00)	1 (6.67)	1.00
SpO2
Preintervention	98.73 ± 1.05	98.53 ± 1.06	98.93 ± 1.03	0.30
10 minutes after start of intervention	98.60 ± 1.22	98.33 ± 1.40	98.87 ± 0.99	0.24
End of VR session	98.57 ± 1.14	98.20 ± 1.37	98.93 ± 0.70	0.076
Change from pre- to 10 minutes	−0.13 ± 0.82	−0.20 ± 1.01	−0.07 ± 0.59	0.66
Change >20%	0 (0.00)	0 (0.00)	0 (0.00)
Change from pre- to end of VR session	−0.17 ± 0.75	−0.33 ± 0.82	0.00 ± 0.65	0.23
Change >20%	0 (0.00)	0 (0.00)	0 (0.00)
RR
Preintervention	13.67 ± 1.92	13.33 ± 1.72	14.00 ± 2.10	0.35
10 minutes after start of intervention	14.73 ± 3.25	14.53 ± 3.40	14.93 ± 3.20	0.74
End of VR session	14.53 ± 2.98	14.40 ± 2.82	14.67 ± 3.22	0.81
Change from pre- to 10 minutes	1.07 ± 2.43	1.20 ± 2.62	0.93 ± 2.31	0.77
Change >20%	3 (10.00)	2 (13.33)	1 (6.67)	1.00
Change from pre- to end of VR session	0.87 ± 2.24	1.07 ± 2.22	0.67 ± 2.32	0.63
Change >20%	4 (13.33)	3 (20.00)	1 (6.67)	0.60
SBP
Pre-intervention	122.37 ± 11.90	117.00 ± 6.34	127.73 ± 13.83	0.011^*
10 minutes after start of intervention	123.17 ± 14.87	118.47 ± 11.77	127.87 ± 16.50	0.083
End of VR session	126.57 ± 13.46	122.40 ± 10.52	130.73 ± 15.09	0.090
Change from pre- to 10 minutes	0.80 ± 9.71	1.47 ± 12.06	0.13 ± 7.00	0.71
Change >20%	1 (3.33)	1 (6.67)	0 (0.00)	1.00
Change from pre- to end of VR session	4.20 ± 9.30	5.40 ± 10.54	3.00 ± 8.07	0.49
Change >20%	1 (3.33)	1 (6.67)	0 (0.00)	1.00
DBP
Preintervention	76.67 ± 12.25	71.73 ± 9.33	81.60 ± 13.10	0.025^*
10 minutes after start of intervention	77.27 ± 12.50	71.40 ± 9.21	83.13 ± 12.86	0.008^*
End of VR session	75.13 ± 11.24	71.67 ± 9.56	78.60 ± 12.02	0.091
Change from pre- to 10 minutes	0.60 ± 5.44	−0.33 ± 5.59	1.53 ± 5.32	0.36
Change >20%	0 (0.00)	0 (0.00)	0 (0.00)
Change from pre- to end of VR session	−1.53 ± 9.26	−0.07 ± 6.52	−3.00 ± 11.43	0.40
Change >20%	1 (3.33)	1 (6.67)	0 (0.00)	1.00
Mean BP
Preintervention	91.90 ± 11.16	86.82 ± 6.69	96.98 ± 12.57	0.010^*
10 minutes after start of intervention	92.57 ± 12.24	87.09 ± 8.78	98.04 ± 12.99	0.011^*
End of VR session	92.28 ± 10.86	88.58 ± 8.55	95.98 ± 11.92	0.061
Change from pre- to 10 minutes	0.67 ± 5.06	0.27 ± 5.31	1.07 ± 4.95	0.67
Change >20%	0 (0.00)	0 (0.00)	0 (0.00)
Change from pre- to end of VR session	0.38 ± 7.36	1.76 ± 5.71	−1.00 ± 8.68	0.31
Change >20%	0 (0.00)	0 (0.00)	0 (0.00)

Data are presented as mean ± SD for all measures and n (%) for the change >20%. Statistical significance (P values) are reported for tests comparing ASA 1 and ASA 2; Fisher’s exact test was used for categorical variables and Mann–Whitney test was used for continuous variables.

P < 0.05.

BP, blood pressure; DBP, diastolic blood pressure; HRV, heart rate variability; RR, respiratory rate; SBP, systolic blood pressure; SpO2, pulse oximetry oxygen saturation; VR, virtual reality.

Only four AEs occurred during the study (Table 4): 3 (20%) in the ASA 1 group and 1 (6.67%) in the ASA 2 group. Although 1/15 (6.67%) of ASA 1 participants experienced transient dizziness and 1/15 (6.67%) of ASA 2 participants had vague, undefined minds/brain fog, both participants completed the study interventions. One study participant, 1/15 (6.67%) from the ASA 1 group, experienced a transient increase in SBP >20% from baseline. None of the ASA 2 group participants experienced any AEs. No participant experienced more than one AE. No serious AEs occurred. The procedure was not interrupted due to an AE.

Table 4.

Adverse Events by American Society of Anesthesiologists

	All	ASA 1	ASA 2
AE	N = 30	N = 15	N = 15
Neuro, n (%)
Numbness^a	1 (3.33)	1 (6.67)^a	0
Nystagmus	0	0	0
Transient dizziness on moving	1 (3.33)	1 (6.67)	0
Persistent dizziness	0	0	0
Other^b	1 (3.33)	0	1 (6.67)^b
Subtotal, neurological	3 (10.00)	2 (13.33)	1 (6.67)
Cardiovascular, n (%)
New bradycardia	0	0	0
New tachycardia	0	0	0
Hypotension, >20% of baseline	0	0	0
Hypertension, >20% of baseline	1 (3.33)	1 (6.67)^c	0
Other	0	0	0
Subtotal, cardiovascular	1 (3.33)	1 (6.67)	0
Pulmonary/respiratory, n (%)
New onset hypoxia	0	0	0
RR >35	0	0	0
New dyspnea	0	0	0
Other	0	0	0
Subtotal, pulmonary/respiratory	0	0	0
Total, n (%)	4 (13.33)	3 (20.00)	1 (6.67)

Transient numbness of arm due to tight blood pressure cuff.

Undefined mind/brain fog.

Undefined transient systolic hypertension secondary to a history of situational hypertension related to testing events.

AE, adverse event.

Discussion

The goal of this feasibility study was twofold: (1) to examine the feasibility, acceptability, and tolerability of ReCognitionVR-based CSEs and (2) to obtain preliminary data on elderly participants’ performance during VR games, which was used as a proxy CS, particularly for attention domain of cognition. The present study supports our hypothesis and demonstrates that ReCognitionVR-based CSEs were highly feasible and acceptable by healthy elderly volunteers. However, the study results rejected our hypothesis that VR-based CSEs would be less tolerable, as our study found that ReCognitionVR-based CSEs were very safe and highly tolerable by healthy elderly volunteers (ASA 2 group). In addition, elderly participants had lower VR games’ accuracy scores and required more attempts to complete the games, but the difference was acceptable.

CS is a nonregimented, nonpharmacological group of techniques that promotes cognitive processing, restores cognitive functions, and improves the performance and effectiveness of cognitive capabilities such as memory, attention, language, and reasoning.²² Several strategies exist for CSEs, ranging from classical CSE workbooks²³ to more dynamic, innovative techniques such as brain training games²⁴ and online programs.²⁵ Conventional CS methods can become repetitive and tedious, leading to decreased motivation and participant disengagement.²⁶ Moreover, implementing CS protocols in inpatient hospitals is challenging due to barriers that include nursing staff-related challenges, documentation burden, a lack of understanding, and not being prioritized.²⁷

Digital technology-based psychological interventions, such as VR games-assisted CS, are emerging as an alternative to traditional CS therapy. VR is a fully digital, computer-generated, 3D experiential environment that allows the user to step inside an experience, become immersed in it, and interact with a 3D world that can either simulate or differ entirely from the real world.²⁸ There are three primary categories (nonimmersive, semi-immersive, and fully immersive) of VR simulations used today. Fully immersive VR can give users the most realistic simulation experience, complete with sight and sound.²⁸ The VR environment offers treatment flexibility and better engagement and tailors the exercise to the needs, cognitive status, and motivation for the participation of individual users, which provides therapy that is more feasible, enjoyable, and effective.^29,30 VR-based psychological therapy has been used as an additional treatment as a part of cognitive behavioral therapy for managing various mental health conditions, including anxiety, depression, post-traumatic stress disorder, and phobia.⁸ However, studies on VR-based psychological intervention are lacking in the elderly population. Moreover, safety and tolerability concerns have been raised regarding using VR-based CSEs in the elderly surgical population. VR experiences can elicit a negative effect called cybersickness, particularly in an older population.¹⁹ Previous research states that approximately 20%–80% of VR users have experienced cybersickness or related discomfort at least once.^31,32

Cybersickness is characterized by severe and frequent disorientation symptoms (dizziness, vertigo, and difficulty in focusing), followed by nausea symptoms (stomach awareness, increased salivation, and nausea itself), and in third place, oculomotor symptoms (eyestrain, headache, and blurred vision), the so-called D > N > O profile.^16,17,32 Cybersickness may hinder VR’s immersive and interactive qualities, limiting the VR system’s usability and real-life applications.³³ Literature suggests younger people are more resistant to cybersickness³⁴ and higher age groups experience more cybersickness.¹⁹ In contrast, our study results showed that ReCognitionVR-based CSEs were safe and tolerable by healthy elderly volunteers. In the present study, all participants, including elderly study subjects, completed the full duration of the ReCognitionVR-based CSEs. Minimal AEs were reported during the study. No sessions were stopped prematurely or early for safety concerns. Four AEs were reported, including double vision and a new-onset, transient systolic hypertension, but these occurred in the younger ASA 1 group, not in older participants. Double vision was found to be related to the misfitting of the Oculus Quest 2 Headset on the participant and promptly resolved after adjustment of the appropriate straps. Systolic hypertension was secondary to a history of situational hypertension related to testing events. Both events were resolved quickly and were not present at the end of the study session. Future studies should ensure proper headset positioning and operator training. HRV has been linked to human performance under stress.

Few studies have examined HRV and performance in VR, which stimulates natural stress conditions.¹⁹ The current study found no statistical difference in change in HRV pre- and postcognitive exercise delivered via ReCognitionVR. Moreover, there were no statistically significant differences in the change from preintervention to 10 minutes or postintervention in other safety measures, including SpO2, SBP, DBP, and mean BP.

Acceptability is one of the main factors considered before adopting and implementing new interventions, specifically computerized cognitive software. SUS scores above 68 are considered average, while an SUS score above 80 is considered above average.²¹ For practical purposes, we selected above-average SUS scores (87.5) to consider our prototype software as an acceptable intervention for elderly participants. In our study, the acceptance survey found a mean SUS score of 88.42 between both ASA groups. Similar results were found by Bacha et al.,²⁰ who evaluated the acceptability of training using Microsoft Kinect Adventures games through a satisfaction questionnaire in community-dwelling older adults. Another study by Gomes et al.²¹ assessed the acceptability of video games using a game satisfaction questionnaire in frail older adults and found that all participants enjoyed the video game.

This study provides essential data about older adults’ initial experience playing VR-based games. The results of our study are consistent with previous findings that support the benefits of VR immersive technology in outpatient settings for rehabilitation.^7,35 We have demonstrated that ReCognitionVR-based CSEs are tolerable, acceptable, and feasible for older populations and offer a promise for CS therapy. However, there are several limitations worth noting. First, this study was a small study designed to inform safety, tolerability, and preliminary data on using VR for CSEs. We did not perform a comprehensive cognitions assessment and used game accuracy scores as a proxy for CS. Therefore, no conclusion can be made regarding the effectiveness of ReCognitionVR in improving cognitive domains needed for CS therapy in clinical practice. Future studies are warranted to evaluate the effects of VR games, including ReCognitionVR software games, on cognitive ability and domains, specifically attention, executive functions, and working memory. This study data will lay the groundwork for the larger trial in which ReCognitionVR-based CSEs will be tested on older hospitalized patients to evaluate the mechanism of VR-based CSEs in more detail. Second, study participants were not screened for anxiety or depression, gender-based analyses were not performed due to the small sample size, and a small sample from a single center limits the generalizability of the specific results of this study. Third, many important questions still need to be addressed, including optimal timing, duration, and intensity of ReCognitionVR-based CS used for psychological interventions. Fourth, we used our prototype platform, the ReCognitionVR, in which games were built considering all the cognitive domains required for CS in clinical practice. We also preferred simple minigames with balloons of different colors and difficulty levels because the prototype ReCognitionVR software was being tested on older volunteers. However, in our study, the accuracy of VR games was more or less the same in easy, medium, and hard modes, which suggests the need to add distraction and incorporate advanced features in minigames of ReCognitionVR software. Lastly, user and operator errors occurred during the execution of ReCognitionVR software, and some users reported that the software is easier to operate with assistance. The main goal of this study was to test the VR-based CSEs in elderly surgical patients. However, our IRB recommended testing the VR-based CSEs in healthy volunteers due to safety and tolerability concerns. In summary, our study findings provide evidence that the accuracy of game scores was acceptable. This shows that playing VR games has the potential to be used for CS in older adults and requires further investigation in inpatient hospital settings.

Conclusion

The feasibility, acceptability, and safety of the ReCognitionVR-based software in this study of healthy volunteers were acceptable. The ASA 2 group participants had lower accuracy scores and required more attempts to complete the exercise, but the difference was acceptable, and no safety concerns were noted. These results indicate that evaluation of the ReCognitionVR-based CSEs is reasonable for study in hospitalized surgical patients.

Footnotes

Acknowledgments

The authors sincerely thank Shaefali P. Rogers, Ph.D. (senior scientific writer, Academic Institute of Houston Methodist), for her diligent proofreading of this article. We also thank Laurie Minz, B.S., M.B.A. (Senior Research Operations Manager, Immunobiology & Transplant Science Center, Houston Methodist Research Institute) for the administrative support and for streamlining all the resources required to conduct this investigation.

Disclaimer

The study’s sponsors and funders had no role in the study design, data collection, analysis, interpretation, report writing, or decision to submit the paper for publication.

Data Sharing

The data will be made available to colleagues and the National Institutes of Health upon request. The data and associated documentation are available to users only under a data-sharing agreement that provides for (1) a commitment to using the data only for research purposes and not to identify any individual participant; (2) a commitment to securing the data using appropriate computer technology; and (3) a commitment to destroying or returning the data after analyses are completed. We will follow Houston Methodist Research Institute’s policy on data-sharing plans.

Authors’ Contributions

H.F. led all aspects of the study, from conceptualization and funding acquisition to article preparation. W.L., A.D., P.L., and R.G. contributed to study design, data collection, visualization, project administration, software design, data validation, and article writing. E.C.L. and S.T.W. contributed to study supervision, data validation, visualization, and article review. J.X. contributed to data curation, formal analysis, and methodology. F.N.M. contributed to study supervision, funding acquisition, visualization, and article review.

Author Disclosure Statement

The content is solely the author’s responsibility and does not necessarily represent the official views of the National Institute on Aging and the National Institutes of Health. All authors have no financial disclosures or any conflicts of interest.

Funding Information

This work received support from the National Institutes of Health/National Institute on Aging R03 grant (R03AG078857) and the Foundation for Anesthesia Education and Research (FAER) grant (GEMSSTAR-11-01-2022-Faisa). Texas A&M Reliant Innovation Funds (26300001) contributed to the VR software’s design and development.

Supplementary Material

References

Woods

, Rai

, Elliott

, et al. Cognitive stimulation to improve cognitive functioning in people with dementia. Cochrane Database Syst Rev, 2023; 2023(1):CD005562; doi: 10.1002/14651858.CD005562.pub3

Khaleghi

, Aghaei

, Mahdavi

. A Gamification framework for cognitive assessment and cognitive training: Qualitative study. JMIR Serious Games, 2021; 9(2):e21900.

Green

, Bavelier

. Action video game modifies visual selective attention. Nature, 2003; 423(6939):534–537.

Lumsden

, Edwards

, Lawrence

, et al. Gamification of cognitive assessment and cognitive training: A systematic review of applications and efficacy. JMIR Serious Games, 2016; 4(2):e11.

Barnes

, Yaffe

, Belfor

, et al. Computer-based cognitive training for mild cognitive impairment: Results from a pilot randomized, controlled trial. Alzheimer Dis Assoc Disord, 2009; 23(3):205–210.

Cho

, Lee

. Effects of virtual reality immersive training with computerized cognitive training on cognitive function and activities of daily living performance in patients with acute stage stroke: A preliminary randomized controlled trial. Medicine (Baltimore), 2019; 98(11):e14752.

Gamito

, Oliveira

, Alves

, et al. Virtual reality-based cognitive stimulation to improve cognitive functioning in community elderly: A controlled study. Cyberpsychol Behav Soc Netw, 2020; 23(3):150–156.

Baghaei

, Chitale

, Hlasnik

, et al. Virtual reality for supporting the treatment of depression and anxiety: Scoping review. JMIR Ment Health, 2021; 8(9):e29681.

Gomes

GCV

, Simões

MDS

, Lin

, et al. Feasibility, safety, acceptability, and functional outcomes of playing Nintendo Wii Fit Plus(TM) for frail older adults: A randomized feasibility clinical trial. Maturitas, 2018; 118:20–28.

10.

Bauer

ACM

, Andringa

. The potential of immersive virtual reality for cognitive training in elderly. Gerontology, 2020; 66(6):614–623.

11.

Catania

, Rundo

, Panerai

, et al. Virtual reality for the rehabilitation of acquired cognitive disorders: A narrative review. Bioengineering (Basel), 2023; 11(1):35.

12.

Chang

C-M

, Hsu

C-H

, Hsu

C-F

, et al. Performance measurements of virtual reality systems: Quantifying the timing and positioning accuracy. 2016; 655–659.

13.

, Liu

. A novel way of estimating a user’s focus of attention in a virtual environment. 2018.

14.

Bigozzi

, Malagoli

, Pecini

, et al. Attention components and spelling accuracy: Which connections matter? Children (Basel), 2021; 8(7):539.

15.

Marougkas

, Troussas

, Krouska

, et al. Virtual reality in education: A review of learning theories, approaches and methodologies for the last decade. Electronics, 2023; 12(13):2832.

16.

, Son

. Cybersickness and its severity arising from virtual reality content: A comprehensive study. Sensors (Basel), 2022; 22(4):1314.

17.

Weech

, Kenny

, Barnett-Cowan

. Presence and cybersickness in virtual reality are negatively related: A review. Front Psychol, 2019; 10:158.

18.

Mazloumi Gavgani

, Hodgson

, Nalivaiko

. Effects of visual flow direction on signs and symptoms of cybersickness. PLoS One, 2017; 12(8):e0182790.

19.

Plechatá

, Sahula

, Fayette

, et al. Age-related differences with immersive and non-immersive virtual reality in memory assessment. Front Psychol, 2019; 10:1330.

20.

Brooke

. SUS: A quick and dirty usability scale. Usability Eval Ind, 1995; 189.

21.

Lewis

, Sauro

. (2009). The factor structure of the system usability scale. In. Human Centered Design, HCII 2009 ( Kurosu

. Ed.), Springer-Verlag: Heidelberg, Germany; pp.94–103.

22.

Gibbor

, Yates

, Volkmer

, et al. Cognitive stimulation therapy (CST) for dementia: A systematic review of qualitative research. Aging Ment Health, 2021; 25(6):980–990.

23.

Orrell

, Yates

, Leung

, et al. The impact of individual cognitive stimulation therapy (iCST) on cognition, quality of life, caregiver health, and family relationships in dementia: A randomised controlled trial. PLoS Med, 2017; 14(3):e1002269.

24.

Al-Thaqib

, Al-Sultan

, Al-Zahrani

, et al. Brain training games enhance cognitive function in healthy subjects. Med Sci Monit Basic Res, 2018; 24:63–69.

25.

Tapia

, Puertas

, Duñabeitia

. Digital therapeutics for insomnia: Assessing the effectiveness of a computerized home-based cognitive stimulation program. J Integr Neurosci, 2023; 22(2):34.

26.

Kallio

, Öhman

, Kautiainen

, et al. Cognitive training interventions for patients with Alzheimer’s disease: A systematic review. J Alzheimers Dis, 2017; 56(4):1349–1372.

27.

Parker

, Aldabain

, Akhlaghi

, et al. Cognitive stimulation in an intensive care unit: A qualitative evaluation of barriers to and facilitators of implementation. Crit Care Nurse, 2021; 41(2):51–60.

28.

Barteit

, Lanfermann

, Bärnighausen

, et al. Augmented, mixed, and virtual reality-based head-mounted devices for medical education: Systematic review. JMIR Serious Games, 2021; 9(3):e29080.

29.

Kueider

, Parisi

, Gross

, et al. Computerized cognitive training with older adults: A systematic review. PLoS One, 2012; 7(7):e40588.

30.

Coyle

, Traynor

, Solowij

. Computerized and virtual reality cognitive training for individuals at high risk of cognitive decline: Systematic review of the literature. Am J Geriatr Psychiatry, 2015; 23(4):335–359.

31.

Kim

, Kim

, Chung

, et al. Clinical predictors of cybersickness in virtual reality (VR) among highly stressed people. Sci Rep, 2021; 11(1):12139.

32.

Rebenitsch

, Owen

. Review on cybersickness in applications and visual displays. Virt Real, 2016; 20(2):101–125.

33.

Yildirim

. Cybersickness during VR gaming undermines game enjoyment: A mediation model. Displays, 2019; 59:35–43.

34.

Lewis-Evans

. Simulation sickness and VR—What is it, and what can developers and players do to reduce it? Game Developer, 2014.

35.

Bacha

JMR

, Gomes

GCV

, de Freitas

, et al. Effects of kinect adventures games versus conventional physical therapy on postural control in elderly people: A randomized controlled trial. Games Health J, 2018; 7(1):24–36.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.13 MB

0.16 MB

Usability and Tolerability of Virtual Reality-Based Cognitive Stimulation in Healthy Elderly Volunteers—A Feasibility Clinical Trial

Abstract

Background:

Methods:

Results:

Conclusion:

Introduction

Methods

Intervention

Acceptability survey

Outcome’s measurement

Statistical analysis

Results

Discussion

Conclusion

Footnotes

Acknowledgments

Disclaimer

Data Sharing

Authors’ Contributions

Author Disclosure Statement

Funding Information

Supplementary Material

References

Supplementary Material