Abstract
Objective:
To evaluate the effectiveness of augmented reality (AR) game based on n-back training paradigm as a training tool for working memory (WM) of Chinese healthy older adults.
Materials and Methods:
One hundred eighteen older adults self-assessed as healthy were included in this study. Individuals were randomly divided into an intervention group (n = 57) and a control group (n = 61). Interventions, consisting of a 30-minute AR game-based training and a 30-minute health science program, were administered three times per week for 4 weeks, whereas the control group was required to view a 60-minute health science program three times per week for 4 weeks. Tests, Digit Span, Corsi Block-Tapping Task (CBT), and Stroop Color and Word Test (SCWT), were conducted for all participants before and after the experiment, and the game accuracy rate of the intervention group before and after intervention was recorded.
Results:
There was a statistically significant difference in terms of both CBT indicators, CBT forward span (z = −2.835, P = 0.005) and CBT backward span (z = 3.285, P = 0.001), and the SCWT indicator of Stroop Words Test (SW) (z = −1.894, P = 0.048) in the two groups. The intervention group showed significant improvements in the game accuracy of both medium level (z = −3.535, P < 0.05) and of high level (z = −3.953, P < 0.05). In addition, differences were observed in subgroup analysis in the accuracy of medium level (H = 6.218, P < 0.05) and high level (H = 8.002, P < 0.05) among older people with different levels of education.
Conclusion:
AR game based on n-back training paradigm could improve WM of Chinese older adults, showing potential for wider promotion and adoption.
Introduction
Aging is associated with a decline in cognitive functions, including memory deterioration, decreased responsiveness and attention, and impaired executive function, according to studies.1,2 Working memory (WM), which refers to the brain system for transient storage and operation of complex cognitive tasks such as language comprehension, learning, and reasoning, 3 is one of the primary intrinsic causes of these cognitive deficits. It can be divided into (i) the central executive system, (ii) visual-spatial WM for processing visual images, and (iii) linguistic WM for processing phonological loops.
Memory training not only aids in the enhancement of mild memory impairment, but also improves cognitive functions. 4 As a type of memory training, n-back paradigm has been shown to effectively enhance WM.5,6 The participant must respond when the current stimulus matches the stimulus at the nth position in the sequence (e.g., 1, 2, or 3). The participants' task performance can be enhanced after 4 weeks of training. 7 The n-back task is commonly used in research on aging, 8 and language memory and processing play a crucial role in n-back tasks. 9 Although n-back training has a positive effect on WM, in traditional practice it must still be performed on a computer, and feedback has indicated that the procedure is relatively simple and monotonous. 10
Integrating augmented reality (AR) into games stands out as an effective way to enhance memory as it improves spatial memory through spatial semiotics. 11 In addition to being training tools, AR-based mobile applications can provide mental state examinations for older adults and act as a treatment assistant tool, which could increase their motivation to complete therapeutic tasks given by occupational therapists. 12 Older adults could make day-to-day decisions more independently through interaction and recall of events, 13 and achieve therapeutic goals while having fun. Studies have shown that AR gaming combined with the WM training paradigm could increase physical activities that prevent cognitive decline in older adults leveraging AR technology that supports both indoor and outdoor activities. 14 In terms of visual presentation, the AR-based n-back test demonstrates stronger potential as it is more engaging and has better effectiveness in WM training and migration improvements compared with the traditional 2D n-back tests. 15
Still, few studies have provided effectiveness validation of the game products that combine AR technology with n-back paradigm for memory training.
Therefore, the project team designed an AR game product based on the n-back paradigm for WM training of older adults and conducted a randomized controlled trial to determine the AR game product's efficacy in enhancing the WM of older adults.
This study seeks to validate the efficacy of memory training AR games for the older adults, as well as examine the opinions of older adults of various ages and backgrounds regarding memory training games.
Materials and Methods
Study design
This study is a randomized controlled trial. Informed consent in written form was provided for older adults capable of providing informed consent to participate, to authorize participation in the experiment. The study was conducted in accordance with the current version of the Declaration of Helsinki and was approved by the Human Research Ethics Committee of Shanghai Jiao Tong University IRB HRP AD01V01 (20200723).
Sample size calculation
During the experimental design phase, the sample size calculation was performed using PASS. Group sample sizes of 55 and 55 achieve 90.118% power to reject the null hypothesis of equal means when the population mean difference is μ1 − μ2 = 4.0 − 5.0 = −1.0 with a standard deviation (SD) for both groups of 1.6 and with a significance level (α) of 0.050 using a two-sided, two-sample, equal-variance t-test.
Randomization and blinding
The trial utilized a zone group randomization procedure with a zone group size of four. An expert in statistics utilized the SAS software analysis system to generate random numbers for a computer simulation. Participants were assigned randomly to groups (intervention and control). The researchers administered the appropriate interventions in the order of the subjects' group enrollment. Number circumventing was not permitted.
Participants
Participants of the research were older adults recruited via the internet and the community from specific Shanghai communities and nursing homes.
Inclusion criteria
Healthy individual aging between 65 and 89 years old.
Normal vision and hearing, able to clearly perceive and differentiate displayed information.
Voluntarily participate in this test and sign the informed consent form.
Exclusion criteria
No self-consciousness or behavioral capabilities, unable to perform normal activities in daily life, diagnosed with diseases that may affect the experiment (e.g., bedridden caused by serious illness, behavioral disorder, inability to take care of themselves), or with score of Activities of Daily Living scale ≥16. 16
With a score of Mental State Examination Scale ≤24 17 (with regard to education level, score of illiteracy is >17, primary school >20, junior high school and above >24), suffering from mild cognitive impairment, cognitive disorder, or dementia manifestations, with a score of simplified Geriatric Depression Scale-15 > 8, 18 with depression tendency, unclear consciousness, or incapability to communicate properly.
Have unhealthy habits such as alcoholism, heavy smoking, or staying up late.
Concurrently performing other similar items.
Exit criteria
Subjectively unwilling to continue the experiment.
Experiencing discomfort during the experiment.
Unable to continue the experiment due to other reasons (e.g., illness, death, relocation).
Procedure
Intervention group
According to the Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE) trial by Rebok et al., 19 the intervention was designed to be a 30-minute AR game-based training and a 30-minute health science program conducted three times a week for 4 weeks, and the experimental time was set from 9:00 am to 18:00 pm. Before the official beginning of the intervention experiment, the subjects were instructed to complete three sets of WM indicator evaluation tests to document their WM status. At the time of their initial introduction to the game, individuals were given ∼10 minutes to comprehend its controls. Each of the 3 levels of difficulty contained 20 questions, and the participant was instructed to play game in ascending order of difficulty. During the experiment, the participant was not disturbed in any way, and when he/she encountered difficulties or felt ill and requested assistance, only minimal intervention was provided.
At the conclusion of the 4-week experiment, the participant retook the evaluation examinations. Each participant's game accuracy before and after the intervention was recorded, and each participant was provided with daily necessities.
Control group
The control group also completed three sets of WM indicator evaluation tests before the formal start of the experiment to record their WM status. Participants were asked to watch a 60-minute health science program three times a week for 4 weeks. At the end of the experiment, the participant completed the evaluation tests again. Also, daily necessities valued at $10 were provided to each participant.
AR memory training game design
In the Chinese Pinyin system, there are initials, finals, and tones. The primary goal of the design is to combine the n-back training paradigm with Chinese Pinyin and to conduct WM training in the form of AR projection and AR recognition. By actively observing and memorizing the pictures, Chinese phrases, and phonetic transcriptions that jump in a specific order on the screen, the users' task is to continuously recall the nth picture and phonetic transcription before the current stimulus, select the correct vowel, rhyme, and tone cards, combine them into a phonetic form, and place them in the designated area for AR recognition. Through the form of “card + picture + text Pinyin,” the game stimulates the ability to “see-think-find-read-spell” as well as the memory of older adults.
The difficulty of the current version of the game is set to three levels of easy, medium, and hard, which correspond to the 0-back, 1-back, and 2-back in the n-back training paradigm, respectively. The training lasts for 30 minutes with 10 minutes for each level.
Easy level (0-back): In this mode, the user does not have to memorize anything. Users just need to spell out the Pinyin of the displayed Chinese character and put it in the right place for AR recognition. Once recognized successfully, the next character will be demonstrated (Fig. 1a).
Medium level (1-back): The interface display is the same as 0-back. A picture will be displayed on the screen 3 seconds after the first picture is shown. The user needs to recall the Pinyin of the previous picture and combine cards of phonetic symbols for recognition. Once recognized successfully, the next picture will be demonstrated (Fig. 1b).
Hard level (2-back): The interface display is the same as 0-back and 1-back. This is the most challenging level with the highest difficulty in the current game. The screen will show a picture and display the next picture every 3 seconds for two rounds. Participants need to actively recall the Pinyin of the very first picture and finish the task successfully so as to start the next round (Fig. 1c).

Interaction flow for easy
The WM training game primarily uses Unity 2018.2.2 and Visual Studio 2019 as development tools, and the main interface visualization, interactive transfer, and function realization of the product are mainly written in C# language, with Vuforia 9 calling external USB camera to realize AR recognition (Fig. 2).

AR memory training game demonstration. AR, augmented reality. Color images are available online.
Evaluation metrics
Digit Span (DS). 20 When testing, DS was measured for both forward span (DS-FS) and reverse (backward) span (DS-BS) of digit sequences. Digit sequences were presented starting with two-digit lengths, and two trials were presented at each increasing list length. Tests stopped when participants failed to accurately report either trial of one sequence length or when the maximum list length (nine digits forward and eight digits backward) was reached. The reported list across the DS-FS and DS-BS was recorded with correct numbers. 21
Corsi Block-Tapping Task (CBT). 22 Participants were shown a board with nine cubes arranged in an asymmetric pattern. In the FS, a series of blocks were tapped by the examiner at a rate of one block per second and participants were asked to repeat the sequence in the same order immediately after presentation. The test stopped when the participant failed on two trials of the same span length. The longest string of blocks that the participant correctly tapped on was used as a measure of memory span, with the list reported as CBT forward span (CBT-FS) and CBT backward span (CBT-BS). 23
The Stroop Color and Word Test (SCWT). Participants were instructed to read three tables as fast as possible. In the first table, participants were required to read names of colors printed in black ink (Stroop Words Test [SW]). In the second table, participants were asked to read names of colors printed in ink of corresponding color (Stroop Color [SC]). In the third table named color-word condition, the color-words were printed in an inconsistent color ink, and participants were required to name the color of the ink (SCW). 24 Brugnolo A evaluated and provided standard data for all SCWT conditions, 25 with items arranged in a matrix of 10 × 10. The number of accurate responses in constant time was recorded.
Statistical analysis
Statistical analysis of data was conducted using SPSS 26. Mean ± SD or intermediate values with ranges were used to describe continuous features. To compare the DS, CBT, and SCWT scores before and after game intervention, a paired t-test or nonparametric paired t-test was used. The significance value was accepted as P < 0.05.
Results
One hundred fifty older individuals self-assessed as healthy were recruited. After excluding substandard individuals, a total of 118 participants were finally included in the study (Supplementary Fig. S1). In terms of education, 21.19% of the total sampling were of primary schooling and below, 55.08% of secondary and high school education background (including technical secondary schools), and 22.03% of higher education background. The 118 participants were eventually included in the experiment, with a mean age of 74.91 ± 7.25 in the intervention group (n = 57) and 73.86 ± 6.01 in the control group (n = 61). In each group, older adults were divided into young-elderly (aged 65–75) and old-elderly (aged above 75). Statistical analysis shows that there were no significant differences in all indicators in the two groups (Table 1). Information of the drop outs in two groups and chi-square test in the basic information are shown in Supplementary Tables S3 and S4. Non-parametric paired t-test was used according to normal-test analysis (Supplementary Table S1).
Baseline Data of the Intervention Group and the Control Group
BS, backward span; CBT, Corsi Block-Tapping Task; DS, Digit Span; FS, forward span; SC, Stroop Color; SCW, Stroop Color and Word; SW, Stroop Words Test.
Between-group comparisons showed significant differences in CBT-FS (z = −2.835, P = 0.005), CBT-BS (z = −3.285, P = 0.001), and SW (z = −1.894, P = 0.048) between the experimental group and the control group after the intervention (Table 2).
Variation Between Experimental and Control Groups
Difference in scores between the experimental and control groups after the intervention, P < 0.05.
Difference in scores between the experimental group before and after the intervention, P < 0.05.
Difference in scores between the control group before and after the intervention, P > 0.05.
Within-group comparisons revealed significant differences in the experimental group's CBT-FS, CBT-BS, SW, medium-level game accuracy rate, and high-level game accuracy rate (Table 3). Before and after the intervention, the control group did not differ significantly (Table 4).
Comparison of the Intervention Group's Pre- and Postintervention Working Memory Test Results
Results in bold represent significant differences.
Comparison of the Control Group's Pre- and Postintervention Working Memory Test Results
Differences were shown in the medium-level game accuracy rate (H = 6.945, P = 0.031) and high-level game accuracy rate (H = 6.939, P = 0.031) of older adults with different levels of education (Table 5 and Supplementary Table S2). After comparison of group results, it was indicated that older adults with a college degree or above received higher scores than those of lower educational background in both medium-level game training (P = 0.026) and high-level game training (P = 0.027).
Comparison of the Intervention Group's Pre- and Postintervention Working Memory Test Results (with Different Education)
Results in bold represent significant differences.
Discussion
A randomized controlled trial was conducted to assess the effectiveness of the product and study the differences in training results among older adults of different ages and educational backgrounds in this study. Results showed that the FS and BS scores of CBT, the SW score of SCWT, and the accuracy of medium- and high-level game training in the WM tests were significantly improved after a month-long experiment.
For the visual stimuli provided by memory training AR games, reliable group-to-group differences were observed with significant enhancement after training in both the CBT-FS and CBT-BS variables. In addition, intragroup comparisons in the experimental group demonstrated a significant increase in FS and BS scores after the intervention compared with preintervention. It was driven predominantly by improvements in visual WM, 26 which were more pronounced in older adults than in younger adults. 27 and in line with previous research. There was a significant correlation between n-back tasks and complex span tasks, and the study demonstrated that the subject's diverted visual attention and simple visual memory expanded following the experiment. 28 The interaction card used in product design was based on visual interaction supplemented by voice interaction, and because elder adults were more familiar with the pronunciation of words, they subconsciously paid less attention to voice interaction. 29
In conclusion, the task stimulates WM by memorizing visual sequences, and training results in enhanced episodic memory and visuospatial ability. 30
SCWT is frequently used to assess cognitive functions such as attention and processing speed.31,32 The study revealed that the SW was substantially different between two groups, while no significant difference was observed in SC and SCW. Also, the within-group comparison of the experimental group exhibited a significant increase in SW scores after the intervention. SW embodies the processing speed in WM, 33 indicating that the AR game-based training could effectively improve the memory processing speed of older adults. The in-game symbol searching behavior increased participant's processing speed, as the participant was instructed to respond quickly to visual stimuli by identifying symbols and matching them to other symbols. 34 In contrast, SC and SCW tasks with color interference are more complex. Higher error rates in high-load tasks suggest that high-load tasks are more challenging than low-level tasks, and further cognitive control cannot be performed after a conflict. 35
Also noteworthy is the n-back game tally. The game scores showed significant improvement on game accuracy before and after the intervention in medium mode (n = 1) and hard mode (n = 2). It is believed that improved attention span is one of the training benefits of n-back tasks. 36 However, this finding conflicts with previous studies, in which Kato et al. showed that older adults performed worse on 2-back tasks and were more susceptible to interference. 37 The relationship between the complexity of Chinese characters and phonetics in game tasks and the effect of intervention is not yet known, and research on this could be carried out in the future.
Age was associated with decreased cognitive performance, 38 but this study did not observe reliable group-to-group differences between sexes and age groups, which may be due to the nonlinear nature of all age-related neural activation increases. 39 Several studies have demonstrated that memory training games can substantially improve the WM of older adults with varying levels of education, that is, education is positively correlated with WM. 40 The brain regions implicated in WM networks become more active with age in highly educated individuals. 41 Our research reaffirmed this argument, in which older adults with a college degree and above scored higher than those of middle or high school education both in medium mode and hard mode.
To summarize, AR-based game could create a relaxing training environment for older adults 42 effectively reduce cognitive load to improve the sense of space.
Contribution and limitations
The contribution of the study is that through a randomized controlled trial, the AR game product based on n-back training paradigm was proven effective for the memory training of Chinese older adults. Based on the results of this experiment, some community caregivers in Shanghai are currently using our product as daily companionship for the older adults, and we will continue to iteratively improve our game content and format in the future.
However, the study still has some limitations. First, the sample size was relatively small, some participants dropped out during the experiment, and the participants were mainly from Shanghai so that regional differences failed to be analyzed. In future, larger sample size and multicenter studies are needed to explore the topic. Second, since the game is designed with Chinese phrases and phonetic transcriptions, the study is not generalizable to non-Chinese populations.
Conclusion
Results of the study have shown that AR game products based on n-back memory training paradigm could effectively enhance the WM of older adults.
Footnotes
Authors' Contributions
D.Z. and J.H. conceived of the presented idea. B.Z. and J.H. collected the data. D.Z. performed the data analysis and statistics. All the authors contributed to the interpretation of the results. All the authors drafted the article. All the authors discussed the results and contributed to the final article.
Author Disclosure Statement
No competing financial interests exist.
Funding Information
This work was supported by funding XYB-DS-202202, Research on the function and mechanism of participatory design in community social innovation: the perspective of social network.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
