Feasibility of a developed cognitive training system based on virtual reality with smart mirror for expert in community older cognitive disabled persons setting

Abstract

Background

Numerous stroke survivors reintegrating into the community experience cognitive challenges that restrict their engagement, subsequently contributing to additional cognitive decline and adversely affecting their quality of life.

Objective

This study seeks to feasibility a cognitive training system based on virtual reality with a smart mirror designed for cognitive disabled persons with chronic stroke in the community setting.

Methods

Ten cognitive disabled persons with chronic stroke aged 60 years older, each with independent mobility in the community, were involved in this study. The validation process included a 30-min cognitive training session administered twice a week for eight weeks. The feasibility of cognitive function assessments employed the MoCA-K and CoSAS. Additionally, a usability test was performed at the end of the experiment using SUS and the Adapted IMI. The Wilcoxon signed rank test was then employed to compare pre- and post-cognitive function results.

Results

The feasibility of the implemented cognitive training system based on virtual reality with smart mirror revealed significant differences in the total score, delayed recall, and orientation items of the MoCA-K (p < 0.05). Additionally, a notable improvement was observed in the accuracy and response time of task performance in the CoSAS (p < 0.05). Usability test results indicated an SUS mean score of 73.5 (SD 17.25) and an Adapted IMI score of 5.63 (SD 1.55), surpassing suggested thresholds for usability tests.

Conclusions

Providing a cognitive training system tailored for the community, this approach aims to the prevention and recovery of cognitive issues in the older cognitive disabled persons with chronic stroke.

Keywords

Alzheimer's disease chronic stroke cognitive disabled person community setting feasibility virtual reality

Introduction

Stroke cognitive impairment, characterized by difficulties in concentration and memory, is a common challenge following a stroke diagnosis, significantly impacting the quality of life for survivors.¹ The treatments and outcome measures employed to address cognitive impairment after stroke exhibit considerable diversity and heterogeneity.² Clinical practices encompass interventions such as cognitive and learning training, compensatory skills training, psychobehavioral therapy, use of external aids, communication training, occupational therapy, and alternative therapies like art and music therapy.^3–5 Notably, there is a growing interest in virtual reality (VR)-based cognitive rehabilitation for stroke survivors and computer-assisted cognitive rehabilitation (CACR), integrating Information & Communications Technology (ICT) and Internet of Things (IoT) technologies.^6–9

CACR systems have gained prominence due to their capacity to offer objective and systematic treatment.¹⁰ These systems provide structured training to enhance problem-solving skills across various scenarios, offering benefits such as repetitive learning, immediate feedback, and objective evaluation. Moreover, they can be utilized by caregivers or patients themselves.^11,12 Additionally, cognitive training with VR, leveraging digital technology, enhances engagement by creating a realistic virtual environment, thereby increasing the sense of immersion and overall enjoyment of the experience.^13,14 The smart healthcare industry is rapidly expanding through the integration of smartphones, diverse wearable devices, and social network services (SNS), leveraging advanced big data technologies developed through multifaceted analysis.⁹

Amid the COVID-19 pandemic, healthcare awareness has surged, propelling heightened expectations for the digital expansion and non-face-to-face evolution of the smart healthcare industry.¹⁵ Broadly defined, smart healthcare entails intelligent services that fuse digital technologies—such as big data, artificial intelligence, the IoT, and cloud computing—into healthcare frameworks. This integration enables the real-time monitoring and management of an individual's health status, facilitating personalized care.^15,16 Particularly noteworthy are cognitive tasks employing computer programming, proven effective in enhancing overall cognition and specific domains like learning, memory, attention, speed, executive function, and practical life skills such as financial literacy and driving ability.¹⁷ Traditional pen-and-paper cognitive training is gradually yielding to computerized cognitive training embedded in digital health applications, accessible across personal computers, tablets, VR, or mobile devices.¹⁸

Despite remarkable technological strides in post-stroke cognitive rehabilitation, there remains a dearth of comprehensive data on their practical scalability. Consequently, a pressing need exists for evidence-based assessments and management strategies for cognitive impairment.² The cognitive training system based on smart mirror developed in this study caters specifically to cognitive disabled adults, incorporating training modules for orientation, memory, attention, visual perception, and high cognition. It also integrates kiosk-based activities relevant to daily life and dual-task training, creating a VR environment to stimulate cognitive functions in both community-dwelling older cognitive disabled persons. Thus, the primary objective of this study is to assess the feasibility and usability of the developed cognitive training system based on virtual reality with smart mirror for Improving efficacy of cognitive disabled person with chronic stroke, confirming its utility, acceptability, and consistency as a consumer-centric system leveraging ICT technology in the community setting.

Methods

Participants

The study protocol was approved by the Institutional Review Board of the Korea National Rehabilitation Center (no.: NRC-2023-04-030), in accordance with the Declaration of Helsinki. The design employed a cognitive training system based on virtual reality with smart mirror developed for a single group to compare cognitive function levels before and after training, with a subsequent usability test conducted after the experiment. Participants in this study were individuals who willingly volunteered for a clinical trial after responding to a recruitment notice at the National Rehabilitation Center in Seoul, Korea, provided they met the specified selection criteria. The eligibility criteria included the following: individuals aged 60 years or older, with a diagnosis of stroke and an onset period of two years or more; a Modified Barthel Index (MBI) score of 75 or more, indicating a mild level of assistance and the ability to move independently in the community; and an absence of unilateral neglect in line bisection and the capability to follow instructions for the test.

The selection of 10 participants for the usability test and intervention in this study aligns with the findings of Virzi's study, which indicated that 80–90% of usability evaluation problems can be identified with a sample size ranging between five and eight participants.¹⁹

Before participating, all participants received a detailed explanation of the study and provided written informed consent (Figure 1).

Figure 1.

Research flow diagram.

Smart mirror training system

The Smart Mirror Training System (SMTS), a cognitive training system based on virtual reality with smart mirror developed in this study, features tailored cognitive programs based on the educational level of cognitive disabled persons, allowing training on specific cognitive areas aligned with individual capabilities. Moreover, it is designed to enhance the skills essential for the social lives of cognitive disabled persons by engaging in Instrumental Activities of Daily Living (IADL) using VR. The SMTS incorporates the Mini-Mental State Examination-Korean version (MMSE-K), IADL, Korean Geriatric Depression Scale (KGDS), and Computer-based Cognitive Rehabilitation Program (CoTras) cognitive assessment systems. These systems are integrated into the equipment to classify the cognitive level of participants through pre-assessment, enabling the application of personalized training (Figure 2).

Figure 2.

Cognitive assessment analysis process.

The cognitive training methodology of the SMTS comprises content for training specific cognitive areas and VR content enabling virtual participation in daily and social life. Content targeting specific cognitive areas includes orientation, attention, memory, visual perception, and high cognition elements (Figure 3). Studies have demonstrated the effectiveness of CACR programs in improving semantic fluency in older adults with mild cognitive impairment.²⁰ Additionally, computer cognitive training programs have shown efficacy in enhancing cognitive function, alleviating depression, improving life satisfaction, and enhancing activities of daily living in older adults with mild cognitive impairment.²¹ VR amalgamates various technologies to enhance specific health-related factors, gaining prominence in healthcare and demonstrating satisfactory results in managing chronic diseases.²² Cognitive rehabilitation training using VR provides a safe and efficient means of creating complex cognitive training environments, offering advantages in cost, motivation, and the ability to provide a comprehensive problem-solving process.²³ The VR content in SMTS encompasses IADLs, involving tasks such as shopping, financial management, and kiosk utilization (Figure 3). In individuals aged 60 years and older, IADL ability correlates with years of education and cognitive function, with cognitive function variables accounting for 70% of the variance in determining IADL levels.²⁴

Figure 3.

Smart mirror training system (SMTS) contents with specific cognitive areas and virtual reality.

SMTS cognitive rehabilitation equipment is well-suited for delivering targeted cognitive rehabilitation programs, benefiting hospitalized patients as well as community-dwelling older disabled persons with chronic stroke, and traumatic brain injury (TBI) patients. SMTS can devise personalized cognitive programs through K-MMSE, IADL, KGDS, and CoTras cognitive assessments. The program is designed to provide virtual training for specific cognitive areas requiring attention and tasks challenging to engage in during daily life. Additionally, it allows for the fine-tuning of parameters such as training frequency and concentration time and facilitates the monitoring of patients’ progress and activity levels.

The purpose of this feasibility and usability test is to assess the acceptance of the developed SMTS cognitive program by community-dwelling cognitive disabled persons with chronic stroke. We aim to discern any differences in operability and motivation compared to existing CACR programs and evaluate the ease of learning and using the developed device's system through feedback on usability, acceptability, and motivation as a system that can be utilized when patients return to the community.

Experimental procedures

For feasibility, based on the initial evaluation results, the training regimen allocated 15 min to target specific cognitive areas (orientation, memory, attention, visual perception, language, high cognition) and another 15 min to simulate real-life scenarios through VR. The study spanned 16 sessions over eight weeks, with sessions occurring twice a week in accordance with participants’ schedules. Feasibility of cognitive training was carried out one day before and one day after the initiation and completion of training, respectively, while usability test was conducted on the final day of training.

The feasibility and usability test sessions took place at the National Rehabilitation Center in Seoul. The cognitive tasks were divided into two segments: a 15-min program focusing on improving specific cognitive domains (Figure 4), and a subsequent 15-min program involving problem-solving and complex cognitive tasks within daily activities using VR (Figure 4). Throughout the intervention, a dedicated therapist operated the SMTS system and guided participants through the program. The cognitive training was tailored based on pre-assessed cognitive assessment baselines, with the program customized for each participant. The therapist, adjusting the frequency, intensity, and programming elements weekly, aimed to stimulate participants’ cognitive functions.

Figure 4.

Specific cognitive areas and virtual reality training examples.

The therapist selected VR programs for specific cognitive domains and IADL training, making decisions based on the following considerations: for specific cognitive training programs, the therapist selected areas targeting the lowest cognitive items based on the MMSE-K and CoTras. In the case of VR training, the therapist utilized KGDS to identify the participants’ interests, selecting contents deemed important by the participants among the IADL elements. The therapist then applied these selections to the training. During the SMTS cognitive training, the therapist continually adapted the system to customize the time and difficulty of each activity.

Outcome measures

To assess cognitive function for confirming feasibility, standardized instruments were employed both before and after the SMTS cognitive training. The Montreal Cognitive Assessment-Korean version (MoCA-K), designed for mild cognitive impairment screening, is available in 61 languages and is utilized globally.²⁵ Notably, MoCA has demonstrated higher diagnostic accuracy for mild cognitive impairment screening compared to MMSE-K.²⁶ MoCA-K, specifically used in this study, is valuable for its inclusion of executive function assessment, addressing a limitation of MMSE-K. Scores on MoCA-K range from 0 to 30, with an adjustment for years of education. An additional point is added for participants with six or fewer years of education, and a cutoff for mild cognitive impairment is set at less than 23, indicating cognitive decline. Cognitive Screening Assessment System (CoSAS) is a computerized cognitive assessment tool designed to assess and analyze the cognitive status of older adults and individuals with cognitive impairments.²⁷ It scores the average ability of normal older and cognitive disabled adults and evaluates six domains: orientation, memory, visual perception, attention, language, and high cognition, with scores ranging from 0 to 100. CoSAS has demonstrated a high correlation with MMSE-K, with a coefficient of r = .73.²⁸

The usability and motivation of the developed SMTS were assessed by the participants through semi-structured interviews and usability questionnaires after the eight-week validation period. The evaluation employed the following scales: The System Usability Scale (SUS)²⁹ and the Adapted Intrinsic Motivation Inventory (adapted IMI).³⁰

The SUS scale utilized in this study ranges from 0 to 100%, with higher scores indicating better usability. The established threshold for SUS is 68%.³¹ The adapted IMI scale is designed as a multidimensional measure, gauging participants’ subjective experiences with a given activity. The full version comprises 45 items and seven subscales, with the scoring for each item ranging from 1 “strongly agree” to 7 “strongly disagree”.^30,32 For this study, a condensed version of the scale with 20 items was used, encompassing six subscales: interest/enjoyment, perceived competence, effort/importance, pressure/tension, perceived choice, and value/usefulness.³⁰

Data analysis

Data processing in this study used Windows SPSS 22.0 (IBM/SPSS Inc., Chicago, IL, USA). Descriptive statistics were used to summarize the general characteristics and usability evaluation of the study participants. The comparison of cognitive function scores before and after the intervention was analyzed using the Wilcoxon signed-rank test. The statistical significance level for this study was set at p < 0.05.

Results

Participant characteristics

The study commenced in August 2023 and concluded in November 2023. Of the 21 individuals who responded to the clinical trial recruitment notice, 11 participants meeting the inclusion criteria were initially selected. However, during the intervention period, one participant was excluded due to a low participation rate, resulting in a final cohort of 10 participants.

The participants’ ages ranged from a minimum of 61 to a maximum of 83 years (mean = 68.8, standard deviation (SD) 7.74), with a gender distribution of 90% male and 10% female. Regarding stroke types, 20% were hemorrhagic, and 80% were infarct cases, with 60% experiencing right hemiparesis and 40% left hemiparesis. Moreover, 40% had right-hand dominance, while 60% had left-hand dominance. The average duration since the onset of stroke was 14.2 years (SD 5.75), and the educational background consisted of 20% middle school and 80% high school. Importantly, all participants in the study were not concurrently undergoing any other form of treatment (Table 1).

Table 1.

Demographic information of participants.

	Age (y)	Gender	Stroke type	Hemiparesis	Dominant hand	Years since stroke	Final academic	Other therapies
S1	66	F	I	R	L (changed)	7	Middle school	N
S2	66	M	I	R	L (changed)	12	High school	N
S3	61	M	I	L	R	6	High school	N
S4	65	M	I	L	R	19	High school	N
S5	71	M	I	R	L (changed)	19	Middle school	N
S6	82	M	I	R	L (changed)	21	High school	N
S7	67	M	I	L	R	22	High school	N
S8	67	M	H	R	L (changed)	14	High school	N
S9	83	M	I	R	L (changed)	11	High school	N
S10	61	M	H	L	R	11	High school	N

F: female; M: male; I: infarction; H: hemorrhage; R: right; L: left; N: no.

Cognitive function evaluation results

The assessment outcomes of the SMTS cognitive training are succinctly presented in Tables 2 and 3. The MoCA-K exhibited significant differences in total score, delayed recall, and orientation (p < 0.05) in Table 2, while the CoSAS demonstrated significant differences in the accuracy of cognitive task performance and reaction time (p < 0.05) in Table 3.

Table 2.

Comparison between pre and post of MoCA-K.

	Total score	Visuospatial/executive	Naming	Attention	Language	Abstraction	Delayed recall	Orientation
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Pre	20.40 (2.07)	2.40 (1.17)	2.90 (0.32)	5.10 (0.88)	2.40 (0.84)	1.40 (0.70)	0.70 (9.45)	5.50 (0.53)
Post	23.90 (2.85)	3.30 (1.06)	3.00 (0.00)	5.20 (0.63)	2.70 (0.67)	1.70 (0.48)	1.90 (1.66)	6.00 (0.00)
p (z)	0.007* (−2.677)	0.101 (−1.638)	0.317 (−1.00)	0.792 (−.264)	0.276 (−1.089)	0.180 (−1.342)	0.031* (−2.154)	0.025* (−2.236)

MoCA-K: Montreal Cognitive Assessment-Korean version. *p < 0.05.

SD: standard deviation.

Table 3.

Comparison between pre and post CoSAS.

	Total score	Accuracy	Response time	Orientation	Memory	Attention	Visual perception	Language	High cognition
	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
Pre	76.81 (12.09)	0.78 (0.12)	18.08 (3.63)	13.64 (2.14)	8.79 (4.15)	16.97 (6.09)	5.15 (4.53)	16.97 (1.56)	15.30 (4.43)
Post	85.60 (11.36)	0.87 (0.11)	14.65 (2.97)	15.15 (0.00)	8.18 (4.53)	21.51 (5.04)	6.37 (4.38)	17.27 (1.46)	16.21 (2.86)
p (z)	0.092 (−1.686)	0.046* (−1.992)	0.009* (−2.599)	0.059 (−1.890)	0.739 (−0.333)	0.074 (−1.786)	0.340 (−0.954)	0.564 (−0.564)	0.735 (−0.339)

*p < 0.05. CoSAS: Cognitive Screening Assessment System; SD: standard deviation.

Results of SUS and adapted IMI

The findings from SUS and adapted IMI are detailed below. Past studies indicate that a SUS mean score of 68 or higher is deemed satisfactory for product usability.³³ In this study, the perceived usefulness averaged 73.5 points (SD 17.25) (Figure 4). Adapted IMI is generally considered satisfactory when it is 4 or higher,³⁴ and the adapted IMI score in this study was 5.63 (SD 1.55). Detailed adapted IMI's subitem scores were as follows: interest/enjoyment 6.19 (SD 1.06), perceived competence 5.78 (SD 0.97), effort/importance 6.3 (SD 0.73), pressure/tension 4.11 (SD 1.21), perceived choice 6.55 (SD 0.96), and value/usefulness 4.79 (SD 1.56) (Table 4 and Figures 5 and 6).

Figure 5.

Results on the SUS (score of 0–100). SUS: system usability scale.

Figure 6.

Results on the subitem of adapted intrinsic motivation inventory (adapted IMI).

Table 4.

Results of adapted intrinsic motivation inventory (adapted IMI).

Adapted Intrinsic Motivation Inventory (adapted IMI)
Interest/enjoyment	Perceived Competence	Effort/importance	Pressure/tension	Value/usefulness	Perceived choice	Total score
Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)	Mean (SD)
6.19 (1.06)	5.78 (0.97)	6.3 (0.73)	4.11 (1.21)	6.55 (0.96)	4.79 (1.56)	5.63 (1.55)

Discussion

The primary objective of this study was to validate a device employing SMTS cognitive training within the community setting context to enhance cognitive function in cognitive disabled persons with chronic stroke. Additionally, the study aimed to analyze the usability, acceptability, and consistency of the developed device. In the realm of medical devices, relying solely on electrical and mechanical safety tests and performance measurements is insufficient to guarantee safety and effectiveness. Validation is crucial to confirm the validity of the intended use.³⁵ It is also common practice in product development to employ a user-centered approach to engage users, assess their needs, and meet those needs. This approach holds significant value.³⁶ This is because users are better equipped to assess and identify issues in using the device. By sharing their experiences, they can offer valuable insights that form a basis for identifying potential risks.³⁷ Likewise, in medical device development, user involvement is a pivotal factor in determining a device's success or failure.³⁸ The usability test is indispensable to create systems that are comfortable, motivating, and conducive to long-term treatment adherence.³¹

The evolving landscape of ICTs has introduced novel models of healthcare service delivery distinct from traditional face-to-face interactions. In the era of chronic diseases, patients are increasingly utilizing self-health management devices for real-time health monitoring and personalized treatment outside medical institutions.³⁹ In recent years, services utilizing ICT have become prevalent in the healthcare sector. Specifically, as healthcare services increasingly focus on prevention and aftercare, a variety of ICT-based solutions such as remote monitoring systems, home care for older and cognitive disabled adults, and the treatment and management of chronic diseases are being offered.⁴⁰ In particular, digital therapies leveraging ICT are rapidly emerging as a focal point in the fourth industrial revolution, presenting a new paradigm for the healthcare system. This swift evolution, encompassing artificial intelligence technologies like machine learning and deep learning, is reshaping environments across various fields. Within the medical sector, it is fundamentally transforming the procedures for screening, diagnosing, and monitoring diseases, enabling more precise profiling of information for personalized treatment.⁴¹ In this study, a cognitive training system using ICT was developed to assess the effectiveness of customized cognitive training for cognitive disabled persons with chronic stroke in the community, coupled with a usability evaluation to build a consumer-centered system.

The feasibility results demonstrated notable improvements in the total score, delayed recall, and orientation on the MoCA-K, as well as in accuracy and response time on the CoSAS. Minor changes were observed in other items. Notably, the study cohort consisted of chronic stroke disabled persons living in the community for more than two years, providing evidence that customized cognitive training remains effective even in the chronic stage. In a study comparing neurological and social cognition in disabled persons with acute and chronic stroke, it was observed that thinking operation, visuomotor organization, and visual perception were linked to social cognition in the chronic stroke group. The study concluded that training aimed at improving social cognition becomes necessary after the recovery of neurological cognition.⁴² Social cognition and specific cognitive training were identified as crucial for improving efficacy of cognitive disabled persons with chronic stroke, emphasizing the need for targeted interventions in this population.

The most recent international standards for medical device quality management underscore the significance of designing for user usability. These standards mandate that usability assessments be carried out from the design stage of medical devices to consider the usability aspects (fitness for use) of users during the development phase.⁴³ In healthcare, the design of equipment and work environments can be technology-driven rather than user-centered. This approach may lead to disruptions, and participants may easily become bored, disregarding established procedures and protocols for performing tasks.⁴⁴ Not only is this inefficient, but it also creates an environment that can lead to bigger problems down the road. To create a product with high usability, usability evaluation is essential. Therefore, in this study, we conducted a usability evaluation of the developed product. As a result, the SUS average for the 10 participants was 73.5, with a median score of 70, surpassing the 68 threshold for usability evaluation, indicating “good” usability.⁴⁵ However, the distribution of individual participant scores revealed that five participants scored below 68. Common comments from this subgroup included concerns about the “use of English icons,” “small font,” and “somewhat inadequate explanations.” Adapted IMI assesses the content and level of motivation participants experience with a product, with a score of 4 indicating a neutral response, and a higher score suggesting a more positive outcome in terms of motivation.^46,47 In this study, the average score was 5.63 out of 7. Across the subscales, value/usefulness, effort/importance, and interest/enjoyment scored above 6, while pressure/tension and perceived choice were rated in the 4 s. Notably, most participants experienced pressure and tension when using SMTS, particularly in comparison to other aspects. However, there were significant differences in the perceived choice aspect of product use. Based on the results of the usability test in this study, to develop a consumer-centered product, identified issues such as the use of English icons and font size adjustment should be addressed to alleviate tension and pressure during product use and enhance user understanding and freedom of choice.

The limitations of this study include a small sample size, variability in the onset dates of participants, and an imbalance in the gender distribution of the subjects. Future research should encompass usability and clinical validation of the product across diverse populations, including not only cognitive disabled person with chronic stroke but also cognitive disabled person with other disease in the community and children with cognitive impairment. It should incorporate a larger sample size and include a control group. Additionally, research efforts should be directed toward substantiating the effectiveness of cognitive training content and developing tailored programs.

Conclusion

This study confirmed the feasibility of cognitive training using SMTS for cognitive disabled persons with chronic stroke and its usability in establishing a user-driven community-based training program. The feasibility results demonstrated enhancements in total score, delayed recall, and orientation in MoCA-K, along with improvements in accuracy and response time in CoSAS. The usability analysis indicated higher scores compared to the baseline on SUS and IMI, signifying that the system is not only useful but also safe and motivating.

In this study, participants engaged in 30 min of combined cognitive and virtual training, focusing on specific areas for 15 min each. Through a participant-centered cognitive function analysis, the study confirmed the efficacy of this approach in enhancing cognitive function. The addition of motivation and enjoyment factors, achieved by integrating problem-solving and memory tasks during IADLs in virtual training alongside specific area training, contributed to the positive usability test results.

Moreover, community-dwelling cognitive disabled persons with chronic stroke face an elevated risk of accelerated secondary cognitive decline, such as dementia, due to limitations on care and social participation resulting from COVID-19. Prevention of this decline is imperative. Given the heightened risk of accelerated secondary cognitive decline, such as dementia, in community-dwelling cognitive disabled persons with chronic stroke due to COVID-19-related restrictions, it becomes imperative to prevent and address these issues. Building a user-friendly training system for cognitive programs can play a pivotal role in improving healthcare for cognitive disabled person. By providing a cognitive training system tailored for the community, this approach aims to reduce social costs with helping expert and contribute to the prevention and recovery of cognitive issues in the cognitive disabled persons with chronic stroke.

Footnotes

Acknowledgments

The authors have no acknowledgments to report.

ORCID iDs

Bo-Ra Kang

Young-Hyeon Bae

Seong Hun Park

Hye-Yun Kang

Ethical considerations

This study was conducted after obtaining approval from the Korea National Rehabilitation Center (no.: NRC-2023-04-030) to minimize ethical issues in the research process.

Consent to participate

This study was conducted after explaining the purpose and methods of the research to participants who voluntarily expressed their intention to participate and obtaining their written informed consent.

Consent for publication

Not applicable.

Author contributions

Young-Hyeon Bae (Formal analysis; Funding acquisition; Investigation; Project administration; Resources; Supervision; Validation; Visualization; Writing – original draft; Writing – review & editing); Bo-Ra Kang (Data curation; Formal analysis; Investigation; Methodology; Validation; Visualization); Seong Hun Park (Data curation; Formal analysis; Investigation; Methodology; Resources; Software; Validation; Visualization); Hye-Yun Kang (Conceptualization; Data curation; Formal analysis; Investigation; Methodology; Resources; Software; Validation; Visualization; Writing – original draft; Writing – review & editing).

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by a grant (#SR202106002) by the Ministry of Culture, Sports and Tourism in South Korea.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability

The datasets generated during and/or analyzed during the current study are available from the corresponding author on reasonable request.

References

Qazzaz

NKAI

Ali

Ahmad

, et al. Cognitive impairment and memory dysfunction after a stroke diagnosis: a post-stroke memory assessment. Neuropsychiatr Dis Treat 2014; 10: 1677–1691.

Dawar

Kamalakannan

Chawla

, et al. A scoping review of recent advancements in intervention and outcome measures for post-stroke cognitive impairments. Ann Indian Acad Neurol 2023; 26: 32–42.

Lee

. The effect of multimodal intervention through virtual reality-based immersion program on cognitive function and brain activity in patients with mild cognitive impairment. J Korean Soc Phys Med 2023; 18: 87–97.

Mun

Chun

. The effect of group art therapy on the emotions and cognition of the elderly with mild cognitive impairment using book art. IKAT 2021; 28: 31–54.

Kwon

Lee

. The effect of nonpharmacologic interventions on behavioral and psychological symptoms of dementia: a meta-analysis. J Korea Contents Assoc 2017; 17: 540–550.

Lohse

Hilderman

CGE

Cheung

, et al. Virtual reality therapy for adults post-stroke: a systematic review and meta-analysis exploring virtual environments and commercial games in therapy. PLoS One 2014; 9: e93318.

Subramanian

Lourenco

Chilingaryan

, et al. Arm motor recovery using a virtual reality intervention in chronic stroke: randomized control trial. Neural Repair 2013; 27: 13–23.

Nijland

Van Wegen

Verbunt

, et al. A comparison of two validated tests for upper limb function after stroke: the Wolf motor function test and the action research arm test. J Rehabil Med 2010; 42: 694–696.

Park

. Present and future of IoT based smart healthcare industry. J Korean Inst Electronics Eng 2017; 44: 24–28.

10.

Choi

Ahn

. Virtual reality cognitive rehabilitation system using memory recall contents. In: Fall conference of the korean society of social welfare, Seoul, South Korea, October 27–28, 2017, pp.1533–1544.

11.

Han

Noh

, et al. Effects of Korean computer-based cognitive rehabilitation program (CoTras) on frontal-executive functions in patients with traumatic brain injury. JKAIS 2015; 16: 3344–3352.

12.

Kim

. Development of immersive cognitive rehabilitation VR contents based on storytelling. JDAEM 2023; 10: 163–173.

13.

Jang

Kim

. Study on the influence of VR characteristics on user satisfaction and intention to use continuously-focusing on VR presence, user characteristics, and VR sickness. J Korea Contents Assoc 2018; 18: 420–431.

14.

Bowman

McMahan

. Virtual reality: how much immersion is enough? IEEE 2007; 40: 36–43.

15.

Hand

Kim

. Smart healthcare. KISTEP 2020; 2020: 1–34.

16.

Ministry of Trade. Industry and energy, strategies for promoting the smart healthcare industry . 2015.

17.

Bodner

Goldberg

Devanand

, et al. Advancing computerized cognitive training for MCI and Alzheimer's disease in a pandemic and post-pandemic world. Front Psychiatry 2020; 11: e557571.

18.

Geng

Yang

, et al. Effectiveness of computerized cognitive training in delaying cognitive function decline in people with mild cognitive impairment: systematic review and meta-analysis. J Med Internet Res 2022; 24: e38624.

19.

Virzi

. Refining the test phase of usability evaluation: how many subjects is enough? Hum Factors 1992; 34: 457–468.

20.

Byeon

. The effects of computer-based cognitive rehabilitation program (CoTras) on the verbal fluency of Korean patients with mild dementia. Indian J Public Health Res Dev 2018; 9: 1033–1040.

21.

Kim

Choi

. Effects of a computerized cognitive training on cognitive, depression, life satisfaction and activity of daily living in older adults with mild dementia. JKAIS 2019; 20: 311–319.

22.

Zhong

Chen

Feng

, et al. Effects of virtual reality cognitive training in individuals with mild cognitive impairment: a systematic review and meta-analysis. Geriatr Psychiatry 2021; 366: 1829–1847.

23.

Bauer

ACM

Andringa

. The potential of immersive virtual reality for cognitive training in elderly. Gerontology 2020; 66: 614–623.

24.

LaBuda

Lichtenberg

. The role of cognition, depression, and awareness of deficit in predicting geriatric rehabilitation patients’ IADL performance. Clin Neuropsychol 1999; 13: 258–267.

25.

Kwak

Kim

. The normative study of the Montreal cognitive assessment-Korea (MoCA-K) as instrument for screening of mild cognitive impairment (MCI). KSIM 2021; 9: 37–45.

26.

Freitas

. Montreal Cognitive assessment: validation study for mild cognitive impairment and Alzheimer disease. Alzheimer Dis Assoc Disord 2013; 27: 37–43.

27.

Jung

Ham

. Usefulness of COSAS test for mild Alzheimer type dementia and mild cognitive impairment. JKSAT 2020; 12: 41–51.

28.

Kim

, et al. A study on the standardization of computer-based on the computer cognitive senior assessment system. JKCBOT 2013; 21: 87–102.

29.

Brooke

. SUS-A quick and dirty usability scale. In: Jordan

Thomas

Weerdmeester

, et al. (eds) Usability evaluation in industry. London: Taylor & Francis, 1996, pp.189–194.

30.

Self-Determination Theory. An approach to human motivation and personality-Questionnaires, Intrinsic Motivation Inventory. https://selfdeterminationtheory.org/intrinsic-motivation-inventory/ (accessed 4 April 2023).

31.

Guillen-Climent

Garzo

Munoz-Alcaraz

, et al. A usability study in patients with stoke using MERLIN, a robotic system based on serious games for upper limb rehabilitation in the home setting. J Neuroeng Rehabil 2021; 18: 41.

32.

Colombo

Pisano

Mazzone

, et al. Design strategies to improve patient motivation during robot-aided rehabilitation. J Neuroeng Rehanil 2007; 4: 12.

33.

Sauro

. Measuring usability with the system usability scale (SUS). https://measuringu.com/sus/ (accessed 4 April 2023).

34.

CSDT. Intrinsic motivation Inventory (IMI). https://selfdeterminationtheory.org/ (accessed 4 April 2023).

35.

Han

Kim

, et al. An empirical study on clinical trial support for digital medical devices. JSTS 2020; 2052–2055.

36.

Brockhoff

. Customers’ perspectives of involvement in new product development. Int J Technol Manag 2003; 26: 464–481.

37.

Gosbee

. Human factors engineering and patient safety. Qual Saf Health Care 2002; 11: 352–354.

38.

Shah

SGS

Robinson

. User involvement in medical device technology development and assessment: structured literature review. Int J Health Care Qual Assur Inc Leadersh Health Serv 2006; 19: 500–515.

39.

Yoon

Song

Choi

, et al. Challenges for activating the convergence of ICT and healthcare service. KIHASA 2016; 51: 1–205.

40.

Korea Health Industry Development Institute. Final report on the establishment and demonstration project of ICT-based public healthcare service support system 2016. KHIDI 2017.

41.

Liu

Ting

DSJ

, et al. Digital technology, tele-medicine and artificial intelligence in ophthalmology: a global perspective. Prog Retin Eye Res 2021; 82: 100900.

42.

Park

. Relations of neurological and social cognitions in patients with acute phase and chronic phase before returning to the community. JKAIS 2017; 18: 549–556.

43.

Kim

Kwon

. Usability test of medical devices by the adaption of the physical load. In: Korean society of ergonomics proceeding, 2019 Korean society of ergonomic spring conference, 2019, pp.160–163.

44.

Jung

. A survey of usability for the medical devices and the systems. In: Proceedings of the conference materials, the ergonomics society of Korea 2009 spring conference, 2009, pp.179–182.

45.

Nasr

Leon

Mountain

, et al. The experience of living with stroke and using technology: opportunities to engage and co-design with end users. Disabil Rehabil Assist Technol 2016; 11: 653–660.

46.

McAuley

Duncan

Tammen

. Psychometric properties of the intrinsic motivation inventory in a competitive sport setting: a confirmatory factor analysis. Res Q Exerc Sport 1989; 60: 48–58.

47.

Nijenhuis

Prange

Amirabdollahian

, et al. Feasibility study into self-administered training at home using an arm and hand device with motivational gaming environment in chronic stroke. J Neuroeng Rehanil 2015; 12: 89.