A usability evaluation framework for a mobile application in supporting home-based rehabilitation for stroke patients: A qualitative study

Abstract

Background

Digital health tools can enhance stroke patient care by promoting home-based self-management for clinicians and patients. However, these tools often face poor market adoption and low adherence due to insufficient analysis and lack of context for end-users.

Objective

To enable stroke patients to manage their health independently after discharge and receive care at home, the purpose of this study was to determine the usability, satisfaction, and performance levels of a mobile app designed for the clinical monitoring of stroke patients. The feedback obtained serves as a reference for design improvements, with the overall aim of assisting and encouraging patients to continue long-term care and rehabilitation plans at home, thereby enhancing the effectiveness and quality of healthcare.

Methods

This study employed a mixed-methods approach in three phases: (1) design of mobile app mockups, (2) execution of a usability test for both healthcare professionals and patients, assessing operational skills and acceptance, and (3) evaluation of satisfaction and performance levels regarding the usability of the app. Participants were selected through purposive sampling to ensure a representative sample and then divided into two groups: healthcare professionals (n = 10) and patients (n = 10). Each participant was provided with a tablet containing the mockup designs, and usability tests were conducted through interviews. Audio recordings of participants were transcribed anonymously and analyzed using NVivo12.0 software, focusing on the application interface and usability test. Task difficulty was assessed from (1) “very easy” to (5) “very difficult,” with task non-completion considered a critical mistake. Satisfaction and performance with the usability testing were measured using a 5-point Likert scale, from (1) “totally disagree” to (5) “totally agree.”

Results

Overall, both groups of participants found the app easy to use, despite some differences in their perceptions of app difficulty, which could have been attributed to the variations in the interface designs for each group. On average, 70% of healthcare professionals described most tasks as “very easy,” compared to only 40% of patients. No participants made critical mistakes or withdrew from the experiment during the trial. In the satisfaction and performance survey, both groups expressed high satisfaction with the interface design and the usability of the app. On average, the healthcare professionals group completed all usability tasks within 20 min, whereas patients required around 40 min to complete all tasks.

Conclusions

Satisfaction and performance surveys and qualitative data analysis revealed that most participants described the interface design of the app as intuitive and easy to use. Although some functions presented challenges, there was still a high level of satisfaction among both groups. These outcomes from usability evaluation contribute to a deeper understanding of the usage patterns of mobile healthcare applications and help to identify areas for improvement in the current design, thereby enhancing the potential for the use of the app in home-based long-term care and rehabilitation.

Keywords

Mobile application interface stroke patients home therapy usability

Introduction

The aftermath of a stroke can cause significant damage to limbs, impacting patients’ ability to carry out the activities of daily life.¹ Currently, strokes are the third-leading cause of death worldwide and the second-leading cause of disability,^2,3 and the incidence of strokes is increasing as the population ages. While advancements in medical technology often prevent fatalities, stroke remains a leading cause of disability in Taiwan, contributing significantly to the rising number of disabled individuals in the country.² At-home care for disabled stroke patients not only affects the quality of life of the patients but also of their caregivers.⁴

Approximately 80% of stroke patients experience impairments to hand movement, with fine motor skills often proving the most challenging to rehabilitate.⁵ Contributing factors include pain during rehabilitation, the monotony of therapy sessions, fatigue from frequent hospital visits, and the lengthy rehabilitation process.⁶ These challenges may lead patients to miss the optimal rehabilitation window, resulting in suboptimal recovery outcomes.⁷ Clinical evidence suggests that appropriate training is still the gold standard for regaining lost motor function.^5,8 To prevent patients from missing the critical rehabilitation period, digital rehabilitation is emerging as a promising field and is expected to become a future trend in medicine. After the pandemic, the clinical community has become more confident that digital tools can effectively mitigate the treatment limitations imposed by hospital closures. Additionally, for discharged patients, digital tools will create a more comprehensive healthcare system for post-discharge care. This is particularly important for patients with chronic diseases who require long-term monitoring to prevent complications such as a second stroke. Remote monitoring not only enhances the effectiveness of monitoring but also accumulates rehabilitation data, which can assist in making optimal medical decisions.

With the significant advancement of digital tools and technological capabilities, the widespread adoption of 5G networks, and the evolution of AI technology, smart healthcare is poised to become the best approach to the diagnosis, treatment, and evaluation of medical conditions.⁹ Digital health tools (e-Health) can potentially aid in the treatment of patients with chronic diseases by facilitating the development of self-management and self-care skills among physicians, patients, and caregivers.^10,11 Development of these skills will likely enhance their knowledge, engagement, and autonomy.¹²E-Health can overcome many challenges associated with the rehabilitation process.^13,14 For instance, e-Health can increase the diversity of rehabilitation therapies and introduce gamified elements to make the rehabilitation process more enjoyable, allowing patients to overcome difficult treatments with greater ease.¹⁵ Additionally, digital tools can assist patients residing in remote areas or with mobility issues by enabling diagnosis, treatment, and health monitoring at home.^16,17 More importantly, digital tools can be tailored to address individual patient differences by offering customized rehabilitation prescriptions.^18,19 These strategies provide healthcare institutions with a safe, reliable, and cost-effective method for follow-up care.^20–22

While there are many e-Health tools available on the market, these applications are usually marketed without evaluate and without providing any context to end-users, which frequently leads to low usage rates and limited therapeutic effectiveness.^23,24 Usability is thus an important reference criterion in the evaluation of medical devices, especially as many accompanying apps are rigorously categorized as one of the critical factors determining whether the device passes certification review. The provision of digital tools for medical use should consider the perspectives of different stakeholders in hardware design through a user-centered methodology.^25–27 Considering users’ needs is necessary for the functions and usability of mobile health apps to meet the expectations of end-users.^28,29 Additionally, the usability of these tools needs to be evaluated before implementation.³⁰ Performing usability testing provides insight into how these mobile apps are used in specific environments.^31–33 As Weichbroth (2020) pointed out, since the advent of the first mobile devices, the usability of built-in applications has become a key concern for both software and hardware manufacturers. With the academic community joining the discussion, leveraging their experience with desktop applications, usability theories and empirical measurements have progressively extended to the domain of mobile applications. However, methodological inconsistencies in existing literature highlight the need to revisit usability definitions, attributes, and evaluation methods. This observation suggests that, despite the numerous definitions of usability, certain aspects remain underexplored. Methodologically, usability evaluations often rely heavily on observational studies, while techniques such as eye-tracking, thinking aloud, and interviews are infrequently employed and are mainly used as supplementary methods for collecting additional data.

These findings underscore the significant potential for further exploration in usability evaluations, particularly in identifying the most appropriate methods for assessing the usability of specific types of applications. Furthermore, usability evaluations are often conflated with user experience assessments. Importantly, user experience surveys go beyond evaluating application design quality to include user beliefs, emotions, and preferences regarding the design. Therefore, it is crucial to select research tools that align with the objectives and requirements of the study while addressing the usability of mobile applications. This emphasizes the importance of establishing a theoretical and practical consensus among all stakeholders in the field.

As highlighted in the study by Mendiola et al. (2015), patients tend to value apps that save time compared to current methods and recognize an app as valuable when it is simple and intuitive to use, provides specific instructions for better managing a condition, and shares data with designated individuals. Furthermore, while tracking is a core function of most health apps, improper execution of this feature may negatively impact the user experience. Therefore, in the design of the app for this study, in addition to considering usability evaluation factors, emphasis was placed on the psychological aspects, emotions, and preferences considered in the user experience, with the aim of enhancing patient acceptance and usability of the app, ultimately providing tangible benefits for health management.

In summary, assessing the usability of a mobile app for the clinical monitoring of stroke patients undergoing hand rehabilitation at home is an important design step.^34,35 Thus, this study recruited patients and healthcare professionals with the purpose of determining the usability and satisfaction and performance of a mobile app for the clinical follow-up of patients prescribed at-home rehabilitation. Therefore, the concept proposed in this study introduces an innovative usability framework designed to support home-based rehabilitation for stroke patients. This framework not only digitizes the rehabilitation treatments that patients are required to undergo but also dynamically adapts to the changes in the rehabilitation environment. It takes into account various factors such as workforce availability, the tools used in rehabilitation, and patient capabilities. In contrast to existing commercial products, which struggle to be effectively implemented in home care, the mobile application developed in this study is specifically designed to address these challenges by providing a comprehensive, user-centric solution that supports long-term rehabilitation at home. This framework is intended to fill the gap left by traditional products by offering a flexible and personalized approach tailored to individual patient needs and circumstances.

Methods and materials

This study aims to evaluate the responses of key stakeholders to the design of the system developed by the researchers. The evaluation includes testing aspects such as system usability, acceptance, and user experience, to assess the potential impact of the design on patient health improvement and its feasibility for future home rehabilitation.

This study employed a mixed-methods approach comprising three phases. First, mobile app mockups were designed. Second, usability testing was conducted with healthcare professionals and patients to assess operational skills and acceptance. Finally, satisfaction and performance levels with the app's usability were evaluated. Participants used a tablet containing the mockup designs, and usability tests were facilitated through interviews. Audio recordings from the interviews were anonymized, transcribed, and analyzed using NVivo 12.0 software, focusing on the app interface, operational skills, and usability. Thematic analysis was conducted, with open coding identifying key themes and axial coding exploring their interrelationships. Two researchers independently coded a subset of the data to ensure reliability, resolving any discrepancies through discussion. The finalized coding scheme highlighted themes related to operational skills, app usability, and acceptance.

This study employs the System Usability Scale (SUS) questionnaire, which is based on the framework originally developed by John Brooke in 1986 at DEC, UK. To ensure the validity and reliability of the experiment, a pilot test involving three participants was conducted prior to the formal implementation. The feedback from this pilot test was used to refine the experimental design. The study aims to evaluate the usability of an application developed by the researchers for use by patients and healthcare providers, focusing on three key dimensions of usability.³⁶ The first aspect is effectiveness, which evaluates users’ ability to achieve their objectives using the technology. The second aspect is efficiency, which examines the amount of effort and resources expended by users in reaching those objectives. The third aspect is satisfaction and performance, which assesses the level of satisfaction and performance with the user's experience.

Task difficulty was rated on a 5-point scale from (1) “very easy” to (5) “very difficult,” with task non-completion considered a critical error reflecting operational challenges. Satisfaction with the usability testing was measured using a 5-point Likert scale ranging from (1) “totally disagree” to (5) “totally agree. This study formed part of a design project aimed at evaluating the performance and effectiveness of a mobile app in clinical settings by examining potential issues in real-world scenarios, thereby optimizing its design. This specific study was a controlled, randomized, and non-pharmacological clinical trial to assess the efficacy of a mobile app in the clinical monitoring of patients undergoing home rehabilitation.³⁷ The methodology focused on end-users. The study was registered as a clinical trial (no: 202301197A3). The research team recruited participants via telephone, explaining the purpose and requirements of the survey. This study, conducted at a rehabilitation medical institution in Taoyuan, Taiwan, employed purposive sampling to obtain a representative sample. Participants were divided into two groups: healthcare professionals (n = 10) and patients (n = 10). The primary goal of the research was to identify the expectations and needs of end-users regarding the developed app.³⁸ On average, healthcare professionals completed all usability tasks in 20 min, while patients required approximately 40 min.

This research contributes to the design of e-Health apps by incorporating usability evaluations as well as the perspectives of patients and professionals in the optimization process.^27,39,40 We aimed to enhance the app's usability and acceptance in clinical settings. Using convenience sampling, we recruited 10 healthcare professionals (including rehabilitation physicians, physical therapists, and/or occupational therapists) and 10 patients. It has been suggested that 5 users are sufficient to identify 85% of usability issues, especially when part of a homogeneous group.^41,42 We thus felt confident in our sample size of n = 20. Inclusion criteria for the healthcare professionals were as follows: (1) participants must hold professional licenses as rehabilitation physicians, occupational therapists, and/or physiotherapists linked to a home care company; (2) participants must have at least 10 years of professional service and over 5 years of experience in caring specifically for stroke patients; and (3) participants must have experience in home care and using digital tools. Inclusion criteria for the patient group were as follows: (1) participants must be over 60 years old, stable in their condition as referred by a physician, and willing to participate in the experiment and interviews; (2) participants must possess experience in using digital tools; (3) participants must have a caregiver; and (4) participants must consent to participating in the study and being audio-recorded in the sessions. All participants had the right to withdraw from the trial and terminate the experiment at any time.

To ensure the smooth progression of the trial, each participant operated the app functions on an iPad, using the interactive and functional mockup designs on the device. Each participant performed user tests according to the on-screen instructions. Figure 1 shows the content and distribution of information on each screen. The screens for the two groups of participants differed slightly. Usability testing involved participants performing various tasks and using case scenarios on the mobile app, with specific tasks assigned to each group of participants (Table 1). During the testing process, the emphasis was not on whether participants answered questions correctly but on observing them while they performed tasks on the app. This allowed for the analysis of task difficulty, execution time, and success rate. We used the following scale to measure task difficulty: 1 = very easy (task completed with no errors); 2 = easy (task completed after one error); 3 = moderately easy (task completed after two errors); 4 = difficult (task completed after three errors); and 5 = very difficult (completed after multiple attempts or else unsuccessful). To evaluate satisfaction and performance with app usability, a survey was developed using questionnaires adapted to the specific objectives of this study. The survey included questions tailored for both healthcare professionals and patients (Table 2). Satisfaction and performance with ease of deployment, navigation, and utility for self-management, monitoring, and decision-making activities related to home rehabilitation were assessed. Responses were evaluated using a 5-point Likert scale ranging from (1) “strongly disagree” to (4) “strongly agree.” Satisfaction and performance levels were determined based on quartile measurements. Additionally, participants were audio-recorded during the usability tests. The audio recordings, transcribed verbatim to maintain participant anonymity, were analyzed using NVivo 12.0. This analysis focused on data related to mockup characteristics to identify critical usability issues. Each participant engaged in a 45-min session, involving operational tasks and interviews. Quantitative survey data were analyzed using SPSS version 22.0 for statistical analysis.

Figure 1.

Mockups used for app usability tests. (a) App interface for healthcare professionals: Users can adjust rehabilitation plans based on a patient's condition, referencing clinical data and psychophysical states. They can also monitor a patient's rehabilitation progress after each session, thereby facilitating self-management by patients. (b) App interface for patients: The upper limb rehabilitation app interface guides patients through gamified screens. There is an offline practice version and an online interactive version, both including features such as receiving rehabilitation prescriptions, activity tracking, and progress monitoring.

Table 1.

Usability items.

Task no.	Usability test for healthcare professionals	Usability test for patients
1	Creating an account	Logging in
2	Logging in	Selecting offline practice or online connection
3	Establishing patient information and management: This includes adding or entering a patient's name, age, sex, identification code, and a brief description of the patient's physical condition.	Initiating pairing with the clinical end through the connection interface^a
4	Generating a pairing code for remote connection	Receiving rehabilitation prescriptions (recommendations) by clicking to enter the screen
5	Backend management (including rehabilitation prescription and remote course adjustment): This includes setting patient prescriptions and transmitting them to the patient's end.	Entering the game screen and starting practice
6	Creating online questions and level-setting	Following the on-screen instructions to proceed with practice
7	Reviewing patient rehabilitation accuracy and tracking their rehabilitation history	Generating rehabilitation outcomes, including accuracy of answers as well as cognitive and hand function analysis
8	Analyzing patient rehabilitation data to make predictions, and providing prescription recommendations	Viewing rehabilitation history and records

This study used connectivity as the testing objective.

Table 2.

User satisfaction assessment.

Questions for healthcare professionals		Questions for patients
1	Overall, this app is user-friendly.	Overall, this app is user-friendly.
2	The app interface is well-designed and aligns with clinical information needs.	The app has a simple and easy-to-understand interface.
3	Through the interface design, it is easy to interpret a patient's rehabilitation status.	The app enables easy self-practice at home.
4	The app is a significant aid to clinical care professionals, especially in setting rehabilitation prescription references.	The recording feature of the app enhances the overall effectiveness of rehabilitation.
5	The app facilitates monitoring patients’ home-based exercise rehabilitation after discharge.	When I encountered operating issues, I could easily resolve them.
6	I would recommend colleagues use this app or support the use of digital tools in stroke rehabilitation for patients.	The game mode contributes to my increased focus and enjoyment during rehabilitation.
7	The operations of the app are straightforward and easy to remember.	I intend to continue using this app.
8	Whether online or offline, operating the app was straightforward for me.	Even without assistance, I can operate the app for both online and offline exercises.
9	The app design is comprehensive, with no missing or incorrect information.	I feel safe using this app at home.
10	I believe that it is safe for patients to use this app at home.	The interface design helps me understand my recovery status.

The ultimate goal of the research design is to explore the long-term feasibility of applying digital tools for home-based rehabilitation in stroke patients from the perspectives of various stakeholders. The transition to digital therapy requires addressing the diverse needs of both clinical and patient sides. To ensure the design is closely aligned with user needs, it is crucial to consider the interaction of participants, environment, and tools. The innovative usability framework proposed in this study uniquely integrates these elements to create a dynamic, adaptable system that meets the specific needs of stroke patients in-home rehabilitation. This framework not only addresses the rehabilitation goals of the patients but also supports healthcare institutions’ care objectives and aligns with market demands, offering a comprehensive and effective solution that bridges the gap left by existing systems.

Results

Our sample of healthcare professionals included three rehabilitation physicians (30%), three physical therapists (30%), and four occupational therapists (40%). Most respondents were female (70%), with a median age of 48 years (interquartile range of 7 years). Among them, 60% had more than 10 years of service and over 5 years of experience providing home care for stroke patients. Additionally, all of the professionals reported having prior experience with digital healthcare tools. The patient group was predominantly male (n = 8, 80%), with an average age of over 60 years. Most of these patients were stable and had been receiving rehabilitation therapy for over one year (60%). Table 3 provides detailed information on the sociodemographic and clinical backgrounds of the two participant groups.

Table 3.

Sociodemographic characteristics of participants.

Healthcare professionals (HCPs) N = 10			Patients (Ps) N = 10
Variable	n	Percentage (%)	Variable	n	Percentage (%)
Gender			Gender
Male	3	30%	Male	8	80%
Female	7	70%	Female	2	20%
Age (SD)	48 years (±7 years)		Age (SD)	62 years (±5 years)
Job title			Education level
Rehabilitation physicians	3	30%	Elementary	4	40%
Physical therapists	3	30%	High school	3	30%
Occupational therapists	4	40%	Bachelor's or higher	3	30%
Experience in caring for stroke patients			Duration of rehabilitation
1–5 years	1	10%	1–3 years	6	60%
6–9 years	3	30%	4–6 years	4	40%
> 10 years	6	60%	> 6 years	0	0%

Participants’ perceptions

Following user interaction with the system app, one-on-one semi-structured interviews were conducted to gather insights into user experiences. Participants described any difficulties encountered and their feelings during task execution. The interviews were recorded, transcribed verbatim, and subjected to qualitative analysis using NVivo 12.0 software for content analysis and synthesis.

Among healthcare professionals, seven out of the ten (n = 7/10, 70%) acknowledged the need for a transitional period to familiarize themselves with the software interface and functionalities. However, they quickly understood the questions posed by the interface and were able to efficiently operate it. Six healthcare professionals (n = 6/10, 60%) suggested that the app could be more effective if it provided auditory or visual cues to indicate whether the operations were performed correctly. In addition, the ability to visualize information differently (such as through pie charts or bar graphs) could help to quickly identify issues such as cognitive or motor coordination problems. Seven out of the ten healthcare professionals (n = 7/10, 70%) agreed this would make the overall treatment process more efficient and assist in identifying the specific abilities that need strengthening at each stage. All the healthcare professionals emphasized the importance of tracking rehabilitation progress and changes over time. Recording patients’ long-term rehabilitation data and transforming the data into graphical representations (such as progress curves) would thus be a useful improvement. Six out of the 10 professionals (n = 6/10, 60%) indicated that progress curves are not only essential for clinical interpretation but also serve as important reference tools when communicating with patients.

Half of the patient group (n = 5/10, 50%) expressed initial nervousness before using the app, fearing that they might press the wrong button and struggle to find the “Back” button. They suggested simplifying and clarifying the interface to alleviate these concerns. Three patients (n = 3/10, 30%) proposed improvements to facilitate purposeful exits from sections of the app, as accidental closures were a recurring issue. Similarly, 3 out of the 10 patients (n = 3/10, 30%) found themselves inadvertently returning to the home page. Six patients (n = 6/10, 60%) expressed a lack of confidence in operating the platform independently, indicating a need for more training time.

Overall, the participants perceived both the hardware setup and the mobile application interface as familiar, simple, and user-friendly. The app was described as “easy to use” and the user interface “understandable” by 9 out of 10 healthcare professionals. Among patients, nine stated that the mobile app was beneficial for self-monitoring, and seven appreciated its well-organized structure and simple interface. They also highlighted that the current interface design, including color settings and graphical annotations, made it easy to understand their progress and described it as a thoughtful design. All healthcare professionals and patients noted that the straightforward design of the mockups enabled them to conduct the tests without extensive prior training, allowing them to quickly familiarize themselves with the presented case scenarios. Detailed comments from participants are provided in Table 4.

Table 4.

Participant feedback from semi-structured interviews.

Participants	Participants’ feedback	Pain points
Healthcare professionals (HCPs)	Honestly, I didn't think it was too difficult when I first saw the screen. However, I wondered why there wasn't an option for “forgot password.” What do I do if I forget my password? (HCP 3)	The initial design may not be fully refined yet, but the overall design is still very user-friendly. Recommendations include anticipating potential scenarios that a user might encounter, such as forgetting a password, to promptly address these situations before they arise.
	I think filling in patient information is easy, but I would hesitate when recording patient symptoms since their condition can change. (HCP 2)	Caution should be exercised during the preliminary determination of a patient's condition and symptoms, but if they are described based on the current stage of the illness, it is not too difficult.
	Well … I think everyone will have the same issue as I do regarding the connection process. I believe it might be somewhat challenging for both doctors and patients, as it requires mutual coordination. If the other side doesn't know how to operate it, it might be difficult to proceed. (HCP 5)	For many elderly patients, using consumer electronics can be challenging; therefore, simpler interface designs are preferable, and guided screens are crucial.
	Based on my experience with digital tools, transmitting commands over a connection is often delayed. I'm unsure if achieving synchronous patient responses in online questioning would be feasible. (HCP 9)Um… Sometimes, I notice a slight delay between what I see on the patient's screen and what I see on mine. It's probably due to internet lag. It's not a significant difference and doesn't compromise safety, and I'm okay with it. But if we need high precision, I might have some concerns about this. (HCPs 7 and 8)	It is essential to ensure data conversion speed and the ability to provide accurate information promptly.
	Well… Currently, some graphs provide a clear view of the patient's rehabilitation status, which I find great. However, seeing patients’ growth curves in clinical settings would be more beneficial. Do you think there will be such interface designs in the future? (HCPs 1 and 3)	To generate recovery status curves, a large amount of data on patient changes is needed, requiring cooperation between patients and the hospital.
	I have always been concerned about the security of digital tools. Is there a possibility of patient data being leaked if it is stored in the cloud? (HCP 5)	Patient safety is always the top priority in clinical practice, and obtaining accurate information is equally important.
Patients (Ps)	I can't control the force I use, so I'm afraid I might break the screen while operating it. (P 3)	Prolonged use of the screen may lead to concerns about damage and cracking.
	Umm… I feel comfortable practicing on my own, but I'm a bit anxious about the connection part. I'm unsure how to proceed and worried about making mistakes and having to start over. I can't manage this without assistance. (P 5)	Elderly individuals often find it challenging to use electronic devices, particularly when it involves connecting to Wi-Fi, which adds to their stress and difficulty in using the technology.
	Initially, I was unclear about the meaning of the connection code. Was I supposed to press something or type it in? I'm not very familiar with using this tablet (referring to the iPad). (Ps 1, 6, and 10)	The connection pairing mechanism seems to be difficult for both professionals and patients.
	Time seemed to pass quickly, and I hadn't finished reacting before the next question appeared. This made me a bit frustrated. (P 4)	Time settings should be established based on patient norms. Due to the limited sample and data available, defaults are currently used for implementation.
	I feel that while the previous method and the current one are similar, the current one feels more challenging and makes me a bit nervous, especially with the countdown timer part. (Ps 2, 7, and 10)	Currently, using a countdown method to alert users is intended to assess their response capabilities. In the future, changing the interface could help to reduce patient anxiety.
	Hmm… Although this method seems more advanced, I prefer having someone beside me. I'm afraid I might break the machine. (P 7)	For many elderly individuals, digital tools are relatively unfamiliar and require more time for adaptation. It would be beneficial to provide educational training even while they are still in hospital.
	Umm… I need more time to get used to the device, but I can learn. Will the hospital teach me? If it doesn't work out, can I switch back to the original method? (P 1)	For many elderly individuals, digital tools are relatively unfamiliar and require more time for adaptation. It would be beneficial to provide educational training even while they are still in hospital.

Evaluation of user operating difficulties

The healthcare professionals group was given tasks such as creating a username and password, logging in, creating patient accounts, reviewing patient rehabilitation, and tracking rehabilitation history; 70% of the healthcare professionals (n = 7/10) assessed these tasks as “very easy” (Table 5a). No task was described as “difficult” or “very difficult” in this test. In the patient group, logging in, choosing offline practice or online connection, entering the game screen, and starting practice were assessed as “very easy” by 60% of the healthcare professionals (n = 6/10). Additionally, 50% (n = 5/10) reported that receiving rehabilitation prescriptions, viewing rehabilitation history and personal information, and following the instructions from physicians’ remote messages were considered “easy.” However, 50% (n = 5/10) indicated that tasks related to pairing connections and understanding rehabilitation results and data were “difficult” (Table 5b).

Table 5.

Perceived difficulty of interacting with app interface.

Task	Very easyn (%)	Easyn (%)	Moderaten (%)	Difficultn (%)	Very difficultn (%)
a. Healthcare professionals (HCPs) (n = 10)
T1. Creating an account	8 (80)	2 (20)	0 (0)	0 (0)	0 (0)
T2. Logging in	10 (100)	0 (0)	0 (0)	0 (0)	0 (0)
T3. Creating patient accounts	7 (70)	3 (30)	0 (0)	0 (0)	0 (0)
T4. Initiating remote connection	6 (60)	2 (20)	2 (20)	0 (0)	0 (0)
T5. Prescription adjustment	5 (50)	4 (40)	10 (10)	0 (0)	0 (0)
T6. Conducting remote questioning and interaction	5 (50)	4 (40)	10 (10)	0 (0)	0 (0)
T7. Reviewing rehabilitation	8 (80)	2 (20)	0 (0)	0 (0)	0 (0)
T8. Recording review and analysis	6 (60)	3 (30)	10 (10)	0 (0)	0 (0)
T9. Tracking patient rehabilitation history	7 (70)	2 (20)	1 (10)	0 (0)	0 (0)
T10. Patient information and prescription management	6 (60)	3 (30)	10 (10)	0 (0)	0 (0)
b. Patients (Ps) (n = 10)
T1. Logging in	7 (70)	3 (30)	0 (0)	0 (0)	0 (0)
T2. Choosing offline practice or online connection	8 (80)	2 (20)	0 (0)	0 (0)	0 (0)
T3. Pairing with physician	0 (0)	1 (10)	2 (20)	5 (50)	2 (20)
T4. Receiving rehabilitation prescriptions	3 (30)	6 (60)	1 (10)	0 (0)	0 (0)
T5. Entering the game screen	7 (70)	2 (20)	1 (10)	0 (0)	0 (0)
T6. Starting practice	6 (60)	4 (40)	0 (00)	0 (0)	0(0)
T7. Following real-time instructions from the physician's end for practice	3 (30)	5 (50)	2 (20)	0 (0)	0 (0)
T8. Understanding final rehabilitation outcomes	0 (0)	0 (0)	2 (20)	5 (50)	3 (30)
T9. Viewing past rehabilitation history	2 (20)	5 (50)	2 (20)	1 (10)	0 (0)
T10. Viewing personal information	2 (20)	5 (50)	2 (20)	1 (10)	0 (0)

Task completion by participants

In usability testing with healthcare professionals, we assessed their ability to create an account, log in, enter patient information, and connect remotely; encouragingly, none of the healthcare professionals failed these tasks. This group of participants showed a higher success rate, reaching 100% for login tasks (Table 6a). During the testing process, each participant had three opportunities to complete the task. The majority of healthcare professionals (n = 6/10, 60%) required some time to think and adapt in order to familiarize themselves with the application: 40% (n = 4/10) made one error during operations but quickly corrected it; 30% made two or more errors (n = 3/10); and 60% (n = 6/10) ultimately succeeded in completing the tasks within the specified limit. The participants attributed these errors primarily to unfamiliarity with the interface and impulsively pressing incorrect buttons. They noted that the design of the system was not complex; other than the connectivity part, which required some thought, the remaining tasks were relatively simple. The app interface, with the inclusion of a “back” mechanism, allowed for quick error correction.

Table 6.

Task completion by participants.

Number of attempts before successful completion	Success upon first attempt		Success upon second or third attempt		Failure to complete task
Number of attempts before successful completion	Frequency	%	Frequency	%	Frequency	%
a. Healthcare professionals (HCPs) n = 10
T1. Creating an account	4	40	6	60	0	0
T2. Logging in	10	100	10	100	0	0
T3. Creating patient accounts	4	40	6	60	0	0
T4. Initiating remote connection	3	30	7	70	0	0
T5. Prescription adjustment	4	40	6	60	0	0
T6. Conducting remote questioning and interaction	5	50	5	50	0	0
T7. Reviewing rehabilitation	4	40	6	60	0	0
T8. Recording review and analysis	7	70	3	30	0	0
T9. Tracking patient rehabilitation history	6	60	4	40	0	0
T10. Patient information and prescription management	4	40	6	60	0	0
b. Patients (Ps) n = 10
T1. Logging in	4	40	6	60	0	0
T2. Choosing offline practice or online connection	0	0	10	100	0	0
T3. Pairing with physician	4	40	0	0	6	60
T4. Receiving rehabilitation prescriptions	3	30	7	70	0	0
T5. Entering the game screen	4	40	6	60	0	0
T6. Starting practice	5	50	5	50	0	0
T7. Following real-time instructions from the physician's end for practice	4	40	0	0	6	60
T8. Viewing final rehabilitation outcomes	7	70	3	30	0	0
T9. Viewing past rehabilitation history	6	60	4	40	0	0
T10. Viewing personal information	4	40	6	60	0	0

In the patient group, the participants successfully completed most tasks, including logging in, choosing offline practice or online connection, entering the game screen, and starting practice. However, tasks related to connectivity posed significant challenges: 60% (n = 6/10) failed to complete these tasks successfully within three attempts. These difficulties are reflected in the visualization of “frequently asked questions.” Among these patients, 60% (n = 6/10) made two to three mistakes. The latter was categorized as unsuccessful (Table 6b). Overall, mistakes in the patient group were related to unfamiliarity with transitioning from traditional treatment methods to digital tools. Additionally, 30% (n = 3/10) failed to complete tasks involving connectivity due to unfamiliarity with the mechanism. Despite these challenges, none of the participants in either group made critical mistakes.

Satisfaction and performance with app

The satisfaction and performance survey comprised 10 items. Because the interface and functions of the app operated by the healthcare professionals group and the patient group differed, the questions for the two groups were also different (as shown in Table 7). Both used a five-point Likert scale ranging from (1) “strongly agree” to (5) “strongly disagree.” Note that none of the items were deemed the highest level of satisfaction and performance (i.e. level 1) by every participant in each group. However, in the healthcare professionals group, there was a high level of satisfaction and performance with the transmission of medical information and the convenience of long-term patient monitoring: 50% selected level 1 for these questions. In the patient group, there was higher satisfaction and performance with the interface design, particularly regarding personal rehabilitation records, historical record queries, and personal information queries, with 50% (n = 5/10) selecting level 2 for these items.

Table 7.

Satisfaction survey.

Task	Strongly agree (1)n (%)	Agree (2)n (%)	Neutral (3)n (%)	Disagree (4)n (%)	Strongly disagree (5)n (%)
a. Healthcare professionals (HCPs) (n = 10)
01. The app's interface is user-friendly.	6 (60)	4 (40)	0 (0)	0 (0)	0 (0)
02. The app has well-designed patient information management.	5 (50)	4 (40)	1 (10)	0 (0)	0 (0)
03. The interface design makes it easy to understand the patient's rehabilitation status.	4 (40)	6 (60)	0 (0)	0 (0)	0 (0)
04. The app design facilitates clinical work and serves as supporting evidence for issuing rehabilitation prescriptions.	3 (30)	5 (50)	2 (20)	0 (0)	0 (0)
05. The app effectively assists patients in continuing their rehabilitation plans and follow-up at home.	5 (50)	4 (40)	1 (10)	0 (0)	0 (0)
06. I would recommend this digital tool to my colleagues.	4 (40)	5 (50)	1 (10)	0 (0)	0 (0)
07. The app's operations are straightforward and easy to learn.	4 (40)	5 (50)	1 (10)	0 (0)	0 (0)
08. Both online and offline modes are straightforward and easy to learn.	4 (40)	4 (40)	1 (10)	1 (10)	0 (0)
09. The app design is comprehensive, with no missing or incorrect information.	6 (60)	4 (40)	0 (0)	0 (0)	0 (0)
10. I believe that it is safe for patients to use this app at home.	5 (50)	5 (50)	0 (0)	0 (0)	0 (0)
b. Patients (Ps) (n = 10)
01. The app's interface is user-friendly.	2 (20)	5 (50)	2 (20)	1 (10)	0 (0)
02. I understand each feature on the interface.	3 (30)	5 (50)	1 (10)	1 (10)	0 (0)
03. The app enables easy self-practice even at home.	3 (30)	5 (50)	1 (10)	0 (0)	1 (10)
04. The app's recording feature enhances the overall effectiveness of rehabilitation.	0 (0)	2 (20)	2 (20)	4 (40)	2 (20)
05. Even if I encounter operating issues, I can easily resolve them.	2 (20)	3 (30)	1 (10)	3 (30)	0 (0)
06. The game mode increases my interest in rehabilitation.	50 (50)	3 (30)	2 (20)	0 (0)	0 (0)
07. I intend to continue using this app.	6 (60)	3 (30)	1 (10)	0 (0)	0 (0)
08. I can operate this app by myself at home, both online and offline.	2 (20)	2 (20)	0 (0)	5 (50)	1 (10)
09. I feel safe using this app at home.	3 (30)	5 (50)	2 (20)	0 (0)	0 (0)
10. The interface design helps me understand my recovery status.	2 (20)	3 (30)	1 (10)	4 (40)	0 (0)

Overall, healthcare professionals viewed this app positively. All of the participants stated that the interface was user-friendly and very easy to operate. The majority of the group (n = 9/10, 90%) believed that the app would assist in managing patients’ personal information. All participants (100%, n = 10) indicated that the app had excellent record-keeping features that facilitated remote rehabilitation and patient condition monitoring. Finally, half of the healthcare professionals (n = 5/10, 50%) agreed that the online functions would make it easier to conduct dynamic patient analysis and aid in clinical treatment interpretation and decision-making.

In the patient group, it appears that more time was needed to adapt to the intervention of digital tools. Despite this, seven out of the ten (n = 7/10, 70%) were satisfied with the interface design of the app, finding it visually appealing as well as easy to understand and use, and over half of the group (n = 8/10, 80%) reported that the clear interface design made it easy to understand the function and meaning of each interface, which would allow them to practice on their own at home. Additionally, 80% (n = 8/10) stated that the digital games would help to increase their interest in the rehabilitation program. Compared to traditional rehabilitation methods, patients preferred this more engaging and challenging approach. Indeed, 9 out of the 10 (n = 9/10, 90%) expressed a desire to continue using this method for the remainder of their treatment and even to use it daily. Of these, 8 out of the 10 patients (n = 8/10, 80%) had confidence in the platform and believed that using the app for rehabilitation at home would be very safe.

Discussion

User responses to this app were generally positive. Overall, both healthcare professionals and patients were fairly satisfied with the interface design of the app. During the testing process, none of the participants made critical errors, such as accidentally deleting data. Notably, both groups indicated that they needed more time to adapt to and learn the connectivity features of the app.⁴³

Both groups consistently expressed that aside from the connectivity features, other interface operations were not overly difficult. They also indicated that the recording, tracking, and monitoring functions could help to develop self-management skills and facilitate clinical monitoring during home care.^44,45 The usability tests allowed us to identify areas for improvement in app design.³¹ For instance, the connectivity feature posed significant challenges for many elderly users, thus highlighting the need for the interface to be as clear and simple as possible.

Improving the usability of healthcare technology through digital tools and design factors

Technological advancements offer various methods to enhance healthcare service quality.⁴⁶ Digital tools play a significant role in both medical and home care settings.^47,48 Despite the longstanding application of digital technology in these areas, there is still room for improvement in practical implementation and user acceptance due to the lack of user-centric interface design.⁴⁹ This study revealed that software functionality, content delivery, and interface design can make it challenging for elderly users to master healthcare technology. Usability depends not only on the technology itself but also on the design and arrangement of the user interface. Furthermore, different users face different challenges, especially those with poor physical and mental health. During usability testing, participants exhibited a positive attitude and high satisfaction and performance towards the app design with its intuitive user interfaces and contextualized content for home care. This suggests that once digital rehabilitation software and hardware are properly designed, many issues caused by diseases (such as COVID-19), such as the inability to receive in-hospital treatment or lack of effective post-discharge tracking, can be alleviated through the introduction of digital technology. In other words, having good hardware, when paired with user-centered app design that adheres to usability principles, can enhance the effectiveness of home care.

Future trends in clinical care suggest that digital tools will be widely applied in home care, telemedicine, and monitoring, with physicians, patients, and caregivers becoming frequent users of new technologies. From a socio-technical perspective, for the human-centered design and usability testing data of health technology to be meaningful, participants must have the knowledge and skills to develop these technologies.⁵⁰ This study found that users, especially the elderly, experienced significant operating difficulties and confusion with the connectivity features of the interface, which could negatively impact their experience and lead them to consider these features useless. Despite this frustration, the participants found the other functions of the app innovative, intuitive, and easy to use. Considering the potential of digital tools in reducing the burden on healthcare professionals and enabling precision medicine, the app included an automatic data recording feature. This feature used accumulated data as a basis for prescriptions, reducing the workload of healthcare professionals and minimizing interpretation errors. Professionals in our study recognized that creating clinical records and medical prescriptions through the app could greatly aid in managing patients with chronic diseases.⁵¹

This study diverged from previous research in which issues such as the cost of digital tools, information security, and the digital knowledge gap are often emphasized.^52,53 Instead, this study suggests that digital technology used at home could pose challenges for bidirectional communication. If one end cannot maintain proper function in an app, the healthcare process halts. Nonetheless, with generational shifts, the current user population has higher educational backgrounds and more exposure to digital tools compared to the past. Therefore, hardware and technology are not the primary issues; rather, the quality of interface design is crucial.

Proper guidance can ensure the smooth transmission of health service models

Unclear hardware instructions and app interfaces can negatively impact user performance, reducing the perceived usefulness of the technology. Consequently, interface design should prioritize the capabilities and preferred modes of communication of users. For example, considering the educational level and physical abilities of the elderly, integrating visuals and incorporating voice functions could potentially improve user effectiveness. This means that product, system, and service interface design are often influenced by factors such as the knowledge background, culture, and abilities of different stakeholders. As highlighted by Mendiola et al. (2015), users, especially patients, tend to prefer designs that are simple and time-efficient. Overly complex and time-consuming designs are more likely to reduce users’ patience, acceptance, and satisfaction and performance with the design.

Technology acceptance by stroke patients for home rehabilitation

Studies have indicated that the application of digital technology not only reduces the workload of clinical personnel but also provides more precise references for clinical decision-making. However, these advantages must be built on a foundation of good design, particularly interface design, as effective communication relies on well-designed interfaces.⁵⁴ According to our results, users generally found the proposed app user-friendly and easy to operate. They regarded the design as innovative, especially in its recording functionality, all of which could enhance the efficiency of healthcare. Additionally, real-time data results not only helped in understanding the current condition of patients but the accumulation of big data provided optimal rehabilitation prescription suggestions, thus avoiding subjective judgments or potential medical errors.

We identified a number of design shortcomings during the trial, such as the setup of connectivity functions between the clinical side and home side, which is currently the major challenge. Effective information communication and exchange rely on the capabilities of both parties.⁵⁵ When connectivity fails for one party, communication breaks down, and the desired outcome cannot be achieved.⁵⁶ Therefore, a well-designed user interface is a crucial element in message transmission. Although users indicated some difficulty with the connectivity function during the trial, both groups of respondents expressed that with more time to learn and adapt, they could master these functions and were generally satisfied with the design. This suggests that interface improvements will help users to accept, trust, and rely on digital technology.

Unlike previous studies, this research found that both healthcare professionals and patients exhibited higher levels of education and familiarity with computers and electronic devices compared to a decade ago.⁵⁷ Furthermore, professionals with higher education levels and familiarity with tablets tended to prefer to utilize visuals to quickly understand the progress of patient recovery, which indicates that the app should utilize a database to generate a historical chart displaying patient clinical data behavior over a specific period. Growth curves can assist medical professionals in readily assessing and predicting patient conditions, improving the precision of medical decisions and significantly impacting patient recovery outcomes and timelines. This would benefit the long-term management and monitoring of patients’ clinical status.

Furthermore, this study highlights a critical issue: elderly individuals may have impaired visual and motor skills, making them less likely to accept and use technology. Patients in this research suggested that incorporating voice guidance, alongside graphical and textual interfaces, might be more effective in achieving the desired outcomes. Overall, the application of digital technology tools in healthcare not only enables patients to access self-management information for their conditions but also positively impacts the health outcomes of stroke patients. This could be achieved by utilizing the characteristics of digital tools to change traditional rehabilitation methods using games to increase patients’ motivation and focus on rehabilitation, thus making the process more engaging and enhancing adherence.

The majority of participants expressed a willingness to adopt this approach as a future method of health management. These findings indicate that the proposed app elicited high levels of satisfaction and performance from participants, suggesting its potential for widespread acceptance in the future.

Recommendations for app design based on usability testing

The usability study revealed that participants experienced hesitation, uncertainty, and fear when using the connectivity feature, despite their familiarity with digital devices and their understanding of connectivity concepts. When their actions did not produce the expected results, they felt helpless and confused.⁵⁸ This issue was more pronounced among elderly patients, who often face physiological limitations such as reduced tactile and visual responses, hindering their comprehension of on-screen information and their confidence in operating the app.⁵⁹ Although previous studies have shown that graphical interfaces can aid in the understanding of information, the physiological limitations of elderly users may necessitate additional assistive tools.^60,61 These tools could include auditory guidance or flashing prompts to enhance their interaction with the app.

The study found that both groups of participants considered the app interface and operations to be simple, yet there remains room for improvement. The healthcare professionals appreciated the current design of color blocks indicating the status of a question; however, they suggested that adding rehabilitation record graphs to the historical records would help clinical staff to quickly understand a patient's condition, predict symptom dynamics, and determine necessary rehabilitation prescriptions. Meanwhile, the patient group expressed confusion about the results, due perhaps to the fact that their educational background made it challenging to comprehend the significance of the data. Therefore, the interface design could be adjusted to present information differently according to a specific group of users, such as using encouraging visuals to indicate areas needing improvement or reward-based screens to boost patients’ confidence in their rehabilitation plans.

It was observed that users generally had few difficulties operating the tablet and interacting with the app, apart from elderly participants, who exhibited slower reaction times and movements. This ease of use can be attributed to the intuitive design of the app, such as the straightforward binary choice between “offline practice” and “online connection,” which facilitated decision-making even among older users. However, challenges arose with the connectivity pairing function, causing confusion among patients and necessitating additional time for healthcare professionals to interpret messages displayed by the app. Another significant issue was the latency in message transmission during real-time interactions, which hindered communication between patients and healthcare professionals. This latency could have affected scoring accuracy if response times had been factored into the evaluation. Therefore, improvements in IT infrastructure are essential, especially considering the planned implementation of the system for home use. Addressing the variability in internet connectivity and resource limitations across different regions will be crucial for the future success of the system.

Regarding the account creation and login process, both groups experienced fewer issues compared to other studies, due perhaps to the use of simple, easy-to-remember login requirements, like birth dates. However, this approach raises concerns about medical information security. The slow adoption of digital tools in clinical settings is primarily due to security concerns surrounding patient information. Future improvements and discussions should consider users’ education levels and experience with digital devices as well as physiological factors such as tactile, visual, and auditory functions.

It seems the effectiveness of digital tools is not solely dependent on interface design and guidance but also closely related to the users’ knowledge and operational skills regarding digital technology.⁶² Therefore, the successful implementation of a healthcare mobile app must consider multiple key factors influencing participants’ final behaviors. The adoption of digital tools in therapeutic practices is feasible, as these tools can aid in data accumulation, identify patient characteristics based on data traits, and subsequently assist in the development of individualized treatment plans for patients with unique needs. This approach provides optimal and precise treatment strategies, enhancing therapeutic efficacy, which is the primary benefit and outcome of this study. Both groups of participants expressed high satisfaction and performance with the proposed digital tools. Despite certain functions requiring additional learning and adaptation time, the participants indicated their willingness to learn and recommend the app to others. Examining the usability of the tool from different perspectives, it is evident that users’ functional operating capabilities and acceptance of the proposed app are indeed feasible and serve as a valuable reference.

Based on the above discussions, a conceptual framework was developed to summarize the main usability challenges and potential design enhancements. As shown in Figure 2, this framework highlights both technical and user-centered factors that are essential for improving the home-based rehabilitation experience for stroke patients.

Figure 2.

Usability framework for Stroke Rehab App.

Study limitations and future work

Though our sample size was relatively small (20 participants), Nielsen (2000) suggested that five participants are typically sufficient to find 80% of usability problems; however, it is important to note that sample sizes vary depending on the specific type of study.^63–65 This perspective is supported by numerous related studies.

We intend to conduct further research to address the identified shortcomings in the study by focusing on design improvements of the proposed app. In our future research, a three-month rehabilitation and follow-up will be conducted in conjunction with a rehabilitation treatment plan.^66,67 This will explore the effectiveness of the app in facilitating rehabilitation for stroke patients at home through remote monitoring as well as assess the impact of the app on their clinical status and self-management skills. Additionally, through extensive user testing and feedback, the study aims to optimize the app and clearly determine whether the proposed usability framework represents a feasible and innovative design model.

Conclusions

Overall, most healthcare professionals and patients indicated that the app's interface design was intuitive and highly user-friendly. The satisfaction and performance survey results demonstrated that both groups consistently rated the design highly and expressed significant satisfaction. Nonetheless, our usability assessments identified certain weaknesses during practical operation, predicting potential challenges in clinical care settings. Therefore, when implementing such systems or tools in home-based environments, it is crucial to address and refine these elements to enhance the feasibility of utilizing digital tools for home rehabilitation. Future research could focus on resolving the issues and challenges identified by users.

The usability tests confirmed that considering individual differences among users is crucial; these include age, education level, and functional conditions according to the health status of patients. Leveraging the characteristics of digital tools to accumulate data, identify key features, and use these features to infer and predict the rehabilitation plans needed for individualized patients were proven to be valuable strategies. This data-driven approach not only allows for rational and scientific estimation of the treatments required by users but also enables long-term monitoring of patients’ post-discharge health status, helping to mitigate the risk of secondary strokes.

Footnotes

Acknowledgements

We sincerely thank all participants for their valuable contributions to this study. Special appreciation goes to the Department of Rehabilitation and the Department of Occupational Therapy at Chang Gung Memorial Hospital for their invaluable clinical support.

ORCID iD

Shih-Ying Chien

Ethical considerations

This research was approved by the Research Ethics Committee of the Chang Gung Medical Foundation (Reference No: IRB/REC 202301197A3).

Author contributions

S.Y. Chien was involved in the study design, conducted the interviews, collected and analyzed the data, and drafted the manuscript. C.Y. Wu provided clinical guidance and manuscript revision.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study received support from the National Science and Technology Council (NSTC) through grant number 113-2221-E-182-057.

Declaration of conflicting interests

The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

To protect the privacy of the participants, participant data will not be made publicly accessible. However, interested parties may obtain access to the datasets by contacting the corresponding author and making a reasonable request.

Informed consent

Before the experiment began, all participants provided informed consent by signing the necessary documentation and agreed to participate in the rehabilitation platform test without compensation. During the trial, participants had the unconditional right to withdraw from the test at any time without any impact on their rights. Additionally, all personal information collected during the experiment was anonymized, and the collected data were processed using identification numbers.

References

Lee

Bae

Jeon

, et al. The effects of cognitive exercise therapy on chronic stroke patients’ upper limb functions, activities of daily living and quality of life. J Phys Ther Sci 2015; 27: 2787–2791.

Katan

Luft

. Global burden of stroke. In Seminars in Neurology. New York, NY: Thieme Medical Publishers, 2018, pp. 208–211

Kuriakose

Xiao

. Pathophysiology and treatment of stroke: present status and future perspectives. Int J Mol Sci 2020; 21: 7609.

Camak

. Addressing the burden of stroke caregivers: a literature review. J Clin Nurs 2015; 24: 2376–2382.

Hatem

, et al. Rehabilitation of motor function after stroke: a multiple systematic review focused on techniques to stimulate upper extremity recovery. Front Hum Neurosci 2016; 10: 42.

Hoff

. Patients’ experiences of group therapy during inpatient stroke rehabilitation. 2021.

Nas

, et al. Rehabilitation of spinal cord injuries. World J Orthop 2015; 6: 8.

Agostini

Moja

Banzi

, et al. Telerehabilitation and recovery of motor function: a systematic review and meta-analysis. J Telemed Telecare 2015; 21: 202–213.

Chui

Alhalabi

Pang

, et al. Disease diagnosis in smart healthcare: innovation, technologies and applications. Sustainability 2017; 9: 2309.

10.

Jiang

, et al. Patients’ and healthcare providers’ perceptions and experiences of telehealth use and online health information use in chronic disease management for older patients with chronic obstructive pulmonary disease: a qualitative study. BMC Geriatr 2022; 22: 1–16.

11.

Stefanicka-Wojtas

Kurpas

. Ehealth and mHealth in chronic diseases—identification of barriers, existing solutions, and promoters based on a survey of EU stakeholders involved in Regions4PerMed (H2020). J Pers Med 2022; 12: 467.

12.

Barello

, et al. Ehealth for patient engagement: a systematic review. Front Psychol 2016; 6: 2013.

13.

Dequanter

Gagnon

M-P

Ndiaye

M-A

, et al. The effectiveness of e-health solutions for aging with cognitive impairment: a systematic review. Gerontologist 2021; 61: e373–e394.

14.

Wentink

Bodegom-Vos

Brouns

, et al. What is important in E-health interventions for stroke rehabilitation? A survey study among patients, informal caregivers and health professionals. Int J Telerehabil 2018; 10: 15.

15.

Tosto-Mancuso

Tabacof

Herrera

, et al. Gamified neurorehabilitation strategies for post-stroke motor recovery: challenges and advantages. Curr Neurol Neurosci Rep 2022; 22: 183–195.

16.

Philip

Rodrigues

JJPC

Wang

, et al. Internet of things for in-home health monitoring systems: current advances, challenges and future directions. IEEE J Sel Areas Commun 2021; 39: 300–310.

17.

Senbekov

, et al. The recent progress and applications of digital technologies in healthcare: a review. Int J Telemed Appl 2020; 2020: 8830200.

18.

Michie

Yardley

West

, et al. Developing and evaluating digital interventions to promote behavior change in health and health care: recommendations resulting from an international workshop. J Med Internet Res 2017; 19: e232.

19.

Chien

S-Y

Wong

AM-K

Tseng

, et al. Feasibility and design factors for home-based pulmonary rehabilitation of patients with chronic obstructive pulmonary disease and chronic lung diseases based on a people-object-environment framework: qualitative interview study. JMIR Human Factors 2024; 11: e51150.

20.

Frederix

Solmi

Piepoli

, et al. Cardiac telerehabilitation: a novel cost-efficient care delivery strategy that can induce long-term health benefits. Eur J Prev Cardiol 2017; 24: 1708–1717.

21.

Dineen-Griffin

Garcia-Cardenas

Williams

, et al. Helping patients help themselves: a systematic review of self-management support strategies in primary health care practice. PLoS One 2019; 14: e0220116.

22.

Al-Jaroodi

Mohamed

Abukhousa

. Health 4.0: on the way to realizing the healthcare of the future. IEEE Access 2020; 8: 211189–211210.

23.

Rosa

Campbell

ANC

Miele

, et al. Using e-technologies in clinical trials. Contemp Clin Trials 2015; 45: 41–54.

24.

Granja

Janssen

Johansen

. Factors determining the success and failure of eHealth interventions: systematic review of the literature. J Med Internet Res 2018; 20: e10235.

25.

Molina-Recio

Molina-Luque

Jiménez-García

, et al. Proposal for the user-centered design approach for health apps based on successful experiences: integrative review. JMIR Mhealth Uhealth 2020; 8: e14376.

26.

Göttgens

Oertelt-Prigione

. The application of human-centered design approaches in health research and innovation: a narrative review of current practices. JMIR Mhealth Uhealth 2021; 9: e28102.

27.

Chien

S-Y

Wong

C-Y

, et al. Interactive electronic pegboard for enhancing manual dexterity and cognitive abilities: instrument usability study. JMIR Human Factors 2024; 11: e56357.

28.

Weichbroth

. Usability of mobile applications: a systematic literature study. IEEE Access 2020; 8: 55563–55577.

29.

Mendiola

Kalnicki

Lindenauer

. Valuable features in mobile health apps for patients and consumers: content analysis of apps and user ratings. JMIR Mhealth Uhealth 2015; 3: e4283.

30.

Maramba

Chatterjee

Newman

. Methods of usability testing in the development of eHealth applications: a scoping review. Int J Med Inf 2019; 126: 95–104.

31.

Hoehle

Venkatesh

. Mobile application usability. MIS Q 2015; 39: 435–472.

32.

Barnum

. Usability testing essentials: Ready, set…test! Cambridge, MA: Morgan Kaufmann, 2020.

33.

Moumane

Idri

Abran

. Usability evaluation of mobile applications using ISO 9241 and ISO 25062 standards. SpringerPlus 2016; 5: 1–15.

34.

Chen

Abel

Janecek

, et al. Home-based technologies for stroke rehabilitation: a systematic review. Int J Med Inf 2019; 123: 11–22.

35.

Segura

Grau-Sánchez

Sanchez-Pinsach

, et al. Designing an app for home-based enriched music-supported therapy in the rehabilitation of patients with chronic stroke: a pilot feasibility study. Brain Inj 2021; 35: 1585–1597.

36.

Brooke

. SUS: a retrospective. J Usability Stud 2013; 8: 29–40.

37.

Naranjo-Rojas

Perula-de-Torres

LÁ

Cruz-Mosquera

, et al. Mobile application for monitoring patients under home oxygen therapy: a protocol for a randomized controlled trial. BMC Fam Pract 2021; 22: 104.

38.

Jin

, et al. EUCA: The end-user-centered explainable AI framework. arXiv preprint arXiv:2102.02437, 2021.

39.

Bombard

, et al. Engaging patients to improve quality of care: a systematic review. Implement Sci 2018; 13: 1–22.

40.

Mercieca-Bebber

, et al. The importance of patient-reported outcomes in clinical trials and strategies for future optimization. Patient Relat Outcome Meas 2018; 9: 353–367.

41.

Vélez

Okyere

Kanter

, et al. A usability study of a mobile health application for rural Ghanaian midwives. J Midwifery Womens Health 2014; 59: 184–191.

42.

Nielsen

Landauer

. A mathematical model of the finding of usability problems. In Proceedings of the INTERACT'93 and CHI'93 conference on Human factors in computing systems. 1993.

43.

Grua

De Sanctis

Lago

. A reference architecture for personalized and self-adaptive e-health apps. In European Conference on Software Architecture. Springer, 2020.

44.

Morgan

. ‘Pushed’ self-tracking using digital technologies for chronic health condition management: a critical interpretive synthesis. Digital health 2016; 2.

45.

Morton

Dennison

May

, et al. Using digital interventions for self-management of chronic physical health conditions: a meta-ethnography review of published studies. Patient Educ Couns 2017; 100: 616–635.

46.

Fatima

Malik

Shabbir

. Hospital healthcare service quality, patient satisfaction and loyalty: an investigation in context of private healthcare systems. Int J Qual Reliab Manage 2018; 35: 1195–1214.

47.

Golinelli

Boetto

Carullo

, et al. Adoption of digital technologies in health care during the COVID-19 pandemic: systematic review of early scientific literature. J Med Internet Res 2020; 22: e22280.

48.

Jnr

. Use of telemedicine and virtual care for remote treatment in response to COVID-19 pandemic. J Med Syst 2020; 44: 32.

49.

Chen

C-H

KKH

, et al. A user-centric design approach for smart product-service systems using virtual reality: a case study. J Cleaner Prod 2021; 280: 124413.

50.

Holden

Valdez

. Human factors engineering and user-centered design for mobile health technology: enhancing effectiveness, efficiency, and satisfaction, in human-automation interaction: mobile computing. Cham, Switzerland: Springer International Publishing, 2022, pp. 97–118.

51.

Leviatan

Oberman

Zimlichman

, et al. Associations of physicians’ prescribing experience, work hours, and workload with prescription errors. J Am Med Inform Assoc 2021; 28: 1074–1080.

52.

Barth

de Jong

MDT

Junger

, et al. Putting the privacy paradox to the test: online privacy and security behaviors among users with technical knowledge, privacy awareness, and financial resources. Telemat Inform 2019; 41: 55–69.

53.

Kache

Seuring

. Challenges and opportunities of digital information at the intersection of big data analytics and supply chain management. Int J Oper Prod Manage 2017; 37: 10–36.

54.

Johnson

. Designing with the mind in mind: simple guide to understanding user interface design guidelines. Cambridge, MA: Morgan Kaufmann, 2020.

55.

Eggert

Ulaga

Frow

, et al. Conceptualizing and communicating value in business markets: from value in exchange to value in use. Ind Mark Manag 2018; 69: 80–90.

56.

Boin

Bynander

. Explaining success and failure in crisis coordination. Geografiska Anna: Ser A Phys Geogr 2015; 97: 123–135.

57.

Tan

HSG

van den Berg

, et al. The influence of product preparation, familiarity and individual traits on the consumer acceptance of insects as food. Food Qual Prefer 2016; 52: 222–231.

58.

Hao

Lee

Chen

S-T

, et al. An evaluative study of a mobile application for middle school students struggling with English vocabulary learning. Comput Human Behav 2019; 95: 208–216.

59.

Saftari

Kwon

O-S

. Ageing vision and falls: a review. J Physiol Anthropol 2018; 37: 1–14.

60.

Machado

, et al. A conceptual framework for adaptive user interfaces for older adults. In 2018 IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops). IEEE, 2018.

61.

Castilla

Suso-Ribera

Zaragoza

, et al. Designing ICTs for users with mild cognitive impairment: a usability study. Int J Environ Res Public Health 2020; 17: 5153.

62.

Lin

Shadiev

Hwang

W-Y

, et al. From knowledge and skills to digital works: an application of design thinking in the information technology course. Think Ski Creat 2020; 36: 100646.

63.

Davids

Harvey

Halperin

, et al. Determining the number of participants needed for the usability evaluation of e-learning resources: a Monte Carlo simulation. Br J Educ Technol 2015; 46: 1051–1055.

64.

Hermawati

Lawson

. Establishing usability heuristics for heuristics evaluation in a specific domain: is there a consensus? Appl Ergon 2016; 56: 34–51.

65.

Henshall

Davey

. Development of an app for lung cancer survivors (iEXHALE) to increase exercise activity and improve symptoms of fatigue, breathlessness and depression. Psycho-Oncology 2020; 29: 139–147.

66.

Scott

Strong

C-AH

Gorter

, et al. Predictors of post-concussion rehabilitation outcomes at three-month follow-up. Clin Neuropsychol 2016; 30: 66–81.

67.

Schow

, et al. Evaluation of a four month rehabilitation program for stroke patients with balance problems and binocular visual dysfunction. NeuroRehabilitation 2016; 38: 331–341.