Human Factors and Requirements of People with Cognitive Impairment,Their Caregivers,and Healthcare Professionals for mHealth Apps Including Reminders,Games,and Geolocation Tracking: A Survey-Questionnaire Study

Abstract

Background:

Mobile Health (mHealth) apps can delay the cognitive decline of people with dementia (PwD), by providing both objective assessment and cognitive enhancement.

Objective:

This patient involvement survey aims to explore human factors, needs and requirements of PwD, their caregivers, and Healthcare Professionals (HCPs) with respect to supportive and interactive mHealth apps, such as brain games, medication reminders, and geolocation trackers through a constructive questionnaire.

Methods:

Following the principles of user-centered design to involve end-users in design we constructed a questionnaire, containing both open-ended and closed-ended questions as well as multiple choice and Likert scale, in order to investigate the specific requirements and preferences for mHealth apps. We recruited 48 participants including people with cognitive impairment (n = 15), caregivers (n = 16), and HCPs (n = 17) and administered the questionnaire.

Results:

All participants are likely to use mHealth apps, with the primary desired features being the improvement of memory and cognition, assistance on medication treatment, and perceived ease to use. HCPs, caregivers, and PwD consider brain games as an important technology-based, non-pharmaceutical intervention. Both caregivers and patients are willing to use a medication reminder app frequently. Finally, caregivers are worried about the patient wandering. Therefore, global positioning system tracking would be particularly important to them. On the other hand, patients are concerned about their privacy, but are still willing to use a geolocation app for cases of emergency.

Conclusion:

This research contributes to mHealth app design and potential adoption. All three groups agree that mHealth services could facilitate care and ameliorate behavioral and cognitive disturbances of patients.

Keywords

Alzheimer’s disease apps eHealth information technologies mHealth mild cognitive impairment remote monitoring serious games telemedicine

INTRODUCTION

Alzheimer’s disease (AD) constitutes an increasing health problem with wide-ranging consequences for the caregivers, health-care systems, and society, with over 50 million cases worldwide [1, 2]. Nevertheless, there is a number of evidence which shows that lifestyle recommendations, qualitative care, and non-pharmaceutical interventions alongside with the effective implementation of information technology (IT) advances, might forestall the onset or at least delay the future progression of cognitive impairment [3 –10].

IT advances focus on developing innovative solutions to improve functional and cognitive capabilities and increase safety and autonomy, while compensating further decline of people with dementia (PwD). In particular, several applications including brain games are considered to be of particular interest for healthcare professionals (HCPs) given that they have proved to be very beneficial for PwD [11, 12]. Additionally, medication reminders and global positioning system (GPS) trackers for PwD are considered of high importance for the caregivers and patients themselves since they provide safety to the second ones [13, 14]. In general, mobile Health (mHealth) apps can ameliorate cognitive and functional decline of PwD by implementing passive apps which target in assessment and monitoring or interactive apps such as brain games and reminders [15 –18]. The mHealth apps can provide insightful information about patients’ health status, send reminders for patients’ daily medication or physical activity, allow both the caregivers and the patients to set alarms in case of emergency, and provide meaningful disorder-specific feedback to HCPs, while at the same time facilitate communication between patients and HCPs [15 , 19–21]. Additionally, mHealth apps and technological interventions in the form of brain games may improve diagnostic accuracy both in demented population as well as in preclinical stages (e.g., mild cognitive impairment, MCI), by assessing in real-time cognitive functions while allowing the HCPs to compare their results with other evidence-based biomarkers [22]. Meanwhile, brain games constitute a last-decade non-pharmaceutical intervention targeting on enhancing patient’s cognitive as well as motor function and coordination [23 –25]. Moreover, mHealth apps are accessible for the majority of people since they are generally affordable, can be easily installed on patients’ phones and can easily integrate with electronic health records [26]. However, the acceptance of the mHealth apps by the elderly population and more specifically by PwD is still a challenging issue, while the investigation of preferable features and requirements constitutes an important topic in IT and clinical research [27, 28].

Recent survey studies have found that although many elders have positive attitudes toward adopting mHealth apps and technology in general, the usage rates for technologies like mobile phones and computers by the seniors are still low [29]. Nevertheless, a lot of research has been done investigating various aspects of health-related technologies, focusing on preferable features and traits that the elders would like to be incorporated in the suggested mHealth apps [9]. Also, the attitudes of elders toward different kinds of technologies ranging from brain games, smart home systems, and remote monitoring and assistive technology, to general information that they would like to receive have also been explored [9 , 31]. Given that the elderly need to be more certain before they act, they are usually among the last to adopt a product, service, or an innovative idea [29 , 33]. In addition, the elderly tend to have relatively negative views toward technology and show less interest in using various new technologies [28]. It has been shown that the computer-using experience, perceived ease of use, perceived usefulness, self-attitude toward new technology, and socialization agents could increase the acceptance of technology [32 , 34–36]. Therefore, the abovementioned research studies have paved the way to explore particular features that are of high importance for the end-users so as to increase patients’ engagement and empowerment in self-monitoring of their health and provide seamless access to health care services to the HCPs and the caregivers. Thus, IT can provide benefits to the elderly population and more specifically to PwD as patients, as well as HCPs and the caregivers in case they are willing to use them. Also investigating the PwD requirements prior to the adoption of an IT product or a mHealth app alongside with the caregivers’ and clinicians’ preferences is of high importance in order to clarify and ensure technology acceptance by the patients.

There is a drive to involve patients and the public in health research, due to recognition that patient and public involvement (PPI) may increase the impact and relevance of health research. Practical issues with the intervention were also identified by PPI feedback, including prioritizing time to complete intervention sessions, and suggestions of ideas of how care partners can overcome barriers to completing the intervention [37]. PPI studies can benefit research focusing on investigation of technology adoption in several ways. The Alzheimer Europe association highlights that involving PwD and caregivers in dementia research is a “win-win for everyone involved” [38]. PPI in IT and mHealth apps studies has become increasingly common since it involves people in research that pertains to them [37 , 40], while making the research more effective suggesting efficient implementation strategies [36 , 41–43]. According to a recent systematic review focusing on PPI in dementia research, a great number of studies have evaluated the impact of PPI in dementia research by employing online surveys, semi-structured interviews, tailored developed questionnaires related to the study’s questions and focus groups to evaluate the research objectives, providing useful evidences in research [37].

Moreover, as suggested by Alzheimer Europe [44], PPI may also draw on a range of research methods such as interviews, focus groups, surveys and questionnaires, Delphi rounds, user-led forums, email, and Skype consultations. In particular, Stevenson et al. [45] used a paper-pencil questionnaire developed by the team, exploring the preferences and worries of PwD while trying to capture their perspectives on the benefits of being a co-researcher in the exercise. On the other hand, Littlechild et al. [46] in order to investigate the viewpoints of co-researchers, organizations, and academic researchers, used semi-structured interviews and focus groups to assess the impact of involvement in all stages of the research process from prioritization and formulation of research questions, study design, recruitment, data analysis and interpretation to dissemination. Another survey [13] argues that it is a must to involve end-users in the co-design of new technologies in order to develop apps, devices, as well as testing them in a real-world context [47]. More specifically, in our previous PPI study, we developed a Human Factors and Technology Requirements Questionnaire (HFTRQ) which included ten dimensions in order to explore the beneficiaries’ requirements towards using a wearable solution and the features they would like to be incorporated in remote monitoring technologies [48]. Given that the previous study was focusing mainly on remote monitoring using contactless sensors, currently there is no PPI study examining HFs, acceptance and willingness of adoption of interactive systems and mHealth apps, such as brain games, medication reminders or GPS trackers by the PwD, HCPs, and the caregivers. Moreover, there is no single, established, universal questionnaire for assessing and evaluating human factors requirements for interactive mHealth apps and how to design and integrate particular applications in daily life of HCPs, caregivers, and PwD.

In this direction, the present PPI study aims to address the need to extract HFs, needs, and requirements of not only the PwD but also their caregivers and HCPs with respect to supportive mHealth apps. In particular, it explores the intentions and preferences of people at a preclinical stage of AD such as MCI and SCI, their caregivers and HCPs with respect to using passive and interactive mHealth apps such as brain games, medication reminders, or GPS trackers through a constructive questionnaire. We hypothesize that their feedback would help shape current and future eHealth and IT research toward more acceptable, usable, and suitable mHealth apps. The present questionnaire does not constitute a psychometric tool, since it aims to examine several factors as well as potential improvements in an application that could be implemented in patients and aspects that could be objectively adopted by a mHealth app by considering preferences from HCPs, caregivers, and people with cognitive impairment in the grounds of a PPI survey study. Therefore, in this paper, we explore the involvement of members of the public and patient groups in shaping research to: 1) inform design and procurement decisions regarding the mHealth app preferences including brain games, medication reminders, and GPS tracking systems and 2) produce general outcomes to optimize study design and improve acceptability for those planning and conducting dementia research using the respective mHealth apps. It was made clear to the participants of this PPI survey that we were interested in their views of current and future mHealth research studies.

MATERIALS AND METHODS

Participants and settings

We recruited 48 participants (34 female and 14 male) from the day centers of the Greek Association of Alzheimer’s Disease and Related Disorders (GAADRD) in Thessaloniki and the Frontida Zois patient center in Patras, between September and October 2020. They were either elderly people with mild (MCI) and subjective cognitive impairment (SCI) who lived independently (n = 15), caregivers taking care of patients with cognitive impairment (n = 16), or HCPs (n = 17) holding great experience in the dementia research field. The distribution of subjects by demographical data in each subcategory is shown in Table 1. We tried to include a sufficient number of participants (at least 15 participants representatives from each group) based on other similar approaches which included 30 [39, 49], 35 [7], 12 [42], and 8 [50]. We obtained ethics approval for our study from the Institutional Review Board (IRB) of the Centre for Research and Technology Hellas (ETH.COM_44/2019). All relevant ethical safeguards have been met in relation to participant protection.

Table 1

Demographic Characteristics of the HCPs (N = 17), caregivers (N = 16), and patients (N = 15)

	HCPs	Caregivers	Patients
Gender (F:M)	14:3	11:5	9:6
Average Age per group	34.05	52.53	69.93

The Human Factors and Requirements for Interactive mHealth Applications including Reminders, Games, and Geolocation for People with Dementia, Caregivers, and Healthcare Professionals Questionnaire (HFIAQ)

We have based our study on similar approaches that included similar open-ended and close-ended questions to assess end-users’ requirements, views, and concerns [51]. In particular, we followed the responsive PPI design, which is most suitable to gain an understanding of the meaning of experiences [21 , 53], and focus mainly on the people’s significant participation in all aspects of the research process [50, 54]. Within this PPI survey design, we did not conduct neuropsychological validation of the generated questionnaire, instead we used responsive research as framework by developing a survey questionnaire related to our study’s objectives. Previous research has shown that the responsive methodology and PPI surveys can be seen as an established method to actively involve patients in research and to get insight in the perspectives of all stakeholder groups involved [49]. In particular, our recent PPI activity with patient advisory group showed that people with dementia and their caregivers can actually provide useful and very important information in research with regards to wearable technology [55].

Therefore, following similar PPI approaches and our recent published study, involving PwD, HCPs, and caregivers [55] in order to explore the benefits and concerns of the technology adoption, we constructed a questionnaire, containing both open-ended and closed-ended questions as well as multiple choice and Likert scale. These aim to prioritize the participants’ views [56] and investigate the specific requirements and preferences for designing an application that accommodates the needs of people with cognitive impairment, the caregivers, and HCPs. In particular, the HFIAQ has been constructed following the user-centered design (UCD) of IT [37, 39]. The pool or set of questions, referred to as a questionnaire in short, does not constitute a validated neuropsychiatric validation questionnaire, but rather one for eliciting requirements for interactive mHealth apps. The HFIAQ includes three dimensions and was used in this study to explore the beneficiaries’ requirements and feature preferences for using both interactive and passive mHealth apps (Fig. 1).

Fig. 1

The three dimensions of HFIAQ as examined in the context of questions for the HCPs, caregivers, and patients.

In detail, the questionnaire contains 5 open-ended questions (e.g., What kind of features would you prefer to be included in a brain game?), 4 multiple choice questions, and 13 close-ended questions (e.g., Would you consider using a medication reminder application? YES/NO) and 40 Likert scale questions (range 1–5). For all Likert scale questions, a higher score indicates a higher intention, satisfaction, agreement, or willingness in response to the question. As such, the scores from 1 to 5 correspond to “Strongly Disagree”, “Disagree”, “Neither Agree nor Disagree”, “Agree”, and “Strongly Agree”. The HFIAQ is further segmented into categories of questions with respect to domains of interest to evaluate, such as “Cognitive Enhancement and Brain Games”, “Medication Reminders”, “GPS Tracking and Restrictions”, to facilitate filling the questionnaire and obtain deeper insights. Table 2 briefly presents the categories and the respective number and types of questions for caregivers, HCP and participants with cognitive impairment.

Table 2

Question Categories and Numbers in HFIAQ

Category	Questions
Cognitive Enhancement and Brain Games	5 close-ended
	2 multiple-choice
	5 open-ended
	20 Likert scale
Medication Reminders	4 close-ended
	1 multiple-choice
	10 Likert scale
GPS Tracking and Restrictions	4 close-ended
	1 multiple-choice
	10 Likert scale

Statistical analysis

SPSS 25.0 was used to conduct all statistical analysis. We performed descriptive analysis for the demographic data and the results of the questionnaire (close-ended, open-ended, multiple choice, and Likert scale questions). In the open-ended questions, we have created clusters of the common answers of the participants. In addition to the statistical analysis, for all the Likert scale questions, principal component analysis (PCA) was conducted, in order to evaluate the prototypes of the answers and investigate how the different types of questions are grouped together using the open source development environment, RStudio. Appropriate packages were incorporated in order to calculate the PCA results and to visualize the Likert scale answers [57, 58].

RESULTS

Cognitive enhancement and brain games

The results demonstrate that UCD plays a pivotal role in shaping ICT research together with the elderly with cognitive impairment related to AD, the caregivers, and the HCPs, by providing insights about user preferences, wishes, real needs in mHealth. Concerning the answers given with regards to the types of brain games, most HCPs (88,2%) and patients (93,8%) and all caregivers (100%) reported “Memory” as a favorable type of brain game (Fig. 2), while “Orientation” games were non-negligible by the caregivers (80%).

Fig. 2

Types of games preference for each end-user category.

More specifically, the caregivers considered having “Enjoyable time” while playing brain games and “Improvement of Cognitive functions” such as memory capacity, response, and attention skills among the most useful areas that brain games would benefit the patients. In turn, the HCPs considered that bridging the gap of “Communication and improving social interaction” with other family members or other seniors were of utmost importance for the patients. Moreover, they considered enhancing the “Feeling of importance” and “Enhance Creativity” and contributing to the “Adoption of new strategy or planning methods”, while ensuring “Enjoyable time” to the patients, as the most important domains that the brain games would contribute to (Fig. 3).

Fig. 3

Brain games usefulness for each end-user category.

Additionally, in a respective Likert scale with questions regarding the features of the brain games, all items were selected as particularly important by all the three groups. In particular, the HCPs and caregivers selected “Easy-to-Use” as the most important trait of the mHealth apps. More specifically, patients reported that “Doctor/Relative inform” regarding their performance, lacing questions with “Big-Letters”, “Showing Errors”, and “Performance Metrics” and the capability to “Adapt to educational level” for each user were of utmost importance for them. Also, HCPs, stated that the “Increasing level of difficulty” and the “Inclusion of Different Questions” for every task holds great importance (Fig. 4). As a result, both HCPs and patients considered receiving information about patient performance as a central part of the brain game.

Fig. 4

Brain games feature score for each end-user category.

Finally, in a multiple-choice question (“In your opinion, what is the appropriate duration for the use of a brain game?”), 53%of the caregivers and 41%of the HCPs reported that they would suggest that patients play brain games for a duration of 3 to 6 months, while 56%of the patients and 41%of HCPs would be willing to use them for 1 year (Fig. 5). Moreover, HCPs and patients considered that brain games would be beneficial for the patients for long-term implementation.

Fig. 5

Percentage regarding the frequency of playing brain games for each end-user category.

In general, HCPs, caregivers, and patients considered brain games as an important IT non-pharmaceutical intervention for the patients. More specifically, the efficient implementation of brain games in daily life of the patients would improve patients’ memory, orientation, response time, and logic skills, while at the same time increase the feeling of importance, creative thinking and will offer enjoyable time to patients to challenge their selves mentally. Furthermore, among the most important features, the three groups selected that presentation of metrics and errors would be particularly important for the patients when receiving feedback, while sending a report or informing the caregivers or HCPs about patient’s progress should be included as well. Finally, most of the participants of the three groups highlighted that more than 3–6 months up to a year was mandatory for the patients to interact with the brain games in order to be able to witness any improvement in their cognitive functions.

Open-ended questions

The answers provided by the participants to the open questions are presented in Table 3. From the open-ended question, “Give reasons why you would be interested in using brain games”, it seems that all participants’ interest in using brain games relied on providing them with daily practice and challenge, pleasure, while contributing in cognitive, functional, and behavioral improvement (e.g., memory, attention), preserving and maintaining cognitive functions and enhancing social interaction of the patient. Moreover, for the HCPs, the brain games would benefit their work since they would provide them a progress report with objective information of the patient’s memory status from which they would be able, in turn, to propose timely and adaptive interventions. Similarly, the patients in the open-ended question related to the area that the brain games would contribute to, highlighted among others that the brain games would enhance their creativity, while offering companionship and cognitive improvement. Additionally, in the respective question, the caregivers supported that incorporating enjoyable activities into patient’s daily routine would help them avoid boredom while maintaining daily functionality and cognitive functions. At the same time, the patient would receive daily feedback about their status through self-assessment, which would enhance patient engagement and user-centered approach.

Table 3

Open-ended questions about the brain

1. Give reasons why you would be interested in using brain games

•Daily practice and improvement of patient’s QoL

•Offer pleasure, strengthening of cognitive functions, improve communication

•Entertainment, memory enhancement

•Forestall further cognitive deterioration

•Improve particular areas of cognition that are affected

•Improvement of mood and behavior

•Maintain daily functionality and autonomy

•Improve attention

•Practices the memory, logic and observation

•Increase awareness and personal skills

•Have fun

•Self - assessment

•Better memory performance

•Preserve memory capabilities

•Gain better knowledge of patient’s health status

2. Describe a game which would be of high interest for you

•Present pictures and then ask patients particular questions

•Personalized design based on the patient’s capabilities

•Spot differences between two similar images

•Increasing level of difficulty

•Play sounds to give feedback to the patient

•Mark patient’s progress every week

•Focus on memory and entertainment

•Easy-to-use

•Include a lot of colors and nice graphics to attract the interest of the patient

•Adapted based on the cognitive level of the user

•Present scores and to keep up with the scores on a continuous basis so that clinicians can have a monthly report about patient’s progress

•To include an artistic part in order to take advantage of the creative mood or to highlight / evaluate its errors

•Adapted to the educational level

•Interactive

•Reinforcement to increase the desire to achieve the ultimate goal of the game

•Easy connectivity to games (without many login procedures, operation and no internet access needed)

•Friendly visualization for people with dementia and the elderly

•Setting goals and incorporate questions related to memory, attention and observation

•An activity that is especially beloved is the orientation to time and events that have happened the same day in the past. A game that could somehow integrate something like this and differentiate daily would be interesting.

•Play sounds, include colorful images, capital letters

•Interesting exercises

•Short and understandable questions

•Be creative and easy to use as well as interesting for the patient

•Related to the patient’s daily life, e.g., house operation and management of personal issues

•To monitor patient’s cognitive status

•Have anagram questions, word completions, videos, pictures crossword questions

•Pleasant and creative and promote complex thinking

3. Brain Games Features that you would like to be included

•Completion of sentences, Puzzle and assembling pieces, puzzles with pictures, find similarities

•Give rewards in case of success

•Have the ability to use them without supervision by the caregiver

•Logic game

•Many pictures and different shapes

•Find the differences games

•Memory exercises between pictures, sudoku, exercises with reflections

•Large screen, quite simple and enjoyable game environment

•Levels of increasing difficulty and place scores for each game

•Adapted based on the educational and cognitive level

•Team games of people with a relatively similar level of cognitive impairment

•Big letters and available on touch screen

•Available for touch screens including the amplifying sounds

•Opposites and synonyms as well as name animal or thing

•Simple questions and answers that strengthen the patient’s memory

4. In what area do you think these games would help you/the patient?

•Patient engagement

•Increase creativity, mood, attention, memory and problem solving

•Cognitive Enhancement

•Improve attention, social skills, communication and orientation

•Empowering the patient at all levels of his life

•Assessment / timely intervention, exchange of experiences / good / bad practices

•Incorporating enjoyable activities into patient’s daily routine and avoiding boredom

•Maintaining daily functionality and cognition

•Improve cognitive functions and thinking using logical combinations

•Companionship and improvement of the patient’s daily life

•Self-assessment

•Integration of what patients learn in their daily life

•Spending time creatively and enjoyable

5. What is that characteristic that would make you play a game every day? / suggest a game to the patient?

•Interesting, helpful, pleasant and useful to the patient

•Not having complicated questions and confuse the patient (log in etc.)

•Not being difficult and stressing out the patient

•Combination of entertainment and cognitive empowerment

•Easy to use from a technological point of view

•Long-term improvement of patient’s cognitive functions

•Setting goals and gives reward (score or some gift)

•Include interaction with other members of the same state

•Different levels

•Try to level up and score better

•Not boring not to have the same things all the time

•Have nice features, different things and meet personalized needs

Finally, the HCPs, the caregivers and the patients highlighted the features: “increasing level of difficulty”, “team games with seniors who share similar cognitive problems”, “inclusion of memory exercises”, “offering rewards after a successful completion”, and “impressive graphics with a simplistic and not confusing interface” as important. Additionally, in the open-ended related question (“What is that trait that would make you play a game every day? / suggest the game to the patient?”), the participants underlined that if the game was pleasant, helpful, allowed them to set goals and receive rewards and to track their performance longitudinally (to better describe the daily problems to their doctor), it would make them play it daily or at least suggest it to another patient.

Medication reminders

Regarding medication reminders (the answers are presented in Fig. 6), the majority of the patients (69%) and caregivers (53%) indicated, in the multiple choice question “How frequent would you like to receive reminders from the medical treatment app?”, that they were willing to receive reminders from the medication treatment app for every medication, while 41%of the HCPs chose the “weekly reminder” and “for every medication” as an important feature of the app for monitoring patients’ treatment. The findings suggest that both caregivers and patients are willing to use a medication reminder app frequently.

Fig. 6

Frequency of receiving reminders for the medication by the end-users category.

Additionally, in the multiple-choice question concerning the features that the medication reminder app could include, all HCP (100%) indicated “Reminders”, “History”, and “Sound” as basic domains, while the 80%of the caregivers selected “Display of medication with image” and 100%“Reminders” and “History” (Fig. 7) as a primary function. On the other hand, several patients considered the image display as a not favorable feature, while they were in favor of medication reminders and history (100%). All participants agreed that medication reminders would be the most important contribution of a tailored app, followed by medication history and sound playing.

Fig. 7

Percentage of preferences for the medication reminder features by the end-users category.

Finally, in the respective 8 Likert scale questions about the potential parameters that the HCPs, caregivers, and patients would find the medication reminder app useful (e.g., “I would like the medication reminder to keep history of my medication”), all caregivers selected the number 5 (the median value for all 8 Likert scale questions), indicating all domains as very important and useful for the proposed solution (Fig. 8). Both HCPs and patients chose the “easy-to-use”, “compliance to medication”, and “reminder of the medication” as a major feature for using the app for monitoring patients’ medical treatment. Additionally, HCPs stated that sending reminders to relatives would be also a useful trait, whereas connection to other devices seemed not so important for patients.

Fig. 8

Usefulness regarding medication reminders by the end-users category.

As a result, HCPs, caregivers, and patients considered medication reminders as an important interactive app feature, particularly by ensuring medication compliance and keeping a history of medications, while at the same time it would increase the feeling of autonomy by also providing information about medical history to HCPs and caregivers. Additionally, regarding the frequency that the end-users would like the patients to receive medication reminders, most of the participants chose “reminder for every medication” during the day. Moreover, among the most important features, the end-users agreed that presentation of history and playing sound for reminding the medication while at the same time showing a banner with the name of the medication would be of outmost importance.

GPS tracking and restrictions

Concerning the answers given about the geographical location and GPS tracking, all HCPs, caregivers, and patients (100%) agreed that calling relatives in case of emergency was an especially important feature. In particular, the caregivers considered all features particularly important with regards to the GPS tracking of the patient, while the patients found less important the implementation of geographical limits and restrictions (43.8%) or the alarm ringing if they move away of the resident area (25%). Both HCPs and caregivers considered real-time GPS tracking of the patient and setting limits would be immensely helpful and important for the app and should not be negligible (Fig. 9). The findings suggest that the majority of the participants were willing to use a GPS tracking app with the most critical feature being calling relatives in case of emergency, while features such as “alarm if the patient goes too far” were not so important.

Fig. 9

Geographical location features by the end-users category.

Additionally, in the respective 8 Likert scale questions about the potential parameters that the HCPs and caregivers would like to be included while using GPS tracking app (e.g., “By using the geographical location app I would like to. . . ”), all HCPs and caregivers selected the number 5 (the median value for all 8 Likert scale questions), indicating total agreement with respect to “Missing”, “Call someone in case of emergency”, “Call someone if the person goes too far”, “Connect to other devices”. In general, all of them underlined and admitted that the “Easy-to-use” feature of the GPS tracking application was extremely important, while most of them would find it useful in cases of “Emergency” (Fig. 10). Finally, keeping “History” of the patient’s route was considered among the least important aspects.

Fig. 10

Geographical location usefulness by the end-users category.

Furthermore, HCPs, caregivers, and patients considered GPS tracking and restriction as an important intervention for the last ones, given that it will contribute particularly in cases of emergency and disorientation. Nevertheless, the patients were not in favor of constant monitoring of their location or setting boundaries and restrictions or alarms in cases where they want to distance from their home. On the other hand, both HCPs and caregivers underlined the importance of the abovementioned features. Finally, all end-users agreed upon calling a relative in case of emergency.

Principal component analysis for Likert scale questions

To evaluate the prototypes of the answers and how the different types of questions are grouped together, PCA was conducted on all the Likert type questions for the different categories. In Fig. 11, the corresponding categories of questions are shown, “Cognitive” - Cognitive Enhancement and Brain Games, “Medication” - Medication Reminders, and “GPS” - GPS Tracking and Restrictions. Those are accompanied by the percentages of participant answers for each Likert score respectively, where “3” stands for “Neither Agree nor Disagree” and is considered a neutral ranking for the PCA analysis. Two principal components were finally used, “Component 1” (PC1) and “Component 2” (PC2), and the PCA parameters were calculated and are shown in Table 4. For this process, regarding the number of principal components selected, it is ensured that Eigenvalues are larger than 1 (8.078, 6.525) and the Variance Explained (Cumulative) adequately represents the dataset (0.406). A categorization emerged for PC1 and PC2 that groups a majority of 13/20 “Cognitive” and 6/8 “Medication” questions in PC1 category and 6/8 “GPS” questions in PC2 category. This signifies a differentiation in regards to patient answers, resulting in a grouping of the answers in two principal components, which seemingly reflects the questions’ initial categorization.

Fig. 11

Principal component analysis for Likert scale questions.

Table 4

Principal Component Analysis Parameters Summary

Parameter	PC1	PC2
Eigenvalues	8.078	6.525
Variance Explained	0.224	0.181
Variance Explained (Cumulative)	0.224	0.406
Variance Explained (Proportion)	0.224	0.181

DISCUSSION

From the results of the present PPI survey, all participants were likely to use mHealth apps, with the primary desired features being the improvement of memory and cognitive functions, assistance on medication treatment and perceived ease to use. In general, caregivers are seriously worried about patient’s wanderings, therefore geographical location and setting restrictions would be of utmost importance to them. On the other hand, patients have a different perspective on setting boundaries, but they are willing to use a geographical location app in case of emergency. Moreover, all three groups agree that mHealth apps could help the patients to ameliorate behavioral and cognitive disturbances. To the best of our knowledge, this is the first PPI survey that explored the preferences, views, and concerns, regarding mHealth apps, of people with cognitive impairment, HCPs, and caregivers. The present PPI survey highlights that people with cognitive impairment as well as HCPs and caregivers can provide valuable and insightful suggestions that can promote the design, quality, and the outcome of a research.

In particular, there are several approaches which have gained interesting insights regarding technology acceptance from PPI activities [20 , 37]. Broadly, their findings are consistent with previous research which has reported that older adults do perceive benefits to technology [59] but raise some concerns that older adults are unable, unwilling, or afraid to use technology [60].

Recent evidence suggests that several factors may influence acceptance of technology including characteristics of the technology together with usage characteristics, perceived complexity, and level of innovation or characteristics of the user such as perceived ease of use and compatibility, experience, personal traits, and fun [21 , 61–64]. Similarly, the results of the present PPI survey provide convergent evidence that ease of use, fun and enjoyment are significant variables in adopting mHealth apps from the end-users’ perspective. Additionally, training and education before the mHealth app installation seems to be necessary to make the end-users familiar in using technology to avoid any potential anxiety of using the suggested mHealth apps. Moreover, contrary to the previous studies, which explored general technology acceptance, the participants of the present PPI survey, were presented with a wide variety of features and questions regarding several mHealth apps, including brain games, medication reminders and GPS tracking systems. Participants reported using the suggested mHealth apps on a daily basis long-term, a finding perhaps reflecting that our participants sample is in need of useful and efficient solutions for the patients.

Moreover, caregivers, playing pivotal role in patients’ care, found the suggested mHealth apps extremely useful especially in cases of emergency. In this common vein, recent findings from similar approaches indicate that mHealth apps can also assist caregivers in releasing their mental and economic burden while at the same time providing useful information about their patients [47, 65]. A recent study by Thorpe et al. explored the adoption of smartphone and smartwatch with six mobile apps (navigation, scheduling, orientation, communication, emergency help, and monitoring) [47]. In their study the most appreciated function by patients was scheduling, which reminded or notified them about the activities they should perform, while the navigation or emergency support functions were not considered that much useful. Finally, the most important feature for the patients was found to be the personalization and adaptation of the apps based on individual preferences and needs. This was in fact confirmed in other research studies as well [13 , 66–68]. Like our PPI survey, the patients as well as the HCPs and the caregivers highlighted that reminders of medication and emergency was of utmost importance.

Other surveys have emphasized that the touch accuracy for these people might cause a difficulty and therefore technical aspects of the devices should be considered, e.g., weight of device, screen size, or the layout of buttons and taskbar [68 –70]. Therefore, important features include content quality, usability, need to match the app to patients’ educational and health background, device connectivity standards, security, and user privacy. Furthermore, the findings of the research studies [69 , 71–73] show evidence that mHealth apps are effective in cognitive screening and the daily monitoring and assessment of dementia, given that these apps are more accurate than the traditional assessment methods and they are easily administered and understood by the end-users. At the same time they can minimize the examiner’s biases, enable patients to stay independent on their tasks of daily living; forestall hospitalization and improve the overall quality of life of people with cognitive impairment [71 , 75]. Our results are also in compliance with the abovementioned approaches who maintain that mHealth apps are more sensitive to detect decline in cognitive functions due to natural variation at the time of testing, given that they provide repeated and reliable assessments of cognitive functions.

In general, according to our study’s findings, the patients are willing to use mHealth apps for enabling homecare and cognitive training instead of visiting a doctor frequently for assessment or reporting their daily problems (e.g., medication) to the clinicians. In turn, they consider mHealth apps as tools for improving their cognition, a finding which is aligned with a recent review assessing potential factors affecting adoption of technology, highlighting that the use of technology also depends on people’s perceived personal need for technology [4 , 76–78].

Since memory and activities of daily living are widely affected within the AD spectrum, the patients outlined that they are willing to adopt brain games to enhance their memory while medical reminders would be of particular interest to them given that they frequently forget to take their prescribed medication. Therefore, mHealth apps should incorporate functions related to medication reminders, geolocation in case of emergency and brain games for cognitive improvement. One of the main potential barriers in adopting mHealth apps that HCP, caregivers, and the patients underlined is that perceived ease of use and reliability of the mHealth apps have significant influence on the intention to use interactive applications technologies. Thus, the fact that technology creates such concerns indicates that designers and ICT researchers play a pivotal role in addressing such worries, by minimizing for instance erroneous messages or sounds as much as possible. In particular, concerning the brain games, HCPs, caregivers, and patients highlighted among others that brain games would constitute as brain workout for the patients, significantly contributing to their memory and organizational skills.

All the three groups underlined that brain games would offer enjoyable time and fun to the patients, strengthen their cognitive and behavioral functions, improve communication, forestall further cognitive deterioration, while at the same time maintain daily functionality, attention, awareness, and autonomy and gain better knowledge of patient’s health status [47]. Additionally, regarding more appealing brain game features, the patients selected, among others, the adapted and personalized design with nice graphics, inclusion of suitable questions based on the patient’s cognitive status and educational background, increasing level of difficulty, tracking patient’s progress by presenting scores every week and playing sounds to give feedback to the patient.

Moreover, the inclusion of a brain game, which would ask the patient orientation to time and events questions, would be interesting as mentioned by the caregivers and HCPs. In conclusion, the findings suggest that the eHealth systems through interactive mHealth apps can assist HCPs, caregivers, and patients in care assessment and decision-making. Therefore, it is important especially for PwD living on their own to have an enjoyable time while playing brain games, which also provides them with tools to enhance interaction in an intuitive and transparent way.

Moreover, this PPI study highlights the importance of a UCD and patient-centered approach in shaping research with mHealth apps. Because of the questionnaire administered to the HCPs, caregivers, and the patients, brain games, medical treatment reminders, and geographical location found to have great potential but can be rendered unacceptable or unsuitable if they are not easy to use both by patients as well as by HCPs and caregivers. However, this study found that both HCPs and caregivers are willing to incorporate mHealth apps in their daily activities in order to have a holistic and objective overview of the patients’ health status while at the same time providing support for their cognitive improvement.

On the other hand, patients are strongly in favor of using interactive and passive mHealth apps in order to track their medical history and play brain games in order to have an enjoyable time while at the same time improving their cognitive performance and feel safer to be located in case of emergency. The present PPI results indicate that mHealth apps can provide support for patients with cognitive impairment in their activities of daily life (e.g., medication management), and especially, in the intervention and the management of cognitive functions through the usage of brain games. Also, interactive technologies can reduce both mental and economic burden of their caregivers, providing a feeling of assistance in case of emergency and offering full access and objective information to HCPs that drive the implementation of personalized and tailored interventions.

CONCLUSIONS AND FUTURE WORK

This PPI research analyzed the main factors influencing individual mHealth apps adoption from a quantitative perspective and put forward a comprehensive user adoption framework. Also, the study analyzed the HCPs as well as caregivers’ perspectives with regards to the mHealth apps tailored to PwD. This research contributes to the mHealth apps services adoption. Future work is to implement the games and apps in the framework of support2Live (https://www.ypostirizo-project.gr/) and evaluate their acceptance and clinical value. Future research should consider the abovementioned concerns and requirements to build such mHealth apps by extending previous work while the findings from this paper could be a useful addition in remote clinical trials and technological solutions to tackle cognitive related issues for PwD or elders in general. Thus, a thorough understanding of older adults’ usage and perception of technology is essential for maximizing the potential that technology has to offer for facilitating independence in everyday life of elder users.

Footnotes

ACKNOWLEDGMENTS

This research has received funding by the Greek national funds through the Operational Program Competitiveness, Entrepreneurship, and Innovation, under the call RESEARCH-CREATE- INNOVATE (project code: T1EDK-02668).

CONFLICT OF INTEREST

The authors have no conflict of interest to report.

References

Prince

, Wimo

, Guerchet

, Gemma-Claire

, Wu

Y-T

, Prina

(2015) World Alzheimer Report 2015. The Global Impact of Dementia - An analysis of prevalence, incidence, cost and trends. Alzheimer’s Disease International, London.

Nichols

, Szoeke

CEI

, Vollset

, Abbasi

, Abd-Allah

, Abdela

, Aichour

MTE

, Akinyemi

, Alahdab

, Asgedom

, Awasthi

, Barker-Collo

, Baune

, Béjot

, Belachew

, Bennett

, Biadgo

, Bijani

, Bin Sayeed

, Brayne

, Carpenter

, Carvalho

, Catalá-López

, Cerin

, Choi

JYJ

, Dang

, Degefa

, Djalalinia

, Dubey

, Duken

, Edvardsson

, Endres

, Eskandarieh

, Faro

, Farzadfar

, Fereshtehnejad

, Fernandes

, Filip

, Fischer

, Gebre

, Geremew

, Ghasemi-Kasman

, Gnedovskaya

E V.

, Gupta

, Hachinski

, Hagos

, Hamidi

, Hankey

, Haro

, Hay

, Irvani

SSN

, Jha

, Jonas

, Kalani

, Karch

, Kasaeian

, Khader

, Khalil

, Khan

, Khanna

, Khoja

TAM

, Khubchandani

, Kisa

, Kissimova-Skarbek

, Kivimäki

, Koyanagi

, Krohn

, Logroscino

, Lorkowski

, Majdan

, Malekzadeh

, März

, Massano

, Mengistu

, Meretoja

, Mohammadi

, Mohammadi-Khanaposhtani

, Mokdad

, Mondello

, Moradi

, Nagel

, Naghavi

, Naik

, Nguyen

, Nirayo

, Nixon

, Ofori-Asenso

, Ogbo

, Olagunju

, Owolabi

, Panda-Jonas

, Passos

VM d. A

, Pereira

, Pinilla-Monsalve

, Piradov

, Pond

, Poustchi

, Qorbani

, Radfar

, Reiner

, Robinson

, Roshandel

, Rostami

, Russ

, Sachdev

, Safari

, Safiri

, Sahathevan

, Salimi

, Satpathy

, Sawhney

, Saylan

, Sepanlou

, Shafieesabet

, Shaikh

, Sahraian

, Shigematsu

, Shiri

, Shiue

, Silva

, Smith

, Sobhani

, Stein

, Tabarés-Seisdedos

, Tovani-Palone

, Tran

, Tsegay

, Ullah

, Venketasubramanian

, Vlassov

, Wang

, Weiss

, Westerman

, Wijeratne

, Wyper

GMA

, Yano

, Yimer

, Yonemoto

, Yousefifard

, Zaidi

, Zare

, Vos

, Feigin

, Murray

CJL

(2019) Global, regional, and national burden of Alzheimer’s disease and other dementias, 1990–2016: A systematic analysis for the Global Burden of Disease Study 2016. Lancet Neurol 18, 88–106.

Lazarou

, Karakostas

, Stavropoulos

, Tsompanidis

, Meditskos

, Kompatsiaris

, Tsolaki

(2016) A novel and intelligent home monitoring system for care support of elders with cognitive impairment. J Alzheimers Dis 54, 1561–1591.

Khosravi

, Ghapanchi Hossein

(2016) Investigating the effectiveness of technologies applied to assist seniors: A systematic literature review. Int J Med Inform 85, 17–26.

Frantzidis

, Ladas

A-KI

, Vivas

, Tsolaki

, Bamidis

(2014) Cognitive and physical training for the elderly: Evaluating outcome efficacy by means of neurophysiological synchronization. Int J Psychophysiol 93, 1–11.

Lazarou

, Stavropoulos

, Meditskos

, Andreadis

, Kompatsiaris (Yiannis)

, Tsolaki

(2019) Long-term impact of intelligent monitoring technology on people with cognitive impairment: An observational study. J Alzheimers Dis 70, 757–792.

Sponselee

(2013) Acceptance and effectiveness of telecare services from the end-user perspective. In Technische Universiteit Eindhoven, Eindhoven, pp. 107-136.

Stern

(2012) Cognitive reserve in ageing and Alzheimer’s disease. Lancet Neurol 11, 1006–1012.

Stucki

, Urwyler

, Rampa

, Müri

, Mosimann

, Nef

, Muri

, Mosimann

, Nef

(2014) A web-based non-intrusive ambient system to measure and classify activities of daily living. J Med Internet Res 16, 1–12.

10.

Karakostas

, König

, Crispim-Junior

, Bremond

, Derreumaux

, Lazarou

, Kompatsiaris

, Tsolaki

, Robert

(2020) A French-Greek cross-site comparison study of the use of automatic video analyses for the assessment of autonomy in dementia patients. Biosensors 10, 103.

11.

Zygouris

, Tsolaki

(2015) Computerized cognitive testing for older adults: A review. Am J Alzheimers Dis Other Demen 30, 13–28.

12.

Zygouris

, Giakoumis

, Votis

, Doumpoulakis

, Ntovas

, Segkouli

, Karagiannidis

, Tzovaras

, Tsolaki

(2015) Can a virtual reality cognitive training application fulfill a dual role? Using the virtual supermarket cognitive training application as a screening tool for mild cognitive impairment. J Alzheimers Dis 44, 1333–1347.

13.

Faucounau

, Riguet

, Orvoen

, Lacombe

, Rialle

, Extra

, Rigaud

A-S

(2009) Electronic tracking system and wandering in Alzheimer’s disease: A case study. Ann Phys Rehabil Med 52, 579–587.

14.

Wong

AMK

, Chang

, Ke

, Huang

, Tsai

, Chang

, Shieh

, Chan

, Chen

, Pei

(2012) Technology acceptance for an intelligent comprehensive interactive care (ICIC) system for care of the elderly: A survey-questionnaire study. PLoS One 7, e40591.

15.

Docking

, Lane

, Schofield

(2018) Usability testing of the iPhone app to improve pain assessment for older adults with cognitive impairment (prehospital setting): A qualitative study. Pain Med 19, 1121–1131.

16.

Bayen

, Jacquemot

, Netscher

, Agrawal

, Noyce

, Bayen

(2017) Reduction in fall rate in dementia managed care through video incident review: Pilot study. J Med Internet Res 19, 339–345.

17.

Atee

, Hoti

, Hughes

(2018) A technical note on the PainChek^TM system: A web portal and mobile medical device for assessing pain in people with dementia. Front Aging Neurosci 10, 117–125.

18.

Alaa

, Zaidan

, Talal

, Kiah

MLM

(2017) A review of smart home applications based on Internet of Things. J Netw Comput Appl 97, 48–65.

19.

Lancioni

, Singh

, O’Reilly

, Sigafoos

, Pangrazio

, Megna

, Zonno

, La Martire

, Pinto

, Minervini

(2009) Persons with moderate Alzheimer’s disease improve activities and mood via instruction technology. Am J Alzheimers Dis Other Demen 24, 246–257.

20.

Zhao

, Ni

, Zhou

(2018) What factors influence the mobile health service adoption? A meta-analysis and the moderating role of age. Int J Inf Manage 43, 342–350.

21.

Alshahrani

, Stewart

, MacLure

(2019) A systematic review of the adoption and acceptance of eHealth in Saudi Arabia: Views of multiple stakeholders. Int J Med Inform 128, 7–17.

22.

Choi

, Ku

, You

, Jo

, Kwon

, Choi

, Jung

, Ryu

, Park

, Go

, Kim

, Cha

, Kim

(2019) Resting-state prefrontal EEG biomarkers in correlation with MMSE scores in elderly individuals. Sci Rep 9, 10468.

23.

Tarnanas

, Tsolakis

, Tsolaki

(2018) Cognitive exercising for patients with MCI using serious games: Design of a pilot study. In Virtual and Augmented Reality: Concepts, Methodologies, Tools, and Applications. IGI Global, pp. 1313–1342.

24.

Billis

, Konstantinidis

, Mouzakidis

, Tsolaki

, Pappas

, Bamidis

(2010) A game-like interface for training seniors’ dynamic balance and coordination. IFMBE Proceedings, pp. 691–694.

25.

Imbeault

, Bouchard

, Bouzouane

(2011) Serious games in cognitive training for Alzheimer’s patients. In 2011 IEEE 1st International Conference on Serious Games and Applications for Health (SeGAH).

26.

Tachakra

, Wang

, Istepanian

RSH

, Song

(2003) Mobile e-Health: The unwired evolution of telemedicine. Telemed J E Health 9, 247–258.

27.

Bettiga

, Lamberti

, Lettieri

(2020) Individuals’ adoption of smart technologies for preventive health care: A structural equation modeling approach. Health Care Manag Sci 23, 203–214.

28.

Mehrabian

, Extra

, Wu

Y-H

, Pino

, Traykov

, Rigaud

A-S

(2015) The perceptions of cognitively impaired patients and their caregivers of a home telecare system. Med Devices (Auckl) 8, 21–29.

29.

Fischer

, David

, Crotty

, Dierks

, Safran

(2014) Acceptance and use of health information technology by community-dwelling elders. Int J Med Inform 83, 624–635.

30.

Villalba-Mora

, Casas

, Lupiañez-Villanueva

, Maghiros

(2015) Adoption of health information technologies by physicians for clinical practice: The Andalusian case. Int J Med Inform 84, 477–485.

31.

Stavropoulos

, Papastergiou

, Mpaltadoros

, Nikolopoulos

, Kompatsiaris

(2020) IoT wearable sensors and devices in elderly care: A literature review. Sensors (Basel) 20, 1–22.

32.

Xue

, Chiuan

, Chang

, Chuan

, Choo

, Tan

, Been

, Duh

, Choolani

, Yen

, Chang

, Chan

, Tai

, Tan

, Duh

HBL

, Choolani

(2012) An exploratory study of ageing women’s perception on access to health informatics via a mobile phone-based intervention. Int J Med Inform 81, 637–648.

33.

Hawley-Hague

, Boulton

, Hall

, Pfeiffer

, Todd

(2014) Older adults’ perceptions of technologies aimed at falls prevention, detection or monitoring: A systematic review. Int J Med Inform 83, 416–426.

34.

Stavropoulos

, Lazarou

, Strantsalis

, Nikolopoulos

, Kompatsiaris

, Koumanakos

, Frouda

, Tsolaki

(2020) Human factors and requirements of people with mild cognitive impairment, their caregivers and healthcare professionals for eHealth systems with wearable trackers. In Proceedings of the 2020 IEEE International Conference on Human-Machine Systems, ICHMS 2020.

35.

Giger

, Pope

, Vogt

, Gutierrez

, Newland

, Lemke

, Lawler

(2015) Remote patient monitoring acceptance trends among older adults residing in a frontier state. Comput Human Behav 44, 174–182.

36.

Liu

C-J

, Yang

(2014) Using the technology acceptance model to examine seniors’ attitudes toward Facebook. World Acad Sci Eng Technol Int J Educ Pedagogic Sci 8, 969–974.

37.

Miah

, Dawes

, Edwards

, Leroi

, Starling

, Parsons

(2019) Patient and public involvement in dementia research in the European Union: A scoping review. BMC Geriatr 19, 1–20.

38.

Roberts

, Rochford-Brennan

, Goodrick

, Gove

, Diaz-Ponce

, Georges

(2020) Our reflections of patient and public involvement in research as members of the European Working Group of People with Dementia. Dementia 19, 10–17.

39.

Hassan

, Swarbrick

, Sanders

, Parker

, Machin

, Tully

, Ainsworth

(2017) Tea, talk and technology: Patient and public involvement to improve connected health ‘wearables’ research in dementia. Res Involv Engagem 3, 12.

40.

Slade

, Bird

, Chandler

, Fox

, Larsen

, Tew

, Leamy

(2010) The contribution of advisory committees and public involvement to large studies: Case study. BMC Health Serv Res 10, 323.

41.

Padilla-Góngora

, López-Liria

, Díaz-López del

M P

, Aguilar-Parra

, Vargas-Muñoz

, Rocamora-Pérez

(2017) Habits of the elderly regarding access to the new information and communication technologies. Procedia Soc Behav Sci 237, 1412–1417.

42.

Giebel

, Roe

, Hodgson

, Britt

, Clarkson

(2019) Effective public involvement in the HoST-D Programme for dementia home care support: From proposal and design to methods of data collection (innovative practice). Dementia 18, 3173–3186.

43.

Hannigan

(2018) Public and patient involvement in quantitative health research: A statistical perspective. Health Expect 21, 939–943.

44.

Gove

, Diaz-Ponce

, Georges

, Moniz-Cook

, Mountain

, Chattat

, Øksnebjerg

(2018) Alzheimer Europe’s position on involving people with dementia in research through PPI (patient and public involvement). Aging Ment Health 22, 723–729.

45.

Stevenson

, Taylor

(2019) Involving individuals with dementia as co-researchers in analysis of findings from a qualitative study. Dementia 18, 701–712.

46.

Littlechild

, Tanner

, Hall

(2014) Co-research with older people: Perspectives on impact. Qual Soc Work 14, 18–35.

47.

Thorpe

, Rønn-Andersen

KVH

, Bień

, Özkil

, Forchhammer

, Maier

(2016) Pervasive assistive technology for people with dementia: A UCD case. Healthc Technol Lett 3, 297–302.

48.

Stavropoulos

, Lazarou

, Strantsalis

, Koumanakos

, Frouda

, Tsolaki

49.

Schipper

, Dauwerse

, Hendrikx

, Leedekerken

, Abma

(2014) Living with Parkinson’s disease: Priorities for research suggested by patients. Parkinsonism Relat Disord 20, 862–866.

50.

Claes

, Devriendt

, Tournoy

, Milisen

(2015) Attitudes and perceptions of adults of 60 years and older towards in-home monitoring of the activities of daily living with contactless sensors: An explorative study. Int J Nurs Stud 52, 134–148.

51.

Shippee

, Domecq Garces

, Prutsky Lopez

, Wang

, Elraiyah

, Nabhan

, Brito

, Boehmer

, Hasan

, Firwana

, Erwin

, Montori

, Murad

(2015) Patient and service user engagement in research: A systematic review and synthesized framework. Health Expect 18, 1151–1166.

52.

Newton

, Dickinson

, Gibson

, Brittain

, Robinson

(2016) Exploring the views of GPs, people with dementia and their carers on assistive technology: A qualitative study. BMJ Open 6, e011132.

53.

Kuper

, Reeves

, Levinson

(2008) Qualitative research: An introduction to reading and appraising qualitative research. BMJ 337, 404–407.

54.

Mertens

(2010) Divergence and mixed methods. J Mix Methods Res 4, 3–5.

55.

Stavropoulos

, Lazarou

, Diaz

, Gove

, Georges

, Manyakov

N V

, Pich

, Hinds

, Tsolaki

, Nikolopoulos

, Kompatsiaris

(2021) Wearable devices for assessing function in Alzheimer’s Disease: A European public involvement activity about the features and preferences of patients and caregivers. Front Aging Neurosci 13, 643135.

56.

Lange

(2002) Review: Richard A. Krueger & Mary Anne Casey (2000). Focus Groups. A Practical Guide for Applied Research (3^rd edition). Forum Qual Soc Res 3, 2–5.

57.

Lüdecke

(2021) sjPlot: Data Visualization for Statistics in Social Science. R package version 2.8.7. 1-106.

58.

Wickham

(2016) ggplot2: Elegant Graphics for Data Analysis.

59.

(2010) Older adults talk technology: Technology usage and attitudes. Comput Hum. Behav 26, 1710–1721.

60.

Crisp

, Griffiths

(2016) Reducing depression through an online intervention: Benefits from a user perspective. JMIR Ment Health 3, e4.

61.

Maresova

, Tomsone

, Lameski

, Madureira

, Mendes

, Zdravevski

, Chorbev

, Trajkovik

, Ellen

, Rodile

(2018) Technological solutions for older people with Alzheimer’s disease: Review. Curr Alzheimer Res 15, 975–983.

62.

Chen

, Chan

AHS

, Chan

(2012) Gerontechnology acceptance by older Hong Kong people. Gerontechnology 11, 1–12.

63.

Freeman

, Clare

, Savitch

, Royan

, Litherland

, Lindsay

, Freeman

, Clare

, Savitch

, Royan

, Litherland

, Lindsay

, Freeman

, Clare

, Savitch

, Royan

, Litherland

, Lindsay

(2005) Improving website accessibility for people with early-stage dementia: A preliminary investigation A preliminary investigation. Aging Ment Health 9, 37–41.

64.

Davis

(1989) Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Q Manag Inf Syst 13, 319–339.

65.

Pitts

, Pudney

, Zachos

, Maiden

, Krogstie

, Jones

, Rose

, Macmanus

, Turner

(2015) Using mobile devices and apps to support reflective learning about older people with dementia. Behav Inf Technol 34, 613–631.

66.

Malinowsky

, Nygård

, Kottorp

(2014) Using a screening tool to evaluate potential use of e-health services for older people with and without cognitive impairment. Aging Ment Health 18, 340–345.

67.

Lanza

, Kn

, Weber

, Riepe

(2014) Autonomous spatial orientation in patients with mild to moderate Alzheimer’s disease by using mobile assistive devices: A pilot study. J Alzheimers Dis 42, 879–884.

68.

Leng

, Yeo

, George

, Barr

(2014) Comparison of iPad applications with traditional activities using person-centred care approach: Impact on well-being for persons with dementia. Dementia 13, 265–273.

69.

Boulos

MNK

, Brewer

, Karimkhani

, Buller

, Robert

(2014) Mobile medical and health apps: State of the art, concerns, regulatory control and certification. Online J Public Health Inform 5, 1–23.

70.

Yamagata

, Coppola

, Kowtko

, Joyce

(2013) Mobile app development and usability research to help dementia and Alzheimer patients. In 9th Annual Conference on Long Island Systems, Applications and Technology, LISAT 2013.

71.

Onoda

, Yamaguchi

(2014) Revision of the cognitive assessment for dementia, iPad version (CADi2). PLoS One 9, e109931.

72.

Sangha

, George

, Winthrop

, Panchal

(2015) Confusion: Delirium and dementia - a smartphone app to improve cognitive assessment. BMJ Qual Improv Rep 4, 1–3.

73.

Zorluoglu

, Kamasak

, Tavacioglu

, Ozanar

(2014) A mobile application for cognitive screening of dementia. Comput Methods Programs Biomed 118, 252–262.

74.

Klimova

(2017) Mobile phone apps in the management and assessment of mild cognitive impairment and/or mild-to-moderate dementia: An opinion article on recent findings. Front Hum Neurosci 11, 461.

75.

Allard

, Husky

, Gwenaelle

, Pelletier

, Dilharreguy

, Amieva

, Peres

, Foubert-Samier

, Dartigues

J-F

, Swedsen

(2014) Mobile technologies in the early detection of cognitive decline. PLoS One 9, e112197.

76.

Gibson

, Newton

, Pritchard

, Finch

, Brittain

, Robinson

(2016) The provision of assistive technology products and services for people with dementia in the United Kingdom. Dementia (London) 15, 681–701.

77.

Liu

, Stroulia

, Nikolaidis

, Miguel-Cruz

, Rios

, Rios Rincon

(2016) Smart homes and home health monitoring technologies for older adults: A systematic review. Int J Med Inform 91, 44–59.

78.

Calvillo

, Román

, Roa

(2015) How technology is empowering patients? A literature review. Health Expect 18, 643–652.