Abstract
PURPOSE:
This mixed methods study explored older adults’ experience using a bespoke active computer gaming (ACG) system designed to deliver falls prevention exercise.
METHODS:
Usability, acceptability and safety were evaluated through observation of system use, questionnaires, in-system ratings, and semi-structured interviews. Quantitative and qualitative data were synthesised concurrently to provide a deeper understanding of older adults’ experience with the system.
RESULTS:
N = 7 older adults (aged 73–88 years, most with increased fear of falling, and over half with reduced physical functioning) completed up to six uses of the system. Observations and qualitative feedback suggested that older adults’ experience with the system was influenced by physical health changes associated with ageing. Overall feedback after using the system was positive. Social support, from either the clinician or a peer, was a key theme influencing experience.
CONCLUSIONS:
Findings suggested that autonomous use of the system may not be feasible given the requirement and preference for social support.
Introduction
Falls are a growing issue for older adults. Each year, approximately a third of over 65s and almost half of over 80s experience a fall [1]. The estimated cost of falls to the NHS is more than £2.3 billion per year [2]; this is likely to increase given the rise in the ageing population. Cochrane review evidence indicates that exercise reduces the rate of falls by 23% and reduces the number of older adults experiencing a fall by 15% [3].
Engagement with exercise interventions can be limited by access difficulties, availability of therapists and cost of delivery [4]. These challenges are compounded by the recent novel coronavirus SARS-CoV-2 (COVID-19) pandemic. Given that this virus causes worse outcomes and a higher mortality rate in older adults [5], public health recommendations include measures to limit the spread of the virus to older adults by reducing their interactions with others and staying at home [6, 7]. These restrictions may have implications on older adults’ physical function, contributing to increased risk of falls [8].
Digital technologies, such as active computer gaming (ACG), may provide an alternative way for older adults to participate in exercise to reduce their risk of falls. Systematic review evidence suggests ACG is a safe and enjoyable way for older people to participate in exercise and activities that may otherwise be difficult, with positive effects on physical and cognitive health outcomes including balance [9]. To date most ACG interventions have been conducted with healthy older adults in a clinical environment with supervision. Previous user-testing by this research team identified that older adults frequently required additional support during supervised use of a bespoke ACG system in a day centre setting [10]. Repeated exposure to the system may improve familiarity and reduce older adults’ requirement of additional instruction and support during use of the ACG system. This may facilitate progression to autonomous use in the home setting. Additionally, there is limited evidence of older adults’ experiences and perceptions of using novel technologies for falls prevention exercises [11]. Feedback from older adults may optimise design to meet the needs of older people to ultimately increase adoption of the ACG technology [12–14].
This paper synthesises quantitative and qualitative data collected during user-testing to explore older adults’ experience using a bespoke ACG system iteratively designed to deliver falls prevention exercises; in particular, its usability, acceptability, and safety with repeated use.
Methods
This section provides an overview of the design and development of the ACG system, and describes the study procedures and how the outcomes of interest were assessed.
ACG system description
The ACG system was developed using Unity 3D software (Unity Technologies SF Inc., San Francisco, USA). It ran on an Alienware PC (Alienware Corps., Miami, USA.) connected to a Microsoft Kinect camera (Microsoft Corps. Redmond, USA) and displayed using a 32” LED screen. Four strength and balance exercises chosen from the Otago Exercise Programme [15] were developed for tracking using the Kinect camera (Supplemental file 1). Users’ body movements were tracked and displayed on the screen in real-time via an avatar. Additionally, each game provided positive or negative feedback indicating the outcome of each action, and a score board provided feedback of results (Supplemental file 2).
Overview of system development
Development of the ACG system was an iterative process by an interdisciplinary team of game developers and clinicians. It was informed by a systematic review of the literature [9]. Older adults were involved from early in the design phase (early development is reported [10]); and, findings from the first phase of user-testing were used to modify the system and inform study design (Fig. 1). Modifications to the system aimed to improve user experience; for example, modification of ACG system feedback to include audio and visual effects, and the addition of user profiling to record scores and ratings of satisfaction and difficulty after each session. The revised system was then tested in a new sample of older adults.

User-centred testing. Findings from user-testing of the first prototype of the ACG system with n = 9 older adults (phase 1) were used to make changes to the system. User-testing of the second prototype was conducted with a new sample of n = 7 older adults.
The study design used mixed methods to explore older adults’ user experience with repeated use of the ACG system. Study visits were carried out at two urban day centres for older adults run by a third sector organisation in the UK. This study was approved by the Office for Research Ethics Committees Northern Ireland (IRAS ID: 187902).
Participants
Participants were recruited from the day centres following group information sessions about the study. Eligible participants were aged 65 years or over, able to walk with or without a walking aid, had stable physical health according to the Physical Activity Readiness Questionnaire [16], and were able to communicate (verbal and written) in English. Participants were excluded if they had a current acute or uncontrolled medical condition that would not tolerate physical activity (determined by their GP who was informed of study participation via letter and asked to indicate any concerns about participation), had significant cognitive impairment according to the Mini-Mental State Examination (MMSE < 21) [17], were unable to follow verbal or written instruction, or were unwilling or unable to consent to participate.
Procedure
Participants were invited to complete six 1-hour sessions with the ACG system. Sessions were delivered twice per week for three weeks in the centre. While independent use was encouraged, one researcher was present to provide supervision to ensure participant safety. The researcher demonstrated and provided instruction on the use of the system. Participants were encouraged to comment on their use of the technology, report problems and request additional instruction, as necessary. Participants were able to use two chairs placed at either side or their walking aid for hand support, with encouragement to use only the hand support they required. Supplemental file 3 includes photographs of a participant using the system with their walking stick.
Initial assessment
Demographic information and participant characteristics were collected before use of the system. Standardised tools validated for the study population included: physical function, using the Short Physical Performance Battery (SPPB) [18]; balance, using the Berg Balance Scale (BBS) [19, 20]; fear of falling, using the Falls Efficacy Scale-International (FES-I) [21]; and depression, using the 15-item Geriatric Depression Scale (GDS-15) [22].
User experience outcomes
Data collected to explore user experience, in terms of usability, acceptability and safety, is summarised in Fig. 2.

Diagram summarising user experience data collection.
The System Usability Scale (SUS) [23] was completed by participants after each use of the system, to explore perceptions of usability with repeated use. The SUS is a standardised questionnaire for the assessment of perceived usability and has been shown to have acceptable levels of reliability [24], concurrent validity [24] and sensitivity [25]. The SUS is scored from 0–100; scores above 70 indicate acceptable usability, while scores below 50 indicate unacceptably low usability [24]. Participants’ scores achieved in each game were logged for each session. Scores were achieved based on the number of repetitions completed within the required time and range of movement for each movement (Supplemental files 1 and 2). Acceptability was measured using the Attitudes to Falls-Related Interventions Scale (AFRIS) [26, 27], which was completed following each use of the system. The AFRIS is a validated questionnaire used to identify factors influencing intervention engagement through 6 items related to attitudes, subjective norms, behavioural control, identity, and intentions, each rated on a Likert scale from 1(strongly disagree) to 7(strongly agree). Additionally, the ACG system allowed participants to rate their satisfaction (6-point Likert scale) and difficulty (9-point Likert scale) following each use of the system. The system also recorded participant scores.
Observation
During system use, the researcher made handwritten notes to record safety concerns, additional instruction and hand support required, and participant comments. Sessions were video recorded for retrospective analysis to supplement these hand-written observations.
Qualitative feedback
Qualitative data were collected via semi-structured interviews with participants (approximately 25 minutes), audio recorded after the practical aspect during the final study visit. Open ended questions were used to explore the factors influencing older adults’ experiences with using the system and enhance understanding of quantitative findings.
Data analysis
Statistical analysis was performed using SPSS software (version 23). The data were checked for normality, then appropriate descriptive analyses were used to summarise participant characteristics and outcomes. Associations between participant characteristics and their use of and feedback on the system were explored using Spearman’s correlation.
Interviews were digitally recorded and transcribed verbatim and verified by the researcher. The Technology Acceptance Model (TAM; Fig. 3) [28] was used to develop a coding framework. The TAM is widely used to understand the factors influencing technology adoption. The definitions used are summarised in Supplemental file 4. Data were coded based on participants’ perceptions of the usefulness and ease of use of the ACG system, and external variables that may influence perceptions and attitudes towards and actual behavioural intentions to use the system. Interpretation, synthesis and data reduction were undertaken by three members of the research team. To ensure reliability of the analysis, all anonymised transcripts were coded by two authors (100% SH, 50% IW, 50% KP). The codes were then compared and agreed by consensus through discussion.

Technology acceptance model [28].
Quantitative and qualitative findings were synthesised concurrently to provide greater understanding of older adults’ experience with the system and the factors influencing their experience. A framework was developed to synthesise the relevant quantitative outcomes and qualitative codes related to the outcomes of interest. The framework was agreed after discussion by three members of the research team. The related qualitative and quantitative outcomes are summarised in Supplemental file 4. The framework was populated with a summary of the quantitative findings, including relevant relationships between quantitative outcomes, and the related qualitative excerpts collected within each code. The mixed methods results were presented based on each key quantitative finding alongside the supporting data collected within the related qualitative code.
Recruitment
Thirty-eight service users attended one of three information sessions held at the day centres to find out more about participation in the study. Fifteen service users declined screening for the study, due to health reasons, including chronic health conditions, musculoskeletal pain or mobility limitations (n = 8), or lack of interest (n = 7). Eligibility screening was completed for the remaining 23 participants; n = 7 individuals were excluded due to cognitive impairment (n = 5), registered blind (n = 1), or inadequate level of mobility (n = 1). Of the n = 16 eligible participants, n = 7 gave written informed consent and participated in the study. Thus, recruitment rates were 18% of the total number of older adults (7/38) who attended the information sessions, and 44% of those who were screened as eligible for the study (7/16). Seven participants (aged 73–88 years, 3 female/4 male) consented to participate and completed at least one session using the ACG system. Reasons provided for non-participation were health-related or related to scheduling, when individuals wished to attend other activities within the day centre.
Participants
Participant characteristics, baseline measures and the number of sessions completed are presented in Table 1. Three participants regularly used a walking aid. According to SPPB scores, n = 4 participants had a lower level of physical functioning (SPPB score < 10). BBS scores indicated that one participant was at high risk of falls. According to FES-I scores, 6/7 had moderate to high concern about falling (FES-I score 20–64); however, none of the participants reported having a fall in the last twelve months. According to the GDS-15, n = 2 participants scored on the cut-off for mild depression (GDS-15 score≥5).
Participant characteristics and session adherence
Participant characteristics and session adherence
¥ - lower score = better. * - higher score = better. BBS – Berg Balance Scale. FES-I – Falls Efficacy Scale International. GDS – Geriatric Depression Scale. SPPB – Short Physical Performance Battery.
The system was used a total of 22 times, representing 52% adherence to the 42 scheduled sessions, with n = 4 participants completing at least half of the scheduled sessions. No participants completed all six sessions. Reasons reported for non-completion of sessions were not related to user experience. Reasons reported included health-related reasons, such as head cold symptoms, a flare-up of osteoarthritic pain and stiffness (n = 12 sessions), or scheduling issues (n = 8), such as participants’ non-attendance at the day centre or preference to attend other activities scheduled at the day centre. Following all but one (21/22) uses of the system, participants agreed they would intend to use the system if given the opportunity (AFRIS item 6); while in qualitative feedback related to behavioural intentions, 5/7 participants did not intend to continue using the system, but two stated they would be interested in future.
Exploring participant characteristics alongside ACG session adherence, all the male participants (n = 4) completed at least half of the scheduled sessions, with n = 3 completing 5/6 sessions. All female participants (n = 3) completed fewer than half the scheduled sessions (1–2). Two of the females (Participants E and F) had scores representing the lowest physical function (SPPB) and balance (BBS), and the highest fear of falling (FES-I) and depression (GDS-15). Qualitative feedback supported this finding and provided greater understanding that the participants experienced changes in their physical health associated with ageing, suggesting this was an external variable influencing their engagement with the system:
Pt A: I could have worked anywhere, no balance trouble at all, but I wouldn’t even try that now because I know that I don’t have the balance.
Pt G: I always did have reasonable health and fitness but over this last, I suppose, this last four or five years things have just deteriorated.
Pt F: The first time I done it I was great, but I wasn’t well last week so that’s why I’m not doing it. I enjoyed doing it, so I did, and turning around and different things, but, not this week. I couldn’t do it this week.
Participant feedback following system use
SUS scores suggested high levels of usability. The median (interquartile range; IQR) SUS score across the completed sessions was 85 (75.6–92.5), which is considered excellent [24, 29]. AFRIS scores suggested high levels of acceptability and positive attitudes (median (IQR) AFRIS score 36 (35–39) out of 42). Participants most frequently rated moderate to high levels of satisfaction following their use of the system (21/22), with one negative rating following a session with technical difficulties that caused the participant some frustration. Participants reported experiencing low to moderate levels of difficulty with their ACG sessions (19/22). Of note, the highest rating of difficulty on the first session was “moderate” which was rated by the three participants with the lowest baseline of balance, physical function and balance confidence.
Perceived usefulness of the system influenced participants’ perceptions of the game. After every (22/22) system use, participants agreed that using the system would be good for them (AFRIS item 1). Qualitative feedback indicated that participants tended to believe that using the system would have a health benefit or that it would make exercise more enjoyable:
Pt B: I thought the game would actually help achieve a bit of fitness for me, bit of strength in my legs.
Pt F: Well, I thought I would get better walking . . . But exercise does do you good.
Pt E: I thought that it would make doing exercises more interesting. . . I wasn’t doing anything I hadn’t done before. I found it quite good, you know. And it’s less boring than doing a thing and doing it on your own.
Pt B: The game is probably different, I think if it’s there you have a visual and you’re trying to do what that visual is doing, and I think it’s a better motivation to do the thing daily, on a daily basis even.
Overall, participants expressed positive perceptions of the ease of use of the system, which was influenced by the visual feedback provided by the system:
Pt B: If you see the object on the screen, what you need to do, then I think it helps you too. It’s better than actually doing it at home, say trying to walk along a table or that. You can see your result on the tv screen.
Pt E: It probably helps to do the things when you’re sort of watching yourself doing it. I think it’s beneficial that way.
Pt B: A score shows, a good score shows that you are capable of participation of the game and you can maybe try and increase it.
Perceived ease of use was reflected in responses to AFRIS item 4; after 21/22 system uses, participants agreed that if they wanted to, it would be easy for them to use the system. During the semi-structured interviews participants described the game they found most difficult, which was usually influenced by their physical limitations, and aspects of use that were challenging, such as requiring thinking speed. They also made suggestions to improve the ACG system based on their experience:
Pt B: I do think it would be advantageous if you had different levels.
Pt E: If they could slow it down a bit at first and have it gradual, and if it started the first couple of times at a slower pace and then speeded it up to what would be the correct speed, just so you would really get into the way of doing it properly and so on.
Use of hand support
A summary of the level of hand support required by participants during ACG system use is shown in Fig. 4. Games that involved standing on one leg (leg abduction and one leg stand) required the most hand support. While the level of hand support used by participants for each game varied, it remained consistent for that individual i.e. it did not tend to change with repeated use of the system. Observation of participants’ use of the system indicated that some participants tended to use the hand support available to them (two chairs placed at either side) even if it was more than they required.

Summary of hand support used by participants at each ACG session.
Qualitative feedback suggested that having a chair available was an important external variable that influenced participants’ perceived ease of use when using the ACG system:
Pt B: If you have the chair. I do need the support, no doubt about it, and I would have to have the chair there all the time. But if I had the chair, I would feel secure enough to do the exercises, there’s no doubt about that.
Pt A: You see, with the least touch [of the chair] I can do it, but without it I can’t.
Pt F: You had the chairs set up to hold on to. I’m no good if I haven’t anything to hold onto.
The median (IQR) frequency of instruction related to set-up in a session was 2 (1–2). The highest frequency of instruction related to set-up was observed on the first use of the system, which consistently decreased on the second use of the system. However, for participants A, B and G, who completed the highest number of ACG sessions, the frequency of instruction related to set-up did not decrease with further uses of the ACG system (Fig. 5A).

Summary of additional instruction during system uses.
The median (IQR) frequency of additional instruction related to play, for example to correct timing or technique of movement to improve success in the game, was 3 (2–8). The frequency of instruction related to play for participants A, B and G, who completed the greatest number of ACG sessions, decreased with use of the ACG system. For example, participant G required n = 23 additional instructions on the first visit, and on the last visit required n = 5 additional instructions (Fig. 5B).
Interestingly, lower in-system difficulty ratings across all completed sessions were associated with increased frequency of instruction (r = –0.525, p = 0.012). Feedback from participants suggested that the availability of someone there to provide additional instruction or support was an external variable influencing perceived ease of use and enjoyment:
Pt B: You had someone there to help and to try and tell you if you were doing something wrong, or if you should be trying it this way. I think that is a great help to people.
Pt B: I think someone there is a help. I couldn’t see myself enjoying it as much, possibly, on my own as I did when you were here when I was doing it. . . . You had someone.
Pt G: [I felt secure] under your instruction; I have to give you your bit as well, because that was totally new to me.
The importance of social interaction was a recurrent theme throughout the qualitative feedback. As described above, availability of the researcher for instruction and support was important to participants; however, many also valued peer support. Reasons for this varied from having company, to encouragement and an element of competition.
Pt B: That applies to anyone, I mean if you’ve anyone in the room at all. If you’ve two people doing it, one encouraging the other. Encouragement is a great thing.
Pt F: I watched her doing the things and she watched me. . . It’s company, isn’t it. Although you’re company. But somebody nearly the same as yourself, you know. . . I had never spoke to that woman, maybe said hello or something. But that’s like a friend I’ve made. I think that’s good.
Pt B: I think if you’re competing with someone else it is that extra bit of motivation. . . I’d look at myself and say that chap can do it, if I knuckle down here I can do it.
There were no adverse events related to use of the ACG system.
Discussion
This study synthesised quantitative and qualitative data to explore older adults’ (aged 73–88, most with increased fear of falling, and over half with reduced physical functioning) experiences with repeated use of an ACG system designed to deliver strength and balance exercises for falls prevention. Of those screened as eligible, 44% took part in the study; seven participants used the system at least once, and participants completed 52% of scheduled sessions. Quantitative and qualitative feedback from participants suggested high levels of usability and acceptability. Nonetheless, additional support and instruction were frequently required, and often preferred by participants. Social support was identified as on overarching theme influencing their experience using the ACG system.
Reasons related to health, physical function and the changes associated with getting older were frequently reported as barriers to engagement with the system. These factors were observed at study recruitment, through adherence rates and observations made during ACG system use, and through feedback provided by participants following system use. Approximately half of the reasons reported for declining to participate in the study were health-related; as were 60% of the reasons reported for non-adherence to scheduled ACG sessions. Adherence in this study (52.3%) was lower than the mean adherence rate (78.8%) reported from n = 17 trials in a systematic review of ACG interventions [9]. Notably, many of the trials in this systematic review included healthy older adults, which may explain this disparity. The current study included participants regardless of their level of physical function and balance, and found that although those with increased fear of falling or reduced balance and physical function were willing to participate in the study and use the ACG system and reported positive perceptions of its usefulness, their overall session adherence was lower. This observation echoes findings of a review of enablers and barriers to older adults’ participation in traditional strength and balance activities which suggested that, while desire to improve health and prevent functional decline are motivators, both fear of falling and complications from comorbidities are barriers to participation in strength and balance exercise [30]. As well as overall participation, physical functioning seemed to influence user experience with the system. Participants with increased fear of falling or reduced balance or physical functioning at baseline chose a higher difficulty rating after using the system. Qualitative data provided during the semi-structured interviews supported these findings, as participants reiterated that changes to their physical health associated with ageing were a barrier to engaging with both the system and traditional exercise or therapy.
Observations during system use alongside quantitative and qualitative participant feedback indicated that social support was an important facilitator to use of the system, and that it influenced user experience. While the frequency of instruction required did tend to reduce as participants became more familiar with the system, it was observed that participants tended to turn to the researcher for support and reassurance. Despite requiring additional support and instruction, participant ratings after using the system suggested high levels of usability and acceptability. Notably, participants tended to rate lower difficulty when they had received more instruction from the researcher. Older adults are typically less familiar with being instructed by a computer screen [31]. Despite attempts to ensure the game was intuitive and that instruction provided by the system was short and clear, the older adults’ computer anxiety and low self-efficacy may have contributed to the preference for human support observed during their system use [32–34]. Responses in the semi-structured interviews also suggested that social support was important to them. Participants reported that social support could be provided by a peer rather than a clinician or researcher, indicating that having company was an important factor and adding that comparing performance or an element of competition could be motivating. On a similar note, alongside system-specific suggestions, such as adding levels of difficulty and options to repeat each game to improve their score, the participants provided general suggestions to improve their experience while using the system. These included having the system set up in the activity room in the day centre, and multiplayer functionality. These suggestions and the observations described above were in line with previous literature that suggested that older adults prefer group-based exercise to individual exercise [35]. Future research could explore the delivery of the intervention within a day centre or rehabilitation group setting, or the influence of a buddy or peer support on engagement and experience when using technology-related interventions, such as active computer gaming.
An alternative that may be particularly timely to explore given the global COVID-19 pandemic and the advice for older adults to “shield” or “cocoon” and reduce their contact with others to reduce their risk [5], is the delivery of social support via virtual platforms. There is growing interest in the use of technology by older adults for social support [36, 37]. Remote monitoring and videoconferencing may provide an alternative to in-person clinician support, while online multiplayer and chat functionalities may provide an alternative means of social interaction and peer support. However, while at-home delivery of an ACG system with online virtual interaction may provide an opportunity for older adults to stay active and complete exercise at home and provide social support, the findings of the current study suggested that human support was required and preferred by older adults using the ACG system. Thus, further research is required to ascertain optimal conditions for delivery of ACG interventions to ensure both safety and long-term engagement.
While this study provides valuable insight into older adults’ perceptions of the usability and acceptability of a specifically designed ACG system using both quantitative and qualitative methods, there are several limitations to its findings. Primarily, the small number of participants recruited providing data over a small number of ACG sessions meant limited ability to perform statistical analysis of usability and acceptability with repeated use. Additionally, as the study design did not include a comparison group, we are unable to make assumptions about this population in terms of their adherence to another exercise programme.
Conclusion
Despite high levels of usability and acceptability of the ACG system in older adults, the findings of this study suggested it was unlikely that this ACG system would be suitable for autonomous use by this study population. Given the low adherence rates to supervised use of the system recorded in this study, it is unlikely that participants would use the system independently. Although there were no safety issues during system use, even after repeated use of the system, participants required support when using the system. This may have been due to physical function limitations, both perceived by the participants and observed in baseline measures, and their reduced confidence with both their balance and the use of technology. However, social support seemed to have an important influence on user experience. Future research should explore ways to include social support in the delivery of ACG interventions.
Footnotes
Acknowledgments
This research was completed as part of a PhD funded by the Department of Employment and Learning. We would like to thank all the staff and clients at the two Age NI day centres where this study was conducted, and in particular those who took part in the study. We would also like to acknowledge Calvin Brannigan (BSc Computing) for his contribution to the development of the system during his undergraduate student placement.
Conflict of interest
The authors have no conflicts of interest to declare.
Ethical considerations
This study was approved by the Office for Research Ethics Committees Northern Ireland (IRAS ID: 187902).
