Abstract
Objective:
Gesture-based serious games can be based on playful and interactive scenarios to enhance user engagement and experience during exercises, thereby increasing efficiency in the motor rehabilitation process. This study aimed to develop the Rehabilite Game (RG) as a complementary therapy tool for upper limb rehabilitation in clinics and home environments and to evaluate aspects of usability and user experience of it.
Materials and Methods:
The evaluation consisted of the use of a gesture-based serious game with motor rehabilitation sessions managed in a web platform. Thirty-three participants were recruited (21 physiotherapists and 12 patients). The protocol allowed each participant to have the experience of playing sessions with different combinations of settings. The User Experience Questionnaire (UEQ) was used to evaluate aspects of usability and user experience. The study was approved by the Research Ethics Board of the Federal University of Piaui (number 3,429,494).
Results:
The level of satisfaction with the RG was positive, with an excellent Net Promoter Score for 85.7% of physiotherapists and 100% of patients. All six UEQ scales (attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty) reflected acceptance.
Conclusion:
The study demonstrated that, according to the results obtained in the experiments, the RG had positive feedback from physiotherapists and patients, indicating that the game can be used in a clinical trial to be compared with other rehabilitation techniques.
Introduction
Every physiologically healthy human being has the ability to perform active movements, a skill that is directly related to the body's functionalities and therefore essential for maintaining health. 1 When evolving to the bipedal position, the human being started to freely use the upper limbs to develop various activities, such as eating and drinking. When the motor capacity of the upper limbs is compromised, there is consequently a loss in the functional independence. The recovery of motor function, particularly the upper limbs, is a key factor in determining the chance of an independent life, 2 since most of the daily routine activities involve functional movements of the upper limbs. 3
Thus, health professionals specialized in the motor rehabilitation process have used several techniques that can enhance the functional recovery of patients' limbs, such as noninvasive brain stimulation, 4 robotic systems, 5 sensory signaling, 6 mirror therapy, 7 telerehabilitation, 8 and game therapy. 9 In particular, game therapy is a method that uses videogame environments (i.e., serious games) to assist in motor rehabilitation. Serious games are computer environments based on the use of videogame technologies and methodologies that allow players to learn and practice skills to improve both human and professional development applicable to real-world situations. 10
By using playful and interactive components, serious games go beyond simple entertainment, 11 they increase the user experience in specific contexts such as well-being, education, and health. 12 The serious games used in both cognitive and motor rehabilitation 13 are presented as a physiotherapy complementary technique. Health solutions based on serious games increase patient involvement and provide immersion, 14 experience, 15 and biofeedback 16 in the motor rehabilitation process of the upper17,18 and lower19,20 limbs, therefore favoring the engagement for the practice of physical exercises in motor rehabilitation treatments. Studies have been conducted using serious games as a complementary therapy for upper limb motor rehabilitation.21,22
However, proposed solutions are based on the use of sensors, 23 wearable devices, 24 robotic arms, 25 and proprietary gaming consoles, such as Nintendo® Wii™ 26 and Xbox® Kinect, 27 which have a high cost, therefore contributing to low adoption of the game therapy technique, especially in home environments. 28 In addition, there is an increasing number of serious games that enable telerehabilitation, especially rehabilitation in the home environment. 29 Home rehabilitation offers the opportunity to increase training frequency, especially for people who have difficulty in attending rehabilitation clinics regularly. 30 However, some solutions do not require monitoring assistance by professionals, 31 which can bring risks related to the aggravation of motor disability or treatment ineffectiveness. 32
The aim of this study was to develop the Rehabilite Game (RG), a serious game for motor rehabilitation of upper limbs, and evaluate the usability and user experience of it with physiotherapists and patients. A machine learning-based computer vision tool is used to capture the patient's gestures by a webcam integrated to a web platform. In the RG, health professionals are required to configure game sessions specific to the treatment appropriate to the patient's clinical conditions (i.e., treatment-specific game mechanics). RG has configuration interfaces that allow professionals to configure and enable exercise sessions according to the protocol already used in classic patient treatment, as a complementary technique. We developed the game to help health professionals in the motor rehabilitation process, enabling the game therapy technique to be present in specialized clinics and home environments.
Materials and Methods
The Rehabilite Game
The serious game named RG (see www.rehabilitegame.com) presents a playful and manageable environment for the practice of exercises that promote motor rehabilitation of upper limbs. It was developed using a machine learning-based computer vision tool, the MediaPipe, 33 integrated with a wide social inclusion technology: cameras embedded in laptops, desktop computers, or integrated into televisions (Fig. 1). To capture movements of the upper limbs during exercises, which generate the necessary dynamics in the game interface, we used the MediaPipe Hands model. 34 The platform was implemented using web technologies.

Volunteer in her home environment using the RG. She is using a laptop camera to capture upper limb movement. RG, Rehabilite Game.
Rehabilitation sessions allow patients to access motor exercises managed by professionals (e.g., physiotherapists, orthopedists, and occupational therapists). Sessions are defined according to a treatment protocol, in which the patient performs exercises according to the schedule defined in the RG. Game mechanics allows players to perform movements of the upper limbs (i.e., a gesture-based serious game), mainly flexion, extension, abduction, and adduction.
The scenario dynamics encourages the patient to pop balloons displayed on the screen by means of arm movement, which makes the visual object of the hand (right or left) to meet balloons, repeating the actual movement of the upper limb (Fig. 2). When starting the game, the camera starts capturing the video movement of the patient's hand. Specifically, the reference point 10 defined by the MediaPipe Hands (i.e., the proximal interphalangeal middle finger) is used to define the movement of the hand object in the game scenario. When the hand finds the balloon, audible and visual pop feedback is emitted to the patient, which indicates that the target has been reached.

Example of a scenario for rehabilitation sessions. The user has a screen with his/her image captured by the webcam. There is a score, time recording, and control buttons during the session (top left). The background can be chosen for each training session.
The current version of the RG has three phases that allow different exercise mechanics for upper limb rehabilitation, which are focused on movements of shoulder and elbow joints. In phase I, the player performs repetitive movements to hit the target (i.e., balloons). Upon reaching the target (i.e., the balloon is popped), the player should return his/her limb to the starting position. In phase II, when the player reaches the balloon, a new one appears on the opposite side, which is a mechanics that works the lateral and/or diagonal displacement of limbs.
Differently, in phase III, the player is free to perform necessary movements to pop balloons with different colors and scores that appear distributed in random positions. The RG allows for configuring patient-specific treatments, thereby enabling the definition and frequency of exercises. Professionals can configure the following definitions in the game interface: (1) day and time that the session will be enabled for access by the patient; (2) duration for each session; (3) limb used by the player (right or left); (4) side and height of balloons in the game area; (5) balloon pop mode, which can be automatic or using a timer; and (6) background image displayed.
To work on different movements (e.g., flexion, extension, abduction, and adduction) and ranges, the game area was developed in quadrants, which can also be configured by professionals according to the patient's profile and treatment goals. On the horizontal axis, the game area is divided into two sides (left and right), that is, balloons appear on the right or left side of the screen, then requiring the limb to be moved in abduction or adduction. On the vertical axis, the game area is delimited at three ranges: 45°, 90°, and 180°, thus requiring the player to move his/her limb in one of these range values to reach a target. A balloon pop may occur immediately or seconds after the player's hand keeps touching a balloon, according to the predefined settings.
This game definition, which causes players to hold the limb in the same position (i.e., for a predefined period) until the balloon pops, was provided to encourage active movement, trains endurance, as well as limb balance.
Participants
For the usability and user experience study, physiotherapists and patients were recruited. Physiotherapists were recruited through a snowball technique, starting with members of the Technological Neuro Innovation Laboratory of the Federal University of Parnaíba Delta (UFDPar). Patients were selected in two school clinics: Centro Universitário Inta (Sobral, CE) and Centro Universitário Maurício de Nassau (Parnaíba, PI).
To be included in the study, physiotherapists should not have visual, hearing, physical, or psychiatric disabilities. Differently, patients should have a motor impairment in an upper limb. All participants signed the free and informed consent form. Experiments were approved by the Research Ethics Committee of the Federal University of Piauí (no. 3,429,494) according to the criteria of ethics in research with human subjects included in the Declaration of Helsinki. The purpose, nature, procedure, and potential risks of the experiments were fully explained to the participants, and all of them provided written informed consent before participating in the study.
Experimental procedure
All participants were trained, online and synchronously, on the use of the platform functionalities. In addition, tutorial videos and user manual for RG were made available. To perform the experimental protocol, participants could access the game in clinics or at home, but preferably in a bright environment, using a laptop or a desktop computer with a front camera. There were no minimum hardware specification restrictions. After completing the protocol, participants had access to a questionnaire, via the Google Forms web tool, to evaluate usability and user experience.
Physiotherapists had between April 19 and May 31, 2021, to join the study and participate in the experimental protocol (to set up exercise sessions and play them) and answer the online questionnaire. The protocol did not present a specific clinical case to be followed by physiotherapists. However, the recommendation was to create two patient profiles considering clinical scenarios already known by the physiotherapist, that is, the participant freely defined two clinical cases based on his/her experience. For each of these profiles, three rehabilitation sessions should be defined, exploring different possibilities of definitions provided by the game platform. Sessions had a time of 3 minutes, one session per day, interspersed every 48 hours (a total of 12 days). Next, by using the access credentials as patients, physiotherapists played the previously configured sessions to have the patient experience of using the game.
In the experiment with patients, participants played the game in sessions with settings defined by their physiotherapists, according to the protocol already used in the classic patient treatment. All patients played the exercise sessions in rehabilitation school clinics for 10 days (from December 6 to 17, 2021). Each patient played three sessions using the RG before answering the questionnaire.
Data collection and analysis
Initially, participants answered three questions to express their general opinion on the RG. Question (1) “What is your overall rating regarding the use of the Rehabilite Game?” was evaluated by physiotherapists and patients using a scale from 0 to 10, based on the Net Promoter Score (NPS). 35 In the experiment with physiotherapists, the following dichotomous questions (answers yes/no) were asked: (2) “Would you use the Rehabilite Game as a complementary technique for the motor rehabilitation of your patients?” and (3) “Would you recommend the use of Rehabilite Game to another professional?” Then, we applied a questionnaire, both to the physiotherapists and patients, to evaluate aspects of usability and user experience, regarding the use of the platform and its features. For this purpose, we applied the User Experience Questionnaire (UEQ). 36
Each item (a total of 26 items) of the UEQ consists of antonym-adjective pairs (e.g., annoying vs. enjoyable), which can be rated on a 7-point Likert scale ranging from −3 (fully agree with a negative term) to +3 (fully agree with a positive term). They are randomly presented. The 26 items are grouped into six scales: attractiveness (general impression of the proposed game), perspicuity (how easy it is to get familiar with the proposed game), efficiency (solving tasks without unnecessary effort), dependability (feeling in control), stimulation (how exciting and motivating the proposed game is), and novelty (how innovative and creative the game is), as seen in Supplementary Table S1.
Furthermore, we compared the results of the RG with the results of other established products using a benchmark data set containing quite different products. The UEQ compares the level of participants' experience, and measured scale means with a benchmark data set of 452 studies related to different products (with a total of 20,190 participants in all evaluations). 37 Finally, to show the research dependability for each scale of the questionnaire, we evaluated the values obtained for Cronbach's alpha coefficient. 38 For data analysis, we used a spreadsheet, a tool available by the UEQ.
Results
A total number of 33 participants (21 physiotherapists and 12 patients) were screened. Table 1 presents an overview of the demographic characteristics of participants. Physiotherapists, aged between 26 and 48 years, were from seven different states of the Brazilian federation. All of them had more than 2 years of undergraduate degree completed, working in clinics or hospitals for at least 2 years in the field of motor rehabilitation. Patients, aged between 23 and 66 years, had a motor impairment in an upper limb. None of the participants reported any adverse events during the game experiment.
Participants' Characteristics (n = 33)
The level of satisfaction with the RG is positive, with an excellent NPS (i.e., ≥7.5) in 85.7% of physiotherapists and 100% of patients. All the physiotherapists answered “Yes” when asked if they would use the RG as a complementary technique for the motor rehabilitation of patients and if they would recommend it to another professional. The UEQ analysis supports to calculate means of all scales, according to the aggregated results of all participants, as shown in Figure 3a and b. Mean values between −0.8 and 0.8 represent a neutral evaluation, values >0.8 correspond to a positive evaluation, and values <−0.8 refer to a negative evaluation. In the evaluation with physiotherapists (Fig. 3a), all scales present values above 1.5, reflecting acceptance. When considering the evaluation with patients (Fig. 3b), the mean value is close to 2.5 for five scales.

Results of the UEQ aggregated into six scales: attractiveness, perspicuity, efficiency, dependability, stimulation, and novelty. Comparison of evaluation results between the RG and the UEQ benchmark.
Distributions of responses for the 26 UEQ items were analyzed, according to their score on the Likert scale, as shown in Figure 4a and b. Figure 4a shows that 20 items had a positive or neutral evaluation ≥80% by physiotherapists. Patients (Fig. 4b) had good feedback with 21 items with a positive or neutral evaluation ≥90%. The comparison of the results with data in the UEQ benchmark allows conclusions about the relative quality of the game compared with other products, as shown in Figure 3a and b. According to the benchmark comparison in the evaluation with physiotherapists, the aspects of attractiveness, efficiency, stimulation, and novelty are in the excellent range, whereas the perspicuity and dependability aspects are in the good range. With patients, all scales reached an excellent range.

UEQ answers per item [
The consistency of the UEQ scale is based on reliability measured using the Cronbach's alpha coefficient. 38 Cronbach's alpha coefficients in evaluation were as follows: attractiveness (physiotherapists [PH] = 0.85; patients [PT] = 0.89), perspicuity (PH = 0.85; PT = 0.75), efficiency (PH = 0.78; PT = 0.92), dependability (PH = 0.40; PT = 0.76), stimulation (PH = 0.71; PT = 0.82), and novelty (PH = 0.75; PT = 0.84). Scales should show an alpha value >0.7 to be considered sufficiently consistent.39,40 The confidence level of the UEQ scales for attractiveness, perspicuity, efficiency, stimulation, and novelty is consistent, as the Cronbach's alpha values indicated. In the evaluation with physiotherapists, the dependability scale is not sufficiently consistent.
Discussion
RG contributions
This study describes the design of the RG, a gesture-based serious game for motor rehabilitation of upper limbs. Different from other proposals,9,26,27 the game developed for motor rehabilitation allows professionals to generate personalized training sessions, according to the limitations and specificities of patients. The RG is not intended to replace the role of specialized health professionals. All experiments with patients were conducted and monitored by professionals (i.e., telerehabilitation model). In pandemic times (e.g., COVID-19), with the need for social isolation, the RG is a viable and adequate solution to maintain the routine of rehabilitation treatments in home environments, remotely guided by professionals. 41 During the experiments with patients, RG demonstrated to be viable for the treatment of different impairments (i.e., stroke, rotator cuff syndrome, radius fracture, humerus fracture, and bursitis).
Experiments indicated that the game, which has different settings, can be used in rehabilitation clinics and home environments. This was motivated by the use of widespread and low-cost webcams already available for laptops and desktop computers. Also, in the game experiments, costs with proprietary technologies or infrastructure adaptations in environments were not necessary. The game has been proposed to be a rehabilitation solution that uses a patient-centered approach, in which physical spaces of the rehabilitation clinic or home environment are taken into account, not requiring any infrastructure changes, qualified relatives or caregivers, and additional financial expenses.
Principal findings
There is a clear polarization in the answers of the UEQ items, with a tendency to positive evaluation, which demonstrates the satisfaction of the participants regarding the game. Results indicated that the serious game is exciting, then creating great motivation (stimulation scale), as well as being friendly, attractive, and pleasant (attractiveness scale). The different motor exercises in sessions were performed successfully, and it is easy to get used to the virtual environment (perspicuity and efficiency are good).
For the dependability scale, which had the lowest alpha values in both experiments (i.e., physiotherapists and patients), a high variance for the item unpredictable/predictable (Fig. 4a and b) was observed, which decreases correlations with the other items, therefore decreasing the alpha value. Possibly from the feedback collected during experiments, this high variance was motivated by a wrong feedback message when recognizing the participant's hand (i.e., right or left). We have already identified and corrected this visual feedback.
A similar solution had not yet been used by the participants, which would allow a comparison of UEQ results between the RG and other games. By comparing the obtained results with the UEQ benchmark, RG presents excellent levels of evaluation. In the evaluation with patients, the game is among the 10% of the products with the best results. When analyzing the results obtained with physiotherapists, four scales had excellent results. In addition, only the perspicuity and dependability scales were good, which means that 75% of the data in the UEQ benchmark were worse than those obtained by the RG. However, even for obtaining a satisfactory evaluation, a better comparative analysis is needed, considering that the types of systems evaluated in the UEQ benchmark data set are not known. UEQ items may favor serious games over classic information systems.
Limitations and future studies
We acknowledge that the restricted number of participants is a study limitation. Although the sample size is considered acceptable when compared with other similar usability evaluation studies,42–44 the number of participants could be larger. The study was applied during the COVID-19 pandemic, which directly or indirectly affected the potential audience of participants in our study. Further investigations using larger sample sizes are recommended.
In future studies, improvement in the game's feedback system is needed. A recent study highlights feedback as a key component in motor control, one of the main goals of motor rehabilitation during game therapy. 45 In addition to text messages, the game needs to include sound alerts and voice messages, thereby contributing to better patient feedback. Furthermore, a trunk stability control system will be implemented. Trunk stability is necessary to maintain balance and posture when moving the upper limb. 46 It is possible that upper limb training with trunk support and visual feedback within the game environment can aid in upper limb stability and balance. 47 Another feature to be implemented is the use of all reference points provided by the MediaPipe Hands. This will enable to create new scenarios and provide additional exercises for the upper limb rehabilitation.
Conclusion
This study developed and evaluated the RG, a complementary therapy tool for upper limb rehabilitation, in clinics and home environments. Measuring usability and user experience is important before the game is fully implemented. According to the results obtained through the UEQ, the game had positive feedback from physiotherapists and patients, thereby indicating that the game can be used in a clinical trial to measure the functional performance of patients before and after treatment using the RG in motor rehabilitation. Finally, the use of serious games from easily accessible technologies, such as webcams, can popularize the game therapy technique in motor rehabilitation treatments.
Footnotes
Authors' Contributions
R.B. conceived the study and drafted the article. J.E. and R.F. drafted the article and were part of the development team. R.M., P.A., and D.S. drafted the article and made critical revisions. A.T., F.S., V.H.B., and S.T. drafted the article and supervised the project. All authors read and approved the final article.
Author Disclosure Statement
No competing financial interests exist.
Funding Information
This work is supported by the State University of Piauí by granting full removal to the first author of his teaching activities. This work is partially funded by the Brazilian National Council for Scientific and Technological Development (CNPq), and the State of Maranhão Research Funding Agency (FAPEMA).
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
