Serious Game-Based Training for Improved Control of a Temporalis Electromyography Interface for Controlling Powered Wheelchairs

Abstract

Background:

Amyotrophic lateral sclerosis (ALS) is characterized by a significant decrease in mobility due to its neurodegenerative etiology. Throughout the progression of ALS, patients lose independence in their motor control. An electromyography (EMG) interface that enables control of a powered wheelchair can restore autonomous movement. The use of serious game-based training that mimics EMG-temporalis wheelchair controls can be used as training prior to wheelchair usage in the real world.

Purpose:

Limbitless Runner, a forearm EMG training game, found improved player performance with no significant difference noted between the free and structured play training. This study aims to investigate the generalizability of this finding when applied to a temporalis EMG system.

Methods:

Participants were given Limbitless Runner’s quantifiable, structured training mode, the “ring challenge,” for a scored pretest. Participants were then assigned to one of three cohorts for training: Limbitless Journey, Limbitless Runner “free play mode,” or the Limbitless Runner “ring challenge.” To assess improvement in user’s performance, the ring challenge was repeated and scored. Quantitative surveys, including the Game User Experience Satisfaction Scale (GUESS) and the System Usability Scale (SUS), were given to gauge users’ perceptions of the games.

Results:

All cohorts showed a user score improvement; however, the ring challenge (Cohort 2) showed the highest improvement, from an average of 22.9 rings in the pretest to 31.1 rings posttraining. The SUS and GUESS scores were not significantly different between the two different video games, demonstrating that both versions can be satisfying or acceptable platforms for users.

Keywords

Game-based training autonomy EMG interactive games games for health prosthetics

Introduction

Amyotrophic lateral sclerosis (ALS) is a neurodegenerative disease that primarily targets the nervous system’s motor neurons, which enable ambulation.¹ ALS is presented in 2 out of every 100,000 people per year and typically manifests late in adults, leading to a mean age range of diagnosis of 54–64.² Though ALS may have various onset types, the most common clinical presentation of ALS includes distal muscle weakness that spreads to proximal muscles with disease progression.¹ Individuals with ALS experience deficits in motor function that can inhibit voluntary muscle control, interrupting daily activities.³

Assistive technology such as electric-powered wheelchairs can play an important role in improving patient mobility in daily life.⁴ However, during later stages of ALS, using a joystick required for directing traditional wheelchairs can be difficult due to muscle deterioration.⁴ Though a variety of alternative control schemes are currently available on the market, the ALS population has the added concern of a continuous disease progression. Interfaces that allow for modulation of their control scheme are thus ideal for the unique nature of neurodegenerative disease. One study proposes the use of electromyography (EMG) for controlling a power wheelchair to increase autonomous mobility.⁵ Surface EMG electrodes placed on a specific muscle group pick up on electrical impulses or spikes in electrical activity.⁶ Electrical activity varies based on how hard the user contracts the targeted muscle. Preset thresholds based on the magnitude of contraction lead to different forms of actuation in the system interface.

This novel wheelchair-EMG interface was developed⁵ using electrodes placed on the temporalis muscle. The temporalis muscle, involved in mastication, was selected as the target for EMG, as individuals with spinal onset of ALS, who comprise the majority of the ALS population,² display a significantly lower reduction of bite force compared to finger force.⁷ This finding suggests that the temporalis muscle provides a viable target for EMG usage; however, the modularity of the interface allows for the usage of any muscle preferred. In this case, the intensity of temporalis muscle flexion determines the direction of wheelchair movement (straight, left, and right). The Dalhousie wheelchair skills test was used to assess mastery of wheelchair controls, in which participants were instructed to perform specific movements.⁸ Results showed that patients perceived a greater degree of independence and that the interface was a viable system to operate for patients with reduced dexterity.⁸ However, researchers reported participants had difficulty with discretizing temporalis muscle contractions and calibrating the system. Training prior to the use of the interface is anticipated to lead to better outcomes for patients.⁸

Serious games provide an interactive learning environment that can be designed to develop specific skills.⁹ To improve user control in navigation of the wheelchair-EMG system, a serious game called Limbitless Journey was specifically developed for ALS patients to improve muscular control.¹⁰ This game features a calm in-game environment, designed with varying obstacles and difficulty as levels progress to mimic the everyday obstacles a wheelchair user might navigate.¹⁰ Similarly, other serious games, such as Limbitless Runner, were developed for the EMG interface of myoelectric prosthesis. To evaluate the efficacy of the serious game in improving discretization of muscle control, one study utilized Limbitless Runner on forearm muscles.¹¹ Participants were instructed to play Limbitless Runner and were split among two cohorts: a structured “ring challenge” mode and a “free-play” mode.¹¹ These modes required participants to flex their forearm muscles at specific intensities in order to prompt on-screen characters to jump through rings or over obstacles.¹¹ Pre- and post-test scores demonstrated significant improvements, establishing that Limbitless Runner could be used to improve muscle contractions in the upper limb muscles of the arm.¹¹ To expand the resources available to ALS patients, this study aims to evaluate the generalizability of these findings to the temporalis muscle group and other serious games.

Methodology

Purpose

The purpose of this research was to determine whether the use of free and structured gameplay modalities yields similar improvement with the temporalis as the target muscle. Since a previous study showed the merits of serious game-based training for the forearm muscles for prosthesis training, another goal was to investigate if those successful methods were generalizable to the temporalis muscle.

Participants

Eligibility for this study included participants between the ages of 18 and 65 who have the ability to flex the temporalis muscle and no prior experience utilizing the EMG-temporalis interface. Able-bodied participants (n = 30) were selected for the completion of this study in order to determine generalizability against past works and prior to the release of the game to the ALS population. Ensuring the efficacy of the serious game before usage by individuals with ALS allows the optimization of the gameplay, eye-tracking technology, and EMG interface. Participants were first taken through a verbal consent process where the procedure, risk, and benefits were explained. Participants’ emails were linked to a randomly generated 7-digit code, which was used for cohort determination and to fill out post-study qualtrics surveys. An EMG video game controller was then placed on the individual’s temporalis muscle of choice (left or right), and researcher-assisted calibration to the EMG-game interface was performed. This procedure was approved by the University of Central Florida Institutional Review Board (STUDY00005987).

A total of 30 participants aged between 18 and 24 participated in the study, with 10 per cohort. Technological issues following consent did not allow 1 participant to finish pre- and post-tests, which were not included in the data analysis.

Detection and processing of electromyographic signals

The in-game actions of both Limbitless Journey and Limbitless Runner are controlled through player muscular contraction. The surface electrodes found in the video game controllers are capable of detecting changes in voltage generated by the recruitment of motor units in a noninvasive manner.⁶ Voltage generated by such actions varies depending on the degree of muscular movement that occurs. To aid in the processing of such signals, the facility has developed a routine featuring the amplification, rectification, application of a band-pass filter, and smoothing. This method, which is utilized by prosthesis and wheelchair interface users alike, allows for the generation of an electromyographic signal with less noise. The simplification of the voltage then allows for more efficient determination of players’ respective calibration thresholds. Limbitless Runner interprets flexes into jumps, with the magnitude of the flex resulting in corresponding jump heights. A large flex will jump high while a smaller flex will jump at a lower height. When calibrating, the system sets a continuous range between the user’s noise at rest and the highest flex they can consistently achieve. The wheelchair-EMG interface requires discrete levels of muscular contraction to enable steering control. As such, three distinguishable calibration thresholds are needed for the ability to turn left, right, and move forward/backward. The thresholds are user configurable both before and during the gameplay. To increase efficiency, researchers aided in the placement of video game controllers and participant calibration.

Framework and training

To establish a baseline score, participants initially completed a pre-test consisting of Limbitless Runner’s quantifiable “ring challenge” mode. The “ring challenge” includes 50 rings placed at varying heights, which the user’s character needs to jump through to earn points. Fundamentally, both games operate similarly. Upon release of the temporalis flexion, the analog EMG signal has a set minimum and maximum numerical value that the flex value would fall within. These values are then normalized to a scale from 0 to 1, which allows for individual variability based on muscle strengths.

In Limbitless Runner, this normalized signal is correlated to jump height via linear interpolation, where stronger flexes result in higher jumps. As previously mentioned, this allows for each user’s temporalis muscle flexion to be directly associated with the jump height of the character in the game. In Limbitless Journey, different ranges of flex intensities from the normalized values correspond to directional movements (e.g., left, right, and forward/backward).

The in-game environment for Limbitless Runner is set in a laboratory-like setting, inspired by manufacturing spaces. Players undergo an endless run, encountering obstacles including robotic grippers, wooden crates, and 3D printers. Limbitless Journey consists of four different levels: Serenity, Ethereal, Shadow, and Warrior. These levels feature calming mountainscapes, pillared architecture among the clouds, village nightscapes, and a lava-filled fortress, respectively.

The pre-test score was recorded before participants were randomly assigned to one of three cohorts: Limbitless Runner “free-play,” Limbitless Runner “ring challenge,” and Limbitless Journey (Cohorts 1, 2, and 3, respectively). After playing through 15 minutes of the serious game according to the assigned cohort, the ring challenge was repeated, and the post-test score was recorded. An example of the testing software and equipment setup are displayed in Figure 1.

Each cohort represented a different gameplay modality in order to evaluate the efficacy of both unstructured and structured training. Cohort 1 included individuals who played through Limbitless Runner “free-play” mode, which consists of an endless runner platform where users’ flexes are used to jump over obstacles and collect stars. Set in a research lab environment, users play from a third-person perspective, flexing In Cohort 2, the training was the Limbitless Runner “ring challenge,” where participants were challenged to match temporalis muscle flexion strength to jump through randomly generated rings. Cohort 3 included Limbitless Journey, in which the magnitude of temporalis muscle flexions corresponds to different movements for the on-screen character as users navigate through an in-game environment. For Limbitless Journey in particular, specific flexion strengths correlate to different directions, with a light flex correlates to a right turn, a medium flex to a left turn, and a hard flex to the forward direction.

Participants then completed several surveys including the Game User Experience Satisfaction Scale (GUESS) and System Usability Scale (SUS). These questionnaires were targeted toward assessing participants’ experiences and understanding interface usability.

Questionnaires

In tandem with the quantitative results gathered from this study, questionnaires were used to assess user’s satisfaction with the gameplay experience and their perception of usability. GUESS was designed to evaluate elements that may contribute to video game enjoyment.¹² GUESS is a 7-point Likert scale that includes nine subcategories: Narrative, Playability, Player Engrossment, Enjoyment, Creative Freedom, Audio Aesthetics, Visual Aesthetics, Social Connectivity, and Player gratification.

SUS is a Likert scale that measures the effectiveness, efficiency, and satisfaction of a given system.¹² The scale ranges from “Strongly Agree” to “Strongly Disagree” on a 100-point scale. Six distinguishable categories have been identified and deemed acceptable to determine perceived usability: worst imaginable, poor, ok, good, excellent, and best imaginable.¹³

The SUS and GUESS are important for making sure that the game training is effective for participants.

Results

Pre- and Posttest Scores

The “ring challenge” Limbitless Runner mode was used as a pre- and post-test for establishing a measure of improvement. Pretest averages for cohorts 1, 2, and 3 recorded at the beginning of each study session. After participants completed 15 minutes of assigned video game training, the ring challenge was repeated for cohorts 1, 2, and 3. Both pre- and post-test scores are listed in Table 1. Significant differences between the cohorts were evaluated using paired t-tests with a significance value of 0.05. Cohorts 1 and 2 showed statistically significant improvements between the pre- and post-tests (P = 0.01) and are shown in Figure 2.

FIG. 1.

Image depicting the video games used in the study and game setup for a participant playing Limbitless Runner freeplay mode: Limbitless Journey (left), Game Setup (middle), and Limbitless Runner (right).

FIG. 2.

Graph depicting Ring Challenge scores before and after completion of respective Cohort training. The max success score is 50 ring jumps. *Statistically significant difference (P < 0.05), error bars represent the standard error.

Table 1.

Average Pre- and Posttest Scores for Each Cohort Type Are Listed

Cohort type	Pretest score	Posttest score	Performance change
Cohort 1	27.5 ± 6.60	32.4 ± 6.35	+4.9
Cohort 2	22.9 ± 5.88	31.1 ± 5.99	+8.2
Cohort 3	29.8 ± 4.76	31.9 ± 5.42	+2.1

Demographic

Demographic categories of video game confidence (low, medium, or high) and temporalis laterality (right vs. left) were identified and compared to determine if these factors contributed to differences in the post-test scores. The video game confidence demographic category showed a statistically significant difference regarding confidence with video games based on prior video game experience. Specifically, participants who indicated lower confidence in video game ability rated usability as higher than those with medium confidence (P = 0.017). A general trend shown in Figure 3 seemed to indicate that individuals with lower video game experience scored usability higher.

FIG. 3.

Participant self-reported video game confidence level and their average reported SUS score.¹⁴ SUS, System Usability Scale.

Overall average SUS score

Participants answered questions based on the usability of the EMG interface with serious game training after the post-test was completed. Participants answered the SUS questions based on cohort-specific experiences. Cohort 1 and 2 correspond to Limbitless Runner, while Cohort 3 pertains to Limbitless Journey. Overall, Limbitless Runner scored 75.25, which is within the “Good” designation. Limbitless Journey scored 70.56, which falls into the “OK” range on the Average SUS score scale. A statistical analysis showed no significant usability differences between the two games.

Two-sample hypothesis tests were conducted to assess inter-cohort usability differences. Cohort 1 versus Cohort 2 and Cohort 1 versus Cohort 3 did not show statistically significant differences (P > 0.05). Cohort 2 had statistically higher SUS scores than Cohort 3 (P = 0.039).

GUESS subscores

GUESS gives insight into participant experience regarding the games. Figure 4 shows Cohort 1, Cohort 2, and Cohort 3 had means of 3.762, 4.092, and 3.994, respectively. Table 2 displays GUESS subscores for each cohort. Overall positive perceptions are indicated by a rating above “Neither agree nor disagree” or a “4” on the graph, which was the case for 8 out of the 9 GUESS categories irrespective of cohort. Social connectivity had an average rating slightly below 4, which may be attributed to the lack of multiplayer capabilities.

FIG. 4.

Average GUESS score per category from each cohort.¹² GUESS, Game User Experience Satisfaction Scale.

Table 2.

Average GUESS Subscale Scores per Cohort

GUESS subscale	Cohort 1	Cohort 2	Cohort 3
Usability/Playability	5.96	6.08	5.89
Narratives	4.98	4.98	5.30
Play engrossment	4.69	5.08	4.50
Enjoyment	6.20	6.36	6.02
Creative freedom	4.67	5.08	5.38
Audio aesthetics	5.88	6.35	5.97
Personal gratification	5.98	6.26	6.09
Social connectivity	3.76	4.09	3.94
Visual aesthetics	6.13	6.30	6.11

GUESS, Game User Experience Satisfaction Scale.

Discussion

Primary findings

This study aimed to determine the efficacy of various training modalities to improve the usage of a temporalis EMG interface. Results displayed an improvement in post-test score in all three cohorts, with varying levels of improvement in each of the cohorts. The Ring challenge mode (Cohort 2) had the highest overall significant improvement of 8.2 rings on average (P = 0.010), which can be attributed to the positive influence of task-based muscle training.¹³ Free play mode (Cohort 1) also had significant improvements with an average increase of 4.6 rings (P = 0.046). Limbitless Journey (Cohort 3) did not demonstrate statistically significant improvements (P = 1.113); however, it still had an average increase of 2.1 rings between pre- and post-tests scores. This may indicate that although there is an overall increase, the influence of this training is not as effective. The reduced improvement of ring challenge scores in Cohorts 1 and 3 may be due to the lack of task-specific training in these games.

SUS

The overall SUS score for Limbitless Runner was 75.25, falling between the “good” and “excellent” categories. Limbitless Journey had a score of 70.56, which fell between “OK” and “Good” on the SUS scale.¹⁴ There was no significant difference between the overall SUS scores between each of the games (P = 0.074), highlighting that both Runner and Journey provided a viable EMG interface training platform for individuals. One significant finding to be noted is the SUS score between Cohorts 2 and 3 (P = 0.039). Participants utilized the same control scheme and had more task-specific training throughout Cohort 2, which may explain the increased SUS score experienced in Cohort 2. This suggests that the training experience with the games impacts the perceived usability of the system overall.

GUESS

Overall scores showed that all cohorts had a rating above “neither agree nor disagree” (ratings above 4) for 8 out of 9 categories, which corresponds to a positive perception of both Runner and Journey. The social connectivity category did have a rating below 4; however, neither game was designed to provide multiplayer capabilities, which usually include social display menus, leaderboards, and ranking badges.¹⁵ These results indicate that the games were successful in delivering single-player gamified training experiences.

Demographics

Three demographic categories were of interest: video game confidence and temporalis muscle laterality. These groups are important toward understanding the differences and similarities between the current and past studies.

Prior video game experience

Main differences between confidence levels with video games were observed in the SUS score and personal gratification category of GUESS. A lower video game confidence score that a participant listed corresponded to higher SUS scores, with usability being scored significantly higher when compared to medium confidence individuals (P = 0.017). On average, individuals with lower confidence levels rated the EMG-temporalis interface to be more usable. This may be due to the increased impact and satisfaction of the novel interface on video game users with little prior experience with other game systems. As such, participants with little experience do not have a strong precedent for interfaces; they may be able to adapt to the EMG interface faster than those who have had extensive video game experience.

Similarly, this trend is reflected in the Personal Gratification category in GUESS. Individuals with lower confidence in video games had a higher personal gratification score than those with medium or high confidence (P = 0.008, P = 0.038, respectively). Again, this can be attributed to lower-confidence individuals having less experience with other game systems, allowing them to feel more satisfied with a novel interface.

Temporalis laterality

Participants were able to choose which temporalis muscle the EMG interface was placed on (left or right). No significant differences were found between participants who used the EMG interface on the right versus the left side.

Limitations

It is important to note that the sample used in this study includes only able-bodied participants, which is not representative of the intended users of this training. The purpose of this study was to determine the generalizability of serious training games for muscular discretization. Evaluation of serious training games for forearm muscle discretization utilized college-aged, nondisabled participants for the study population.¹¹ In order to then simulate the prior study’s conditions and eliminate confounding variables, a similar population was recruited for this study. The questionnaires involved in this study, the SUS¹⁴ and GUESS,¹² also provide insight into player’s perceptions, particularly on usability. Though Limbitless Journey has an additional, hands-free option for in-game navigation, Limbitless Runner does not yet have such inclusions. Though individuals with ALS were not the original target audience of the endless runner game, a hands-free modality should be considered to allow players with a range of functional control of their limbs to play. The lesser likelihood in force reduction for the temporalis muscle compared to the muscles of the extremities implies that such an EMG-wheelchair interface may still prove efficacious for the ALS population. The recruitment of individuals with tetraplegia, the absence of voluntary movement of the extremities, will be considered in subsequent projects for the EMG-wheelchair interface. The implementation of other hands-free selection modalities in the game to ensure participants with a range of upper extremity control can play will be considered. Future work seeks to provide access to individuals with ALS to the EMG-wheelchair interface with these serious training games in order to determine whether improved acclimation to the control scheme occurs.

However, these findings were targeted toward establishing that the temporalis interface was a muscle group that can be improved by game-based training prior to burdening vulnerable patient populations with the time required to be accustomed to the system. The next steps would be to implement this game-based training in ALS populations and observe if there is an increased improvement in wheelchair usage.

Another limitation is the power of task-specific training in the improvement of pre- and post-test scores. Since the participants in the ring challenge cohort experienced specific training, improvement in scores may be attributed to familiarity with this game rather than an actual increase in skill or ability to control. This may limit the generalizability of these results when applied to a different game or control scheme. The current study was designed in this manner based on the availability of performance testing games that were developed in-house. In future studies, the implementation of a different post-test task that tests the same flexion controls but varies in gameplay could control for this confounding training effect.

Conclusion and Future Work

Comparative testing was performed to compare this study with the previously mentioned Limbitless Runner study for upper limb muscle applications. A 2-sample t-test indicated that there was a significant difference in pre- and post-test scores on average. The temporalis EMG interface participants had significantly higher pre- and post-test scores (P < 0.001); however, the improvement in scores was not statistically significant (P = 0.059). SUS scores were reportedly higher in the previous study (P = 0.028), indicating that participants found the forearm-EMG interface as a more usable system. GUESS scores indicated no inter-study differences, highlighting that the temporalis and forearm EMG interfaces had similar perceptions of enjoyability.

Overall, an improvement in temporalis-EMG interface usage is seen with the implementation of serious game-based training on a temporalis-EMG interface. Additionally, SUS and GUESS surveys showed a positive perception of both the games usability and enjoyment. This shows that the findings of the prior Runner¹¹ study that tested training outcomes of forearm flexions were generalizable to the temporalis muscle and can be used to improve temporalis muscle flex discretization among participants. Such results suggest that the training of facial muscles can be considered for other research ventures at a relatively fast rate, including rehabilitation. Next steps include using survey and participant feedback to make adjustments to Runner and Journey accordingly.

Authors’ Contributions

C.M.D.: Writing—original draft (equal), visualization (lead). E.B.: Formal analysis (lead). M.P.: Writing—original draft (equal), visualization. V.R.: Writing—original draft (equal), visualization (equal). P.S.: Writing—original draft (equal), conceptualization (lead), methodology (lead). M.D.: Software (lead). J.S.: Software (lead), hardware (lead). A.M.: Project administration (lead), supervision (lead), funding acquisition (lead), conceptualization (lead), methodology (lead).

Footnotes

Acknowledgments

The researchers appreciate the philanthropic support for undergraduate research from the Paul B. Hunter & Constance D. Hunter Charitable Foundation, the Don & Lorraine Freeberg Foundation, and the Albert E. & Birdie W. Einstein Foundation, as well as the Pabst Steinmetz Foundation for their support of undergraduate research. The authors declare no competing interests and want to disclose that Limbitless Solutions has received a Unity for Humanity Grant for an unrelated training game, along with financial support from Autodesk (3D design software company), Adobe, Microsoft, and Stratasys (3D printer manufacturer).

Author Disclosure Statement

The researchers have no conflict of interests exist.

Funding Information

This research was internally funded.

Data Availability

Data Supporting this study is available upon request.

References

Masrori

, Van Damme

. Amyotrophic lateral sclerosis: A clinical review. Eur J Neurol, 2020; 27(10):1918–1929; doi: 10.1111/ene.14393

Longinetti

, Fang

. Epidemiology of amyotrophic lateral sclerosis: An update of recent literature. Curr Opin Neurol, 2019; 32(5):771–776; doi: 10.1097/WCO.0000000000000730

Duranti

, Villa

. Muscle involvement in amyotrophic lateral sclerosis: Understanding the pathogenesis and advancing therapeutics. Biomolecules, 2023; 13(11):1582; doi: 10.3390/biom13111582

Bona

, Donvito

, Cozza

, et al. The development of an augmented reality device for the autonomous management of the electric bed and the electric wheelchair for patients with amyotrophic lateral sclerosis: A pilot study. Disabil Rehabil Assist Technol, 2021; 16(5):513–519; doi: 10.1080/17483107.2019.1683237

Manero

, Oskarsson

, Sparkman

, et al. Xavier Electromyographic wheelchair control and virtual training. In: Virtual, Augmented and Mixed Reality. Multimodal Interaction. ( Chen

JYC

, Fragomeni

, eds.) Springer International Publishing: Cham; 2019; pp. 133–142; doi: 10.1007/978-3-030-21607-8_10

Zheng

, Crouch

, Eggleston

. Surface electromyography as a natural human–machine interface: A review. IEEE Sensors J, 2022; 22(10):9198–9214; doi: 10.1109/JSEN.2022.3165988

Riera-Punet

, Martinez-Gomis

, Paipa

, et al. Alterations in the masticatory system in patients with amyotrophic lateral sclerosis. J Oral Facial Pain Headache, 2018; 32(1):84–90; doi: 10.11607/ofph.1882

Manero

, McLinden

, Sparkman

, et al. Evaluating surface EMG control of motorized wheelchairs for amyotrophic lateral sclerosis patients. J Neuroeng Rehabil, 2022; 19(1):88; doi: 10.1186/s12984-022-01066-8

Ricciardi

, De Paolis

. A comprehensive review of serious games in health professions. Int J Comput Games Technol, 2014; 2014:1–11; doi: 10.1155/2014/787968

10.

Smith

, Dombrowski

, McLinden

, et al. Advancing dignity for adaptive wheelchair users via a hybrid eye tracking and electromyography training game. In: 2022 Symposium on Eye Tracking Research and Applications. Association for Computing Machinery: New York, NY; 2022; pp. 1–7.

11.

McLinden

, Smith

, Dombrowski

, et al. Utilizing Electromyographic video games controllers to improve outcomes for prosthesis users. Appl Psychophysiol Biofeedback, 2024; 49(1):63–69; doi: 10.1007/s10484-023-09598-y

12.

Phan

, Keebler

, Chaparro

. The development and validation of the Game User Experience Satisfaction Scale (GUESS). Hum Factors, 2016; 58(8):1217–1247; doi: 10.1177/0018720816669646

13.

Jeon

, Kim

, Park

. Effect of task-oriented training for people with stroke: A meta-analysis focused on repetitive or circuit training. Top Stroke Rehabil, 2015; 22(1):34–43; doi: 10.1179/1074935714Z.0000000035

14.

Bangor

, Kortum

, Miller

. The System Usability Scale (SUS): An empirical evaluation. Int J Hum Comput Interact, 2008; 24(6):574–594; doi: 10.1080/10447310802205776

15.

Haque

, O’Broin

, Kehoe

. Using game elements to guide postgraduate research students to promote progression and social connectedness. In: European Conference on Games Based Learning. Graz, Austria; 2017.