Patient Engagement Is Related to Impairment Reduction During Digital Game-Based Therapy in Stroke

Abstract

Objective:

Upper limb impairment in the chronic phase of stroke recovery is persistent, disabling, and difficult to treat. The objectives of this study were to determine whether therapeutic enjoyment is related to clinical improvement after upper limb rehabilitation and to assess the feasibility of a therapy gaming system.

Materials and Methods:

Ten chronic stroke survivors with persistent upper limb impairment were enrolled in the study. Upper limb impairment was evaluated by using the Fugl-Meyer Assessment of Upper Extremity Function (FMA-UE). The Physical Activity Enjoyment Scale (PACES) assessed the level of therapy enjoyment, and the System Usability Scale (SUS) measured the ease of operation of the game. Upper limb therapy involved 30 minutes of novel digital gaming therapy, three times per week, for 6 weeks.

Results:

The average improvement in the FMA-UE after the digital gaming therapy was 2.8 (±2.1) points. Participants scored the digital gaming system as having good usability (SUS: 72 ± 7.9), and the physical activity as enjoyable (PACES: 65.8 ± 10.6). There was a strong positive correlation between improvement in the FMA-UE score and the PACES (Spearman's Rho = 0.84; P < 0.002).

Conclusion:

This pilot study demonstrates the feasibility and potential for improvements in upper limb motor function by using digital gaming in the chronic stroke patient population. The positive correlation found between therapy enjoyment and clinical gains highlights the importance of engagement in therapy to optimize outcomes in chronic stroke survivors.

Introduction

Stroke is the leading cause of permanent motor disability in adults, and the number of stroke survivors is projected to increase significantly to almost 80,000,000 by 2030.^1,2 A major hallmark of stroke is loss-of-motor function, with 70%–85% of strokes leading to impairments in voluntary movement.^3,4 Indeed, 60% of chronic stroke survivors report significant upper limb impairments that interfere with activities of daily living and functional independence,^5–10 which is associated with decreased quality of life¹¹ and increased anxiety.¹² Thus, improvement of upper limb function in chronic stroke survivors is crucial to maximizing long-term recovery, independence, and quality of life.¹³

Despite the importance of upper limb recovery, little is known about how to optimize rehabilitation of the upper limb in chronic stroke survivors.^13,14 Multiple meta-analyses have failed to isolate a single therapeutic approach that is reliably effective for upper limb recovery after chronic stroke.^13,15–17 Literature from animal models advocates for high repetitions and enriched environments, in combination with task-specific training, to boost chronic stroke recovery.^18,19

Unfortunately, human studies with chronic stroke survivors are less consistent: Increased intensity of therapy does not always equate to a better outcome.²⁰ Nonetheless, existing therapies that show promise for improving functional outcomes for chronic stroke survivors include repetitive task specific training,^21–23 robotic-assisted training,^16,24 and Constraint Induced Movement Therapy (CIMT).^3,25–27 Elements of these interventions represent the principles of neuro-plasticity found to be successful in animal models; however, evidence proving their clinical efficacy for a broad range of chronic stroke survivors remains unclear.^28–30

In addition, conventional therapeutic approaches to chronic stroke rehabilitation suffer from several logistical and ideological flaws. They often require regular clinical visits, which can be expensive and inconvenient. Alternatively, they may involve self-directed home exercises, which are often monotonous and difficult to complete without feedback.^31,32 In both cases, engagement in therapy is a key factor in ensuring rehabilitation is completed with adequate intensity and regularity.

Consequently, an emerging approach to stroke rehabilitation is therapy gaming. Therapy gaming has the potential to provide rehabilitation services and improve motor function at a cost and quality that is comparable or superior to conventional therapy.^33,34 Evidence suggests that exercise performed in an engaged state can significantly improve therapeutic outcome,^35–38 patient motivation, and participation in therapy exercises.^33,39,40 Thus, we propose that enjoyment is a crucial, and often overlooked feature of successful rehabilitation in stroke survivors.

The objectives of this study were to investigate the feasibility and clinical outcomes of a novel digital therapy gaming system on upper limb function and to explore the correlation between therapy engagement and recovery in chronic stroke survivors.

Methods

Ethics statement

Approval for this study was granted by the Burke Rehabilitation Hospital Committee for Human Rights in Research, under IRB approval number BRC-483. Appropriate subjects were selected from an existing registry of chronic stroke survivors maintained by our clinic. All participants signed informed consent documents before study inclusion.

Inclusion and exclusion criteria

Subjects were enrolled in the study if they met the following inclusion criteria: (1) diagnosis of first, single, and unilateral ischemic or hemorrhagic stroke, at least 6 months prior; (2) residual motor impairment of the left or right upper limb; (3) at least 10° of active wrist extension and supination; and (4) ability to follow instructions (minimum score of 24 on the Mini Mental State Examination). Stroke survivors were excluded if they had a fixed joint contracture of the affected wrist, hypertonicity (measured by a score higher than 2 on the Modified Ashworth Scale), or were enrolled in concurrent upper limb therapy.

Description of the digital game: “GesAircraft”

The system used in this study was a custom-designed game called “GesAircraft.” GesAircraft is a digital gaming environment that is designed to train three upper limb joint movements: wrist flexion/extension, ulnar/radial deviation, and forearm pronation/supination. The objective of the game is to use therapist-designated movements to control an airplane avatar through ring-shaped obstacles. The level of task difficulty is calibrated according to the individual's wrist and forearm range of motion (ROM), obtained by completing a short calibration game called “test flight.” The game is completed in levels, with each level advancement bringing incremental increases in task difficulty by altering ring obstacle size and depth (to train movement precision), airplane movement speed (to train rapid movement), and dispersion of the rings (to train a larger range of movement).

The leap motion controller (Fig. 1A) identified hand movements (Fig. 1B), and used them to control the game's avatar. The game was installed on a Microsoft Surface Pro 3 (Fig. 1C) to provide a portable and convenient therapy interface.

FIG. 1.

The LMC. (A) The LMC (inset) is a markerless motion capture device that tracks hand, wrist, and forearm motion in real time. (B) The hand is detected by the LMC and converted into centroids in three-dimensional space that represent hand, wrist, and digit segments. These data can be transferred into a custom gaming environment. (C) We used low-cost, highly portable Microsoft Surface Pro 3 tablets to play the GesAircraft game. LMC, leap motion controller.

Study intervention

Subjects were seated at a table with a Microsoft Surface Pro 3 tablet set up directly in front. The hemiplegic forearm was positioned 6 cm over the end on the armrest, allowing unrestricted forearm rotation, wrist, and finger ROM, while preventing compensatory elbow or shoulder movements with the use of light constraints (Fig. 2).

FIG. 2.

Subject position during therapy gaming protocol. Subjects were seated in a chair with armrests, with the LMC position below the armrest. If the subject was unable to keep the forearm resting on the armrest during the exercise session, we used soft restraints to encourage the position seen in the image.

Subjects participated in 30 minutes of GesAircraft therapy, three times per week, for 6 weeks. Although the system was designed for home use, participants attended the clinic for this pilot study to allow observation of user–system interaction. The clinical observer did not provide additional therapy or encouragement, but was available to assist with the correct use of the system. The “test flight” was conducted before each therapy session to allow calibration of the participants' ROM. The 30-minute therapy session was broken into two 15-minute components: training wrist flexion/extension (Fig. 3A) and forearm pronation/supination (Fig. 3B). If a participant did not complete enough tasks (6/10 rings) successfully to progress to the next game level, the current level was repeated. If the same level was repeated three consecutive times, the participant went down a level of difficulty.

FIG. 3.

Examples of GesAircraft gameplay. GesAircraft was designed to train movements in three planes of motion: (A) wrist flexion/extension, (B) forearm pronation/supination, and (C) wrist ulnar/radial deviation. Only the movements represented in (A, B) were trained in this study.

Outcome measures

The primary outcome measure was a change in the Fugl-Meyer Assessment of Upper Extremity Function (FMA-UE)⁴¹ after 6 weeks, compared with the baseline evaluation. The FMA-UE assesses 33 items of motor function of the shoulder/elbow (36 points) and wrist/hand (30 points) with a maximum score of 66. The FMA-UE has been shown to have high test–retest reliability, interrater reliability, and construct validity in the chronic stroke population.^42,43 The Action Research Arm Test (ARAT) and Wolf Motor Function Test (WMFT) were collected as secondary outcome measures. The baseline and 6-week assessments were performed by a blinded evaluator.

Other secondary outcome measures were collected to determine how usable and enjoyable the subjects found the system. These measures included the System Usability Scale (SUS), the Physical Activity Enjoyment Scale (PACES), and the Beck Depression Inventory (BDI). The SUS is a 10-item scale (with a score from 0 to 100) that evaluates the subjective usability of software,^44,45 whereby subjects rate their agreement or disagreement on a five-point Likert scale from strongly agree to strongly disagree.⁴⁴ The SUS has been shown to be both reliable and valid for assessing perceived usability.^45–47 Subjects completed the SUS at the end of each week to obtain the patient's perspective on the ease of use and system implementation.

The PACES is an 18-item self-administered scale assessing the enjoyment of a physical activity. Research reports the PACES to be a valid and reliable tool of enjoyment.^48,49 The PACES uses a seven-point semantic differential approach. The total score is calculated so that a higher score indicates a higher level of enjoyment.⁵⁰ The PACES was completed at the end of each week of the study to determine whether enjoyment of the therapy changed within subjects over the course of the 6-week trial.

The BDI is a self-report measure of depression with strong sensitivity,⁵¹ predictive validity,⁵² and construct validity⁵³ in stroke survivors. The BDI was recorded to determine whether psychosocial factors influenced PACES scores or clinical outcomes.

To investigate the safety and feasibility of the therapy gaming system, subjective reports of pain, adverse reactions, or worsening of post-stroke symptoms were collected after each therapy session.

Statistical analysis

Statistical analysis was performed by using MATLAB software.⁵⁴ For all measures, except the WMFT, a paired t-test (P < 0.05) was chosen to determine statistically significant differences between pre- and post-therapy outcomes. A paired t-test was selected as significance scores of >0.05 were calculated for most outcome measures when assessed with the Shapiro-Wilk test of normality.

The “time” score for the WMFT did not follow a normal distribution according to the Shapiro-Wilk test. Therefore, WMFT scores were analyzed for statistical significance by using a Wilcoxon signed-rank test. We also assessed for correlations between certain metrics:

• Change in FMA-UE and PACES

• PACES and BDI

• Baseline FMA-UE and PACES

• Baseline FMA-UE and change in FMA-UE

We calculated a Spearman's rank correlation as a nonparametric method for measuring variable dependence. The descriptive statistics for the patient characteristics, compliance, and outcome measures are summarized in Table 1.

Table 1.

Subject Demographics and Baseline Outcome Measure Results

Gender	Seven male; three female
Age (years, SD)	69.5 (±9.9)
Time since stroke (months, SD)	78.4 (±52)
Stroke type	Eight ischemic; two hemorrhagic
Stroke side (hemisphere)	Nine left; one right
Baseline FMA-UE ± SD (range)	38.8/66 ± 12.6 (17–56)
Baseline WMFT score (SD)	Quality: 2.2/5 (±0.8); Time: 33.7/120 (±45.9)
Baseline PACES score (SD)	65.1/80 (±10.6)
Baseline SUS (SD)	72/100 (±7.9)
Baseline BDI (SD)	6.51/63 (±3.4)
Protocol compliance (%, SD)	100 (±0)

SD, standard deviation; FMA-UE, Fugl-Meyer Assessment of Upper Extremity Function; WMFT, Wolf Motor Function Test; PACES, Physical Activity Enjoyment Scale; SUS, System Usability Scale; BDI, Beck Depression Inventory.

A subject-size power analysis was not performed. This pilot trial aimed at establishing the approximate clinical effect of a novel emerging therapy approach, to allow an appropriately powered clinical trial to be conducted in future investigations.

Results

Demographics

Ten chronic stroke survivors (seven men) were recruited for the study. All subjects were at least 6 months post-stroke (mean: 78.4 months; range: 6–162 months) and had a wide range of upper limb impairment (FMA-UE; mean: 38.8; range: 17–56). Participants reported minimal-mild signs of depression during the study according to the BDI (6.5 ± 3.4). A detailed account of subject demographics can be seen in Table 1.

System usability, enjoyment metrics, and compliance: feasibility

The system was rated to have “good” usability according to the SUS scores (72/100 ± 7.9). All participants completed the 6-week protocol with a session compliance of 100%. No pain, adverse reactions, or worsening of post-stroke symptoms were reported. No significant changes in depression scores were measured.

Improvement in upper limb outcomes (Table 2)

There was a significant (P < 0.003), 2.8-point increase in the total FMA-UE between pre- and post-therapy measures. The observed improvements were more evident and statistically significant when wrist and hand FMA metrics were measured (average 6.5% increase; P < 0.003), compared with shoulder and elbow (average 2.5% increase). No significant improvement was reported in the ARAT. Although no movement quality improvements were evident in the WMFT, subjects demonstrated a significant (P < 0.05), 20% decrease in the time to complete the test.

Table 2.

Paired I Test Comparison Using the Pre- and Post-Therapy Outcome Measures

Metric	Pre-therapy	Post-therapy
FMA-UE (total/66)	38.8	41.6
FMA-UE (wrist-hand portion/30)	13.4	15.3
FMA-UE (shoulder-elbow portion/36)	25.4	26.3
WMFT (time/120 seconds)	33.7	24.2
WMFT (quality/5)	2.2	2.7
ARAT (time/120 seconds)	1.4	1.5
ARAT (quality/3)	28.9	28.6
BDI	6.5	6
PACES	65.1	69.5

Significant results are indicated in bold.

ARAT, Action Research Arm Test.

Improvement in Fugl-Meyer scores correlated with PACES

A key finding of this study was that change in “total” and “wrist-hand” FMA-UE scores correlated strongly and significantly with the PACES (Fig. 4A, B). No correlation was seen between baseline FMA-UE score and PACES (Fig. 4C), indicating that the initial level of impairment did not influence the subject's degree of enjoyment (Rho: −0.43, P = 0.21; Fig. 4C). Correlation coefficients and P-values are presented in Table 3.

FIG. 4.

Therapeutic enjoyment correlates with gains in FMA. We saw strong, positive correlations between increasing FMA-UE (A) and wrist-hand FMA-UE (B) scores, and subject-reported PACES. No such correlations were seen between pre-study FMA and PACES (C) or FMA-UE gains (D), indicating that they were not confounds to the correlations seen in (A, B). FMA, Fugl-Meyer Assessment; PACES, Physical Activity Enjoyment Scale.

Table 3.

Variables Correlated Against the Fugl-Meyer Assessment of Upper Extremity Function

Variables correlated	Spearman's Rho	P-value
Change in FMA-UE “total” and PACES	0.82	<0.005
Change in FMA-UE “wrist-hand” and PACES	0.74	<0.01
Baseline FMA-UE and PACES	−0.43	0.21
Baseline FMA-UE and change in FMA-UE “wrist-hand”	0.12	0.46
Baseline FMA-UE and change in FMA-UE “total”	−0.46	0.19
PACES and BDI	0.65	0.3

The only significant correlation was the change in FMA-UE total scores and PACES and the change in wrist-hand only FAM-UE scores and PACES.

Discussion

This pilot study suggests that improvements in upper limb impairment are related to the level of therapy enjoyment of chronic stroke survivors. Our results indicate that the GesAircraft system is a safe, usable, and enjoyable therapy approach to treat upper limb impairment in this patient population. Completion of our protocol did not result in adverse effects or worsening of function. The 100% compliance and retention of subjects during the 6-week trial indicates that the therapy was well accepted.

These results indicate that well-designed therapy games may lead to a reduction in upper limb impairment, even long after the onset of stroke. Although the average improvement in the FMA-UE did not reach the minimally clinically important difference of 5.25 points,⁵⁵ impairment reduction was comparable to results seen in robotic rehabilitation trials for chronic stroke.⁵⁶ The lack of significant improvements in upper limb impairment observed in the present study were likely related to several factors, including small sample size and a relatively short duration of both the therapy sessions (30 minutes) and the intervention period (6 weeks). Previous studies involving both robotic and conventional upper limb therapy for chronic stroke survivors advocate for a more intense and 12-week therapy program.^56,57 Since participants did not withdraw or report adverse effects when completing 30 minutes of the gaming intervention, a longer therapy session and intervention period is feasible for future trials, and may prove beneficial in promoting further upper limb recovery.

To date, there has been limited clinical implementation of digital gaming therapy due to difficulty with usability and implementation of the technology.^58–60 The SUS was administered to capture the appropriateness of purpose,⁴⁴ the effectiveness, efficiency, and satisfaction of using the GesAircraft system. A calculation of the scores from the SUS indicates that participants found the system to have “good” usability, which shows that the system exhibits “above average” usability in comparison to other systems across many industries.^45,46,61 The SUS guidelines⁴⁵ state that an average SUS score is 68, based on an analysis of 500 studies in various industries that employed the SUS, and recommend that any score above 68 is “above average” and can be considered for clinical use or further rehabilitation studies.^45,46

Growing evidence in the literature supports the importance of patient engagement during rehabilitation to optimize functional outcomes and patient-centered care.^62–65 Engagement is often described as multidimensional, for which enjoyment is one of the three major factors, along with cognitive and behavioral components.^64,66 Engagement has been proposed as an important feature of all rehabilitation programs, regardless of patient population or rehabilitation setting, to produce meaningful therapy outcomes.^64,67 Luker et al.⁶⁵ collated the perspectives, experiences, and preferences of stroke survivors and found that participants wanted enjoyment in therapy as part of their patient-centered care.⁶⁵

Despite the increasing value of enjoyment of therapy in the literature,^50,64,65,67 very few rehabilitation studies have assessed the level of patient enjoyment when investigating the effectiveness of a therapeutic approach. Hung et al.⁶⁸ used the PACES to investigate the enjoyment level of a Wii-fit™ intervention compared with conventional balance training for patients with chronic stroke. Results showed a higher level of enjoyment in the Wii fit group (PACES = 79.15 ± 7.49 compared with 72.47 ± 7.63) and a significant difference in static standing stability; however, there was no analysis of the correlation between enjoyment and improvements in balance.⁶⁸

Secondary analysis revealed a positive correlation between therapy enjoyment and upper limb functional improvement. This correlation is significant as it enhances our understanding of how to augment and optimize therapy for chronic stroke survivors. There are different possible explanations for the correlation observed between therapy enjoyment and upper limb improvements during therapy gaming. A previous review of video gaming in neurorehabilitation outlined the biological (dopaminergic) and behavioral changes that occur because of engagement during enjoyable game-play, which can lead to greater repetitions and duration of therapy.⁶⁹ It is possible that participants who found the GesAircraft game enjoyable completed more repetitions within the 30 minutes of therapy and actively engaged throughout the session to optimize their motor learning potential.⁷⁰

With more evidence for effective therapy methods, we can optimize productivity in rehabilitation that could potentially reduce length of hospital stay, medical expenses, and the burden of care after discharge. Here, we have associated therapeutic enjoyment and favorable rehabilitation outcomes. Future work should be directed toward developing actionable strategies to optimize enjoyment during therapy.

Study limitations

This study had several limitations that reduce the generalization and impact of the study results. A previous report indicates that the mean improvement in FMA-UE scores after upper limb rehabilitation reduces with increasing time post-stroke.⁵⁵ In this study, the inclusion of subjects with a wide range of baseline upper limb impairments and with a broad timeline since onset of stroke (from 6 months to 13.5 years) allowed the results to be applied to a broader population, but may have led to smaller gains in the primary outcome measure at 6 weeks. Furthermore, the duration of the therapy session was only 30 minutes. Some may argue that this short therapy duration was not sufficient to capture disengagement or boredom with the therapy interface.

Nonetheless, Lohse et al.⁶⁹ propose that gaming has the potential to prolong the time engaged in therapy and to delay the onset of disengagement within a session. Therefore, future studies could assess a longer intervention period to evaluate for the impact of time on the level of therapy enjoyment.

Another limitation was that the number of repetitions during the therapy was unknown. Although the duration of the activity was standardized, the quantity of activity was not recorded, and, therefore, we are unable to establish whether greater enjoyment led to more repetitions of the activity, or whether those who had greater gains in upper limb impairment performed more repetitions. Of note, two subjects did not improve on the FMA-UE during the trial. These subjects reported “average” enjoyment scores on the PACES (Fig. 4A, B)^45,46 and were consistently ranked within the lowest three PACES scores in the study, supporting the finding that therapy outcomes correlate to enjoyment levels.

Further limitations of this study include the predominance of male participants and left hemisphere strokes, the lack of a control group, and the small sample size. Therefore, the results of this study must be interpreted with caution. Nonetheless, these limitations justify further work in this field to verify the correlation between therapy enjoyment and improvements in upper limb impairment of chronic stroke survivors.

Conclusion

Digital gaming is an emerging therapy approach that shows increasing promise for stroke rehabilitation. This pilot study confirmed that the GesAircraft gaming interface was appropriate and usable for stroke rehabilitation, but there was an exciting positive correlation between therapy enjoyment and gains seen in motor function. Previous studies^62,63 have discussed the importance of patient engagement during rehabilitation to optimize functional outcomes. However, to our knowledge, this is the first study to demonstrate a significant correlation between therapeutic enjoyment and outcome in chronic stroke survivors. Further investigation in controlled trials with larger patient groups is required to confirm these findings.

Footnotes

Acknowledgments

The authors would like to acknowledge the Burke Foundation for funding. They would like to thank the clinicians and participants who took part in this study. In addition, the authors would like to acknowledge their programming team, Luis Disla, Tanjin Panna, and Owais Naeem, for developing the therapy gaming environment for them. D.J.E was supported by NICHD of the National Institutes of Health – USA, under award number R01HD069776.

Author Disclosure Statement

No competing financial interests exist.

References

Mukherjee

, Patil

. Epidemiology and the global burden of stroke. World Neurosurg, 2011; 76(6 Suppl):S85–S90.

Feigin

, Forouzanfar

, Krishnamurthi

, et al. Global and regional burden of stroke during 1990–2010: Findings from the Global Burden of Disease Study 2010. Lancet, 2014; 383:245–254.

Langhorne

, Coupar

, Pollock

. Motor recovery after stroke: A systematic review. Lancet Neurol, 2009; 8:741–754.

Dobkin

. Neurobiology of rehabilitation. Ann N Y Acad Sci, 2004; 1038:148–170.

Rosamond

, Flegal

, Furie

, et al. Heart disease and stroke statistics—2008 update: A report from the American Heart Association Statistics Committee and Stroke Statistics Subcommittee. Circulation, 2008; 117:e25–e146.

Kwakkel

, Kollen

, Van der Grond J

, Prevo

AJH

. Probability of regaining dexterity in the flaccid upper limb: Impact of severity of paresis and time since onset in acute stroke. Stroke, 2003; 34:2181–2186.

Sveen

, Bautz-Holter

, Sødring

, et al. Association between impairments, self-care ability and social activities 1 year after stroke. Disabil Rehabil, 1999; 21:372–377.

Broeks

, Lankhorst

, Rumping

, Prevo

. The long-term outcome of arm function after stroke: Results of a follow-up study. Disabil Rehabil, 1999; 21:357–364.

van Kuijk

, Pasman

, Hendricks

, et al. Predicting hand motor recovery in severe stroke: The role of motor evoked potentials in relation to early clinical assessment. Neurorehabil Neural Repair, 2009; 23:45–51.

10.

Patel

, Duncan

, Lai

, Studenski

. The relation between impairments and functional outcomes poststroke. Arch Phys Med Rehabil, 2000; 81:1357–1363.

11.

Samsa

, Matchar

. How strong is the relationship between functional status and quality of life among persons with stroke?. J Rehabil Res Dev, 2004; 41(3A):279–282.

12.

Morris

, van Wijck

, Joice

, Donaghy

. Predicting health related quality of life 6 months after stroke: The role of anxiety and upper limb dysfunction. Disabil Rehabil, 2013; 35:291–299.

13.

Pollock

, Farmer

, Brady

, et al. Interventions for improving upper limb function after stroke. Cochrane Database Syst Rev, 2014:CD010820.

14.

Dobkin

, Dobkin

, Patel

, et al. Strategies for stroke rehabilitation. Lancet Neurol, 2004; 3:528–536.

15.

Pollock

, Baer

, Campbell

, et al. Physical rehabilitation approaches for the recovery of function and mobility following stroke. Cochrane Database Syst Rev, 2014:CD001920.

16.

Mehrholz

, Hädrich

, Platz

, et al. Electromechanical and robot-assisted arm training for improving generic activities of daily living, arm function, and arm muscle strength after stroke. Cochrane Database Syst Rev, 2012:CD006876.

17.

Aziz

, Leonardi-Bee

, Phillips

, et al. Therapy-based rehabilitation services for patients living at home more than one year after stroke. Cochrane Database Syst Rev, 2008:CD005952.

18.

Plautz

, Milliken

, Nudo

. Effects of repetitive motor training on movement representations in adult squirrel monkeys: Role of use versus learning. Neurobiol Learn Mem, 2000; 74:27–55.

19.

Biernaskie

, Corbett

. Enriched rehabilitative training promotes improved forelimb motor function and enhanced dendritic growth after focal ischemic injury. J Neurosci, 2001; 21:5272–5280.

20.

Winstein

, Wolf

, Dromerick

, et al. Effect of a task-oriented rehabilitation program on upper extremity recovery following motor stroke: The ICARE randomized clinical trial. JAMA, 2016; 315:571–581.

21.

French

, Thomas

, Leathley

, et al. Repetitive task training for improving functional ability after stroke. Cochrane Database Syst Rev, 2007:CD006073.

22.

Arya

, Verma

, Garg

, et al. Meaningful task-specific training (MTST) for stroke rehabilitation: A randomized controlled trial. Top Stroke Rehabil, 2012; 19:193–211.

23.

Page

. Intensity versus task-specificity after stroke: How important is intensity?. Am J Phys Med Rehabil, 2003; 82:730–732.

24.

Prange

, Jannink

MJA

, Groothuis-Oudshoorn

CGM

, et al. Systematic review of the effect of robot-aided therapy on recovery of the hemiparetic arm after stroke. J Rehabil Res Dev, 43:171–184.

25.

Wolf

, Winstein

, Miller

, et al. Effect of constraint-induced movement therapy on upper extremity function 3 to 9 months after stroke: The EXCITE randomized clinical trial. JAMA, 2006; 296:2095–2104.

26.

Wolf

, Winstein

, Miller

, et al. Retention of upper limb function in stroke survivors who have received constraint-induced movement therapy: The EXCITE randomised trial. Lancet Neurol, 2008; 7:33–40.

27.

Grotta

, Noser

, Ro

, et al. Constraint-induced movement therapy. Stroke, 2004; 35(11 Suppl 1):2699–2701.

28.

Teasell

, Foley

, Bhogal

, Speechley

. Evidence-based review of stroke rehabilitation. Top Stroke Rehabil, 2003; 10:29–58.

29.

Miller

, Murray

, Richards

, et al. Comprehensive overview of nursing and interdisciplinary rehabilitation care of the stroke patient: A scientific statement from the American Heart Association. Stroke, 2010; 41:2402–2448.

30.

National Stroke Foundation. Clinical Guidelines for Stroke Management. Melbourne: NSF. 2010; http://strokefoundation.com.au/site/media/clinical_guidelines_stroke_management_2010_interactive.pdf.

31.

Page

, Levine

, Sisto

, et al. Stroke patients' and therapists' opinions of constraint-induced movement therapy. Clin Rehabil, 2002; 16:55–60.

32.

Hung

Y-X

, Huang

P-C

, Chen

K-T

, Chu

W-C

. What do stroke patients look for in game-based rehabilitation: A survey study. Medicine (Baltimore), 2016; 95:e3032.

33.

Putrino

, Zanders

, Rykman

, et al. Use of the GesAircraft video game for upper limb rehabilitation in stroke: A pilot study. Neurorehabil Neural Repair, 2015; 30:36–37.

34.

Putrino

. Telerehabilitation and emerging virtual reality approaches to stroke rehabilitation. Curr Opin Neurol, 2014; 27:631–636.

35.

Burdea

. Virtual rehabilitation—benefits and challenges. Methods Inf Med, 2003; 42:519–523.

36.

Burke

, McNeill

MDJ

, Charles

, et al. Optimising engagement for stroke rehabilitation using serious games. Vis Comput, 2009; 25:1085–1099.

37.

Maclean

, Pound

, Wolfe

, Rudd

. Qualitative analysis of stroke patients' motivation for rehabilitation. BMJ, 2000; 321:1051–1054.

38.

Burke

, McNeill

MDJ

, Charles

, et al. Optimising engagement for stroke rehabilitation using serious games. Vis Comput, 2009; 25(12):1085–1099.

39.

Smith

, Schoene

. The use of exercise-based videogames for training and rehabilitation of physical function in older adults: Current practice and guidelines for future research. Aging Health, 2012; 8:243–252.

40.

Hsu

, Thibodeau

, Wong

, et al. A “Wii” bit of fun: The effects of adding Nintendo Wii(^®) bowling to a standard exercise regimen for residents of long-term care with upper extremity dysfunction. Physiother Theory Pract, 2011; 27:185–193.

41.

Fugl-Meyer

, Jääskö L, Leyman I, et al. The post-stroke hemiplegic patient. 1. A method for evaluation of physical performance. Scand J Rehabil Med, 1975; 7:13–31.

42.

Duncan

, Propst

, Nelson

. Reliability of the Fugl-Meyer assessment of sensorimotor recovery following cerebrovascular accident. Phys Ther, 1983; 63:1606–1610.

43.

Hsieh

, Wu

, Lin

, et al. Responsiveness and validity of three outcome measures of motor function after stroke rehabilitation. Stroke, 2009; 40:1386–1391.

44.

Brooke

. SUS: A “quick and dirty” usability scale. In: Jordan

, Thomas

, McClelland

WB (eds.) Usability Evaluation in Industry. London, United Kingdom: CRC Press; 1996, pp. 189–194.

45.

Sauro

. Measuring Usability with the System Usability Scale (SUS). Measuring U. 2011. www.measuringu.com/sus.php (accessed January 9, 2016).

46.

Bangor

, Kortum

, Miller

. An empirical evaluation of the system usability scale. Int J Hum Comput Interact, 2008; 24:574–594.

47.

Lewis

, Sauro

. The factor structure of the system usability scale. In: Kurosu

(ed.) Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol 5619 LNCS; Springer, Berlin, Heidelberg; 2009, pp. 94–103.

48.

Kendzierski

, DeCarlo

. Physical activity enjoyment scale: Two validation studies. J Sport Exerc Psychol, 1991; 13:50–65.

49.

Heesch

, Mâsse

, Dunn

. Using Rasch modeling to re-evaluate three scales related to physical activity: Enjoyment, perceived benefits and perceived barriers. Health Educ Res, 2006; 21 Suppl 1:i58–i72.

50.

Mullen

, Olson

, Phillips

, et al. Measuring enjoyment of physical activity in older adults: Invariance of the physical activity enjoyment scale (paces) across groups and time. Int J Behav Nutr Phys Act, 2011; 8:103.

51.

House

, Dennis

, Mogridge

, et al. Mood disorders in the year after first stroke. Br J Psychiatry, 1991; 158:83–92.

52.

Desrosiers

, Noreau

, Rochette

, et al. Predictors of handicap situations following post-stroke rehabilitation. Disabil Rehabil, 2002; 24:774–785.

53.

Aben

, Verhey

, Lousberg

, et al. Validity of the beck depression inventory, hospital anxiety and depression scale, SCL-90, and hamilton depression rating scale as screening instruments for depression in stroke patients. Psychosomatics, 2002; 43:386–393.

54.

MathWorks.. MATLAB 9.1 and Statistics Toolbox 9.0. 2014; www.mathworks.com/products/matlab/ (accessed June 30, 2016).

55.

Page

, Fulk

, Boyne

. Clinically important differences for the upper-extremity Fugl-Meyer scale in people with minimal to moderate impairment due to chronic stroke. Phys Ther, 2012; 92:791–798.

56.

, Guarino

, Richards

, et al. Robot-assisted therapy for long-term upper-limb impairment after stroke. N Engl J Med, 2010; 362:1772–1783.

57.

Hsieh

, Wu

, Liao

, et al. Effects of treatment intensity in upper limb robot-assisted therapy for chronic stroke: A pilot randomized controlled trial. Neurorehabil Neural Repair, 2011; 25:503–511.

58.

Forsberg

, Nilsagård

, Boström

. Perceptions of using videogames in rehabilitation: A dual perspective of people with multiple sclerosis and physiotherapists. Disabil Rehabil, 2015; 37:338–344.

59.

Thomson

, Pollock

, Bugge

, Brady

. Commercial gaming devices for stroke upper limb rehabilitation: A survey of current practice. Disabil Rehabil Assist Technol, 2016; 11:454–461.

60.

Putnam

, Cheng

, Seymour

. Therapist perspectives: Wii active videogames use in inpatient settings with people who have had a brain injury. Games Health J, 2014; 3:366–370.

61.

Bangor

, Kortum

, Miller

. Determining what individual SUS scores mean: Adding an adjective rating scale. J Usability Stud, 2009; 4:114–123.

62.

Lequerica

, Rapport

, Loeher

, et al. Agitation in acquired brain injury: Impact on acute rehabilitation therapies. J Head Trauma Rehabil, 22:177–183.

63.

Kortte

, Falk

, Castillo

, et al. The hopkins rehabilitation engagement rating scale: Development and psychometric properties. Arch Phys Med Rehabil, 2007; 88:877–884.

64.

Lequerica

, Kortte

. Therapeutic engagement: A proposed model of engagement in medical rehabilitation. Am J Phys Med Rehabil, 2010; 89:415–422.

65.

Luker

, Lynch

, Bernhardsson

, et al. Stroke survivors' experiences of physical rehabilitation: A systematic review of qualitative studies. Arch Phys Med Rehabil, 2015; 96:1698–1708.e10.

66.

Frydenberg

, Ainley

MRV

. Student Motivation and Engagement. Schooling Issues Digest: Department of Education, Science and Training.

67.

Danzl

, Etter

, Andreatta

, Kitzman

. Facilitating neurorehabilitation through principles of engagement. J Allied Health, 2012; 41:35–41.

68.

Hung

J-W

, Chou

C-X

, Hsieh

Y-W

, et al. Randomized comparison trial of balance training by using exergaming and conventional weight-shift therapy in patients with chronic stroke. Arch Phys Med Rehabil, 2014; 95:1629–1637.

69.

Lohse

, Shirzad

, Verster

, et al. Video games and rehabilitation. J Neurol Phys Ther, 2013; 37:166–175.

70.

Kleim

, Jones

. Principles of experience-dependent neural plasticity: Implications for rehabilitation after brain damage. J Speech Lang Hear Res, 2008; 51:S225–S239.