Kinematic measures for upper limb motor assessment during robot-mediated training in patients with severe sub-acute stroke

Abstract

Purpose: Kinematic assessments are increasingly used as motor outcome measures during upper limb robot-assisted training, in addition to clinical scales. However, their relevance has not been evaluated much.

Methods: Thirty-eight patients with severe sub-acute stroke (age 56 ± 17 [19–87] years; time since stroke, 55 ± 22 days) carried out 16 sessions (average 3/week, 35 ± 15 days) of upper limb robot-assisted training combined with standard therapy. Pre/post motor performance was evaluated using the Fugl-Meyer Assessment scale, Motor Status Scale (MSS) and kinematic measures. Motor outcomes were compared and relationships between clinical and kinematic outcomes were analyzed.

Results: All clinical and kinematic outcomes improved after training (p < 0.01). FM score increased from 17.7 ± 10.0 to 28.6 ± 15.4. All baseline kinematic measures were strongly correlated with clinical scores. Correlations between clinical and kinematic changes were moderate (r = –0.65 for change in FM Proximal score and change in accuracy measure). However, smoothness and accuracy indicators were shown to be responsive measures.

Conclusion: This study demonstrated that baseline kinematic measures and their pre/post training changes were significantly correlated with clinical motor outcome measures. However, even if kinematic measures are valid for the evaluation of motor impairment we cannot propose to substitute common clinical measures of motor function which also evaluate functional abilities of the upper limb.

Keywords

Hemiparesis upper limb sub-acute stroke robotic therapy kinematics

1 Introduction

Over the last 2 decades, upper limb rehabilitation following stroke has evolved significantly, benefiting from numerous scientific advances in the domain of brain functioning (including imaging) and from a larger evidence base. Numerous studies both in animals (Cramer & Riley, 2008; Murphy & Corbett, 2009; Nishibe, Barbay, Guggenmos, & Nudo, 2010; Nishibe, Urban, Barbay, & Nudo, 2014; Nudo, Milliken, Jenkins, & Merzenich, 1996; Nudo, 2007; Plautz, Milliken, & Nudo, 2000) and humans (Carey et al., 2007; Cramer, 2010; Levy, Nichols, Schmalbrock, Keller, & Chakeres, 2001; Liepert et al., 1998; Liepert, Graef, Uhde, Leidner, & Weiller, 2000; Nelles, 2004; Takeuchi & Izumi, 2013; Turner, Ramos-Murguialday, Birbaumer, Hoffmann, & Luft, 2013) have indicated that repetitive stimulation, intensive and task-specific training (Buma, Kwakkel, & Ramsey, 2013; Bütefisch, Hummelsheim, Denzler, & Mauritz, 1995; Cooke, Mares, Clark, Tallis, & Pomeroy, 2010; Feys et al., 2004; Kwakkel, Wagenaar, Twisk, Lankhorst, & Koestler, 1999; Kwakkel et al., 2004; Lincoln, Parry, & Vass, 1999; Miller, Lincoln, Partridge, Wellwood, & Langhorne, 2004) promote neural re-organization and improve motor outcomes. Moreover, a dose-response relationship has been demonstrated in recent studies (Han, Wang, Meng, & Qi, 2013; Lohse, Lang, & Boyd, 2014).

Robotic systems are becoming increasingly integrated in post-stroke rehabilitation programs to address the need to incorporate elements of motor relearning into exercise-based training. Advanced robotic systems can offer safe, intensive and repetitive rehabilitation for the paretic limb. Since 1997, a growing number of studies have demonstrated improvements in upper limb motor function when post-stroke conventional rehabilitation therapies are augmented with or partly substituted by a robot-mediated rehabilitation program in both the acute/subacute phase of recovery (Aisen, Krebs, Hogan, McDowell, & Volpe, 1997; Lum et al., 2006; Masiero, Celia, Rosati, & Armani, 2007; Masiero, Armani, & Rosati, 2011; Masiero, Armani, Ferlini, Rosati, & Rossi, 2014, Sale et al., 2014, Volpe et al., 2000) and the chronic phase (Klamroth-Marganska et al., 2014; Lo et al., 2010; Lum, Burgar, Shor, Majmundar, & Van der Loos, 2002).

Another interesting advantage of robots is their ability to provide a simple and affordable measurement of motion kinematics during the therapy session, complementing human-administered clinical scales of motor function with more objective and reliable measures in standardized conditions (Gilliaux et al., 2014; Krebs, Aisen, Volpe, & Hogan, 1999; Zollo et al., 2011). Most advanced robotic systems include sensors which measure and record the kinematics of upper limb trajectories in order to derive indicators of movement features. Kinematic variables provide a valuable quantification of motor performance and movement quality. However, the discriminant validity of kinematic variables as measures of upper limb impairment is still unclear. Whilst Krebs et al. (2014) demonstrated that robotic measures might establish biomarkers of motor recovery after stroke, other authors (Zollo et al., 2011) showed that kinematic measures were only moderately correlated with Fugl-Meyer scores, particularly with the proximal part of the FMA upper extremity scores (Krabben et al., 2011; Subramanian, Yamanaka, Chilingaryan, & Levin, 2010). As neurorehabilitation is becoming more rigorous, the use of both objective and less time-consuming evaluations of motor outcomes is of crucial interest. Robotic devices might help to achieve this goal, providing numerous opportunities to improve rehabilitation protocols.

The aim of this study was to investigate the relationships between clinical and kinematic motor outcomes after a 16-session upper limb robot-assisted training program (3 days/week) added to usual care in patients with severe paresis, in the sub-acute phase of stroke.

2 Materials and methods

2.1 Subjects

Between October 2010 and April 2014, 76 inpatients underwent a sub-acute post-stroke upper limb robotic program in the Neurorehabilitation unit at Les Trois Soleils Rehabilitation Center (Boissise Le Roi, France). The program consisted of usual care and in addition, a robot-assisted rehabilitation training using a shoulder/elbow module. Their data were screened for inclusion in this retrospective study. The inclusion criteria were: over 18 years old, with moderate to severe upper limb paresis defined as a score below 35 out of 66 on the Fugl-Meyer (FM) Assessment scale, in the sub-acute phase of a first-ever stroke confirmed on CT scan or MRI, and with sufficient understanding to participate in rehabilitation exercises. Institutional ethics committee approval was obtained for this observational study and all subjects gave informed consent.

Data from 38 patients (19 females, age 56 ± 17 [19–87] years; time from stroke, 55 ± 22 days, type of stroke, 29 ischemic, 9 hemorrhagic; side of hemiparesis, 23 left, 15 right; see Table 1 for all characteristics) met the inclusion criteria. The remaining 38 patients were excluded because of missing data, severe aphasia or an insufficient number of training sessions.

2.2 Robotic device

The robotic training was carried out on the InMotion 2.0 arm robot (Interactive Motion Technologies, Inc., Watertown, MA, Fig. 1), the commercial version of the MIT Manus. This system consists of a manipulator with two active translational degrees-of-freedom to assist shoulder (flexion/extension) and elbow (flexion/extension) movements in the horizontal plane. It was designed by engineers at MIT (Massachusetts Institute of technology) (Krebs, 2003) and has low mechanical inertia. This device has several programs, including an adaptive regimen based on an assist-as-needed algorithm that adjusts the assistance provided according to the patient’s motor performance, and a point-to-point unconstrained reaching program mainly used for the evaluation of motion kinematics.

2.3 Clinical evaluations

A team of 5 trained evaluators (physical and occupational therapists) measured upper limb motor impairment before and after the 16 training sessions usingboth the upper limb total and subcomponents of the Fugl-Meyer Assessment (FMA) scale (Fugl-Meyer, Jaasko, Leyman, Olsson, & Steglind, 1975; Lindmark & Hamrin, 1988) and the Motor Status Scale (MSS) (Ferraro et al., 2002). The pre/post evaluations were carried out by the same therapist for a given patient.

The FMA and the MSS are among the most validated and commonly used ICF (International Classification of Functioning) body function measures in post-stroke rehabilitation studies, including studies using rehabilitation robots (Sivan, O’Connor, Makower, Levesley, & Bhakta, 2011). The FMA scale measures the ability to move the paretic arm and consists of items related to movements of the shoulder, elbow, wrist and hand. 22 items are rated on a 3-point scale (maximum score, 66 points). The Motor Status Scale (MSS) is divided into 4 sections and assesses shoulder, elbow/forearm, wrist and hand movements. This clinical scale provides a more complete measurement of upper-limb motor function than the FMA scale by grading motor abilities on a 6-point scale (maximum score, 82 points). In addition, The MSS fingers section consists of a sum of the scores of 12 hand movements (36/82 points). Wei, Tong, and Hu (2011) demonstrated that the FMA and MSS scales were the best choice for evaluating motor improvement in stroke patients, with high responsiveness and a good correlation with the Action Research Arm Test (Lyle, 1981), a well established upper extremity disability scale.

2.4 Kinematic outcome measures

All 38 patients underwent a pre/post evaluation session during which robot-mediated kinematic measurements were conducted. During these sessions, the patients performed center-out point-to-point unconstrained reaching tasks (i.e. without assistance) towards visual targets set in 8 compass directions (14 cm apart) and presented in a clockwise order. The targets were displayed in the same order 5 times for total of 80 recorded movements. The results were averaged across directions. Five kinematic variables were extracted from the on-line recordings of the hand trajectory towards each target:

a) Mean movement speed. b) Peak movement speed, c) Path error, a measure of movement accuracy, calculated as the mean deviation from the straight line (m). It measures the patient’s ability to move accurately along a straight path towards a goal/target/object. d) Reach error, a measure of efficacy which evaluates the patient’s ability to precisely reach the target. The robot measures the average distance (over 80 movements) which the patient reaches from the center towards the target. e) Speed shape, a smoothness metric calculated as mean speed divided by peak speed. For the point-to-point task the ideal smooth movement (minimum jerk, good timing) score is 0.533.

2.5 Interventions

The study period lasted 35 ± 15 days for each patient. Patients underwent conventional rehabilitationof the upper limb care consisting of one-hour occupational therapy sessions, 5 times per week on average, involving passive stretching of the paretic limb, active assisted movements, targeted reaching movements with or without elbow support and grasping tasks (Semans, 1967; Shepherd & Carr, 2006).

In addition, patients underwent sixteen robot-assisted sessions (3 sessions per week on average). Each session lasted 45 minutes and consisted of series of 320 (4 blocks of 80) goal-directed movements using the adaptive mode. The patients were instructed to perform as many accurate movements as possible in the allocated training time.

The post-stroke multidisciplinary program also included one-hour daily (5 days a week) sessions of physical therapy based on lower limb rehabilitation (without upper limb therapy) and, if necessary, one hour of speech therapy 3-4 times a week.

2.6 Statistical analysis

Results are expressed as means ± standard deviations (SD) or numbers and percentages. Pre and post-treatment values of the clinical outcome measures (total scores and subcomponents of the FMA and MSS) and the kinematic variables were compared using Anova and a Post-hoc Tukey test was carried out on significant variables. Correlations between clinical and kinematic outcomes at baseline and between clinical changes and kinematic changes were assessed using a Spearman’s test. Multiple regressions (without selection) were performed between clinical outcomes and kinematic outcomes and between changes in clinical outcomes and changes in kinematic outcomes. The predictive value of kinematic variables on change in FMA score and change in MSS score was assessed by multiple regression adjusted on the baseline value. Principal component analysis with varimax rotation was conducted on clinical and kinematic variables to determine underlying dimensions. The capacity of baseline kinematic measures to predict favorable clinical outcomes (increase of at least 9 points in FMA score and change in MSS score greater than the median value) was assessed by a logistic regression. Responsiveness to change of kinematic measures was assessed by a logistic regression model and areceiver-operating-characteristic (ROC) curve.

Statistical analyses were performed with SAS 9.3. software (SAS Institute, Cary, NC, USA).

3 Results

3.1 Baseline and changes in clinical outcome measures

Table 2 summarizes the pre/post FMA and MSS total scores and sub scores. Mean total and sub-scores of the FMA increased (p < 0.01) with the total score going from 17.7 ± 10.0 to 28.6 ± 15.4. The MSS total score increased from 18.1 ± 12.5 to 32.3 ± 20.2 and all sub scores increased significantly (p < 0.01).

3.2 Baseline and changes in kinematic metrics

All kinematic indicators improved significantly (p < 0.01) during the training (Table 3).

3.3 Correlations

3.3.1 Correlations between kinematic measures and clinical scores at baseline

At baseline, kinematic measures were strongly correlated with clinical scores. FMA total score was significantly correlated with reach error (r = –0.79; Fig. 1), smoothness (r = 0.75), mean velocity (r = 0.73), path error (r = –0.63) and correlations remained high with all FMA sub scores. MSS total score was also strongly correlated with reach error (r = –0.79), mean velocity (r = 0.73), smoothness (r = 0.72), max velocity (r = 0.60) and path error (r = –0.63).

The principal component analysis (PCA) indicated that the kinematic measures and clinical (both FMA and MSS) scores did not provide the same measurement “dimension”. Each was projected onto a distinct axis of the PCA (66.1% of FMA variance was explained by kinematics: axis 1; and 11, 3% was explained by the clinical scores and sub-scores: axis 2).

3.3.2 Correlation between changes in clinical scores and changes in kinematic measures

Correlations between changes in clinical scores and kinematic measures were weak. However, changes in FMA Shoulder/Elbow score and FMA total score were negatively correlated with change in path error (respectively r = –0.65, Fig. 2, and r = –0.51). Moreover, change in MSS total score was moderately correlated with change in path error (r = –0.49). Multiple regression analysis showed that a low proportion of the variance of clinical changes was explained by changes in kinematics (R² <0.25).

3.3.3 Analysis of patients with significant increases in clinical scores

3.3.3.1 Minimal Clinically Important difference (MCID) for the FMA score An increase of at least 9 points in the FMA total score between the pre and post-training evaluations occurred in 18 patients (9 points is considered as the Minimal Clinically Important Difference, (MCID) for sub-acute patients (Arya et al., 2011)). Two baseline kinematic indicators predicted if this value would be reached: smoothness (p = 0.04, OR = 1.079) and path error (p = 0.03, OR = –0.705).

3.3.3.2 Increase in MSS score of more than 8.4 points (median increase) We stratified the population depending on the median of increase in the MSS score because this scale provides a better assessment of hand motor function than the FM scale.

Change in MMS score was greater than 8.4 points in 19 patients. This magnitude of change was associated with a decrease in reach error and path error (OR = 1.172 and 1.376 respectively).

3.4 Responsiveness of kinematic measures in the sub-acute phase of stroke

ROC curves were used to evaluate the responsiveness of smoothness and path error, the most relevant kinematics. The ROC curves were examined to determine cut-off values for the change in smoothness and path error with regard to the achievement of the MCID in FM score.

The ROC curves showed that a cut-off value of 14.7-units for the change in smoothness (sensitivity, 0.71 and specificity, 0.69) and a cut-off value of –1.3-cm (sensitivity, 0.76 and specificity, 0.75) for path error variable were associated with the probability to reach the MCID value (Table 4).

4 Discussion

This study assessed motor recovery in patients with sub-acute stroke using common clinical motor scales as well as kinematic measures computed fromrobot-based recordings, before and after a 16-session upper limb robot-assisted training program added to conventional rehabilitation. The main aim was to investigate relationships between the results of different types of evaluation tools, clinical and kinematic, and to determine the relevance of kinematic parameters for the evaluation of motor performance.

The results showed that total and sub-scale clinical scores increased significantly during the training period, indicating an overall reduction in upper limb motor impairments. These results are consistent with most other studies (Aisen, Krebs, Hogan, McDowell,& Volpe, 1997; Lum et al., 2006; Masiero, Celia, Rosati, & Armani, 2007; Masiero, Armani, & Rosati, 2011; Masiero, Armani, Ferlini, Rosati, & Rossi, 2014, Sale et al., 2014, Volpe et al., 2000), systematic reviews (Kwakkel, Kollen, & Krebs, 2008, Mehrholz, Hadrich, Platz, Kugler, & Pohl, 2012) and meta-analyses (Norouzi-Gheidari, Archambault, & Fung, 2012) involving robotic therapy in sub-acute stroke survivors. Improvements of motor function occurred in both the proximal and distal limb segments, including the hand (FM and MSS scales) despite the fact that the robotic device only trained movements of the shoulder and elbow joints. This suggests that motor skills transferred from the proximal to the distal part of the limb. Moreover, the mean increase in FMA total score exceeded 9 points (i.e. it was above the Minimal Clinically Important difference, MCID, Arya, Verma, & Garg, 2011). It is thus likely that patients also experienced improvements in activities of daily living. In a Cochrane analysis, Mehrholz, Hadrich, Platz, Kugler, and Pohl (2012) concluded that patients in the sub-acute phase of stroke were more likely to improve in activities of daily living if the rehabilitation regimens involved robot-assisted upper limb rehabilitation.

All kinematic measures also improved after training, providing valuable objective information on quantitative and qualitative changes in motor performance after the robotic rehabilitation program. The results demonstrated that movement quality was improved after robot-assisted training with larger, faster, more accurate and smoother movements. These results are consistent with several other studies of robot therapy in sub-acute patients (Alt Murphy, Willén, & Sunnerhagen, 2013; Colombo et al., 2008; Dipietro et al., 2011; Mazzoleni et al., 2013; Rohrer et al., 2002).

Clinical scores were strongly correlated with all kinematic parameters at baseline. Celik et al. (2010) also found strong correlations between robotic measures, mostly smoothness of movement and trajectory error, and FM scores both at pre- and post-treatment in chronic stroke survivors. Bosecker, Dipietro, Volpe, and Krebs (2010) found strong correlations between selected kinematic variables and both the FM and the MSS scores. In contrast, Colombo et al. (2008) found moderate correlations between kinematic variables and FM scores. The Reach error, a measure related to active range of motion, was a good predictor of both FM and MSS scores at baseline. Despite these correlations, an interesting finding was that kinematic and clinical measures did not appear to evaluate the same dimensions of motor impairment. This suggests that the kinematic assessment of upper limb motion, which is more reliable and has greater resolution than clinical measures, provides a distinct quantitative evaluation of motor performance, thus supplementing the results of clinical scales. This result was not, however, surprising as clinical scales not only provide measures of motor impairment but also include items relating to function, including hand movements. Thus stroke-related motor assessment scales evaluate both impairment and function while kinematic measures only evaluate impairment.

Correlations between changes in clinical scores and changes in kinematic parameters were more moderate; however, considering the psychometric properties of human-administered clinical scales (Lin et al., 2009), the correlation coefficient between the path error variable, a measure of accuracy, and the clinical scores seems acceptable. Moreover, the more significant and relevant correlations were found between the synergistic pattern of the proximal upper extremity (coordination of shoulder and elbow motions) and the measure of accuracy, a result confirming the added and additional value of kinematic data in the assessment of motor performance and giving any more insight into the qualitative pattern of recovery. Otherwise, this finding would also suggest that this training might have induced focal brain plasticity as patients underwent the intensive training at the proximal joints, leading to “local” quantitative and qualitative motor gains.

Other results of this study also suggest that kinematic measures should be integrated into clinical practice. The analysis of the subgroup of patients with good motor outcomes (score exceeding the MCID for the FM score and higher than the median value for the MSS) indicated that change in smoothness (p = 0.04) and path error (p = 0.03) predicted the achievement of these outcomes. These measures were also shown to be responsive for the detection of motor improvementand to classify patients who were more likely to experience clinically meaningful improvements in the sub-acute phase after a stroke. Cistea and Levin (2000) reported that the degree of movement accuracy is correlated with the severity of the clinical impairment; however there is a lack of literature on this. Our results are in agreement with recent studies (Alt Murphy, Willén, & Sunnerhagen, 2013) which demonstrated that kinematic measures are sensitive for the evaluation of improvements in upper extremity motion during the first 3 months after stroke. Van Dokkum et al. (2014) showed that kinematic measures of hand trajectories are sensitive to change over time and Celik et al. (2010) demonstrated that trajectory error is a pertinent kinematic-based measure of motor impairment.

Clinical scales are currently considered as the gold standard for the measurement of impairment; however recently, some researchers have aimed to develop and assess the validity of scales based on robotic recordings (Einav, Geva, Yoeli, Kerzhner, & Mauritz, 2011). Others have calculated clinical scores from kinematic measures and demonstrated the validity of developing such models (Bosecker, Dipietro, Volpe, & Krebs, 2010; Krebs et al., 2014). It is likely that some substantial disadvantages of traditional clinical scales (subjectivity, floor and ceiling effects and time-consuming evaluation sessions) may push clinicians to use more objective and reproducible kinematic measures if they have access to appropriate devices. However, we do not propose that robotic measures should substitute for clinical measures. Clinical measures provide different information as demonstrated in the present study and by Celik et al. (2010) who found weak correlations between clinical measures of functional use and kinematics.

The study has some limitations, including the small sample size and the retrospective methodology. However, the recording of data and the statistical analysis were comprehensive and robust. The results revealed correlations between clinical and kinematic measures of motor function at baseline and between changes in these measures, which until now had been little studied. The finding is likely to have implications for daily practice.

Declaration of conflicting interests

The authors have no conflict of interest to declare, or any financial or other interest in the manufacturer or distributor of the device used in the present study.

Funding

This research received no specific grant.

Footnotes

Acknowledgments

The authors thank Mr Pierre Clerson from Orgamétrie for his help with the statistical analysis and the preparation of this manuscript for submission. This was funded by an unrestricted grant from Ipsen Pharma, Paris, France without any content review. The authors designed and performed the study, interpreted the data and wrote the initial draft of this manuscript.

References

Aisen

M.L.

, Krebs

H.I.

, Hogan

, McDowell

, & Volpe

B.T.

(1997). The effect of robot-assisted therapy and rehabilitative training on motor recovery following stroke. Archives of Neurology, 54, 443–446.

Alt Murphy

, Willén

, & Sunnerhagen

K.S.

(2013). Responsiveness of Upper Extremity Kinematic Measures and Clinical Improvement During the First Three Months After Stroke. Neurorehabilitation and Neural Repair, 27, 844–853.

Arya

K.N.

, Verma

, & Garg

R.K.

(2011). Estimating the minimal clinically important difference of an upper extremity recovery measure in subacute stroke patients. Top Stroke Rehabilitation, 18, 599–610.

Bosecker

, Dipietro

, Volpe

, & Krebs

H.I.

(2010). Kinematic robot-based evaluation scales and clinical counterparts to measure upper limb motor performance in patients with chronic stroke. Neurorehabilitation and Neural Repair, 24, 62–69.

Buma

, Kwakkel

, & Ramsey

(2013). Understanding upper limb recovery after stroke. Restorative Neurology Neuroscience, 31, 707–722.

Bütefisch

, Hummelsheim

, Denzler

, & Mauritz

K. H.

(1995). Repetitive training of isolated movements improves the outcome of motor rehabilitation of the centrally paretic hand. Journal of the Neurological Sciences, 130, 59–68.

Carey

J.R.

, Durfee

W.K.

, Bhatt

, Nagpal

, Weinstein

S.A.

, Anderson

K.M.

, & Lewis

S.M.

(2007). Comparison of finger tracking versus simple movement training via telerehabilitation to alter hand function and cortical reorganization after stroke. Neurorehabilitation and Neural Repair, 21, 216–232.

Celik

, O’Malley

M.K.

, Boake

, Levin

H.S.

, Yozbatiran

, Reistetter

T.A.

(2010). Normalized movement quality measures for therapeutic robots strongly correlate with clinical motor impairment measures. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 18, 433–444.

Cirstea

M.C.

, & Levin

M.F.

(2000). Compensatory strategies for reaching in stroke. Brain, 123, 940–953.

10.

Colombo

, Pisano

, Micera

, Mazzone

, Delconte

, Carrozza

M.C.

, Dario

, & Minuco

(2008). Assessing mechanisms of recovery during robot-aided neurorehabilitation of the upper limb. Neurorehabilitation and Neural Repair, 22, 50–63.

11.

Cooke

E.V.

, Mares

, Clark

, Tallis

R.C.

, & Pomeroy

V.M.

(2010). The effects of increased dose of exercise-based therapies to enhance motor recovery after stroke: A systematic review and meta-analysis. BMC Med, 8, 60.

12.

Cramer

S.C.

, & Riley

J.D.

(2008). Neuroplasticity and brain repair after stroke. Current Opinion in Neurology, 21, 76–82.

13.

Cramer

S.C.

(2010). Brain repair after stroke. The New England Journal of Medicine, 362, 1827–1829.

14.

Dipietro

, Krebs

H.I.

, Volpe

B.T.

, Stein

, Bever

, Mernoff

S.T.

, Fasoli

S.E.

, & Hogan

(2012). Learning, not adaptation, characterizes stroke motor recovery: Evidence from kinematic changes induced by robot-assisted therapy in trained and untrained task in the same workspace. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 20, 48–57.

15.

Einav

, Geva

, Yoeli

, Kerzhner

, & Mauritz

K.H.

(2011). Development and validation of the first robotic scale for the clinical assessment of upper extremity motor impairments in stroke patients. Top Stroke Rehabilitation, 18, 587–598.

16.

Ferraro

, Demaio

J.H.

, Krol

, Trudell

, Rannekleiv

, Edelstein

, Christos

, Aisen

, England

, Fasoli

, Krebs

H.I.

, Hogan

, & Volpe

B.T.

(2002). Assessing the motor status score: A scale for the evaluation of upper limb motor outcomes in patients after stroke. Neurorehabilitation and Neural Repair, 16, 283–289.

17.

Feys

, De Weerdt

, Verbeke

, Steck

G.C.

, Capiau

, Kiekens

, Dejaeger

, Van Hoydonck

, Vermeersch

, & Cras

(2004). Early and Repetitive Stimulation of the Arm Can Substantially Improve the Long-Term Outcome After Stroke: A 5-year follow-up study of a Randomized Trial. Stroke, 35, 924–929.

18.

Fugl-Meyer

A.R.

, Jaasko

, Leyman

, Olsson

, & Steglind

(1975). The post-stroke hemiplegic patient 1. A method for evaluation of physical performance. Scandinavian Journal of Rehabilitation Medicine, 7, 13–31.

19.

Gilliaux

, Lejeune

T.M.

, Detrembleur

, Sapin

, Dehez

, Selves

, & Stocart

(2014). Using the robotic device REAplan as a valid, reliable, and sensitive tool to quantify upper limb impairments in stroke patients. Journal of Rehabilitation Medicine, 46, 117–125.

20.

Han

, Wang

, Meng

P.P.

, & Qi

M.Z.

(2013). Effects of intensity of arm training on hemiplegic upper extremity motor recovery in stroke patients: A randomized controlled trial. Clinical Rehabilitation, 27, 75–81.

21.

Klamroth-Marganska

, Blanco

, Campen

, Curt

, Dietz

, Ettlin

, Felder

, Fellinghauer

, Guidali

, Kollmar

, Luft

, Nef

, Schuster-Amft

, Stahel

, & Riener

(2014). Three-dimensional, task-specific robot therapy of the arm after stroke: A multicentre, parallel-group randomised trial. Lancet Neurology, 13, 159–166.

22.

Krabben

, Molier

B.I.

, Houwink

, Rietman

J.S.

, Buurke

J.H.

, & Prange

G.B.

(2011). Circle drawing as evaluative movement task in stroke rehabilitation: An explorative study. Journal of NeuroEngineering and Rehabilitation, 24, 15.

23.

Krebs

H.I.

, Aisen

M.L.

, Volpe

B.T.

, & Hogan

(1999). Quantization of continuous arm movements in humans with brain injury. Proceedings of the National Academy of Sciences of the U S A, 96, 4645–4649.

24.

Krebs

H.I.

(2003). Rehabilitation Robotics: Performance-Based Progressive. Robot-Assisted Therapy. Autonomous Robots, 15, 7–20.

25.

Krebs

H.I.

, Krams

, Agrafiotis

D.K.

, DiBernardo

, Chavez

J.C.

, Littman

G.S.

, Yang

, Byttebier

, Dipietro

, Rykman

, McArthur

, Hajjar

, Lees

K.R.

, & Volpe

B.T.

(2014). Robotic measurement of arm movements after stroke establishes biomarkers of motor recovery. Stroke, 5, 200–204.

26.

Kwakkel

, Wagenaar

R.C.

, Twisk

J.W.

, Lankhorst

G.J.

, & Koestler

J.C.

(1999). Intensity of leg and arm training after primary middle-cerebral-artery stroke: A randomised trial. Lancet, 354, 189–194.

27.

Kwakkel

, van Peppen

, Wagenaar

R.C.

, Dauphinee

S.W.

, Richards

, Ashburn

, Miller

, Lincoln

, Partridge

, Wellwood

, & Langhorne

(2004). Effects of augmented exercise therapy time after stroke. A meta-analysis. Stroke, 35, 2529–2536.

28.

Kwakkel

, Kollen

, & Krebs

H.I.

(2008). Effects of robot-assisted therapy on upper limb recovery after stroke: A systematic review. Neurorehabilitation and Neural Repair, 22, 111–121.

29.

Levy

C.E.

, Nichols

D.S.

, Schmalbrock

P.M.

, Keller

, & Chakeres

D.W.

(2001). Functional MRI Evidence of Cortical Reorganization in Upper-Limb Stroke Hemiplegia Treated with Constraint-Induced Movement Therapy. American Journal of Physical Medicine Rehabilitation, 80, 4–12.

30.

Liepert

, Miltner

W.H.

, Bauder

, Sommer

, Dettmers

, Taub

, & Weiller

(1998). Motor cortex plasticity during constraint-induced movement therapy in stroke patients. Neuroscience Letters, 250, 5–8.

31.

Liepert

, Graef

, Uhde

, Leidner

, & Weiller

(2000). Training-induced changes of motor cortex representations in stroke patients. Acta Neurologica Scandinavia, 101, 321–326.

32.

Lin

J.H.

, Hsu

M.J.

, Sheu

C.F.

, Wu

T.S.

, Lin

R.T.

, Chen

C.H.

, & Hsieh

C.L.

(2009). Psychometric comparisons of 4 measures for assessing upper-extremity function in people with stroke. Physical Therapy, 89, 840–850.

33.

Lincoln

N.B.

, Parry

R.H.

, & Vass

C.D.

(1999). Randomized, controlled trial to evaluate increased intensity of physiotherapy treatment of arm function after stroke. Stroke, 30, 573–579.

34.

Lindmark

, & Hamrin

(1988). Evaluation of functional capacity after stroke as a basis for active intervention. Validation of a modified chart for motor capacity assessment. Scandinavian Journal of Rehabilitation Medicine, 20, 111–115.

35.

A.C.

, Guarino

P.D.

, Richards

L.G.

, Haselkorn

J.K.

, Wittenberg

G.F.

, Federman

D.G.

, Ringer

R.J.

, Wagner

T.H.

, Krebs

H.I.

, Volpe

B.T.

, Bever

C.T.

Jr. , Bravata

D.M.

, Duncan

P.W.

, Corn

B.H.

, Maffucci

A.D.

, Nadeau

S.E.

, Conroy

S.S.

, Powell

J.M.

, Huang

G.D.

, & Peduzzi

(2010). Robot-assisted therapy for long-term upper-limb Impairment after stroke. The New England Journal of Medicine, 362, 1772–1783.

36.

Lohse

K.R.

, Lang

C.E.

, & Boyd

L.A.

(2014). Is more better? Using metadata to explore dose-response relationships in stroke rehabilitation. Stroke, 45, 2053–2058.

37.

Lum

P.S.

, Burgar

C.G.

, Shor

P.C.

, Majmundar

, & Van der Loos

(2002). Robot-Assisted Movement Training Compared With Conventional Therapy Techniques for the Rehabilitation of Upper-Limb Motor Function After Stroke. Archives of Physical Medicine and Rehabilitation, 83, 952–959.

38.

Lum

P.S.

, Burgar

C.G.

, van der Loos

, Shor

P.C.

, Majmumdar

, & Yap

(2006). MIME robotic device for upper-limb neurorehabilitation in subacute stroke subjects: A follow-up study. Journal of Rehabilitation Research and Development, 43, 631–642.

39.

Lyle

R.C.

(1981). A performance test for assessment of upper limb function in physical rehabilitation treatment and research. International Journal of Rehabilitation Research, 4, 483–492.

40.

Masiero

, Celia

, Rosati

, & Armani

(2007). Robotic-assisted rehabilitation of the upper limb after acute stroke. Archives of Physical Medicine and Rehabilitation, 88, 142–149.

41.

Masiero

, Armani

, & Rosati

(2011). Upper-limb robot-assisted therapy in rehabilitation of acute stroke patients: Focused review and results of new randomized controlled trial. Journal of Rehabilitation Research and Development, 48, 355–366.

42.

Masiero

, Armani

, Ferlini

, Rosati

, & Rossi

(2014). Randomized trial of a robotic assistive device for the upper extremity during early inpatient stroke rehabilitation. Neurorehabilitation and Neural Repair, 28, 377–386.

43.

Mazzoleni

, Carrozza

M.C.

, Sale

, Franceschini

, Posteraro

, & Tiboni

(2013). Effects of upper limb robot-assisted therapy on motor recovery of subacute stroke patients: A kinematic approach. IEEE International Conference on Rehabilitation Robotics, 6650503.

44.

Mehrholz

, Hadrich

, Platz

, Kugler

, & Pohl

(2012). Electromechanical and robot-assisted arm training for improving generic activities of daily living, arm function and muscle arm strength after stroke. Cochrane Database Syst Rev, 6, CD006876.

45.

Miller

, Lincoln

, Partridge

, Wellwood

, & Langhorne

(2004). Effects of augmented exercise therapy time after stroke. A meta-analysis. Stroke, 35, 2529–2536.

46.

Murphy

T.H.

, & Corbett

(2009). Plasticity during stroke recovery: From synapse to behaviour. Nature Reviews Neuroscience, 10, 861–872.

47.

Nelles

(2004). Cortical reorganization–effects of intensive therapy. Restorative Neurology Neuroscience, 22, 239–244.

48.

Nishibe

, Barbay

, Guggenmos

, & Nudo

R.J.

(2010). Reorganization of motor cortex after controlled cortical impact in rats and implications for functional recovery. Journal of Neurotrauma, 27, 2221–2232.

49.

Nishibe

, Urban

E.T.

3rd , Barbay

, & Nudo

R.J.

(2014). Rehabilitative Training Promotes Rapid Motor Recovery but Delayed Motor Map Reorganization in a Rat Cortical Ischemic Infarct Model. Neurorehabilitation and Neural Repair, 29, 472–482.

50.

Norouzi-Gheidari

, & Archambault

P.S.

, Fung

(2012). Effects of robot-assisted therapy on stroke rehabilitation in upper limbs: Systematic review and meta-analysis of the literature. Journal of Rehabilitation Research and Development, 49, 479–496.

51.

Nudo

R.J.

, Milliken

G.W.

, Jenkins

W.M.

, & Merzenich

M.M.

(1996). Use-dependent alterations of movement representations in primary motor cortex of adult squirrel monkeys. Journal of Neuroscience, 16, 785–807.

52.

Nudo

R.J.

(2007). Postinfarct cortical plasticity and behavioural recovery. Stroke, 38, 840–845.

53.

Plautz

E.J.

, Milliken

G.W.

, & Nudo

R.J.

(2000). Effects of repetitive motor training on movement representations in adult Squirrel monkeys: Role of use versus learning. Neurobiology and Learning and Memory, 74, 27–55.

54.

Rohrer

, Fasoli

, Krebs

H.I.

, Hughes

, Volpe

, Frontera

W.R.

, Stein

, & Hogan

(2002). Movement smoothness changes during stroke recovery. Journal of Neuroscience, 22, 8297–8304.

55.

Sale

, Franceschini

, Mazzoleni

, Palma

, Agosti

, & Posteraro

(2014). Effects of upper limb robot-assisted therapy on motor recovery in subacute stroke patients. Journal of NeuroEngineering and Rehabilitation, 11, 104.

56.

Semans

(1967). The Bobath concept in treatment of neurological disorders; a neuro-developmental treatment. American Journal of Physical Medicine, 46, 732–788.

57.

Shepherd

R.B.

, & Carr

J.H.

(2006). Neurological rehabilitation. Disability and Rehabilitation, 28, 811–812.

58.

Sivan

, O’Connor

R.J.

, Makower

, Levesley

, & Bhakta

(2011). Systematic review of outcome measures used in the evaluation of robot-assisted upper limb exercise in stroke. Journal of Rehabilitation Medicine, 43, 181–189.

59.

Subramanian

S.K.

, Yamanaka

, Chilingaryan

, & Levin

M. F.

(2010). Validity of movement pattern kinematics as measures of arm motor impairment poststroke. Stroke, 41, 2303–2308.

60.

Takeuchi

, & Izumi

S.I.

(2013). Rehabilitation with poststroke motor recovery: A review with a focus on neural plasticity. Stroke Research and Treatment, 2013, 128641.

61.

Turner

D.L.

, Ramos-Murguialday

, Birbaumer

, Hoffmann

, & Luft

(2013). Neurophysiology of robot-mediated training and therapy: A perspective for future use in clinical populations. Frontiers in neurology, 4.

62.

van Dokkum

, Hauret

, Mottet

, Froger

, Métrot

, & Laffont

(2014). The contribution of kinematics in the assessment of upper limb motor recovery early after stroke. Neurorehabilitation and Neural Repair, 28, 4–12.

63.

Volpe

B.T.

, Krebs

H.I.

, Hogan

, Edelstein

, Diels

, & Aisen

(2000). A novel approach to stroke rehabilitation: Robot-aided sensorimotor stimulation. Neurology, 54, 1938–1944.

64.

Wei

X.J.

, Tong

K.Y.

, & Hu

X.L.

(2011). The responsiveness and correlation between Fugl-Meyer Assessment, Motor Status Scale, and the Action Research Arm Test in chronic stroke with upper-extremity rehabilitation robotic training. International Journal of Rehabilitation Research, 34, 349–356.

65.

Zollo

, Rossini

, Bravi

, Magrone

, Sterzi

, & Guglielmelli

(2011). Quantitative evaluation of upper-limb motor control in robot-aided rehabilitation. Medical & Biological Engineering & Computing, 49, 1131–1144.