Abstract
Background
Chronic upper extremity motor deficits are present in up to 65% of stroke survivors, and cognitive impairment is prevalent in 46-61% of stroke survivors even 10 years after their stroke. Robot-assisted therapy programs tend to focus on motor recovery and do not include stroke patients with cognitive impairment.
Objective
This study aims to investigate performance on the individual cognitive domains evaluated in the MoCA and their relation to upper-limb motor performance on a robotic system.
Methods
Participants were recruited from the stroke population with a wide range of cognitive and motor levels to complete a trajectory tracking task using the Haptic TheraDrive rehabilitation robot system. Motor performance was evaluated against standard clinical cognitive and motor assessments. Our hypothesis is that the cognitive domains involved in the visuomotor tracking task are significant predictors of performance on the robot-based task and that impairment in these domains results in worse motor performance on the task compared to subjects with no cognitive impairment.
Results
Our results support the hypothesis that visuospatial and executive function have a significant impact on motor performance, with differences emerging between different functional groups on the various robot-based metrics. We also show that the kinematic metrics from this task differentiate cognitive-motor functional groups differently.
Conclusion
This study demonstrates that performance on a motor-based robotic assessment task also involves a significant visuospatial and executive function component and highlights the need to account for cognitive impairment in the assessment of motor performance.
Keywords
Introduction
Motor and cognitive impairments are common occurrences after stroke. Long-term upper extremity motor deficits persist in up to 65% of stroke survivors, 1 while cognitive impairment is prevalent in 46-61% of stroke survivors even 10 years after their stroke. 2 These impairments can impact the ability to perform activities of daily living (ADLs) and to live independently. Mariani et al. showed that deficits in ADLs are highly correlated with memory, psychomotor speed, and executive functions. 3 Many ADLs especially Instrumental ADLs (IADLs) such as medication management, using a phone, and meal preparation require the use of both motor and cognitive skills especially the executive functioning cognitive skills. Executive functioning is often assess by examining a person being able to process information, do visuomotor and visuospatial tracking tasks, and perform abstract thinking. Lee et al. have been one of the few studies that investigated the combined effects of motor and cognitive deficits on ADLs. 4 They specifically look at the co-presentation of stroke and cognitive impairment in a cohort of 3331 community dwelling older adults and found that those with combined deficits had difficulty in dressing, using the telephone, transport, and managing finances.
Robotic therapy systems have emerged as an approach to address the motor impairments that result from stroke. Previous robot-assisted therapy studies have demonstrated improvements in motor capacity with similar efficacy to conventional, high-intensity therapy.5,6 However, these studies largely ignore the presence of cognitive impairments. A recent systematic review showed that 10 out of 66 clinical trials involving robotic therapy systems included participants with impaired cognition, and only five of those used cognitive measures as outcomes. 7 The presence of cognitive impairment has been shown to negatively influence motor outcomes after upper limb therapy, including robotic therapy. 8
Exploring cognitive function through rehabilitation robotics remains an emerging area and there is a need to develop tools to study how cognitive deficits impact motor performance and outcomes. Aprile et al. demonstrated improvements in episodic memory, calculation, and visual attention in a pilot study of 51 stroke subjects going through a combined cognitive training and upper limb robotic therapy regimen. 9 Another pilot study explored the use of an active learning protocol as a cognitive training tool during upper limb robotic therapy, demonstrating that this approach was well-tolerated and resulted in significant gains in upper extremity function. 6 Other works have demonstrated moderate relationships between overall cognition scores using the Montreal Cognitive Assessment (MoCA) and robot-based metrics in stroke and traumatic brain injury populations.10,11
A major barrier to widespread clinical adoption of rehabilitation robotics is the lack of evidence that motor capacity improvements transfer to untrained tasks or ADLs. 6 One potential explanation for this barrier is that the presence of cognitive impairments could be preventing this transfer. Thus, there is a subset of stroke patients presenting with both cognitive and motor impairments for which existing rehabilitation robotic strategies are not currently effective. There is a need to better understand the interactions between specific cognitive and motor impairments to develop more effective neurorehabilitation strategies to improve patient outcomes.
Cognitive-motor interactions have been explored in the cognitive neuroscience field, where studies have demonstrated relationships such as that of secondary motor network supporting working memory tasks. Studies have shown that people with lower working memory capacity recruit motor networks more actively and at lower thresholds of cognitive difficulty than people with higher working memory capacity. 12
A common clinical method of screening for cognitive impairment is the Montreal Cognitive Assessment (MoCA). 13 The MoCA has been validated to identify the likelihood of mild to moderate cognitive impairment across various domains—visuospatial ability, executive function, naming, attention, language, abstraction, delayed recall, and orientation—in the elderly, stroke, traumatic brain injury, and other populations. A recent study by VanGilder demonstrated a relationship between the visuospatial and executive function section of the MoCA to motor skill transfer. 14 To date, there has not been a study examining performance on the individual domains evaluated in the MoCA and their relation to motor performance on a robotic system in the chronic stroke population.
This study aims to explore this cognitive-motor interaction in the context of a visuomotor trajectory tracking assessment task. Individuals recruited from the stroke population with a wide range of cognitive and motor levels completed a classic trajectory tracking task using the Haptic TheraDrive rehabilitation robot system.15,16 Performance on this task was evaluated against clinical cognitive and motor assessment scores. While the trajectory tracking task has traditionally been thought of as a purely motor-based assessment, the first hypothesis we test is that there is substantial cognitive involvement in the visuomotor tracking task—namely visuospatial ability, executive function, and attention domains—that is a significant predictor of performance on the task. We also hypothesize that mild to moderate impairment in these domains results in worse motor performance on the task compared to individuals with no cognitive impairment.
Methods
Subject population
Individuals were eligible for the study if they were older than 18 years of age and were at least 3 months past their stroke. Individuals were excluded if they were unable to sit upright for more than 2 hours at a time, had received a Botox injection within the past 3 months, experienced severe spasticity in the upper-limb, or experienced greater than mild pain as identified by the Visual Analog Scale. This protocol was approved by the Internal Review Board of the University of Pennsylvania (Protocol numbers 819787 and 823511).
A total of 31 individuals—16 males and 15 females—participated across the two studies. 17 were from Protocol 819787 and 14 were from Protocol 823511. The average age of the combined patient population was 57.06 years old. After written informed consent was obtained in-person, a clinical evaluation was performed, followed by the robot assessment portion. Table 1 includes descriptive statistics of the demographics and clinical assessments. Subjects from the two protocols were not significantly different in age or clinical characteristics.
Subject Demographics and Clinical Assessment Scores.
Clinical evaluation
Montreal Cognitive Assessment (MoCA): The MoCA is a cognitive screening tool to detect impairment in various cognitive domains—visuospatial and executive function, naming, memory, attention, language, abstraction, delayed recall, and orientation—and reflects the degree of cognitive impairment in an individual. 13 A score above 25 out of 30 generally indicates normal cognitive function, while a score below 19 indicates a high likelihood of severe cognitive impairment. Both the total score and individual domain subscores were recorded. The subscores were determined by summing the points of the individual tasks for each domain section according to the manufacturer’s guidelines. 13
Box and Blocks (BBT): The BBT is a test of gross motor function measuring how many blocks subjects are able to transfer across a partition in 1 minute, with a higher number of transferred blocks indicating better motor function. 17 Scores were normalized by age, gender, and limb. It is typically used to measure reach and grasp function in the stroke population.
Upper Extremity Fugl Meyer (UE-FM): The UE-FM is a scored index that assesses upper limb motor control in stroke patients. 18 The maximum score for upper limb is 66. A cutoff score of 48 and below was used to determine the presence of moderate motor impairment. 19
Grip Strength: Grip strength was measured with a Jamar (Chicago, IL) digital hand dynamometer. Three trials were taken with each hand, with the average being recorded and standard deviation calculated.
Trajectory Tracking Assessment
After clinical assessment, participants then completed a tracking task on a rehabilitation robot. The robot used in this study, the Haptic TheraDrive, is a one degree-of-freedom robot for upper limb stroke rehabilitation (Figure 1). 15 The user operates the TheraDrive by manipulating a vertically mounted crank handle equipped with force sensors and an optical encoder. For assessment purposes, it is run in a gravity-compensation mode, which uses force sensors as an input signal to a proportional-integral-derivative controller to calculate the necessary response by the motor to give the sensation that there is no resistance or assistance while the user manipulates the handle.
An individual performing the trajectory tracking task on the Haptic TheraDrive, a one-degree-of-freedom robot system used in this study.
The trajectory tracking task is designed to assess upper limb motor performance. A single trial consists of the user moving the crank arm forward and backward to follow a sinusoidal path that vertically scrolls at a fixed speed. There were slight differences in how the tasks were administered between the two protocols, but the equation to generate the trajectory and robotic system were identical. In Protocol 819787, subjects performed three trials that lasted 90 seconds each (270 seconds total). In Protocol 823511, subjects performed 15 trials that lasted 15 seconds each (225 seconds total). To standardize the analysis, the last 45 seconds from the last trial were omitted from those who completed the task with Protocol 819787 such that 225 seconds of trial data matched that in Protocol 823511. A set of kinematic measures were then extracted.
The outcome measures from the trajectory tracking task included performance error, the distance traversed, and mean velocity. Performance error was calculated as the root mean square error (RMSE) of the position relative to the displayed trajectory. The RMSE is then normalized by the maximum RMSE value that results when we assume an angular position of zero degrees for the duration of the task (i.e. no movement relative to the displayed trajectory). A lower performance error indicates better tracking performance.
The distance traversed was as the total angular distance that the subject traversed and normalized by the expected angular distance of the displayed trajectory path. A normalized value closer to one reflects that the actual distance traversed matched the expected distance. A lower value could reflect moderate motor impairment, while a higher value could reflect inefficient movement. A recent review showed multiple studies that demonstrate a relationship between kinematic measures like those used in this study and clinical motor assessments such as the Fugl–Meyer Assessment, Motor Status Score, Modified Ashworth Scale, and Motor Power. 20
Data Analysis
Multiple Linear Regression
To investigate the relationship between clinical scores and robot-based metrics, a forward stepwise multiple linear regression approach was used to identify the clinical scores that were significant predictors of performance on the trajectory tracking task. This consisted of individually testing each clinical score and subsequently adding it to the model only if it was a statistically significant individual predictor (P < .05) and also increased the adjusted coefficient of determination (adjusted (adj) R2) compared to the model without that term. The adjusted R2 is reported to allow for comparison of performance between models with different numbers of terms.
Given the sample size of the subject population, the linear regression model was limited to a maximum of three terms. A sample size analysis determined that the linear regression models were powered to detect a minimum R2 of 0.22 with one predictor, 0.25 with two predictors, and 0.29 with three predictors (n = 31, power = 0.80, alpha = 0.05). Small, medium, and large effect sizes were defined as an R2 value of 0.01, 0.25, and 0.50, respectively. All analysis was conducted in Matlab 2019A (MathWorks, Inc).
Cognitive-Motor Subgroup Analysis
All study participants were categorized by their cognitive and motor status based on clinical score cutoffs. To categorize subjects by motor status, a UE-FM score above 48 was classified as low motor impairment, while a score at or below 48 with moderate motor impairment. Because the MoCA-Visuospatial/Executive Function subscore emerged from the linear regression analysis as the only cognitive metric to be a significant predictor across all robot-based metrics, that score was used to categorize subjects by cognitive status, with a cutoff of 3.5 and below out of five categorized as moderate visuospatial and/or executive function impairment. 20 Lam et al. 21 previously showed that MoCA subscores demonstrated convergent validity with standardized neuropsychological tests. Subjects were then categorized into one of four cognitive-motor functional subgroups based on the possible combinations of cognitive and motor status. There were 15 subjects in the low cognitive and low motor impairment group, seven subjects in the low cognitive and moderate motor impairment group, five subjects in the moderate cognitive and low motor impairment group, and five subjects in the moderate cognitive and moderate motor impairment group.
For each robot-based metric, a one-way analysis of variance (ANOVA) was conducted with the cognitive-motor functional group as the factor. To correct for all pairwise comparisons between the four functional groups, a Tukey–Kramer honest significance difference test was applied to identify significant differences between groups. An alpha level of 0.05 was used to establish statistical significance on the Tukey–Kramer test.
Results
Representative Examples and Functional Group Breakdown
Figure 2 shows the average trajectory across the trials of a representative subject from each of the four functional groups. The example low motor and low cognitive impairment subject (blue trace) is a 58-year-old with a UE-FM score of 66 and a MoCA-Visuospatial/Executive Function subscore of 4. The average trace tracks well with the desired trajectory, represented by the dotted line. The example moderate motor and low cognitive impairment subject (red trace) is a 53-year-old with a UE-FM score of 42 and a MoCA-Visuospatial/Executive Function subscore of 4. Qualitatively, while the individual is able to perform the task, they are not able to navigate the full range of motion and display a large variance across trials as demonstrated by the shaded region. The example low motor and moderate cognitive impairment subject (yellow trace) is a 43-year-old with a UE-FM score of 66 and a MoCA-Visuospatial/Executive Function subscore of 2. Their performance falls between that of the example low motor and low cognitive impairment subject and the moderate motor and low cognitive impairment subject. The variance across the trials is also low. The example moderate motor and moderate cognitive impairment subject (purple trace) is a 58-year-old with a UE-FM score of 25 and a MoCA-Visuospatial/Executive Function subscore of 2. Their performance indicates an inability to follow the trajectory after the first part. Figure 3 shows the distribution of subjects by their functional groups as determined by the cognitive and motor cutoff scores.
Mean trajectories for representative subjects from each functional group recreated from raw position data (degrees) collected from the robot. The shaded region represents the standard deviation across all trials for a particular subject. The displayed trajectory is shown as a black dotted line. (LMI = low motor impairment; LCI = low cognitive impairment; MMI = moderate motor impairment; MCI = moderate cognitive impairment).
Distribution of subjects by cognitive and motor function, using a score of 3.5 for the MoCA-Visuospatial/Executive Function cutoff and 48 as the UE-FM cutoff. (LMI = low motor impairment; LCI = low cognitive impairment; MMI = moderate motor impairment; MCI = moderate cognitive impairment).
Identifying Relationships Between Clinical Scores and Trajectory Tracking Performance
Figure 4 shows the multiple linear regression models for each of the robot-based metrics using the clinical cognitive and motor scores as predictors.
Predicted robot-based scores plotted against actual robot-based scores for performance error (top left, normalized distance traversed (top right), and mean velocity (bottom left). Cognitive-motor functional groups are also identified by different colors and shapes. The multiple linear regression equation is included at the top of each plot. (**P < .001; LMI = low motor impairment; LCI = low cognitive impairment; MMI = moderate motor impairment; MCI = moderate cognitive impairment).
A combination of non-dominant BBT and MoCA Visuospatial/Executive Function subscores accounted for 46% of the variance observed in trajectory tracking performance error scores (adj R2 = 0.46, p = 6.64 × 10−5). This model performed better than the model with non-dominant BBT as the only predictor (adj R2 = 0.37).
A combination of UE-FM and MoCA Visuospatial/Executive Function subscores accounted for 73% of the variance observed in normalized distance scores (adj R2 = 0.73, p = 3.84 × 10−9). This model performed better than the model with UE-FM as the only predictor (adj R2 = 0.66).
A combination of UE-FM and MoCA Visuospatial/Executive Function accounted for 68% of the variance observed in trajectory tracking mean velocity (adj R2 = 0.67, p = 6.48 × 10−8). This model performed better than the model with UE-FM as the only predictor (adj R2 = 0.60).
Differences Between Cognitive-Motor Functional Groups
Figure 5 shows the performance across the different robot-based metrics according to the cognitive-motor functional groups. There was a statistically significant effect of functional group on trajectory tracking performance error. The low cognitive and low motor impairment group had significantly lower performance error scores compared to the low motor and moderate cognitive impairment group (0.42 ± 0.23 vs 0.96 ± 0.38, P = .02) and the moderate motor and moderate cognitive impairment group (0.42 ± 0.23 vs 0.90 ± 0.31, P = .04).
Box and whisker plots showing performance by cognitive-motor functional group on trajectory tracking metrics. The central red line is the median, the edges of the box are the 25th and 75th percentiles, and the whiskers extend to the most extreme non-outlier data points. Outliers are plotted individually as a red cross. (*P < .05; **P < .001).
There was a statistically significant effect of functional group on trajectory tracking normalized distance. The low cognitive and low motor impairment group had higher normalized distance scores compared to the low cognitive and moderate motor impairment group (1.03 ± 0.09 vs 0.69 ± 0.40, P = .05) and moderate cognitive and moderate motor impairment group (1.03 ± 0.09 vs 0.29 ± 0.37, P = .0001).
There was a statistically significant effect of functional group on trajectory tracking mean velocity. The low cognitive and low motor impairment group had higher mean velocity scores compared to the moderate cognitive and moderate motor impairment group (46.99 ± 4.29 deg/s vs 14.41 ± 18.70 deg/s, P = .0002).
The moderate motor impairment and low cognitive group was indistinguishable from the low motor impairment and moderate cognitive group for performance error (0.80 ± 0.46 vs 0.96 ± 0.38, P = .83), normalized distance traversed (0.69 ± 0.40 vs 0.68 ± 0.33, P = .99) and mean velocity (31.28 ± 18.61 deg/s vs. 30.93 ± 15.25 deg/s, P = .99).
Discussion
Visuospatial and Executive Function Significantly Influence Motor Performance
This study aimed to explore how cognitive impairments in specific domains affect performance across different groups of cognitive and motor function. We found that executive function and visuospatial ability significantly contributed to performance on the trajectory tracking task. While clinical measures of motor function still strongly predicted performance, we found that the visuospatial-executive function subscore on the MoCA was an independent predictor of motor performance for all three robot-based metrics. The linear regression models for normalized distance traversed and mean velocity demonstrated strong relationships between clinical scores and the robot-based measures, while the model for performance error demonstrated a moderate relationship. All multiple linear regression models exceeded the effect size for which the study was powered. These results support our first hypothesis that visuospatial and executive function play a role in the performance of the trajectory tracking task that cannot be overlooked. Our results did not support the role of attention, but other robotic assessment experiments suggest that attention can play a large role in performance. 10
Our results also support the findings from VanGilder and colleagues that demonstrated the visuospatial-executive function subscore on the MoCA was related to motor skill training. 14 Their best performing multiple linear regression model had an adjusted R2 of 0.16, while the values in our study had a range of 0.46-0.72. However, this can partially be explained by the different populations evaluated in the two studies (healthy aging vs chronic stroke), with a wider range of impairments evaluated in this study. Given the evidence in this study that visuospatial and executive function has a significant influence on motor performance, this suggests the importance of actively measuring these and other relevant cognitive domains in the context of developing robot-assisted neurorehabilitation strategies.
Some Kinematic Metrics may be More Sensitive to Cognitive Performance than Others
Another key observation from this study is that some kinematic metrics used in robot-based assessments may be more sensitive to cognitive performance than others. This study demonstrated that kinematic metrics from the trajectory tracking task were sensitive to differences between the various cognitive-motor functional groups. Taken together, the results demonstrate that the presence of mild-to-moderate cognitive impairment can influence the interpretation of results. However, one thing to note is that the robot-based metrics did not perform the same in differentiating between the cognitive-motor functional groups. Compared to the low cognitive and low motor impairment group, the moderate cognitive and low motor impairment group performed worse on performance error, while the low cognitive and moderate motor impairment group performed worse on the normalized distance traversed metric. This result suggests different kinematic metrics may be more sensitive to cognitive status, while others may be more sensitive to motor impairment. More specifically, the performance error metric requires the ability to stay on the path for the entire time of the task. This ability requires an integration of visuospatial ability, motor planning, and motor execution, which taps into executive function domains. Thus, it would be expected—and seen in the results of this study—that those with executive function impairments perform worse compared to those without executive function impairments. On the other hand, the normalized distance is independent of the error relative to the desired position, making it more sensitive to motor impairment.
While there are a variety of kinematic metrics that can be used to assess upper-limb performance during robot-assisted rehabilitation, 20 the impact of cognitive impairment on these metrics is not fully known. Going forward, more work needs to be done to determine how other kinematic metrics relate to specific cognitive and motor impairments. This may require strategies to perform neuroimaging concurrently during these tasks, such as through functional MRI, electroencephalography (EEG), or functional near-infrared spectroscopy (fNIRS). The knowledge of how impairment in various cognitive domains influences moto performance will allow for better treatment of people living with stroke and other neurological injuries that result in motor and cognitive impairments. This will require expanding the current cognitive evaluation tools beyond those that are traditionally used (i.e. the MoCA, Mini-Mental State Exam, etc.) to more targeted evaluations of cognitive function that assess specific cognitive domains such as information processing, working memory, and executive function. Examples of such tests include the Color Trails, Digit Symbol—Coding, Spatial Span, and Stroop tests.22 –24 Given that the metrics in this study could not differentiate between the group with low cognitive and moderate motor impairment and the group with moderate cognitive and low motor impairment, more work needs to be done in identifying differences in performance between these groups. It also suggests the need for metrics that are sensitive to the co-occurrence of motor and cognitive impairments.
Limitations of Existing Robotic Assessments
Given its significant involvement, the presence of cognitive impairment can confound results on a task that has traditionally been used to assess motor function in robot-assisted neurorehabilitation. This was shown across all robot-based metrics, as the group with moderate motor and low cognitive impairment was indistinguishable from the group with low motor impairment and moderate cognitive impairment. The difficulty in separating cognitive influence during motor performance highlights the need for new approaches to robotic assessments. Possible approaches include assessing both limbs to remove the impaired limb as a confounding factor or developing tasks that more specifically target working memory, attention, or executive function. Our group has developed such an approach, expanding the robot-assisted technologies to the assessment of cognitive and motor impairments in the HIV and HIV-stroke populations. 25 Consideration should be given as to how to measure these domains in isolation as well as when motor demands are jointly present. Addressing these barriers will allow for broader populations to benefit from robotic therapy systems.
Expansion of robot-assisted neurorehabilitation to stroke survivors with motor impairments and mild to moderate cognitive impairments is possible if we consider what aspect of motor and cognitive domain is being trained. Failure to account for cognitive impairment, which can mask motor ability, may mean the failure to see transfer of any improvements in motor performance to everyday ADLs that have both cognitive and physical demands.
Study Limitations
Given the small sample size, we may not be able to fully generalize these results. While we had adequate distribution across the various variables, the results could have been biased from an uneven distribution across the different cognitive-motor functional groups. Another limitation was the grouping of the visuospatial and executive function domains on the MoCA, which did not allow for examination of each individual domain’s contribution to motor performance. Given the difference in trial length between the two protocols, there could be multiple factors affecting the first protocol such as fatigue, loss of interest, and attention more than the other protocol. The study would also benefit from other clinical cognitive metrics, as the MoCA is a screening tool that does not extensively evaluate individual cognitive domains. Despite these limitations, these results lay the groundwork for future studies to further explore the role of cognitive function on motor performance.
Conclusion
In this study in a chronic stroke population with a range of cognitive and motor impairment levels, we demonstrate that performance on a motor-based robotic assessment task also involves a significant visuospatial and executive function component. We also show that the kinematic metrics from these tasks differentiate performance by cognitive-motor functional group in different ways, indicating that some metrics may be more sensitive to cognitive impairment while others more sensitive to motor impairment. These findings warrant further exploration of the role impairments in visuospatial and executive function—as well as other cognitive domains beyond these—have on motor performance.
Footnotes
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: Michelle Johnson, PhD and Rochelle Mendonca, OTR, PhD are founders of Recupero Robotics, LLC, a spinoff company from the University of Pennsylvania concerned with commercializing affordable robots developed in the University of Pennsylvania Rehabilitation Robotics Lab.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was made possible through core services and support from the National Institute of Neurological Disorders and Stroke of the National Institutes of Health (T32NS091006); the University of Pennsylvania’s Center for AIDS Research (P30AI045008); and the University of Pennsylvania’s Departments of Bioengineering and Physical Medicine and Rehabilitation. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
