Abstract
Background:
Studies have found that individuals with mild cognitive impairment (MCI) exhibit a range of deficits outside the realm of primary explicit memory, yet the role of response speed and implicit learning in older adults with MCI have not been established.
Objective:
The current study aims to explore and document response speed and implicit learning in older adults with neuropsychologically defined MCI using a simple serial reaction (SRT) task. In addition, the study aims to explore the feasibility of a novel utilization of the simple cognitive task using machine learning procedures as a proof of concept.
Method:
Participants were 22 cognitively healthy older adults and 20 older adults with MCI confirmed through comprehensive neuropsychological evaluation. Two-sample t-test, multivariate regression, and mixed-effect models were used to investigate group difference in response speed and implicit learning on the SRT task. We also explored the potential utility of SRT feature analysis through random forest classification.
Results:
With demographic variables controlled, the MCI group showed overall slower reaction time and higher error rate compared to the cognitively healthy volunteers. Both groups showed significant simple motor learning and implicit learning. The learning patterns were not statistically different between the two groups. Random forest classification achieved overall accuracy of 80.9%.
Conclusions:
Individuals with MCI demonstrated slower reaction time and higher error rate compared to cognitively healthy volunteers but demonstrated largely preserved motor learning and implicit sequence learning. Preliminary results from random forest classification using features from SRT performance supported further research in this area.
INTRODUCTION
Cognitive decline is a common phenomenon in aging. Although not all age-accompanied cognitive decline is indicative of underlying pathology, a substantial portion is indeed a sign of impending dementia such as Alzheimer’s disease (AD) [1]. Research in AD has strongly suggested that the pathological process of neurodegeneration begins years, if not decades, before reaching the diagnosis of clinical dementia [2]. Although effective treatments capable of reversing or ceasing the degenerative process is still lacking, interventions applied earlier in the course are believed to have higher likelihood of disease modification [2]. In order to better describe and understand such cognitive changes potentially related to early-stage dementia, the concept of mild cognitive impairment (MCI) was operationalized [3] and has since been widely studied by researchers and clinicians. The original conceptualization of MCI mainly focused on the cognitive decline involving learning and memory; and therefore, episodic memory tests were traditionally used to screen for individuals who are at risk of developing dementia [4]. More recent studies call for the need to consider the heterogeneity of MCI when screening, and have found that individuals with MCI exhibit a range of deficits outside the realm of primary explicit memory [5], including impaired focused attention [6, 7], working memory [7], semantic language [7, 8], and executive function [8, 9]. The role of response speed and implicit learning, however, have not been clearly established in the MCI population [10].
In the AD population, existing evidence shows generally slower response speed [11, 12] and higher error rate [11]. However, three independent groups [13–15] found no significant difference on response speed between individuals with MCI and cognitively healthy controls. Close examination by Fernaeus and colleagues [16], however, found that individuals with MCI exhibit slower reaction time than normal health controls only in the later stage of a repetitive simple reaction time task. In terms of implicit learning, two studies [11, 17] found it largely preserved in the AD population, whereas one study [12] found it impaired. Evidence is similarly inconclusive in the MCI population. Two studies [18, 19] found that individuals with MCI demonstrated robust implicit sequence learning, equivalent to cognitive healthy controls. In contrast, Nemeth and colleagues [10] found that when motor learning (i.e., general skill learning) was controlled, individuals with MCI demonstrated poorer implicit sequence learning. The role of response speed and implicit learning is important to understand, as it can enhance understanding of the pathological process of MCI and may help develop tools for MCI screening. A few factors likely contributed to the mixed findings seen in existing literature. For instance, despite a nearly unanimous reference to the original Petersen [3] guideline, the specific assessment tools and diagnostic criteria for MCI vary greatly among studies. In addition, paradigms used to examine response speed and implicit learning also vary across studies, and the multiple processes involved in implicit learning are not always differentiated in analysis. As Nemeth et al. [10] pointed out, implicit sequence learning involves acquisition of both motor and cognitive skills. Additionally, Negash et al. [19] described different levels of cognitive implicit learning besides simple motor learning involving zero-order (i.e., the relative frequency of individual events) and first order (i.e., which pairs of events are more likely to occur) information.
In the current study, we aim to build upon available research and further understand response speed and implicit learning in the MCI population. Specifically, we compare the performance on a simple computer-administered cognitive task, the serial reaction time (SRT) task, between older adults with neuropsychologically defined MCI and age-matched cognitively healthy adults. SRT task is a simple task specifically designed to investigate reaction time and implicit sequence learning [20], and has been widely implemented in both nonclinical and clinical populations [21]. It is considered to be a particularly well suited research paradigm for clinical populations given its modest cognitive and motor demand [12]. The classic protocol for the SRT task requires participants to respond to a visual stimulus repeatedly shown on the screen. Stimulus presentations are usually grouped into blocks, with some blocks containing a predefined sequence whereas others containing pseudorandom sequences. The key performance outcome is participant’s time to react in each trial. Historically, neurologically healthy individuals are found to exhibit a decrease in reaction time across sequenced blocks and perform relatively slower when switched from sequenced blocks to random blocks [22, 23]. Such a contrast is believed to indicate that information about the “sequence” have been learned despite participants’ lack of explicit awareness. To contribute to existing literature, we design a few methodological revisions to better capture the component of high-order cognitive implicit learning. First, we arranged the predefined sequences in the SRT task in a way that eliminates learning based on zero-order and first-order information. Second, we differentiated motor learning from cognitive learning through statistical analysis to maximize the cognitive component. We also address the issue with participant selection in existing literature by performing comprehensive neuropsychological evaluation with explicit diagnostic criteria to identify individuals with all variations of MCI. As a preliminary test, we are also interested in a novel way of utilizing the large amount of data (i.e., 384 trials) available through the SRT task without subsuming them under a few global scores. Specifically, we explored the feasibility of analyzing SRT performance data and identifying individuals with MCI through supervised machine learning.
Our hypotheses with the current study design include the following: 1) the MCI group is predicted to exhibit relatively weaker overall performance on the SRT task compared to their neurologically healthy peers, indicated by slower reaction time and/or higher error rate; 2) implicit sequence learning in the MCI group is predicted to be weaker comparing to their cognitively healthy peers after controlling for motor learning and lower-order cognitive learning; and 3) supervised machine learning is expected to be able to generate a classifier algorithm to identify MCI based on the SRT task performance with reasonable accuracy.
METHODS
Participants
Participants in this work were drawn from a longitudinal cohort at a major academic medical center for a study of brain aging and cognition. Participants were recruited through various sources including the affiliated hospitals and other local advertisements and resources. Informed consents were obtained from the participants and institutional review board approval was obtained from the relevant institute. Participants were generally neurologically and psychiatrically healthy as determined by a medical screen and a neurological evaluation and exhibited broadly normal global cognitive functioning at the time of the assessment (MMSE: 24–30). Participants were excluded if they reported or presented significant acute and/or chronic medical conditions, severe psychiatric or neurological history aside from MCI, current use of psychotropic substances, and current use of other medications that indicate the existence of significant acute and/or chronic medical conditions. All participants who completed the neuropsychological assessment and the SRT task were included in this study, resulting in a sample of 42 adults (21 men and 21women) aged 60 to 80 years.
We divided the 42 participants into two groups, cognitively healthy volunteers (CHV) and adults with mild cognitive impairment (MCI). Group membership was determined through neuropsychological evidence following the “comprehensive criteria” proposed by Jak et al. [24]. Specifically, all the participants were given a comprehensive neuropsychological evaluation battery containing 14 neuropsychological tests assessing global cognitive functioning, premorbid intelligence, and four specific cognitive domains. All tests were scored using standardized norms adjusted for demographic variables including age, sex, education level, and ethnicity when appropriate. Sixteen standardized performance scores from the ten tests assessing the four specific cognitive domains were used to make neuropsychological classification. See Table 1 for a complete list of the performance scores and the statistics by group. The “comprehensive criteria” described by Jak et al. [24] required that at least two performance scores within one cognitive domain fall below the established cutoff score of 1 SD below normative mean in order for that domain to contribute to the MCI classification. The criteria, however, did not explicitly specify the number of “impaired domain” required for the MCI classification. In our study, we adopted Jak et al.’s [24] “comprehensive criteria” and further standardized our procedure to require at least two domains with two or more impaired scores (i.e., 1 standard deviation (SD) below normative mean) in order for that participant to be classified as MCI. In the case where only one test was included in a given domain (i.e., visuo-spatial perception), a score that is 1 SD below normative mean on that one test would be sufficient for the domain to be considered impaired. This set of criteria was applied to all participants in this study except for one, who scored at least 1 SD below normative mean in every test of the memory domain but had no impaired scores in any other domains. This participant was still classified as MCI after consensus consultation given the consistency of this single domain impairment. Based on the classification described above, our sample was divided into the MCI and CHV groups with 20 and 22 participants, respectively.
Neuropsychological tests used for classifying mild cognitive impairment
MCI, mild cognitive impairment group; CHV, CHV, cognitively healthy volunteer group; WAIS-IV, Wechsler Adult Intelligence Scale-Fourth Edition; TMT, Trail Making Test; SDMT, Symbol Digit Modality Test; HVLT-R, Hopkins Verbal Learning Test-Revised; BVMT-R, Brief Visuospatial Memory Test-Revised; WMS-IV, Wechsler Memory Scale-Fourth Edition; D-KEFS VF, Delis–Kaplan Executive Function System, Verbal Fluency; SCWT, Stroop Color and Word Test; JoLO, Judgement of Line Orientation. *p < 0.05; **p < 0.01.
Material and Procedure
Participants enrolled performed the SRT task and underwent a medical and physiological test, a brief neurological and physical examination, a fasting blood draw, a 1- to 2-h 3T MRI session, and a comprehensive neuropsychological assessment. All neuropsychological assessments, including the SRT task, were administered on the same day within a single test session lasting approximately 2 h.
SRT task
The SRT task was administered on a desktop computer using E-prime (version 2.0). In this implementation of the task, four empty boxes were horizontally presented on a black background, and four buttons on a five-button button box were designated to each of the four locations. One of the four boxes was filled in solid white for each trial, and participants were instructed to respond to the location of the target stimulus (i.e., the solid box) as quickly as possible by pressing the corresponding button (see Fig. 1). Once a response was given, the original stimulus disappeared, and the next stimulus appeared after an approximately 280-ms interval. All participants completed a practice run to ensure accurate understanding of the instructions. For the formal SRT task, participants completed eight blocks, each consisting of alternating 24 and 72 trial segments resulting in a total of 384 trials. In the 1st, 3rd, 5th, and 7th blocks, 24 trials were contained in each block and the location of the target stimulus was determined pseudorandomly. In the 2nd, 4th, 6th, and 8th blocks, 72 trials were contained in each block and the target stimulus appeared in a repeated sequential manner (1-2-1-5-2-4-5-1-4-2-5-4). The structure of the blocks differs from traditional SRT paradigm in two important ways [11]. In the current study, we used an alternating sequence of random versus repeated block, whereas traditional SRT paradigms had participants complete multiple repeated blocks consecutively before switching to a random block to test implicit learning. This revision allows better visibility of simple motor learning demonstrated through random blocks. Secondly, the sequences for both repeated blocks and random blocks were designed in a way that the frequencies of the four stimulus positions were equally likely within each block, and the first-order association (i.e., the likelihood that one stimulus is followed by another stimulus position) was balanced so that the frequency of pairing between two positions are equally distributed across both random and repeated blocks. In this manner, learning based on zero-order and first-order information was minimized and implicit sequence learning of complex structure was better captured. Consistent with previous studies, participants remained uninformed of the transition between blocks and the sequence of repeated stimuli throughout the study. Recorded data included the time point a stimulus appeared, participant’s reaction time, the time point a participant responded, the correct response, and a participant’s actual response for each trial. This study was supported by National Institutes of Health/National Institute of Nursing Research. Study design, data collection, and data analysis were independent from the funding source.
Illustration of the serial reaction task paradigm used in this study.
Statistical analysis
Statistical analyses were performed to investigate group differences on several parameters of the SRT task. Two-sample t-test was performed to establish baseline group comparison on reaction time and error numbers. Multivariate regression was then performed to further examine group differences on reaction time and error numbers while controlling for the effect of block type and demographic variables. Mixed-effects models were used to investigate the learning process as proxied by changes in reaction time per block over time. Mixed-effect models were selected as they allow for differed intercept and learning slope at the individual subject level rather than aggregated change at the group level, and they address the correlation of reaction time between different time points of assessment [25]. Visual inspection of the data with interpolation lines suggested that the learning slope changed direction at the third random block (see Fig. 2). To better capture the learning process, piecewise mixed-effect model was selected using the third random blocks as the knot. We first examined simple motor learning before and after the third random block. Mean reaction time per random block was entered as dependent variable. Fixed effects for time (i.e., block number), diagnostic group dummy variable (i.e., CHV = 0; MCI = 1), and the time×diagnostic group interaction were entered as parameters. Random effects of subjects were included to capture individual performance variability. Restricted maximum likelihood method was used for parameter estimation. Implicit learning was then examined with a similar model construction, with dependent variable replaced by mean reaction time per repeated block. Statistical analysis was performed using the IBM SPSS Statistics, Version 25.0.
Simple motor learning and implicit learning was proxied by the changes in mean reaction time over four random and repeated blocks, respectively. Interpolation lines suggested that the learning trajectory shifted direction at the third random block, which was confirmed by piecewise mixed-effect model analysis. The trajectory of learning was not statistically distinctive between individuals with mild cognitive impairment (MCI) and cognitively healthy volunteers (CHV) in either random or repeated blocks.
Supervised machine learning for classification
Random forest is used to explore the potential utility of supervised machine learning procedure in analyzing features of the SRT task. An advantage of using such a procedure is that all available features of a simple task can be included in the analysis as opposed to being subsumed under a few global scores, and those with the highest values to provide the most accurate classification (as opposed to features that are considered conceptually important) are selected to build the classifier. As a preliminary test, we included only one dimension of the SRT task performance (i.e., reaction time) for feature selection and classification training; error rate data was not included. Reaction time for each individual was adjusted for demographic variance. Four features with the highest feature importance values were selected as the final set of features for classification and a classifier algorithm with the highest accuracy was generated. We then conducted a leave one out cross validation experiment repeated for all 42 participants. The predicted label for each participant was compared to their true label based on neuropsychological classification and prediction accuracy was calculated through 100*correctly labeled participants/42. We also calculated precision, recall, positive predictive value, and negative predictive value as classification metrics.
RESULTS
Overall performance comparison
The MCI and CHV groups were matched for age; however, the CHV group had significantly higher education level as well as higher oral reading ability and marginally differed in sex distribution. Demographic variables were therefore entered as covariates in the statistical analysis in order to account for potential bias. Detailed demographic characteristics can be found in Table 2.
Participants demographics
MCI, mild cognitive impairment group; CHV, cognitively healthy volunteer group. *p < 0.05; **p < 0.01.
With all trials included, regardless of the block type, the MCI group showed overall significantly slower reaction time (t(40) = 2.17, p = 0.02). Both groups achieved high accuracy overall (MCI, 94.44%; CHV, 97.20%), which is indicative of the capacity across all participants to understand and perform the task. Nevertheless, the MCI group had higher number of total commission errors per block (t(40) = 3.06, p < 0.01) and total omission errors per block (t(40) = 1.65, p = 0.05) compared to the CHV group despite of the overall high accuracy. See Table 3 for mean comparison statistics.
Serial reaction time task performance comparison by group
MCI, mild cognitive impairment group; CHV, cognitively healthy volunteer group. *p < 0.05; **p < 0.01.
We next performed three multivariate regression analyses using group membership and block type as categorical predictors, and reaction time, commission error number, and omission error number as dependent variables, respectively. Interaction term of group membership by block type was entered to assess if group membership has a different effect in random block performance versus repeated block performance. Demographic variables including age, sex, and education level were entered as covariates to account for potential confounding effects. AMNART score was excluded as it had a moderate-to-strong correlation with education level (r = 0.57). The main effect of group membership remained significant for reaction time (b = 45.50, p < 0.01) while interaction term was non-significant (b = 15.91, p = 0.38), indicating that MCI group reacted relatively slower than the CHV group regardless of block type. The main effect of group membership on number commission and omission errors cannot be interpreted directly as the interaction effect between group and block type was significant for numbers of commission errors (b = –1.07, p = 0.02) and approaching significance for number of omission errors (b = –0.39, p = 0.07). The significant interaction effect suggest that group membership and block type produce additional effect on each other regarding their impacts on the error numbers. In the current sample, the most errors were observed in the MCI group during the repeated blocks (see Table 4).
Serial reaction time task performance as a function of group, block type, and demographics
†Dummy variable that equals one if a participant is in the mild cognitive impairment group, and zero otherwise. ††Dummy variable that equals one if a block is a random block, and zero otherwise. †††Dummy variable that equals one if a participant is male, and zero otherwise *p < 0.05; **p < 0.01.
Simple motor learning and implicit learning
Piecewise fixed-effects model with the third random block as the knot was selected as it best captures the learning trajectories for both groups (see Fig. 2). For simple motor learning, significant reaction time reduction was observed as a function of time from the first to the third random block (β= –50.48, SE = 9.30, p < 0.01). After the third random block, however, reaction time started to increase as a function of time (β= 50.22, SE = 14.15, p < 0.01). The interaction effect between group and time was non-significant at both stages, suggesting that the learning trajectory across random blocks were not distinguishable between the two groups. See Table 5 for detailed model statistics. When examining implicit learning during the first stage (i.e., across the first three repeated blocks) with a similar model construction, mean reaction time did not exhibit a significant change from the first to the third repeated block. Considering the significant changes in motor speed over time, the mean reaction time of previous random block was then entered as covariates to control for simple motor learning. As a result, the residual learning effect was reversed (β= 17.93, SE = 5.81, p < 0.01), suggesting a lack of implicit learning beyond simple motor learning. The lack of implicit learning during the first stage was not found to be significantly different between MCI and CHV groups. During the second stage (i.e., the third to the fourth repeated block), significant changes in reaction time was not observed without controlling for the motor speed in random blocks. After controlling for motor speed, however, significant implicit learning was observed in both MCI and CHV groups (β= –16.72, SE = 7.68, p = 0.04). The learning slope was, again, not significantly different between the two groups as indicated by a non-significant interaction term between time and group. See Table 6 for detailed model statistics.
Piecewise mixed-effect model for simple motor learning as a function of time and group membership
†Dummy variable that equals one if a participant is in the mild cognitive impairment group, and zero otherwise. *p < 0.05; **p < 0.01.
Piecewise mixed-effect model for implicit learning as a function of time and group membership with and without controlling for simple motor speed
†Dummy variable that equals one if a participant is in the mild cognitive impairment group, and zero otherwise. *p < 0.05; **p < 0.01.
Random forest classification
In the feature selection phase, the four features with highest prediction value were
DISCUSSION
This study examined response speed and implicit learning in a group of individuals with neuropsychologically-defined MCI through a simple SRT paradigm. We additionally explored the feasibility of using supervised machine learning to analyze the SRT performance data. In this study, we made a few important revisions to the traditional SRT paradigm to better capture high-order implicit sequence learning while controlling for simple motor learning and minimizing zero-order and first-order learning. With this paradigm, we found that both groups achieved high overall accuracy, showing that it is an easy-to-administer task even for people with neuropsychologically defined cognitive impairment. Despite of mixed findings in previous studies, the MCI group in our study exhibited significantly poorer performance on all three general outcome measures (i.e., mean reaction time, total commission errors, and total omission errors) than the age-matched control group. We found that individuals with MCI performed generally slower regardless of the type of block (i.e., random versus repeated), whereas their error rates were considerably higher in repeated blocks comparing to random blocks. These findings held true after controlling for demographic variables.
Implicit sequence learning ability in individuals with MCI was mixed in previous studies. As noted before, this study took into consideration multiple layers of processes involved in implicit learning. Additionally, we examined the trajectory of both simple motor learning and implicit learning and employed mixed-effect model analysis to allow for the variation in individual learning slopes in each group. We found that both MCI and CHV groups exhibited significant simple motor learning (i.e., significant reaction time reduction) during the first half of the test; however, their reaction time increased during the last quarter of the SRT task, which is likely a result of motor fatigue. This pattern was not statistically distinguishable between MCI and CHV groups. When exploring implicit learning, our study design of alternating random and repeated blocks allowed us to control for the simple motor speed using the mean reaction time from previous random block. We found that neither MCI nor CHV group exhibited effective implicit learning beyond simple motor learning during the first half of the task. However, towards the end of the SRT task, both groups exhibited significant implicit learning and the learning slopes were not distinguishable between the groups. In sum, our results on learning was mostly supportive of previous literature on the preserved implicit learning in MCI and even mild AD populations. At the same time, our results shed lights on the possibility that simple motor learning and implicit learning occur at different stages of the task, and the interaction between the two may mask the representation of overall learning effect if not examined carefully. Another goal of this study was to explore the feasibility of utilizing supervised machine learning to analyze SRT task features. With features selected from demographically-adjusted reaction time, we were able to achieve a classifier with sensitivity of 85.0% and specificity of 77.3% using random forest method. Given the small sample size available, the results are considered preliminary and should not be generalized. However, the current results provided support to the concept of enhancing predictive values of simple cognitive tasks through advanced statistical tools.
In conclusion, the current study compared and documented the performance in various parameters of the SRT task performance between older adults with MCI and age-matched cognitively healthy volunteers. We demonstrated here that a simple, brief test of visuospatial attention and motor learning is sensitive to some deficits in neuropsychologically defined MCI, such that the MCI group exhibited overall slower reaction time, higher error rate in repeated blocks, but exhibited simple motor learning and implicit learning abilities that are comparable to their age-matched peers. These findings are relatively subtle given that these individuals were in the early stages of neuropathology. However, our results provide support to the possibility that, with sound methodological design and advanced statistical tools, less traditional and less overt cognitive markers may be useful in early detection of cognitive abnormality through a multidimensional and data-driven approach.
The current study has a few limitations. Firstly, the sample size in this study is rather small, which limits the generalizability of the findings, particularly on the machine learning procedure. Secondly, the cross-sectional design does not allow for analysis of rate of progression and thus poses limitations on the clinical implication of the results. Thirdly, data from a non-behavioral domain such as biomarkers were not available at the time of this study. Future research will follow up with goals of a much larger sample, a longitudinal study design, and incorporation of AD biomarkers.
Footnotes
ACKNOWLEDGMENTS
This study was supported by National Institutes of Health/National Institute of Nursing Research R01NR010827, National Institute of Biomedical Imaging and Bioengineering R01EB023281, National Institute of Neurological Disorders and Stroke R01NS105820 R01NS083534, National Institute on Drug Abuse U24DA041123, and National Institute on Aging U01AG052564. We are thankful to the Massachusetts General Hospital Alzheimer’s Disease Research Center for assistance in enrollment of participants for this study.
