Cognitive Speed in Neurodegenerative Disease: Comparing Mean Rate and Inconsistency Within and Across the Alzheimer’s and Lewy Body Spectra in the COMPASS-ND Study

Mean rate and inconsistency in the AD spectrum

Performance differences in mean rate and inconsistency have been reported among cohorts of the AD spectrum. Some studies show slower mean rate (about 100 ms slower) in subjective cognitive impairment (SCI) cohorts compared to CU cohorts [27, 28]. However, a study using the Go-No-Go task demonstrated that these cohorts did not differ significantly in inconsistency (measured as intraindividual standard deviation; ISD): T-score of 8.78 (SD = 1.87) for CU and 9.02 (SD = 2.12) for SCI [29]. A recent meta-analysis comparing CU (M age = 68.2 years) and mild cognitive impairment (MCI; M age = 72.3 years) cohorts found that mean rate was slower by 11% for the latter cohort [30]. This general slowing is evident for both relatively simple tasks (e.g., SRT) [13, 31] and more complex tasks (e.g., CRT) [32, 33]. Conflicting evidence has appeared regarding inconsistency differences between CU and MCI cohorts. Some studies have established that inconsistency in cognitive speed is greater in the MCI cohort compared to the CU cohort [10, 34–36]. However, this has not been confirmed in other studies [37–40]. The discrepancy may be related to differential (a) classification procedures for the unimpaired and impaired cohorts, (b) inconsistency calculations, and (c) speeded task characteristics. The greatest disturbances in mean rate and inconsistency are observed in AD cohorts. Compared to both the CU and MCI cohorts, the AD cohort is slower (more than 200 ms in complex RT tasks) [37] and exhibits greater inconsistency (more than 2.50 difference in mean intraindividual standard deviation for SRT and CRT) [41; see also 30, 42]. For information about the neural bases of mean rate and inconsistency in the AD spectrum, see the Supplementary Material.

Mean rate and inconsistency in the LBD spectrum

Performance differences in mean rate and inconsistency have also been examined among cohorts of the LBD spectrum. Compared to CU cohorts, Parkinson’s disease (PD) cohorts exhibit slower mean rate and make more errors in both SRT and CRT tasks [43–46]. Furthermore, greater inconsistency is evident in PD cohorts than CU cohorts [47–49]. Mild cognitive impairment in PD (PD-MCI) cohorts can exhibit deficits in attention, which is important for faster orienting and reaction time (RT) [50, 51]. Using an attentional network task assessing alerting, orienting, and executive control, a recent study demonstrated that the PD-MCI cohort has slower RT compared to the PD and CU cohorts [52]. However, inconsistency in these cohorts was not examined. PD cohorts who convert to dementia show worse performance in mean rate and inconsistency. For instance, in CRT tasks, PDD cohorts show slower mean rate (57.22 ms, SE = 1.06) compared to both the PD (47.85 ms, SE = 0.87) and CU (47.15 ms, SE = 0.64) cohorts [53]. Changes in inconsistency are also greater among the PDD cohort (M= 8.82, SE = 0.46) than the PD (M= 6.92, SE = 0.38) and CU (5.79, SE = 0.28) cohorts. For information about the neural bases of mean rate and inconsistency in the LBD spectrum, see the Supplementary Material.

The present study

The two facets of cognitive speed performance (mean rate and inconsistency) may be differentially disrupted across the AD and LBD spectra. Although significant progress has been made in differentiating cognitive speed performance among cohorts of each spectrum, there are still important gaps within and across the two conditions. First, it has not been established when the earliest changes of mean rate and inconsistency in the AD and LBD spectra occur. Incorporating cohorts representing the full spectrum of AD and LBD will provide useful information regarding this goal. Second, reports suggest that both inconsistency in global cognitive performance (e.g., Mini-Mental State Exam) and self-rated everyday fluctuations are higher in LBD patients compared to AD patients [51, 55]. More detailed investigations are required to evaluate this possible pattern with both mean rate and inconsistency as measured by in performance on both simple and complex RT tasks. Furthermore, corresponding cohort differences across both spectra have not been examined (i.e., AD versus LBD, MCI versus PD-MCI) in the same study with the same tasks. Third, discrimination analyses can be performed to further validate cohort difference results and provide supplemental information pertaining to the question of whether mean rate or inconsistency differ in their sensitivity for discriminating cohort membership. However, not all studies conduct discrimination analyses on both mean rate and inconsistency, and none compare within and across cohorts in one study. Thus, four important contributions of this research are that it (a) assembles six well-characterized clinical cohorts representing the full spectrum of AD and LBD, (b) deploys the same cognitively unimpaired cohort as benchmark, (c) analyzes both mean rate and inconsistency performance derived with the same procedures (across the same two cognitive speed tasks), and (d) compares across a full complement of carefully diagnosed cohorts within each spectrum and selectively across spectra.

Three research goals guided this study. First, we examined cohort differences in mean rate. We expected to find slower mean rate in the AD cohort compared to the CU, SCI, and MCI cohorts. Similarly, we expected to find slower mean rate in the LBD cohort compared to the CU, PD, and PD-MCI cohorts. Second, we examined cohort differences in inconsistency. We expected to find greater inconsistency in the AD cohort compared to the CU, SCI, and MCI cohorts. We also expected greater inconsistency in the LBD cohort compared to the CU, PD, and PD-MCI cohorts. Third, we performed validation tests of the relative predictive potential of mean rate and inconsistency in discriminating cohort membership within each spectrum. Due to discrepancies and insufficient analyses in the literature, we did not specify an expectation about which indicator would be more sensitive in predicting cohort membership.

Participants

Participants were recruited in the context of the Canadian Consortium on Neurodegeneration in Aging (CCNA). The CCNA clinical cohort study, referred to as Comprehensive Assessment of Neurodegeneration and Dementia (COMPASS-ND), collects clinical, neuropsychological, biological, genetic and other data for individuals aged 50–90 years with the goal of recruiting and diagnose or classify participants fitting one of several pre-selected impairment or neurodegenerative disease groups [56]. COMPASS-ND data are stored and retrieved from LORIS, a modular data management system that integrates acquisition, storage, curation, and dissemination across multiple modalities [57]. For this study, we used data released in May 2021. Participants were excluded from COMPASS-ND if they expressed (a) significant known chronic brain disease conditions (e.g., multiple sclerosis, Huntington’s disease, traumatic brain injury, other brain illnesses), (b) alcohol or drug abuse, (c) lack of English or French proficiency, (d) symptomatic stroke within the previous year, or (e) unwillingness to undergo MRI scan [56]. All participants provided written informed consent to participant in the study. Local approval was received by the University of Alberta Health Research Ethics Board. We included COMPASS-ND cohorts representing the spectra of AD and LBD (see Table 1 for detailed participant characteristics). A CU cohort (n = 39, M age = 70.4, 15.4% male) provided a benchmark for analyses in both spectra. Within the AD spectrum, we included SCI (n = 28, M age = 70.90, 17.2% male), MCI (n = 121, M age = 71.13, 50.4% male), and AD (n = 45, M age = 74.91, 68.9% male) cohorts. Within the LBD spectrum, we included PD (n = 32, M age = 66.77, 53.1% male), PD-MCI (n = 21, M age = 71.57, 81.0% male), and LBD (n = 11 DLB + n = 7 PDD, M age = 73.57, 88.2% male) cohorts. Consideration of sex differences were limited due to distribution restrictions in the database.

Clinical diagnostic criteria

Established diagnostic criteria were assembled and consensually applied for all classification decisions by a team of CCNA clinicians experienced in aging and neurodegenerative diseases [56, 58]. The harmonized criteria for each cohort were as follows.

Descriptive information

We collected the following descriptive information about the participants (see Table 1): (a) MoCA [71]; (b) Geriatric Depression Scale (GDS) [72]; (c) Physical Activity Scale for the Elderly (PASE) [73]; (d) Generalized Anxiety Disorder 7-item Scale (GAD-7) [74]; (e) Pittsburgh Sleep Quality Index (PSQI) [75]; (f) Mayo Fluctuation Scale (MFS) [76]; and (g) pulse pressure (PP; equals systolic blood pressure-diastolic blood pressure, in mmHg) [77, 78]. For PD, PD-MCI, and LBD cohorts, we included the following: (a) calculation of levodopa equivalent dose (LED, in mg) [79]; (b) MDS-UPDRS parts IA and IB: Non-Motor Aspects of Daily Living; (c) MDS-UPDRS part II: Motor Aspects of Experiences of Daily Living; (d) MDS-UPDRS part III: Motor Examination; and (f) MDS-UPDRS part IV: Motor Complications of Dopaminergic Therapy [65].

Cognitive speed tasks

We used two established cognitive speed tasks, SRT and CRT. Our principal performance indicators of mean rate and inconsistency were derived from both tasks. These and similar tasks have been previously used in related research in aging and neurodegenerative disorders [10, 81]. The tasks were administered on a Dell XPS 9343 laptop running Ubuntu 16.04 LTS operating system with a 13-inch display. Responses were entered using a Targus numeric keypad (SKU: AKP10CA). The task procedures supported the intended difference in task complexity. For the SRT, participants responded to a signal stimulus (+) by pressing a green key on a keypad. Responses were made after a warning stimulus (III) to pay attention was presented. After responding, the warning stimulus appeared again and participants were ready to respond again. There was a total of 5 practice trials. For the actual test trials, we administered 50 trials with 10 randomly arranged trials presented at shorter and longer intervals separating the warning and signal stimuli (500, 625, 750, 875, and 1,000 milliseconds). We used the latency scores of the 50 test trials. For the CRT, participants were presented with a row of four black squares (warning stimuli, 1000 milliseconds delay). The appearance of the row with four black squares was the cue to pay attention. There were four colored buttons on the keypad positioned in the following order: red, yellow, blue, and orange. One of the four squares randomly changed to either a red, yellow, blue, or orange “X”, indicating the same position as the four colored buttons. In this 1x4 grid, participants were required to make a speeded decision exclusively based on the color (not the position) of the “X” presented on the screen. Specifically, responses were made by pressing one of the four colored buttons on the keypad that corresponded to the colored “X” on the screen. We administered 10 practice trials and 60 test trials for the CRT. We used the latency scores of the 60 test trials for correct and incorrect responses.

For both tasks, the COMPASS-ND data collection protocol was designed to minimize the extent to which participants could anticipate the onset of the target stimuli. For the SRT implementation, the intervals between the warning and signal stimuli were randomized and thus not the same across trials. Although participants pressed the same button for every trial, the different randomized (unrecorded) delays reduced the possibility of effective anticipation. For the CRT implementation, participants were unaware of the color of the “X” that would appear on the screen and thus not able to anticipate the target stimuli. For both tasks, reaction time was quantified as the interval between the target stimulus onset and the onset of the response.

Data preparation: Mean rate and inconsistency

Following previous protocols [10, 82], we conducted a sequence of data preparation procedures for mean rate and inconsistency in the two cognitive speed tasks. For mean rate, we first examined the distribution of raw latency scores for outliers. In computer-based RT tasks, errors made by accidental key presses or task interruptions could lead to extremely slow or fast responses. Accordingly, we applied specific procedures established in previous aging and dementia studies [11, 53] for (a) detecting aberrant trials for all participants and then (b) imputing values for the detected aberrant trials. Specifically, a lower bound limit for legitimate responses was set for both tasks at 150 ms. This lower limit was the same for all cohorts across both spectra. A value below 150 ms is considered unrepresentative (too fast) to be a typical or legitimate RT to a stimulus change, even for CU older adults [10, 53]. A priori upper limits (2500 ms for SRT and 4000 ms for CRT) were applied exclusively to the CU (benchmark) cohort as informed by prior research [10, 83]. For the clinical cohorts (SCI, MCI, AD; PD, PD-MCI, LBD), we used an empirical approach to detecting within-sample aberrations specific to these study tasks and participants as implemented in previous cognitive speed studies [53, 84]. In this approach, upper bounds were determined for each task and individual by computing the intraindividual mean and standard deviation. RTs for trials that exceeded three standard deviations above a given individual’s mean were replaced. The empirically derived upper bounds were different for the two spectra. Within the AD spectrum, the upper bound response was of 1093 ms for SRT and of 1553 ms for CRT. Within the LBD spectrum, the upper bound response was of 737 ms for SRT and of 2066 ms for CRT. The percentage of detected and replaced values was less than 1% for both spectra (i.e., 0.35% for the AD and 0.53% for the LBD spectra). We imputed values for the aberrant trials using regression imputation/substitution. This procedure estimates missing values based on the relationship among responses across trials [85]. Missing values were imputed using data from all available individuals and trials. The number of values imputed across the entire Persons X Trials data matrix was 2.50% (1.62% lower; 0.88% upper).

For inconsistency (i.e., intraindividual variability), we computed the residualized ISD for each participant (separately for each cognitive speed task). The residualized ISD is a quantification of RT inconsistency that controls for systematic confounds in both between- (e.g., cohort differences in mean rate) and within-subjects (e.g., polynomial time effects) in the raw data [86]. Specifically, during the ISD-creation process, we controlled for any practice or learning effects to ensure that differences in inconsistency were not simply a statistical artifact of differences in individual or group mean performance. Higher ISD scores indicated greater inconsistency across trials, whereas lower scores indicated more consistent performance. We reported all mean ISD scores as T-scores.

Statistical analyses

All analyses were performed on IBM SPSS Statistics Version 28. The dataset included available data for all participants. Therefore, no missing data imputation was necessary. We examined cohort differences within two spectra, with the same CU cohort included as comparison benchmark in both: AD (SCI, MCI, AD) and LBD (PD, PD-MCI, LBD). Furthermore, we selectively examined cohort differences across spectra, based on global cognitive impairment groupings, in (a) AD with LBD and (b) MCI with PD-MCI.

For research goal 1 (cohort differences in mean rate), we conducted one-way multivariate analyses of variance (MANOVAs) on the mean latency scores (for both RT tasks) separately for the AD spectrum, LBD spectrum, dementia (AD, LBD), and impairment (MCI, PD-MCI) cohorts. For research goal 2 (cohort differences in inconsistency), we conducted one-way MANOVAs on the ISD scores (for both RT tasks) separately for the AD spectrum, LBD spectrum, dementia, and impairment cohorts. We conducted MANOVAs rather than multiple ANOVAs for three important reasons. First, MANOVAs can produce greater statistical power than multiple ANOVAs as the correlation structure between dependent variables provides additional information to the model, which could help detect smaller effects [87]. This applies when the dependent variables are correlated, which was the case between SRT and CRT for both speed indicators: mean rate, r (302) = 0.47, p < 0.001; inconsistency, r (302) = 0.40, p < 0.001. Second, an alternative approach of conducting ANOVAs with cohort X task interaction terms was considered. Using MANOVAs permitted the detection of similar or different patterns for both speed indicators and between both RT tasks. Third, given the series of comparisons required to test within- and across-spectrum differences, MANOVA reduced type 1 error rates as both speed tasks could be included simultaneously for within- and across-spectrum comparisons.

We tested the assumptions of multivariate normality and homogeneity of variance-covariance matrices. The Shapiro-Wilk test showed that the distribution of residual scores for both RT tasks departed from normality (SRT: W = 0.87, p < 0.001; CRT: W = 0.94, p < 0.001). Similarly, the Box’s M test showed that the assumption of equality of variance-covariance matrices was not met (M = 183.97; F (18, 634) = 9.92; p < 0.001). Accordingly, we used Pillai’s trace, a robust test statistic adequate for departures from assumptions in data with unbalanced and small sample sizes [88]. As some of the cohorts were relatively small, we conducted unadjusted analyses for research goal l and research goal 2. The results of the unadjusted models are shown in Supplementary Table 1. Overall, the unadjusted and adjusted models identified the same significant cohort differences in mean rate and inconsistency for (a) within and across spectra comparisons and (b) both speed tasks.

For research goal 3 (relative predictive potential of mean rate and inconsistency in discriminating cohort membership), we conducted receiver operating characteristic (ROC) analyses to validate and supplement cohort difference results. ROC analysis evaluates the accuracy of a binary classification [89, 90]. The ROC curve uses three metrics to assess the performance of a prediction model and determine its ability to predict a dichotomous outcome: area under the curve (AUC), sensitivity, and specificity. As such, it does not test for statistical significance (e.g., in multiple comparisons) or requires methods to control for the family-wise error rate. The ROC approach provided information on the extent to which two behavioral markers (mean rate and inconsistency) discriminate among the groups in each spectrum (and selectively across parallel groups from the spectra). The AUC results were influenced by the lack of homogeneity within each subgroup, but this is indeed the nature of the long transitional onset phase of neurodegenerative diseases. Notably, integrated into the analyses for both spectra was one common unimpaired group (CU). This group served as a control group in that it is common to the analyses in both spectra, and it is currently non-overlapping with disease features.

Cross-validation procedures are useful in estimating the performance (or accuracy) of machine learning and other prediction models to support potential generalizability. In practice, external cross-validation is rarely possible as independently collected data (e.g., from different studies) typically present multiple challenges for harmonization. Furthermore, we are unaware of any similar multi-cohort data set with the present set of key variables. Internal cross-validation in which the data is divided into two equivalent parts for training and testing is another procedure used in machine learning [91–93]. The “Leave-One-Out-Cross-Validation” is an approach used particularly for small sample data [94]. This procedure was not preferred for this study, as the ROC analysis was strategically supplemental to the first and second research goals. Specifically, after examining cohort differences in mean rate and inconsistency, the ROC analysis was performed to validate cohort difference results and provide supplemental information pertaining to the question of whether mean rate or inconsistency differ in their sensitivity for discriminating cohort membership. Sensitivity patterns in mean rate and inconsistency within and across spectra were confirmed with the sensitivity, specificity, and AUC metrics provided by the multiple ROC analyses.

For all three research goals, we adopted analysis plans from previous related studies [10, 95] that examined mean rate and inconsistency sensitivity patterns separately for speed tasks varying in complexity. The general notion in related aging and neurodegeneration research is that sensitivity may be more pronounced in the CRT task compared to SRT task because of the increased number of mental operations required for the latter. Accordingly, the CRT task may provide more sensitivity for detecting response distribution differences (i.e., slower mean rate and greater inconsistency) among the multiple clinical cohorts of the AD and LBD spectra. We tested this notion systematically for mean rate and inconsistency comparisons within and across spectra.

A series of steps were taken to address the issue of multiple comparisons. First, we set statistical significance at p≤0.01 for MANOVAs. Second, we reported measures of effect size (partial eta squared; partial η²) with confidence intervals for all MANOVAs. Effect size estimation is a useful method for interpreting meaningful statistical outcomes through an alternative process to that of controlling for type 1 error rates [96]. Essentially, this metric quantifies the magnitude of group differences, independent of significance testing [97]. Third, for post-hoc analyses, we used the Bonferroni method to control for type 1 error. The Bonferroni method is useful when multiple groups are compared at baseline and the comparisons are based on results from the data or all combination of comparisons [96].

Cohort differences in participant characteristics

Analyses for research goals 1 and 2 evaluated cohort differences in mean rate and inconsistency within and across spectra. Such differences may be influenced by other participant characteristics, including measurable demographic and clinical attributes. Accordingly, we also examined cohort differences in participant characteristics within and across spectra. For those participant characteristics in which there were significant cohort differences, we included them as covariates in subsequent analyses to control for their potential influence. As can be seen in Supplementary Table 2, within the AD spectrum, there were significant cohort differences in age, sex, MoCA, GDS, and GAD-7. Similarly, within the LBD spectrum, there were significant cohort differences in age, sex, MoCA, PASE, GDS, GAD-7, PSQI, and MFS. For the dementia comparisons, there were significant cohort differences in GDS, GAD-7, and PSQI. For the impairment comparisons, there were significant cohort differences in sex, MoCA, PASE, and PSQI.

Research goal 1: Cohort differences in mean rate

Within the AD spectrum, the MANOVA comparing mean rate performance in the four cohorts revealed a significant multivariate effect: Pillai’s trace = 0.36, F(6, 454) = 16.82, p < 0.001, partial η² = 0.18 (99% CI: 0.10–0.25). Univariate tests showed significant cohort differences for performance in both tasks: SRT, F(3, 227) = 12.88, p < 0.001, partial η² = 0.15 (99% CI: 0.04–0.25); CRT, F(3, 227) = 35.01, p < 0.001, partial η² = 0.32 (99% CI: 0.19–0.42). Figures 1A and 1B display mean rate performance by cohort in the SRT and CTR tasks, respectively. For both tasks, the AD cohort exhibited slower mean rate compared to the CU, SCI, and MCI cohorts (p < 0.001). Results specific to the SRT task revealed that the MCI cohort exhibited slower mean rate compared to the CU cohort (p = 0.01). No significant differences were found between the CU and SCI cohorts (SRT: p = 0.49; CRT: p = 0.95).

OPEN IN VIEWER

Fig. 1

Mean Rate by Spectra and Cohort for Each Task. The figure displays the mean rate performance for the AD spectrum (top row, A and B) and LBD spectrum (bottom row, C and D) on the SRT (left column) and CRT (right column) tasks. Error bars represent standard error of mean. AD spectrum results: AD exhibited slower mean rate than CU, SCI, and MCI (for SRT and CRT); MCI exhibited slower mean rate than CU (for SRT). LBD spectrum results: LBD exhibited slower mean rate than CU, PD, and PD-MCI (for SRT and CRT); PD-MCI exhibited slower mean rate than CU (for SRT and CRT); PD-MCI exhibited slower mean rate than PD (for CRT). MANOVA effect: [AD spectrum comparisons, F(6, 454) = 16.82; LBD spectrum comparisons, F(6, 202) = 5.12]. ^*p≤0.01; ^**p≤0.001.

Within the LBD spectrum, the MANOVA comparing mean rate performance in the four cohorts revealed a significant multivariate effect: Pillai’s trace = 0.26, F(6, 202) = 5.12, p < 0.001, partial η² = 0.13 (99% CI: 0.02–0.23). Univariate tests showed significant cohort differences for performance in both tasks: SRT, F(3, 101) = 9.51, p < 0.001, partial η² = 0.22 (99% CI: 0.05–0.37); CRT, F(3, 101) = 6.98, p < 0.001, partial η² = 0.17 (99% CI: 0.02–0.32). Figures 1C and 1D display mean rate performance by cohort in the SRT and CRT tasks, respectively. For both tasks, the LBD cohort exhibited slower mean rate compared to the CU, PD, and PD-MCI cohorts (p < 0.01). Furthermore, the PD-MCI cohort exhibited slower mean rate compared to the CU cohort (p = 0.01). Results specific to the CRT task revealed that the PD-MCI cohort exhibited slower mean rate compared to the PD cohort (p < 0.01). No significant differences were found between the CU and PD cohorts (SRT: p = 0.21; CRT: p = 0.97).

For the comparisons across spectra, the MANOVA comparing mean rate performance in the two dementia cohorts revealed a significant multivariate effect: Pillai’s trace = 0.13, F(2, 59) = 4.46, p = 0.01, partial η² = 0.13 (99% CI: 0.00–0.33). Univariate tests showed that the LBD cohort was significantly slower than the AD cohort for performance in the SRT task, F(1, 60) = 9.05, p < 0.01, partial η² = 0.13 (99% CI: 0.00–0.34); but not in the CRT task, F(1, 60) = 2.94, p = 0.09, partial η² = 0.05 (99% CI: 0.00–0.23). Figures 2A and 2B display mean rate performance of both the AD and LBD cohorts in the SRT and CRT tasks, respectively. The MANOVA comparing mean rate performance in the two impairment cohorts revealed a significant multivariate effect: Pillai’s trace = 0.10, F(2, 139) = 7.46, p < 0.001, partial η² = 0.09 (99% CI: 0.01–0.22). Univariate tests showed that the PD-MCI cohort was significantly slower than the MCI cohort for performance in the CRT task, F(1, 140) = 14.78, p < 0.001, partial η² = 0.10 (99% CI: 0.01–0.23); but not in the SRT task, F(1, 140) = 2.63, p = 0.11, partial η² = 0.02 (99% CI: 0.00–0.11). Figures 2C and 2D display mean rate performance of both the MCI and PD-MCI cohorts in the SRT and CRT tasks, respectively.

OPEN IN VIEWER

Fig. 2

Mean Rate Across Spectra and by Cohort for Each Task. The figure displays the mean rate performance for the dementia cohorts (A and B: AD and LBD) in top row and for the cognitive impairment cohorts (C and D: MCI and PD-MCI) in bottom row. Results for the SRT (left column) and CRT (right column) are displayed. Error bars represent standard error of mean. LBD exhibited slower mean rate than AD (for SRT); PD-MCI exhibited slower mean rate than MCI (for CRT). MANOVA effect: [dementia cohorts, F(2, 59) = 4.46; impairment cohorts, F(2, 139) = 7.46]. ^*p≤0.01; ^**p≤0.001.

Research goal 2: Cohort differences in inconsistency

Within the AD spectrum, the MANOVA comparing inconsistency performance in the four cohorts revealed a significant multivariate effect: Pillai’s trace = 0.16, F(6, 454) = 6.41, p < 0.001, partial η² = 0.08 (99% CI: 0.02–0.13). Univariate tests showed significant cohort differences for performance in both tasks: SRT, F(3, 227) = 8.19, p < 0.001, partial η² = 0.09 (99% CI: 0.02–0.19); CRT, F(3, 227) = 9.95, p < 0.001, partial η² = 0.12 (99% CI: 0.03–0.26). Figures 3A and 3B display inconsistency performance by cohort in the SRT and CRT tasks, respectively. For both tasks, the AD cohort exhibited greater inconsistency compared to the CU, SCI, and MCI cohorts (p < 0.001). No additional significant differences were observed.

OPEN IN VIEWER

Fig. 3

Inconsistency by Spectra and Cohort for Each Task. The figure displays the RT inconsistency performance for the AD spectrum (top row, A and B) and LBD spectrum (bottom row, C and D) on the SRT (left panels) and CRT (right panels) tasks. Error bars represent standard error of mean. AD spectrum results: AD exhibited greater inconsistency than CU, SCI, and MCI (for SRT and CRT). LBD spectrum results: LBD exhibited greater inconsistency than CU, PD, and PD-MCI (for SRT and CRT). MANOVA effect: [AD spectrum comparisons, F(6, 454) = 6.41; LBD spectrum comparisons, F(6, 202) = 5.21]. ^**p≤0.001.

Within the LBD spectrum, the MANOVA comparing inconsistency performance in the four cohorts revealed a significant multivariate effect: Pillai’s trace = 0.27, F(6, 202) = 5.21, p < 0.001, partial η² = 0.13 (99% CI: 0.02–0.23). Univariate tests showed significant cohort differences for performance in both tasks: SRT, F(3, 101) = 8.67, p < 0.001, partial η² = 0.20 (99% CI: 0.04–0.36); CRT, F(3, 101) = 6.31, p < 0.001, partial η² = 0.16 (99% CI: 0.01–0.31). Figures 3C and 3D display inconsistency performance by cohort in the SRT and CRT tasks, respectively. For both tasks, the LBD cohort exhibited greater inconsistency compared to the CU, PD, and PD-MCI cohorts (p < 0.001). No additional significant differences were observed.

For the comparisons across spectra, the MANOVA comparing inconsistency performance in the two dementia cohorts revealed a significant multivariate effect: Pillai’s trace = 0.12, F(2, 59) = 4.31, p = 0.01, partial η² = 0.13 (99% CI: 0.00–0.32). Univariate tests showed that the LBD cohort exhibited significantly greater inconsistency than the AD cohort for performance in both tasks: SRT, F(1, 60) = 8.57, p < 0.01, partial η² = 0.12 (99% CI: 0.00–0.33); CRT, F(1, 60) = 5.48, p = 0.01, partial η² = 0.08 (99% CI: 0.00–0.28). Figures 4A and 4B display inconsistency performance of both the AD and LBD cohorts in the SRT and CRT tasks, respectively. The MANOVA comparing inconsistency performance in the two impairment cohorts did not reveal a significant multivariate effect: Pillai’s trace = 0.03, F(2, 122) = 1.93, p = 0.15, partial η² = 0.03 (99% CI: 0.00–0.13). Consequently, we did not proceed to examine univariate results. For comparison purposes, Figures 4C and 4D display inconsistency performance of both the MCI and PD-MCI cohorts in the SRT and CRT tasks, respectively.

OPEN IN VIEWER

Fig. 4

Inconsistency Across Spectra and by Cohort for Each Task. The figure displays the RT inconsistency performance for the dementia cohorts (A and B: AD and LBD) in top row and for the cognitive impairment cohorts (C and D: MCI and PD-MCI) in bottom row. Error bars represent standard error of mean. Results for the SRT (left panels) and CRT (right panels) tasks are displayed. LBD exhibited greater inconsistency than AD (for SRT and CRT). MANOVA effect: [dementia cohorts, F(2, 59) = 4.31; impairment cohorts, F(2, 122) = 1.93]. ^*p≤0.01.

Research goal 3: Validation: Relative predictive potential of mean rate and inconsistency in discriminating cohort membership

Research goal 1: Mean rate comparisons in the AD and LBD spectra

Within the AD spectrum, mean rate comparisons for both speed tasks revealed the robust finding that the AD cohort was slower than the MCI, SCI, and the CU cohorts. Notably, only one other within-spectrum (non-AD-related) comparison was significant: the MCI cohort was slower (on SRT performance) than the CU cohort. This pattern accentuates probable dementia-intensive neurocognitive and fine motor perturbations and may reflect substantial compromise to multiple large-scale networks for diagnosed members of the AD cohort. Markedly slower mean rate of speed performance among persons with AD may occur as a function of progressive neuropathological compromises, including hypometabolism of inferior parietal lobe, medial prefrontal cortex and insula, subcortical white matter hyperintensities in the periventricular region, and reduced grey matter volume in right frontal pole and left supramarginal gyrus [98, 99]. The observed slower mean rate in the MCI cohort compared to the CU cohort may reflect the elevated AD risk of the former cohort and impairment-related neuropathology, including increased activation of temporoparietal junction and posterior parietal regions, less activation of prefrontal and anterior cingulate cortices, and attenuation of the medial prefrontal cortex and default mode networks [100–102]. Some studies have reported slower mean rate in SCI than CU cohorts [27, 28], which differ primarily in the extent of their concern about subjectively experienced cognitive deficits. However, these studies either (a) used different tests to measure RT (e.g., attentional network test); (b) included older SCI cohorts (Mage = 71.16 years) or (c) included CU and SCI cohorts with fewer years of education (M = 9.40 years and M = 9.80 years, respectively). The relatively similar performance observed among the present SCI and MCI cohorts could be further investigated regarding potential neural compensatory mechanisms. One potential research direction relates to the previous finding that for both SCI and MCI cohorts, functional activity increases in fronto-parietal regions represent a compensatory recruitment during RT and other attention-related tasks [103–106]. In general, within-spectrum cohort differences may appear or not depending on several factors but will reliably appear when the comparison involves a diagnosedAD group.

Within the LBD spectrum, mean rate comparisons for both speed tasks revealed the robust finding that the LBD cohort was significantly slower than the PD-MCI, PD, and CU cohorts. Notably, the cognitively impaired PD cohort (PD-MCI) was slower than the CU cohort (for both tasks) and the PD cohort (for CRT only). Overall, the LBD pattern is similar to the AD pattern, with the qualification that PD-related neuropathological impairment may convey greater speed-related disturbance than AD-related impairment. Slower performance in the LBD cohort than the non-LBD cohorts in the spectrum may reflect LBD-related pathology involving progressive deactivation of the posterior default mode network [51, 107] and grey matter atrophy involving the caudate nucleus, putamen, and thalamus bilaterally, and right globus pallidus [108, 109]. In addition to dementia, some exacerbated motor-specific disturbances may contribute to the LBD (and possibly PD-MCI) slowing effects. Indeed, linkages among motor and cognitive performance have been reported for LBD cohorts [58, 111]. A recent study found that atrophy of motor-related brain regions, particularly reduced grey matter volumes in the caudate and subthalamic nucleus, are related to cognitive slowing in LBD [108]. Reports indicate that slower mean rate in PD-MCI than CU cohorts may relate to specific brain-related changes in PD-MCI, such as suppressed activation in the left postcentral gyrus, dopamine depletion in the caudate nucleus, and more activation in the bilateral cerebellum crus 1 [52, 113]. Furthermore, slower mean rate in PD-MCI than PD cohorts has been shown to occur as a result of more damage in PD-MCI to regions involved in the performance of complex RT tasks, such as the orbitofrontal and anterior cingulate cortex, basal ganglia, and thalamus [114–117]. As the mean rate performance of the CU and PD cohorts was similar, a potential research direction relates to checking selective compensatory mechanisms in PD, one which has been demonstrated to be increased activity in the cerebellum and middle frontalgyrus [52, 118].

Research goal 2: Inconsistency comparisons in the AD and LBD spectra

Within the AD spectrum, inconsistency comparisons for both speed tasks revealed the robust finding that the AD cohort exhibited greater inconsistency than each of the MCI, SCI, and CU cohorts. Results indicate that within-spectrum inconsistency differences emerge primarily when there is dementia-specific neurocognitive impairment. Indeed, greater inconsistency in the AD cohort than the non-AD cohorts in the spectrum has been attributed to substantial AD-related white matter changes in the ventral and dorsolateral prefrontal cortex, superior frontal gyrus, and posterior cingulate [42, 129]. A recent finding also demonstrated that changes in grey matter, such as atrophy in the right frontal pole, left supramarginal gyrus, and precuneus can lead to greater inconsistency among persons with AD [99]. For the non-AD cohorts, our results support previous findings showing no differences in inconsistency between CU and SCI [130] or CU and MCI [38, 40] cohorts. A plausible future direction is to investigate inconsistency differences among CU, SCI, and multi-domain (e.g., amnestic and non-amnestic) MCI cohorts, as one study demonstrated that in the Go-No-Go and flanker attentional tasks, multi-domain MCI cohorts have significantly greater speed inconsistency than both single-domain MCI and CU cohorts [34]. In the COMPASS-ND study, participant criteria emphasized amnestic (single or multi) MCI.

Within the LBD spectrum, inconsistency comparisons for both speed tasks revealed the robust finding that the LBD cohort exhibited greater inconsistency than each of the PD-MCI, PD, and CU cohorts. Overall, this pattern highlights the cognitive and motor perturbations common in LBD-related pathology. Several reports indicate that greater inconsistency in diagnosed LBD-related dementia cohorts, as compared to the non-dementia cohorts in the spectrum, may be associated with significant reductions of grey matter volume in the striatum (caudate nucleus), subthalamic nucleus, left thalamus, right inferior frontal gyrus, as well as reduced integration in dorsal and ventral attention, visual, and cingulo-parietal networks [108, 131]. Regarding the non-LBD cohorts, more complex RT tasks may be required to detect inconsistency differences earlier in the LBD spectrum. Some studies have shown that PD cohorts are more inconsistent than CU cohorts when examined using either an eight-choice RT task (CRT-8) [47, 48] or a combination (adding mean-level scores) of multiple complex RT tasks [53]. In the case of the CRT-8 task, participants must attend to stimuli in eight (rather than four) black squares, a more attention-demanding task [10].

Research goal 3: Validation: Relative predictive potential of mean rate and inconsistency in the AD and LBD spectra

We performed validation analyses by using the ROC AUC approach to test the relative predictive potential of the two cognitive speed indicators for discriminating cohort membership within each spectrum. The discrimination patterns closely tracked the cohort difference results reported for the first two research goals. Most discriminations were observed for the within-spectrum pairwise comparisons of the two dementia cohorts. Within the AD spectrum, both mean rate and inconsistency discriminated between the three AD dementia-related comparisons (i.e., AD versus CU, SCI and MCI) for both tasks. Two additional discrimination findings were consistent only for the SRT task: mean rate and inconsistency for (a) CU and SCI and (b) CU and MCI. Within the LBD spectrum, both mean rate and inconsistency discriminated cohort membership in two dementia-related comparisons (LBD versus CU and PD), but only inconsistency discriminated LBD and PD-MCI for both tasks. One non-LBD related discrimination was consistent for both tasks: mean rate and inconsistency for CU and PD-MCI. For the two across-spectrum dementia cohorts (AD, LBD), inconsistency but not mean rate, discriminated them. For the two impairment cohorts (MCI, PD-MCI), both mean rate and inconsistency discriminated them. In general, by using each of the two cognitive speed indicators as individual predictors via the ROC approach, we demonstrated that mean rate and inconsistency do not confer unique prediction but are rather substantially similar in their sensitivity to discriminate cohorts in both spectra of neurodegenerative diseases.

Integrated summary: Comparing mean rate and inconsistency sensitivity patterns for the AD and LBD spectra

Because of the unique range of cohorts and the density of comparisons, we aim to contribute to the general discussion of whether mean rate or inconsistency are different in their sensitivity across multiple aging and dementia cohorts. We report four phases for these sensitivity pattern comparisons: (a) main cohorts within the AD and LBD spectra, (b) across two sets of parallel cohorts, (c) significant test and AUC results, and (d) SRT and CRT results. We base these comparisons on the summary results (see Supplementary Tables 4–9).

Within the AD spectrum, of the 12 significance tests computed for the comparisons of each indicator, (a) about half (7) were significant for mean rate and (b) half (6) were significant for inconsistency. Notably, substantial commonality in which comparisons were significant was observed for both indicators in the three dementia-related comparisons: AD versus CU, SCI, and MCI. Within the LBD spectrum, mean rate produced 9 of 12 significant comparisons whereas inconsistency produced 6. Notably, both indicators produced significant results for the three dementia-related comparisons: LBD versus CU, PD, and PD-MCI. Regarding the non-dementia comparisons, mean rate produced one significant result in the AD spectrum (i.e., CU versus MCI) and two in the LBD spectrum (i.e., CU versus PD-MCI and PD versus PD-MCI). Although these patterns indicate substantial similarity in sensitivity to cohort differences in the AD spectrum, mean rate was somewhat more discriminative in the LBD spectrum. For the cross-cohort comparisons, of the 4 significance tests computed for both mean rate and inconsistency comparisons, half were significant. Notably, the significant difference for each was in the different cross-cohort comparison.

Regarding the AUC results, within both spectra, the ranges were either acceptable (0.65 to 0.80) or excellent (>0.80) for mean rate and inconsistency. Similarly, the AUC results confirmed overall sensitivity of both indicators for the three dementia-related comparisons within each spectrum. Overall, these patterns indicate (a) relatively parallel significant test and AUC results and (b) similar discriminative potential for mean rate and inconsistency for cohorts in the AD and LBD spectra.

Finally, we included two task versions (SRT and CRT) in order to check whether they produced different patterns of group differences in the two performance indicators. A noted point of interest has been that a more complex speeded task could be associated with either (or both) slower and more inconsistent performance, perhaps especially in populations with impaired cognitive or fine motor deficits [10, 53]. Results indicated that both tasks produced similar results when comparing mean rate and inconsistency differences in the AD and LBD spectra (see the Supplementary Material for a summary).

Limitations and strengths

Several limitations should be noted. First, because we examined cross-sectional data, we were unable to examine whether mean rate and inconsistency differentially change over time in these cohorts. Second, the available data were unbalanced in terms of sample size across some cohorts. Although our sample size per cohort is relatively large compared to other recent studies [39, 53], the smaller sample size of some groups may have restricted the detection of significant findings. However, 7 of the 22 effect sizes reported were in the large range (0.14–0.32), with 12 being in the medium range (0.08–0.13). Notably, only 3 were in the small range (0.02–0.05). Therefore, although there were sample size constraints, the magnitude of the reported effects indicate that our results have interpretable significance. Third, the available COMPASS-ND data were also unbalanced in terms of sex representation. Therefore, we were not able to examine female and male differences in mean rate and inconsistency between cohorts. Future research should explicitly examine sex differences by, for example, disaggregation. Fourth, multiple comparisons were performed, thus increasing the probability of a type 1 error. However, we limited this potential issue by (a) setting statistical significance at p≤0.01, (b) reporting effect sizes to quantify the magnitude of the differences independent of significance testing, and (c) using the Bonferroni method for post-hoc analyses. Fifth, by study design, the two cognitive speed tasks were similar in several foundational characteristics (i.e., technical implementation, speeded fine motor responses, and multiple trials) but somewhat different in extent of required motor planning and action (i.e., CRT required slightly more than the SRT). For persons in the LBD spectrum, cognitive slowing (and perhaps inconsistency) deficits may be exacerbated as more complex movement is required for rapid, accurate and repeated performance. That cognitive speed is expressed in part through motor performance is an especially relevant consideration when examining speed in persons with motor impairment. An alternative approach to measuring cognitive speed—a rate-accuracy trade-off paradigm [136]—is designed to minimize potential motor confounds by measuring motor accuracy as a function of visual stimulus rate. This approach could be explored in future research, especially with cohorts experiencing motor deficits. However, the current approach has the advantage of using well-documented tasks that have been previously applied in aging, AD and LBD cohorts [10, 80–84].

Among strengths, we first note the well-characterized COMPASS-ND cohorts representing both the AD and LBD spectra. The comprehensive clinical diagnostic criteria resulted in robust and reliable classifications. Second, we examined cognitive speed mean rate and inconsistency within and (selectively) across the AD and LBD spectra, derived with the same procedures from the same two speed tasks. This thorough investigative approach was important to advance our knowledge of speed rate and inconsistency performance for simple and complex RT tasks in these prevalent dementia spectra. Third, we used robust data preparation methods to control for outliers and systematic confounds. Specifically, for mean rate, we implemented lower bound and upper bound limits for legitimate responses [47]. For inconsistency, we computed the residualized ISD, which is known to be a viable and valid quantification of RT inconsistency [86].

Conclusion

We assembled multiple cohorts from the COMPASS-ND database representing two prominent spectra of neurodegenerative disease (AD and LBD). In a single study, we compared them within and (selectively) across cohorts in two indicators of cognitive speed performance (mean rate and inconsistency) as measured with a common set of two tasks varying moderately in complexity (SRT and CRT). Results showed that mean rate and inconsistency have (a) substantial similarity in sensitivity to differences across the dementia-related comparisons (i.e., AD versus CU, SCI, MCI; LBD versus CU, PD, PD-MCI); (b) notable subtle differences in their sensitivity to conditions within the LBD spectrum (e.g., mean rate was more sensitive to differences in the non-LBD related comparisons), and (c) similar patterns of group differences across parallel impairment (MCI, PD-MCI) and dementia (AD, LBD) cohorts. In addition to the dementia-related comparisons, mean rate and inconsistency discriminated cohort membership between CU versus SCI, CU versus MCI, CU versus PD-MCI, and PD versus PD-MCI. This study contributes to the cognitive speed literature in aging and neurodegeneration by applying a comparative, systematic, and comprehensive approach to the investigation of mean rate and inconsistency differences within and across the two important spectra of neurodegenerative diseases. Both indicators as measured by simple and complex tasks may be sensitive to AD and LBD dementia risk.