ADSS: A Composite Score to Detect Disease Progression in Alzheimer’s Disease

Data sets

Two data sets were used to develop and validate new composite scores to detect clinical progression in early stages of AD. First, in the development of the new ADSS, we included data of MCI patients from the ADNI database (downloaded on the date of November 29, 2022) [16, 17]. The total number of MCI patients was 1096 with n = 396 from ADNI-1, n = 129 from ADNI-Go, n = 342 from ADNI-2, and n = 229 from ADNI-3. Among them, there were over 800 participants having visits at the 6-month follow-up, 788 participants having year 1 visit, and 676 participants completed year 2 visit. Among 676 participants with year 2 follow-up, 624 participants had complete sub-scales for the outcomes of our interest, and they are MCI patients in building the model. Out of these 624 MCI patients, 173 patients progressed to dementia due to AD at the year 2 follow-up with the estimated conversion rate of 27.7%.

Second, in validating the ADSS, we utilized data of the MCI participants from the Alzheimer’s Disease Cooperative Study (ADCS) to evaluate the efficacy of vitamin E and donepezil for the treatment of MCI in a randomized placebo-controlled three-arm study. That study enrolled a total of 769 MCI patients.

Baseline characteristics of these two cohorts are presented in Table 1. The proportion of female in the ADCS study is slightly higher than that in the ADNI MCI cohort. Other than that, these two studies have similar study populations.

Models for longitudinal data

Wang et al. [3] developed a statistical model to detect clinical decline with the outcome as the time from baseline. Let t be the time at the follow-up and t₀ be the time at baseline. For a study with K outcome measures (e.g., CDR-SB, ADAS-cog, their sub-scales), their proposed model is presented as t - t 0 = ∑ i = 1 k w i Δ x i ( t ) ,(1) where ΔX _i (t) is the score change of the i-th cognitive score at time t from baseline (t₀), and w _i is its associated weight. The model was derived by using the linear item of the score change from the Taylor series [3]. In the ADNI data used to construct the ADSS, we used the exam date of the MMSE test to calculate the month t and t₀. The scheduled visit often does not occur exactly as scheduled. In the ADNI MCI patients, the average time for the 1-year follow-up was estimated as 13.73 months with the standard deviation of 1.5 months. For that reason, we used the exact months from baseline in this model to calculate t−  t₀ [18–21]. For example, one participant has the visit time of t = 13.2 months. Then, ΔX₁(t = 13.2) is the difference of CDR-SB (the first cognitive test) at t = 13.2 and CDR-SB at baseline.

Outcome measures

We included the commonly used clinical trial total score measures: (I) CDR-SB, (II) the 13-item ADAS-cog, and (III) MMSE. Their sub-scales were included in the model. The CDR-SB has 6 domains: 1) memory, 2) orientation, 3) judgment and problem solving, 4) community, 5) home and hobbies, and 6) personal care. The ADAS-cog-13 include 13 domain areas: word recall, commands, constructional Praxis, delayed recall, naming, ideational praxis, orientation, word recognition, recall instructions, spoken language, word finding, comprehension, and number cancellation. The MMSE has 7 domain areas: orientation to time, orientation to place, language, attention and calculation, registration, recall, and constructional praxis [3, 22].

We also investigated the additional prediction power gain by adding other measures, such as Functional Activities Questionnaire (FAQ) and Neuropsychiatric Inventory (NPI) [23]. The FAQ includes 10 domain areas: manage finances, complete forms, shop, perform games of skill or hobbies, prepare hot beverages, prepare a balanced meal, follow current events, attend to television programs, books or magazines, remember appointments, and travel out of the neighborhood. Each domain has the score from 0 to 5, with the maximum total score of 50. The FAQ scores were used in predicting the disease progression [24]. NPI consists of 12 sub-scales to assess neuro-psychiatric symptoms [23].

Partial least square regression and model selection

The statistical model in Equation (1) was used in the PLS regression to derive the ADSS using the ADNI MCI data. PLS regression is used to find independent components from the existing variables. The first component was used as the composite score in this research. This component represents the eigenvector (e.g., the weights) for the largest eigenvalue of the covariance matrix between time (t − t₀) and cognitive scores. It is possible that the weight for each cognitive score may be negative although a positive correlation coefficient between time and each cognitive score is assumed. The backward variable selection approach was utilized to remove cognitive measures whose parameter estimates are negative in the fitted statistical model. The following steps were used to derive the final composite score based on the PLS and the backward variable selection approach.

Step 1: Select the clinical scores that have a positive correlation with time t−t₀. These X _i s are the initial set of the outcome measures which is a subset of {X _i :  = 1, 2, …, K}.

This can be viewed as an iterative approach as Step 3 could be used multiple times. The backward selection approach was used in removing cognitive outcome measures with negative parameter estimates or low VIP values.

To avoid including similar orientation sub-scales from the three cognitive measures, we run this model selection approach three times by including the orientation sub-scale from one of the three measures in Step 1.

Comparison with other measures

We first compared the sensitivity of the developed ADSS with the existing measures by using the mean to standard deviation ratio (MSDR). We calculated the bias-corrected MSDR by using 10,000 bootstrap samples. The existing cognitive scores include three widely used total score outcome measures (CDR-SB, ADAS-Cog, and MMSE), and the composite score ADCOMS [3]). A larger MSDR value represents a larger effect size, which leads to greater sensitivity. In addition to the MCI population, we also compared the MSDR between the new composite scores and the existing scores in some enriched population (e.g., Apolipoprotein E ɛ4 (APOE4) carries and Amyloid-β positive patients assessed with positron emission tomography (PET)).

We then compared the sample sizes required for a parallel randomization study by using the ADNI MCI patients as the control group. The sample size for a randomized study was calculated by assuming a 25% slowing of cognitive decline as the observed benefit in a new treatment group as compared to the control group. In addition to the traditional subtraction method for sample size calculation, we considered the approach based on analysis of covariance (ANCOVA) where the baseline measure is the only covariate in the sample size calculation [15]. After analysis of the ADNI MCI data, we used the SMC cohort from the ADNI to validate the newly derived ADSS.

In a two-arm randomized trial, suppose μ _{G 0} and μ _{G 1} are the mean of the outcome measure at baseline and that at the follow-up for the group G, G = c for the control group and G = t for the treatment group. Then, μ _t  = μ _{t 1} − μ _{t 0} (color blue in Fig. 1) and μ _c  = μ _{c 1} − μ _{c 0} (color red in Fig. 1) are the change of the outcome at the follow-up from baseline in the treatment group and the control group, respectively. Their difference δ  = μ _c − μ _t is the parameter of interest (green line in Fig. 1). The statistical hypothesis is:

OPEN IN VIEWER

Fig. 1

A randomized two-arm study. Higher scores indicate worse cognitive performance.

H 0 : δ = 0 VS H a : δ ≠ 0 .

The traditional subtraction method calculates the sample size by using the estimated value of δ ˆ and its variance estimate [9, 15]. Suppose S 0 2 and S 1 2 are the variance estimate of outcome at baseline and that at the follow-up. Then, the variance estimate of the score change ( S d 2 ) is calculated as S d 2 = S 0 2 + S 1 2 - 2 ρ S 0 S 1 , where ρ is the correlation coefficient of outcome at baseline and outcome at the follow-up. The traditional approach is the one based on the two-sample z test [25, 26]. Suppose δ _d is the clinically meaningful improvement of the treatment as compared to the control group (e.g., 25% benefit in the treatment as compared to the control group with μ _c  - μ _t  = 0.25μ _c ). The required sample size per arm for a two-arm study based on the subtraction method is n s = 2 ( z α / 2 + z β ) s d 2 δ d 2(2) where z _c is the 100(1−c)th percentile of the standard normal distribution. The subtraction method is easy to implement with a closed formula, but it has several limitations including the assumption of population homogeneity and consequent under- or over-estimate of sample sizes [13]. When the baseline outcome measure affects the disease progression, the subtraction method may lose statistical power substantially [27]. When the baseline cognitive measure is considered as a covariate in the ANCOVA sample size calculation, the required sample size per arm is presented as [14, 15] n c = ( n s + 1 ) ( 1 - ρ 2 ) ,(3) where n _s is the sample size using the subtraction method in Equation (2), and one additional participant is added to maintain the statistical power level [14]. The sample size n _c goes down as the correlation coefficient increases. In addition to n _c , another exact approach based on the ratio of mean squares could be utilized, but the exact approach does not have a closed formula. For a study with a large sample size, their results are very close to each other [15].

Validation

The developed ADSS was validated by using MCI patients’ data from the ADCS [28]. We identified 536 MCI participants having complete CDR-SB, ADAS-cog, MMSE, and their sub-scales at 18 months. At the 18-month follow-up visit, the donepezil treatment group (n = 168) had the lowest MCI-to-AD conversion rate of 14% as compared to the AD onset rate of 24% and 22% in the vitamin E arm (n = 184) and the placebo arm (n = 184), respectively. When combining data from all the three arms, the new ADSS has the highest MSDR of 0.405, followed by CDR-SB (0.383), MMSE (0.270), and ADAS-cog (0.231). In this example, the ADCOMS has a slightly higher MSDR than the ADSS. Within each arm, the new ADSS and the CDR-SB often have larger MSDR values than others. The ADSS has similar MSDR values as the CDR-SB in the donepezil arm and the vitamin E arm, but the ADSS has 15% higher MSDR than CDR-SB in the placebo arm.

We calculated the predictive validity of the ADSS by using the 624 MCI patients from the ADNI [29]. They can be separated into two subgroups: 173 patients progressed to dementia due to AD at the year 2 follow-up visit, and the remaining 451 patients who remained MCI status. In these 2 years, MCI patients who remained stable in that 2 years had very slight decline of the ADSS (mean change of 0.35 with SD of 1.41), while the ADSS was increased by 3.56 (SD = 2.07) in the 173 patients progressed to dementia due to AD. These results indicate high predictive validity of the proposed ADSS.