Diagnostic and Prognostic Accuracy of the Cogstate Brief Battery and Auditory Verbal Learning Test in Preclinical Alzheimer’s Disease and Incident Mild Cognitive Impairment: Implications for Defining Subtle Objective Cognitive Impairment

Abstract

Background:

There are detectable cognitive differences in cognitively unimpaired (CU) individuals with preclinical Alzheimer’s disease (AD).

Objective:

To determine whether cross-sectional performance on the Cogstate Brief Battery (CBB) and Auditory Verbal Learning Test (AVLT) could identify 1) CU participants with preclinical AD defined by neuroimaging biomarkers of amyloid and tau, and 2) incident mild cognitive impairment (MCI)/dementia.

Method:

CU participants age 50+ were eligible if they had 1) amyloid (A) and tau (T) imaging within two years of their baseline CBB or 2) at least one follow-up visit. AUROC analyses assessed the ability of measures to differentiate groups. We explored the frequency of cross-sectional subtle objective cognitive impairment (sOBJ) defined as performance ≤–1 SD on CBB Learning/Working Memory Composite (Lrn/WM) or AVLT delayed recall using age-corrected normative data.

Results:

A+T+ (n = 33, mean age 79.5) and A+T– (n = 61, mean age 77.8) participants were older than A–T– participants (n = 146, mean age 66.3), and comparable on sex and education. Lrn/WM did not differentiate A + T+or A+T– from A–T– participants. AVLT differentiated both A+T+ and A+T– from A–T– participants; 45% of A+T+ and 25% of A+T– participants met sOBJ criteria. The follow-up cohort included 150 CU individuals who converted to MCI/dementia and 450 age, sex, and education matched controls. Lrn/WM and AVLT differentiated between stable and converter CU participants.

Conclusion:

Among CU participants, AVLT helped differentiate A+T+ and A+T– from A–T– participants. The CBB did not differentiate biomarker subgroups, but showed potential for predicting incident MCI/dementia. Results inform future definitions of sOBJ.

Keywords

Amyloid Cognigram conversion memory neuropsychology one back one card learning sensitivity and specificity subtle cognitive decline tau

INTRODUCTION

There is a genuine need to develop an efficient way to screen the population for elevated brain amyloid and tau in order to 1) identify appropriate individuals with preclinical Alzheimer’s disease (AD) for anti-amyloid and anti-tau clinical trials, and 2) identify those with AD pathology and thus at highest risk of developing AD dementia once anti-amyloid or anti-tau therapies are identified. Simple cognitive assessment methods show promise in this regard. First, there are detectable cognitive differences in amyloid positive versus negative cognitively unimpaired (CU) individuals, which are most robust for memory measures [1, 2]. Effect sizes are small in amyloid positive CU individuals (d =–0.17), but are larger in individuals with both amyloid and additional disease burden in the form of tau or neurodegeneration (d = –0.47) [1]. Second, subtle objective cognitive impairment (sOBJ), also referred to as subtle cognitive decline, shows promise for contributing to detection of preclinical AD. sOBJ based on a single baseline assessment confers risk of converting from CU to mild cognitive impairment (MCI)/AD at follow-up [3]. For example, Edmonds and colleagues showed that 46% of individuals with sOBJ alone, and 84% of individuals with sOBJ, neurodegeneration, and amyloidosis converted to MCI/AD dementia [3]. Further, individuals with cross-sectional sOBJ show faster accumulation of amyloid per PET neuroimaging, faster entorhinal cortical thinning, and greater risk of conversion to MCI/AD dementia over follow-up relative to CU participants without sOBJ [4].

Various definitions of sOBJ have been suggested based on traditional neuropsychological tests [3 , 5–7], but there is no clear agreement on the best definition or measures to use. Cross-sectional definitions have included a <10th percentile cut-off score on composite measures of memory [7] or global cognition [6]. Other approaches have combined cognitive and functional measures, requiring two out of six scores <1 SD below published normative data in two separate cognitive domains or a subtle decline in functional abilities based on a Functional Assessment Questionnaire score of 6–8 [3]. Intra-individual definitions of objective longitudinal cognitive decline (ΔOBJ) may confer greater sensitivity relative to cross-sectional approaches and represent an important method for defining transitional cognitive decline for individuals with preclinical AD [8]. However, whether intra-individual decline is superior compared to a cross-sectional approach remains a hypothesis to be tested and no clear a priori cutoffs have been recommended for how to define longitudinal decline at this time. Despite this ambiguity and prior evidence of the utility of cross-sectional sOBJ [3, 4], there is no cross-sectional option for defining Stage 2 transitional cognitive decline based on cognitive measures as recently proposed by the National Institute on Aging-Alzheimer’s Association (NIA-AA) Working Group [8]. Therefore, additional information about the frequency of cross-sectional sOBJ in preclinical AD will help address whether cross-sectional definitions of sOBJ should be considered as another way to define transitional cognitive decline and will inform development of a generalizable, widely accepted definition of sOBJ.

Computerized cognitive measures, such as the Cogstate Brief Battery (CBB), may be a more efficient method for identifying sOBJ relative to traditional, standardized person-administered measures [9, 10]. Further, computerized measures can be incorporated into study designs independent of diagnosis, which helps avoid the frequent circularity between measures used to define sOBJ and those used for current and future diagnosis. If shown to be reliable and valid, computerized measures that provide the option for unsupervised administration may be better suited for large-scale cognitive screening and monitoring than traditional neuropsychological measures or other computerized measures that require in-person administration. The CBB is one such tool that can be administered with or without supervision and consists of four cognitive tasks that measure psychomotor function, attention, working memory, and visual memory [11]. The CBB is already being used for at-home enrichment of clinical trial enrollment through the Brain Health Registry and has been implemented in the next phase of the Alzheimer’s Disease Neuroimaging Initiative (ADNI-3) [12 –14]. Early Cogstate Ltd.-affiliated studies suggest that the CBB may show promise for detecting early preclinical cognitive change; individuals identified as memory decliners based on longitudinal performance on the One Card Learning subtest from the CBB had a higher frequency of amyloid positivity in CU individuals relative to individuals with stable performance [15, 16]. Cogstate has FDA approval to market the CBB as a medical device under the name Cognigramtrademark for use as a digital cognitive assessment tool that can be completed in clinic or at home through prescription use in individuals 6–99 years of age. Although the CBB was originally developed to detect change over time, Cognigram is marketed for use on a single occasion or to determine cognitive change over time. One study showed that a single supervised administration of the Learning/Working Memory (Lrn/WM) Composite from the CBB showed good diagnostic accuracy for AD dementia and MCI [17], but this has not yet been replicated. It remains unclear whether a single CBB administration can help predict elevated brain amyloid or conversion to MCI/dementia in CU individuals.

The primary aim of this study was to help inform evolving definitions of sOBJ by determining whether baseline performance on a computerized measure, the CBB, can help identify CU participants with 1) preclinical AD defined by neuroimaging biomarkers of amyloid and tau, and 2) incident MCI/dementia. We predicted that the CBB Lrn/WM Composite and a traditional neuropsychological measure, Auditory Verbal Learning Test (AVLT) delayed recall, would show comparable 1) diagnostic accuracy and 2) prognostic accuracy based on AUROC analyses. We also applied a conventional cut-off of ≤ –1 SD below age-adjusted normative scores to the CBB and AVLT to investigate the frequency of sOBJ and to facilitate clinical translation of study results. We predicted that CU individuals meeting criteria for a biological diagnosis of AD (CU A+T+), but not those with Alzheimer pathologic change alone (CU A+T–), would show a higher frequency of sOBJ relative to CU A–T– individuals. We also predicted that individuals with incident MCI/dementia defined as CU individuals who later convert to MCI or dementia (CU converters) would show a higher frequency of sOBJ relative to CU stable participants at baseline.

METHODS

The Mayo Clinic Study of Aging (MCSA) is a prospective population-based study of cognitive aging among Olmsted County, MN, residents following an age- and sex-stratified random sampling design [18]. Individuals randomly chosen for recruitment were invited to participate in the MCSA and those without a medical contraindication were invited to participate in imaging studies. PiB-PET began in 2008 and tau-PET imaging began in 2016. The MCSA began administering Cogstate measures in the clinic in 2012 to the newly enrolled 50–69-year-olds during clinic visits. Cogstate administration for those aged 70 and older began the following year.

MCSA study visits included a neurologic evaluation by a physician, an interview by a study coordinator, and neuropsychological testing by a psychometrist [18]. The physician examination included a medical history review, complete neurological examination, and administration of the Short Test of Mental Status (STMS) [19]. The study coordinator interview included demographic information and medical history, and questions about memory to both the participant and informant using the Clinical Dementia Rating scale [20]. Details about the neuropsychological battery have been previously reported [18].

For each participant, performance in a cognitive domain was compared with age-adjusted scores of cognitively unimpaired (CU) individuals using Mayo’s Older Americans Normative Studies [21]. Participants with scores of ≥1.0 SD below the age-specific mean in the general population were considered for possible cognitive impairment, taking into account education, prior occupation, visual or hearing deficits, and other information. A diagnostic determination of CU, MCI, or dementia is made by a consensus agreement between the study coordinator, examining physician, and neuropsychologist using published criteria [18, 22]. The diagnosis at each study visit is conducted blind to prior clinical information, diagnosis, or knowledge of biomarkers. Performance on the neuropsychological battery is considered by the neuropsychologist for diagnostic recommendation, but not by the physician or study coordinator. Performance on Cogstate was not considered for diagnosis.

The study protocols were approved by the Mayo Clinic and Olmsted Medical Center Institutional Review Boards. All participants provided written informed consent.

Inclusion criteria

Two partially overlapping study samples from the MCSA were derived to address study hypotheses: 1) participants with biomarker data, and 2) all available participants, grouped by future diagnosis (see Table 1, which includes information about sample overlap).

Table 1

Study definitions

Consensus Diagnosis: Cognitively Unimpaired (CU)
Grouped by Future Diagnosis	Grouped by Biomarker Profile (AT)¹
CU Stable² (n = 450)	CU A– T– (n = 146)
CU Converter³ (n = 150)	CU A+T– (n = 61)
	CU A+T+ (n = 33)
Exploring the utility of possible definitions of cross-sectional
subtle objective cognitive impairment (sOBJ)
1. CBB Lrn/WM Composite⁴ ≤-1 SD
2. AVLT Delay⁵ ≤-1 SD

¹Biomarker profile defined by neuroimaging biomarkers of amyloid and tau within two years of baseline Cogstate. ²CU at baseline Cogstate and remained CU at all available follow-up visits. N = 51 participants with biomarker data were also represented in this group (n = 29 CU A– T– , n = 14 CU A+T– , n = 8 CU A+T+). ³CU at baseline Cogstate and converted to MCI/dementia at any follow-up visit. N = 18 participants with biomarker data were also represented in this group (n = 5 CU A– T– , n = 5 CU A+T– , n = 8 CU A+T+). ⁴Cogstate is independent of consensus diagnosis; ≤– 1 SD cut-off is equivalent to an age-corrected z-score of ≤– 1 derived from normative data provided by Cogstate. ⁵AVLT is considered as part of consensus diagnosis; ≤– 1 SD cut-off is equivalent to a Mayo’s Older Americans Normative Studies age-corrected scaled score of ≤7. Note. CU = Cognitively Unimpaired; A = amyloid; T = tau; sOBJ = subtle objective cognitive impairment; CBB = Cogstate Brief Battery; Lrn/WM = Learning/Working Memory; AVLT = Auditory Verbal Learning Test 30-Minute Delayed Recall.

Participants with biomarker data

Individuals were eligible if they were CU, aged 50 or older at the time of their baseline CBB session, and had amyloid PET (using Pittsburgh Compound B [PiB-PET]) and tau PET (AV1451) scans within two years of their baseline Cogstate evaluation (n = 240). After application of the biomarker cut-offs described below to eligible participants, biomarker subgroups included CU individuals with a biological definition of AD (n = 33 A+T+), CU individuals with AD pathologic change (n = 61 A+T–), and CU individuals with normal AD biomarkers (n = 146 A–T–).

Participants grouped by future diagnosis

For this analysis, MCSA participants aged 50+ who were CU at baseline CBB and had at least one MCSA follow-up visit were eligible for inclusion (N = 2,328). A cohort of 150 individuals with a diagnosis of MCI/dementia at any follow-up visit were identified and labeled as CU converters; the majority of these participants converted to MCI (n = 141). The remaining participants (n = 2,178) represented a pool of participants who remained CU at all available follow-up visits and were eligible for matching (CU stable). Each converter was matched to three stable CU participants on age (±5 years), sex and education (< = 12 years, 13–14, 15–16, 17+).

Biomarker methods

PiB-PET and tau-PET are acquired with a PET/CT operating in three-dimensional mode. The details of the acquisition, processing, and cut-off derivation have been previously published [23 –25]. Individuals are considered amyloid positive if they have an SUVR > 1.48 in a previously defined meta-ROI [24] and tau positive if they have an SUVR > 1.25 in a previously define meta-ROI including temporal lobe structures [24]. To simplify the study design and maintain a focus on individuals on the AD continuum, we grouped participants according to amyloid and tau (AT) imaging status, instead of using the full AT(N) classification scheme [26]. Unspecified or unavailable biomarker data is denoted*; we did not consider N status and thus use N* to denote this. We limited the current study to individuals meeting the biological definition of AD (A+T+N*) or with AD pathologic change (A+T–N*) per the recently proposed research framework for a biological diagnosis of AD [8]. We refer to individuals who are A–T– as biomarker negative, but in doing so likely include some individuals with non-AD pathologic change (A–T– N+). Because of our focus on individuals within the AD continuum, we did not include A–T+ individuals.

Cognitive measures

Cogstate Brief Battery (CBB)

Only baseline Cogstate performance is included in the current study. Cogstate version 7 (v7) was used, which provides personal computer (PC), iPad, and web-based administration options. Most participants completed their first Cogstate session in clinic on a PC or iPad (see Stricker, et al. [27] for details about Cogstate procedures). Less than 1% completed their baseline Cogstate session at home through web-based administration on a PC, thus the small differences in performance at home versus in clinic that we previously reported on measures of accuracy are unlikely to impact results presented [28]. The ability to reliably complete and adhere to the requirements of each task was determined by completion checks as previously described [27]. All data values with a failed completion flag were removed from subsequent analyses.

The CBB includes the following tasks: 1) Detection (DET): A simple reaction time paradigm that measures psychomotor function. 2) Identification (IDN): A choice reaction time paradigm that measures visual attention. 3) One Card Learning (OCL): A continuous visual recognition learning task that assesses visual recognition memory and attention. 4) One Back (ONB): A task that assesses working memory and attention. Detection and Identification previously showed limited diagnostic accuracy for MCI and were not included in analyses [17]. Accuracy data (OCL and ONB) were transformed using arcsine transformation to normalize the variables.

The CBB Lrn/WM composite was the primary measure of interest. It is derived using the average of age corrected z-scores for One Card Learning Accuracy and One Back Accuracy. Normative data were provided by Cogstate [29] in the form of means and SDs by age groups for arcsine transformed OCL accuracy and ONB accuracy scores. Secondary exploratory analyses also used only One Card Learning accuracy as an alternative primary outcome variable since measures of episodic memory are typically more sensitive to early decline in preclinical AD relative to measures of working memory [1, 2], and to allow direct comparison to raw-score analyses. One Card Learning z– scores and arcsine transformed raw scores were used in this analysis.

Auditory verbal learning test

The AVLT is a 15-item word list memory test that is part of the neuropsychological battery used at each MCSA study visit. Mayo’s Older Americans Normative Studies (MOANS) age-corrected scaled scores [21] for 30 min delayed recall was the primary measure of interest; these scaled scores have a mean of 10 and SD of 3. Because many participants were already enrolled in the MCSA and followed longitudinally at the time of the baseline Cogstate session that is the focus of this study, a portion of participants had prior exposure to the AVLT (see Results). The normative data applied [21] does not take prior test exposure into account, which likely decreases the sensitivity of this measure as used in the current study, particularly in the subgroups with greater frequency of prior exposure.

Subtle Objective Cognitive Impairment (sOBJ)

sOBJ was defined using a conventional cut-off of performance of less than or equal to 1 standard deviation below the mean (≤–1 SD) based on age-corrected normative scores. This translates to an age-corrected z-score of≤–1 for Cogstate measures [29] and an age-corrected MOANS scaled score of ≤7 for AVLT 30 min delayed recall [21].

Statistical methods

Demographic and clinical differences between groups were assessed by linear model ANOVA test for means and Chi-squared test for frequencies. Effect size (Hedge’s g) was computed using a weighted and pooled standard deviation. AUROC analyses were conducted to assess the ability of measures to differentiate two groups. The Youden index method was used to identify cut points [30]. This method defines the optimal cut-point as the point maximizing the Youden function, which is the difference between the true positive rate (TPR) and false positive rate (FPR) over all possible cut-point values. We also directly tested biomarker group-wise discrimination as summarized by the AUROC for select comparisons [31]. Frequency of sOBJ using a conventional cut-off was compared across groups.

For participants with biomarker data, we also used a multivariate logistic regression model to predict group membership (A–T– versus A+T+, A–T– versus A+T–) and to compute odds ratios (OR) and 95% confidence intervals (CI). Multivariate logistic regression models were adjusted for age, sex, and years of education and included a single cognitive measure of interest as the predictor. This allowed us to compute a demographics-adjusted OR for the Lrn/WM composite and AVLT delayed recall. In order to better compare to Lrn/WM, AVLT delayed recall was rescaled to z-score units. For every 1.0 z-score less, the odds of being in the biomarker positive group increases by (OR-1)%.

Secondary analyses were performed with select raw scores to ensure results were not driven by use of age-corrected scores. Secondary analyses also included the STMS to facilitate direct comparison of the CBB results to a public domain, traditional person-administered screening measure.

RESULTS

CU participants with biomarker data

Demographics and group mean comparisons

A+T+ and A+T– participants were older than A–T– participants and were comparable on education and sex (see Table 2). Performance on Lrn/WM across A+T– and A–T– groups was comparable (p = 0.27, Hedge’s g = 0.17). Lrn/WM performance in the A+T+ group was marginally lower (p = 0.07) than in the A–T– group, with a small to medium effect size (Hedge’s g = 0.35). Both of the biomarker positive groups (i.e., A+T+ and A+T–) showed significantly lower AVLT delayed recall performance (ps < 0.01) relative to the A–T– group, with medium effect sizes for both the A+T+ (Hedge’s g = 0.60) and A+T– (Hedge’s g = 0.54) comparisons. Of note, 40.6% of participants with biomarker data had prior exposure to the AVLT overall, with increasing frequency of prior exposure in the biomarker positive groups (see Table 2). Although direct comparison of the A+T+ and A+T– groups was not part of our a priori planned hypotheses, we did include some comparisons of these groups for the interested reader (see Table 2). Mean comparisons of the A+T+ and A+T– groups showed comparable performances on Lrn/WM (Hedge’s g = 0.11) and AVLT delayed recall (Hedge’s g = 0.08; both ps > 0.05).

Table 2

Demographic characteristics and mean performance on memory measures among cognitively unimpaired (CU) participants

	CU A–T–	CU A+T+	p CU A–T–	CU A+T–	p CU A–T–	p CU A+T–
	M (SD)	M (SD)	versus	M (SD)	versus	versus
	(n = 146)	(n = 33)	A+T+	(n = 61)	A+T–	A+T+
Age Mean (SD)	66.32 (12.06)	79.52 (6.27)	<0.001	77.78 (9.41)	<0.001	0.341
Education Mean (SD)	15.24 (2.44)	14.79 (3.08)	0.362	14.77 (2.81)	0.229	0.978
Sex (% Male)	78 (53.4%)	20 (60.6%)	0.454	37 (60.7%)	0.340	0.996
Short Test of Mental Status¹	36.18 (1.99)	35.69 (2.12)	0.214	35.08 (2.31)	<0.001	0.220
AVLT Delay MOANS SS¹	10.58 (2.92)	8.82 (3.05)	0.002	9.05 (2.58)	<0.001	0.699
Prior AVLT exposure (%)	26.7%	72.7%	<0.001	56.7%	<0.001	0.126
CBB Learning/Working Memory	0.22 (0.78)	–0.07 (0.98)	0.072	0.06 (1.27)	0.274	0.617
Composite z-score
CBB Learning/Working Memory Composite			0.008		0.083	0.369
Normal z> –1	138 (95.2%)	27 (81.8%)		54 (88.5%)
sOBJ z≤–1	7 (4.8%)	6 (18.2%)		7 (11.5%)
AVLT 30 minute delay¹			<0.001		0.016	0.043
Normal SS > 7	129 (88.4%)	18 (54.5%)		45 (75.0%)
sOBJ SS≤7	17 (11.6%)	15 (45.5%)		15 (25.0%)

p-values represent linear model ANOVAs for mean comparisons or Pearson’s Chi-squared test for frequency comparisons of amyloid positive groups with the biomarker negative group. A, amyloid; T, tau; CBB, Cogstate Brief Battery; AVLT, Auditory Verbal Learning Test 30-Minute Delayed Recall; MOANS, Mayo’s Older Americans Normative Studies; SS, scaled score;- sOBJ, objective subtle cognitive impairment. ¹Performance is considered as part of consensus diagnosis.

sOBJ frequency

For Lrn/WM, the A+T+ group showed a significantly higher frequency of sOBJ relative to the A–T– group (see Table 2). Despite this, sensitivity was very low, with 18% of A+T+ participants meeting sOBJ criteria on Lrn/WM. Only 5% of CU A–T– participants met this cut-off, thus specificity of Lrn/WM was excellent (95%). A+T– and A–T– groups showed a comparable frequency of sOBJ on Lrn/WM. For AVLT delayed recall, both of the biomarker positive groups showed a significantly higher frequency of sOBJ relative to the A–T– group. AVLT sensitivity was 45% for the A+T+ group, 25% for the A+T– group, and only 12% of A–T– participants met the sOBJ cut-off (88% specificity).

Diagnostic accuracy

Overall diagnostic accuracy of Lrn/WM for differentiating both A+T+ and A+T– from A–T– participants was very low and not better than chance (see Table 3). Diagnostic accuracy for the AVLT was low, but AVLT delayed recall discriminated both biomarker positive (A+T+ and A+T–) groups from the A–T– group better than chance (CIs did not include 0.50). Comparing total AUC values for both measures, AVLT delayed recall differentiated A+T– and A–T– groups significantly better than Lrn/WM Composite (p = 0.03). However, both measures comparably differentiated A+T+ and A–T– groups (p = 0.40).

Table 3

Diagnostic accuracy of memory measures based on derived, optimal cut-off scores and conventional cut-off scores (≤–1 SD) indicative of subtle objective cognitive impairment (sOBJ)

Measure	Group comparison	AUC (95% CI)	Threshold	Sensitivity (95% CI)	Specificity (95% CI)
CBB Lrn/WM composite z-score	A–T– versus A+T– among CU	0.51 (0.42,0.60)	–0.23 z	0.38 (0.26,0.51)	0.75 (0.68,0.82)
			Conventional	0.11 (0.05,0.20)	0.95 (0.92,0.99)
	A–T– versus A+T+ among CU	0.59 (0.47,0.72)	–0.06 z	0.55 (0.36,0.73)	0.68 (0.61,0.76)
			Conventional	0.18 (0.06,0.33)	0.95 (0.91,0.99)
	CU stable versus converter	0.70 (0.65,0.74)	–0.04	0.77 (0.70,0.83)	0.58 (0.53, 0.62)
			Conventional	0.25 (0.18,0.32)	0.88 (0.85,0.91)
AVLT delay MOANS Scaled Score¹	A–T– versus A+T– among CU	0.64 (0.57,0.72)	11.5 SS	0.83 (0.73,0.92)	0.42 (0.34,0.50)
			Conventional	0.25 (0.15,0.37)	0.88 (0.83,0.93)
	A–T– versus A+T+ among CU	0.66 (0.56, 0.77)	7.5 SS	0.45 (0.27, 0.64)	0.88 (0.83, 0.93)
			Conventional	0.45 (0.27, 0.61)	0.88 (0.83, 0.93)
	CU stable versus converter	0.70 (0.65, 0.75)	9.5 SS	0.70 (0.63, 0.77)	0.61 (0.56, 0.65)
			Conventional	0.38 (0.31, 0.46)	0.86 (0.83, 0.89)

A, amyloid; T, tau; CU, cognitively unimpaired; CBB, Cogstate Brief Battery; Lrn/WM, Learning/Working Memory; AVLT, Auditory Verbal Learning Test 30-Minute Delayed Recall; MOANS, Mayo’s Older Americans Normative Studies. ¹Performance is considered as part of consensus diagnosis.

Logistic regression analyses controlling for age, sex, and education confirmed that these diagnostic accuracy results cannot be fully explained by age differences across groups (see Table 4). AVLT delayed recall significantly separates A+T– and A–T– groups, above and beyond age, education and sex. For differentiation of A+T+ and A–T– groups, the pattern of results remained consistent with frequency and diagnostic accuracy analyses for AVLT delayed recall, but the p-value was at trend level (p = 0.08). Lrn/WM did not significantly separate groups, confirming the above analyses.

Table 4

Odds ratios from logistic regression models after adjusting for age, education, and sex

Groups compared	Term	OR (95% CI)	p
CU A–T– versus CU A+T+	CBB Lrn/WMcomposite z 1 unit less	1.25 (0.78, 2.05)	0.35
CU A–T– versus CU A+T–	CBB Lrn/WM composite z 1 unit less	1.03 (0.74, 1.46)	0.85
CU A–T– versus CU A+T+	AVLT Delay z 1 unit less	1.54 (0.96, 2.53)	0.08
CU A–T– versus CU A+T–	AVLT Delay z 1 unit less	1.58 (1.07, 2.39)	0.03

For logistic regression analyses, AVLT delayed recall scaled score was rescaled to z-score units to aid interpretation. For every 1.0 z-score less, the odds of being in the biomarker positive group increases by the OR reported above. A, amyloid; T, tau; CU, Cognitively Unimpaired; CBB, Cogstate Brief Battery; AVLT, Auditory Verbal Learning Test 30-Minute Delayed Recall. AVLT performance is considered as part of consensus diagnosis.

To investigate whether the pattern of results would change if raw scores were used and to ensure that the sensitivity of Lrn/WM was not artificially decreased by including a measure of working memory in the composite, we completed secondary analyses using One Card Learning accuracy raw scores and z-scores, as well as AVLT delayed recall raw scores (see Fig. 1 and Table 5). Total AUC values were similar between the Lrn/WM Composite and OCL z-score, between OCL z-score and raw score, and between AVLT delayed recall scaled score and raw score. In secondary analyses, the STMS did not differentiate A–T– and A+T+ groups better than chance. Overall diagnostic accuracy of the STMS for differentiating A+T– from A–T– participants was low, but better than chance (CI did not include 0.50). Sensitivity and specificity were low to poor for the optimally derived cut-off (A+T– versus A–T–), suggesting limited clinical utility.

Fig.1

AVLT 30-minute delayed recall sensitivity/specificity plot for differentiating CU A–T– and CU A+T+ groups. Panel A shows age-corrected scaled scores and panel B shows raw scores. The dotted line indicates the derived optimal cut-off per Youden’s index, which jointly maximizes the sum of sensitivity and specificity.

Table 5

Diagnostic accuracy of secondary measures

Description	Threshold (Optimal)	Sensitivity (95% CI)	Specificity (95% CI)	AUC (95% CI)
OCL accuracy (z-score)
A–T– versus A+T– among CU	0.18	0.56 (0.44, 0.69)	0.58 (0.50, 0.66)	0.53 (0.44, 0.62)
A–T– versus A+T+ among CU	–0.21	0.45 (0.27, 0.64)	0.77 (0.70, 0.83)	0.60 (0.49, 0.71)
CU stable versus converter	–0.13	0.63 (0.55, 0.70)	0.67 (0.63, 0.72)	0.68 (0.63, 0.72)
OCL accuracy (Arcsine transformed raw score)
A–T– versus A+T– among CU	0.66	0.48 (0.34, 0.61)	0.68 (0.61, 0.76)	0.55 (0.46, 0.64)
A–T– versus A+T+ among CU	0.66	0.55 (0.39, 0.70)	0.68 (0.60, 0.75)	0.62 (0.51, 0.73)
CU stable vs converter	0.64	0.60 (0.51, 0.68)	0.70 (0.65, 0.74)	0.67 (0.62, 0.72)
AVLT delay (raw score)¹
A–T– versus A+T– among CU	9.50	0.83 (0.73, 0.92)	0.42 (0.34, 0.49)	0.64 (0.57, 0.72)
A–T– versus A+T+ among CU	4.50	0.45 (0.30, 0.64)	0.88 (0.83, 0.93)	0.67 (0.56, 0.78)
CU stable versus converter	7.50	0.70 (0.63, 0.78)	0.61 (0.56, 0.65)	0.70 (0.65, 0.75)
STMS (raw score)¹
A–T– versus A+T– among CU	36.5	0.67(0.56,0.79)	0.58(0.50,0.66)	0.67(0.59,0.74)
A–T– versus A+T+ among CU	35.5	0.41(0.25,0.56)	0.71(0.64,0.78)	0.57(0.46,0.68)
CU stable versus converter	34.5	0.62(0.55,0.70)	0.75(0.71,0.79)	0.74(0.69,0.78)

A, amyloid; T, tau; OCL, One Card Learning; AVLT, Auditory Verbal Learning Test 30-Minute Delayed Recall; STMS, Short Test of Mental Status. ¹Performance is considered as part of consensus diagnosis.

CU participants with follow-up data

Demographics and group mean comparisons

Converter and stable CU groups were comparable on age, education, sex, and years of follow-up available (see Table 6). Time to conversion was 2.7 years after baseline Cogstate, on average. The CU converter group showed lower performance relative to the stable CU group (p’s<0.001) on Lrn/WM (Hedge’s g = 0.59) and AVLT delayed recall (Hedge’s g = 0.72). Across CU participants with follow-up data, 57.0% of participants had prior AVLT exposure overall, with comparable frequency of prior exposure across stable and converter groups.

Table 6

Demographic characteristics and mean performance on memory measures among CU stable and converter participants

	CU Stable M (SD) (n = 450)	CU Converter M (SD) (n = 150)	p
Age Mean (SD)	75.437 (10.345)	75.502 (10.493)	0.947
Education Mean (SD)	14.171 (2.528)	13.940 (2.499)	0.331
Sex (% Male)	240 (53.3%)	80 (53.3%)	1.00
Time to conversion (y)	–	2.723 (1.517)
Total follow up (y)	3.664 (1.691)	3.848 (1.801)	0.257
Short Test of Mental Status¹	35.563 (1.980)	33.624 (2.431)	<0.001
AVLT 30 MOANS SS¹	10.282 (2.771)	8.245 (2.918)	<0.001
Prior AVLT exposure (%)	56.5%	58.5%	0.676
Learning/Working Memory Composite (z-score)	0.014 (0.974)	–0.552 (0.873)	<0.001
Learning/Working Memory Composite			<0.001
Normal z> –1	396 (88.4%)	113 (75.3%)
sOBJ z≤–1	52 (11.6%)	37 (24.7%)
AVLT 30 minute delay¹			<0.001
Normal SS > 7	382 (86.0%)	91 (61.9%)
sOBJ SS≤7	62 (14.0%)	56 (38.1%)

p-values represent linear model ANOVAs for mean comparisons or Pearson’s Chi-squared test for frequency comparisons. MOANS, Mayo’s Older Americans Normative Studies; SS, scaled score;- sOBJ, objective subtle cognitive impairment. ¹Performance is considered as part of consensus diagnosis.

sOBJ frequency

The CU converter group showed a significantly higher frequency of sOBJ on both Lrn/WM and AVLT relative to the CU stable group (see Table 6). Using a conventional ≤–1 SD cut-off, Lrn/WM showed 25% sensitivity for predicting conversion. Only 12% of CU stable participants met the sOBJ cut-off for Lrn/WM, thus specificity was good (88%). AVLT delayed recall had slightly higher sensitivity (38%) for predicting conversion. Only 14% of CU stable participants met the sOBJ cut-off for AVLT delayed recall, thus specificity was good (86%).

Prognostic accuracy for predicting incident MCI/dementia

Prognostic accuracy of baseline memory performance for differentiating CU stable and CU converter participants was moderate based on total AUC and better than chance (total AUC CIs did not include 0.5) for both Lrn/WM and AVLT (see Table 3).

DISCUSSION

The main finding of this study is that sOBJ, defined based on a single administration of a traditional neuropsychological memory test (AVLT), may help identify CU individuals with 1) preclinical AD as defined by neuroimaging biomarkers of amyloid and tau, and 2) incident MCI/dementia. In accordance with the primary study aim to determine whether baseline CBB performance can help identify CU participants with preclinical AD, a single CBB assessment was not useful in predicting elevated brain amyloid either alone or in combination with tau (A+T– or A+T+). However, CBB Lrn/WM Composite significantly contributed to predicting conversion from CU at baseline to MCI/dementia over follow-up.

Our finding that a single baseline memory test performance could identify 45% of CU individuals with a biological diagnosis of AD based on neuroimaging biomarkers of amyloid and tau (A+T+) and 25% with AD pathologic change alone (A+T–) was somewhat unexpected. Prior studies have shown that individuals with AD pathologic change (A+) but unknown T status do not show impairment in verbal memory when assessed on a single occasion [32 –34]. Consistent with our hypothesis, we demonstrated that CU individuals with a biological diagnosis of AD based on positive amyloid and tau biomarkers (A+T+) showed a higher frequency of sOBJ relative to CU A–T– individuals on AVLT delayed recall. However, despite our prediction to the contrary, CU individuals with AD pathologic change alone (A+T–) also showed a higher frequency of sOBJ on AVLT delayed recall. Prior reports have largely shown small effect sizes in studies comparing CU individuals with and without positive amyloid status, with meta-analytic results ranging from –0.14 [2] to –0.17 [1]. Based on a smaller pool of 7 studies using the 2011 preclinical AD framework [35], a medium effect size (–0.47) was demonstrated for Stage 2 (A+ and evidence of either positive neurodegeneration or tau) versus Stage 0 [1]. Our findings suggest a medium effect size on AVLT delayed recall for both CU individuals with AD (A+T+) and AD pathologic change (A+T–) relative to CU individuals without AD biomarkers (A–T–).

Inconsistent with our hypothesis, the total AUC was significantly greater for AVLT delayed recall than for Lrn/WM Composite for differentiating A+T– and A–T– CU groups. Statistical comparison of the total AUC suggested that both AVLT delayed recall and the Lrn/WM Composite comparably differentiated A+T+ and A–T– groups, However, overall diagnostic accuracy of the Lrn/WM Composite was not better than chance and only 18% of the A+T+ group met the sOBJ criterion, suggesting very limited clinical utility. Thus, when considering single assessment results, the AVLT shows greater clinical utility than the CBB for predicting biomarker positive status. Word list memory tests typically have greater sensitivity to MCI and AD dementia relative to other memory testing paradigms, particularly recognition memory [36]. Thus, this difference in clinical utility may simply reflect the type of memory measure used (word list delayed recall versus visual recognition memory) and does not necessarily imply a difference between traditional and computerized cognitive testing. Importantly, the CBB was designed with a focus on sensitivity to detecting change versus sensitivity for use at a single time point. However, given the CBB’s burgeoning use as a cognitive screening measure for clinical trials and for patient care, a better understanding of its clinical utility based on one time point is important. Further, although the CBB has shown some promise in identifying risk of elevated brain amyloid based on memory decline over time on OCL accuracy [15], the initial data presented in this regard suffered from some circularity in study design and there has not been a direct comparison of the added benefit of longitudinal CBB assessment relative to baseline CBB or baseline traditional memory measures. We plan to directly compare the sensitivity of cross-sectional and longitudinal approaches to defining sOBJ and subtle objective cognitive decline in future work once additional follow-up data have been collected. Relatively few studies have used true longitudinal change over time to define objective subtle cognitive decline [15 , 37] even though intra-individual decline is expected to provide a more sensitive means for detecting preclinical cognitive changes.

Although this study presents data regarding the diagnostic and prognostic accuracy of memory measures, memory measures are non-specific with regards to etiology. Low memory performance can be due to a number of factors including numerous other biological etiologies, demographic variables, and transient situational factors. Clinical diagnosis remains the gold standard for describing clinical syndromes. However, the need for data-driven and efficient approaches to identifying risk of preclinical AD based on cognitive measures or other methods is paramount to ongoing clinical trial efforts, and for identifying individuals who will most benefit from treatment once a viable treatment is identified. Predictive algorithms also show some promise in helping to predict the likelihood of elevated brain amyloid. However, in addition to data from in-person cognitive assessments that are typically included in these algorithms, additional information such as genetic testing (i.e., APOE genotype) and/or knowledge of other medical conditions [38, 39] is also often required, and thus may not be as simple and efficient as cognitive-based methods alone.

Our results showed that AVLT delayed recall alone could help predict, better than chance, which CU individuals had elevated brain amyloid. Although the total AUC values of 66% for A+T+ and 64% for A+T– are “low” per traditional standards, these results are promising for discriminating among CU individuals. For example, a recent study reported an AUC of 66% for plasma Aβ₄₂ alone and 68% for plasma Aβ₄₂/Aβ₄₀ ratio for discriminating subjects with normal versus abnormal amyloid PET scans in CU individuals with subjective cognitive decline [40]. In addition, the sensitivity of the AVLT in our study is likely artificially lowered in two ways. First, a portion of the sample had prior exposure to the AVLT, thus unaccounted for practice effects likely led to slightly improved performance for the assessment used for these participants [41]. Also, AVLT performance was considered for diagnosis in a way that would decrease the sensitivity of this measure, as AVLT performance was considered during consensus diagnosis conference and participants were still given a CU diagnosis. Within ADNI, Kandel et al. [42] also demonstrated that AVLT performance has been shown to help predict CSF Aβ_1–42 among patients with MCI, with predictive utility comparable to neuroimaging biomarkers (FDG-PET and hippocampal volume).

A secondary aim of this study was to determine whether performance on the CBB Lrn/WM Composite at baseline could predict conversion from CU to MCI/dementia. As a reference, we also examined AVLT delayed recall performance. Despite the circularity resulting from the AVLT being considered in the diagnosis of MCI, the results were encouraging as both measures achieved moderate overall diagnostic accuracy, with total AUC of 70%. Other cognitive screening measures have similar predictive accuracy values. Both the Montreal Cognitive Assessment (MoCA) and STMS performed similarly for detecting incident MCI with AUCs of 0.70 and 0.71, respectively [43]. Our secondary analyses confirm that the STMS performed similarly within this sample, with a total AUC of 0.74. However, STMS performance is considered as part of the consensus diagnosis. Rizk-Jackson and colleagues [44] found that MRI volumes and FDG PET measures alone showed 65% overall accuracy to predict conversion to MCI within two years, although a model combining MRI and FDG-PET measures achieved 81% total accuracy. Our results are particularly notable given that prior research in this area demonstrated similar AUC values when employing numerous clinical variables. Prediction of incident MCI in CU individuals from the NACC database using 14 non-invasive, clinical variables including performance on various neuropsychological measures resulted in 75% total AUC [45]. Despite our encouraging AUC results for predicting incident MCI/dementia with a single administration of the CBB, derived optimal cut-offs for the CBB fall well within the range of normal performance, complicating meaningful clinical application.

Although this study provides a unique look at the frequency of sOBJ in CU individuals with a biological diagnosis of AD, AD pathologic change, and incident MCI/dementia, there are also important limitations. Most notably, biomarker positive groups were significantly older than biomarker negative groups. Although age-corrected normative scores help account for the substantial age differences across groups, this may have had unintended consequences. For example, biomarker status is known to be significantly associated with age, which precluded use of an age-matched sample for biomarker groups. We aimed to further address this important confound by performing logistic regression equations that controlled for age, education, and sex, and the pattern of results did not change. Also of note, although we report analyses for 3 : 1 age, sex, and education-matched groups for identification of CU stable versus CU converter individuals, unreported preliminary results without this matching procedure showed equivalent findings. Finally, because the current study’s primary focus was on the CBB, a sizeable portion of AVLT performances were derived from individuals who had already been exposed to the AVLT. Application of normative data based on baseline performance therefore likely reduced the sensitivity of AVLT results by failing to take into account the known influence of practice effects [46, 47]. Of note, this may be particularly relevant for the biomarker subgroups, as both the A+T+ and A+T– groups were more likely to have had prior exposure to the AVLT relative to the A–T– group. Prior work by our group showed that among CU individuals, biomarker subgroups may be differentially sensitive to practice effects, with A–N– and A+N– subgroups both showing continued practice effects on memory measures across multiple visits, whereas A+N+ and A–N+ showed relatively stable performances over time [48]. Future work considering the important role of practice effects when optimizing definitions of sOBJ and ΔOBJ is needed.

Despite these limitations, this study contributes to the rapidly growing body of research focused on how to best define sOBJ in preclinical AD and fosters further research in the area. For instance, results suggest that normative data with better sensitivity to preclinical AD are needed. Alternative cut-offs may also need to be considered. For example, a –0.5 SD cut-off may be appropriate for sOBJ [49, 50]. Knopman and colleagues [51] showed that individuals with sOBJ based on a –0.5 SD cutoff have an increased risk of incident MCI/dementia. Use of norms that adjust for sex and education, in addition to age, is recommended for future studies but was not an option for the independent normative data used in this study from the MOANS and Cogstate [21, 29]. Intra-individual definitions of objective subtle cognitive decline may also enhance the sensitivity of cognitive measures to preclinical AD [37] and represents an important future direction.

In summary, results suggest that a single CBB session may show some utility in predicting conversion to MCI/dementia over follow-up, but is unlikely to be helpful in predicting which CU individuals are more likely to have AD pathologic change or AD based on AD neuroimaging biomarkers of amyloid and tau. In contrast, a traditional word list memory test shows promise in this regard, particularly if normative data can be refined to enhance sensitivity.

Footnotes

ACKNOWLEDGMENTS

The authors wish to thank the participants and staff at the Mayo Clinic Study of Aging. This work was supported by the Rochester Epidemiology Project (R01 AG034676), the National Institutes of Health (grant numbers P50 AG016574, P30 AG062677, U01 AG006786, R37 AG011378, R01 AG041851, RF1 AG55151), a grant from the Alzheimer’s Association (AARG-17-531322), Zenith Award from the Alzheimer’s Association, the Robert Wood Johnson Foundation, The Elsie and Marvin Dekelboum Family Foundation, Alexander Family Alzheimer’s Disease Research Professorship of the Mayo Clinic, Liston Award, Schuler Foundation, GHR Foundation, AVID Radiopharmaceuticals, and the Mayo Foundation for Education and Research. We would like to greatly thank AVID Radiopharmaceuticals, Inc., for their support in supplying AV-1451 precursor, chemistry production advice and oversight, and FDA regulatory cross-filing permission and documentation needed for this work. NHS and MMMi serve as consultants to Biogen and Lundbeck. DSK serves on a Data Safety Monitoring Board for the DIAN-TU study and is an investigator in clinical trials sponsored by Lilly Pharmaceuticals, Biogen, and the University of Southern California. RCP has served as a consultant for Hoffman-La Roche Inc., Merck Inc., Genentech Inc., Biogen Inc., Eisai, Inc., and GE Healthcare.

Authors’ disclosures available online ().

References

Duke Han

, Nguyen

, Stricker

, Nation

(2017) Detectable neuropsychological differences in early preclinical Alzheimer’s disease: A meta-analysis. Neuropsychol Rev 27, 305–325.

Baker

, Lim

, Pietrzak

, Hassenstab

, Snyder

, Masters

, Maruff

(2017) Cognitive impairment and decline in cognitively normal older adults with high amyloid-beta: A meta-analysis. Alzheimers Dement (Amst) 6, 108–121.

Edmonds

, Delano-Wood

, Galasko

, Salmon

, Bondi

(2015) Subtle cognitive decline and biomarker staging in preclinical Alzheimer’s disease. J Alzheimers Dis 47, 231–242.

Thomas

, Bangen

, Weigand

, Edmonds

, Wong

, Cooper

, Delano-Wood

, Bondi

, Alzheimer’s Disease Neuroimaging Initiative (2020) Objective subtle cognitive difficulties predict future amyloid accumulation and neurodegeneration. Neurology 94, e397–e406.

Hassenstab

, Ruvolo

, Jasielec

, Xiong

, Grant

, Morris

(2015) Absence of practice effects in preclinical Alzheimer’s disease. Neuropsychology 29, 940–948.

Jack

, Knopman

, Weigand

, Wiste

, Vemuri

, Lowe

, Kantarci

, Gunter

, Senjem

, Ivnik

, Roberts

, Rocca

, Boeve

, Petersen

(2012) An operational approach to National Institute on Aging–Alzheimer’s Association criteria for preclinical Alzheimer disease. Ann Neurol 71, 765–775.

Vos

SJB

, Xiong

, Visser

, Jasielec

, Hassenstab

, Grant

, Cairns

, Morris

, Holtzman

, Fagan

(2013) Preclinical Alzheimer’s disease and its outcome: A longitudinal cohort study. Lancet Neurol 12, 957–965.

Jack

Jr. , Bennett

, Blennow

, Carrillo

, Dunn

, Haeberlein

, Holtzman

, Jagust

, Jessen

, Karlawish

, Liu

, Molinuevo

, Montine

, Phelps

, Rankin

, Rowe

, Scheltens

, Siemers

, Snyder

, Sperling

(2018) NIA-AA Research Framework: Toward a biological definition of Alzheimer’s disease. Alzheimers Dement 14, 535–562.

Bauer

, Iverson

, Cernich

, Binder

, Ruff

, Naugle

(2012) Computerized neuropsychological assessment devices: Joint position paper of the American Academy of Clinical Neuropsychology and the National Academy of Neuropsychology. Arch Clin Neuropsychol 27, 362–373.

10.

Zygouris

, Tsolaki

(2015) Computerized cognitive testing for older adults: A review. Am J Alzheimers Dis Other Demen 30, 13–28.

11.

Maruff

, Thomas

, Cysique

, Brew

, Collie

, Snyder

, Pietrzak

(2009) Validity of the CogState brief battery: Relationship to standardized tests and sensitivity to cognitive impairment in mild traumatic brain injury, schizophrenia, and AIDS dementia complex. Arch Clin Neuropsychol 24, 165–178.

12.

Mackin

, Insel

, Truran

, Finley

, Flenniken

, Nosheny

, Ulbright

, Comacho

, Bickford

, Harel

, Maruff

, Weiner

(2018) Unsupervised online neuropsychological test performance for individuals with mild cognitive impairment and dementia: Results from the Brain Health Registry. Alzheimers Dement (Amst) 10, 573–582.

13.

Weiner

, Nosheny

, Camacho

, Truran-Sacrey

, Mackin

, Flenniken

, Ulbricht

, Insel

, Finley

, Fockler

, Veitch

(2018) The Brain Health Registry: An internet-based platform for recruitment, assessment, and longitudinal monitoring of participants for neuroscience studies. Alzheimers Dement 14, 1063–1076.

14.

Weiner

, Veitch

, Aisen

, Beckett

, Cairns

, Green

, Harvey

, Jack

, Jagust

, Morris

, Petersen

, Salazar

, Saykin

, Shaw

, Toga

, Trojanowski

(2017) The Alzheimer’s Disease Neuroimaging Initiative 3: Continued innovation for clinical trial improvement. Alzheimers Dement 13, 561–571.

15.

Darby

, Brodtmann

, Pietrzak

, Fredrickson

, Woodward

, Villemagne

, Fredrickson

, Maruff

, Rowe

(2011) Episodic memory decline predicts cortical amyloid status in community-dwelling older adults. J Alzheimers Dis 27, 627–637.

16.

Darby

, Pietrzak

, Fredrickson

, Woodward

, Moore

, Fredrickson

, Sach

, Maruff

(2012) Intraindividual cognitive decline using a brief computerized cognitive screening test. Alzheimers Dement 8, 95–104.

17.

Maruff

, Lim

, Darby

, Ellis

, Pietrzak

, Snyder

, Bush

, Szoeke

, Schembri

, Ames

, Masters

(2013) Clinical utility of the cogstate brief battery in identifying cognitive impairment in mild cognitive impairment and Alzheimer’s disease. BMC Psychology 1, 30.

18.

Roberts

, Geda

, Knopman

, Cha

, Pankratz

, Boeve

, Ivnik

, Tangalos

, Petersen

, Rocca

(2008) The Mayo Clinic Study of Aging: Design and sampling, participation, baseline measures and sample characteristics. Neuroepidemiology 30, 58–69.

19.

Kokmen

, Smith

, Petersen

, Tangalos

, Ivnik

(1991) The short test of mental status: Correlations with standardized psychometric testing. Arch Neurol 48, 725–728.

20.

Morris

(1993) The Clinical Dementia Rating (CDR): Current version and scoring rules. Neurology 43, 2412–2414.

21.

Ivnik

, Malec

, Smith

, Tangalos

, Petersen

, Kokmen

, Kurland

(1992) Mayo’s Older Americans Normative Studies: WAIS-R, WMS-R and AVLT norms for ages 56 through 97. Clin Neuropsychol 6, 1–104.

22.

Petersen

, Roberts

, Knopman

, Geda

, Cha

, Pankratz

, Boeve

, Tangalos

, Ivnik

, Rocca

(2010) Prevalence of mild cognitive impairment is higher in men: The Mayo Clinic Study of Aging. Neurology 75, 889–897.

23.

Jack

Jr. , Lowe

, Senjem

, Weigand

, Kemp

, Shiung

, Knopman

, Boeve

, Klunk

, Mathis

, Petersen

(2008) 11C PiB and structural MRI provide complementary information in imaging of Alzheimer’s disease and amnestic mild cognitive impairment. Brain 131, 665–680.

24.

Jack

Jr. , Wiste

, Weigand

, Therneau

, Lowe

, Knopman

, Gunter

, Senjem

, Jones

, Kantarci

, Machulda

, Mielke

, Roberts

, Vemuri

, Reyes

, Petersen

(2017) Defining imaging biomarker cut points for brain aging and Alzheimer’s disease. Alzheimers Dement 13, 205–216.

25.

Vemuri

, Lowe

, Knopman

, Senjem

, Kemp

, Schwarz

, Przybelski

, Machulda

, Petersen

, Jack

Jr. (2017) Tau-PET uptake: Regional variation in average SUVR and impact of amyloid deposition. Alzheimers Dement (Amst) 6, 21–30.

26.

Jack

Jr. , Bennett

, Blennow

, Carrillo

, Feldman

, Frisoni

, Hampel

, Jagust

, Johnson

, Knopman

, Petersen

, Scheltens

, Sperling

, Dubois

(2016) A/T/N: An unbiased descriptive classification scheme for Alzheimer disease biomarkers. Neurology 87, 539–547.

27.

Stricker

, Lundt

, Edwards

, Machulda

, Kremers

, Roberts

, Knopman

, Petersen

, Mielke

(2019) Comparison of PC and iPad administrations of the Cogstate Brief Battery in the Mayo Clinic Study of Aging: Assessing cross-modality equivalence of computerized neuropsychological tests. Clin Neuropsychol 33, 1102–1126.

28.

Stricker

, Lundt

, Alden

, Albertson

, Machulda

, Kremers

, Knopman

, Petersen

, Mielke

(2020) Longitudinal comparison of in clinic and at home administration of the cogstate brief battery and demonstrated practice effects in the Mayo Clinic Study of Aging. J Prev Alzheimers Dis 7, 21–28.

29.

Cogstate (2018) Cogstate Pediatric and Adult Normative Data, Cogstate, Inc, New Haven, CT.

30.

Perkins

, Schisterman

(2005) The Youden Index and the optimal cut-point corrected for measurement error. Biom J 47, 428–441.

31.

DeLong

, DeLong

, Clarke-Pearson

(1988) Comparing the areas under two or more correlated receiver operating characteristic curves: A nonparametric approach. Biometrics 44, 837–845.

32.

Lim

, Ellis

, Harrington

, Kamer

, Pietrzak

, Bush

, Darby

, Martins

, Masters

, Rowe

, Savage

, Szoeke

, Villemagne

, Ames

, Maruff

(2013) Cognitive consequences of high Aβ amyloid in mild cognitive impairment and healthy older adults: Implications for early detection of Alzheimer’s disease. Neuropsychology 27, 322–332.

33.

Baker

, Lim

, Jaeger

, Ames

, Lautenschlager

, Robertson

, Pietrzak

, Snyder

, Villemagne

, Rowe

, Masters

, Maruff

(2018) Episodic memory and learning dysfunction over an 18-month period in preclinical and prodromal Alzheimer’s disease. J Alzheimers Dis 65, 977–988.

34.

Insel

, Donohue

, Mackin

, Aisen

, Hansson

, Weiner

, Mattsson

, Alzheimer’s Disease Neuroimaging Initiative (2016) Cognitive and functional changes associated with Abeta pathology and the progression to mild cognitive impairment. Neurobiol Aging 48, 172–181.

35.

Sperling

, Aisen

, Beckett

, Bennett

, Craft

, Fagan

, Iwatsubo

, Jack

Jr. , Kaye

, Montine

, Park

, Reiman

, Rowe

, Siemers

, Stern

, Yaffe

, Carrillo

, Thies

, Morrison-Bogorad

, Wagster

, Phelps

(2011) Toward defining the preclinical stages of Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 7, 280–292.

36.

Weissberger

, Strong

, Stefanidis

, Summers

, Bondi

, Stricker

(2017) Diagnostic accuracy of memory measures in Alzheimer’s dementia and mild cognitive impairment: A systematic review and meta-analysis. Neuropsychol Rev 27, 354–388.

37.

Nation

, Ho

, Dutt

, Han

, Lai

MHC

(2019) Neuropsychological decline improves prediction of dementia beyond Alzheimer’s disease biomarker and mild cognitive impairment diagnoses. J Alzheimers Dis 69, 1171–1182.

38.

Lee

, Byun

, Yi

, Sohn

, Jeon

, Lee

, Kim

, Lee

(2018) Prediction of cerebral amyloid with common information obtained from memory clinic practice. Front Aging Neurosci 10, 309.

39.

Carr

SJA

, Jaeger

, Bian

, He

, Maserejian

, Wang

, Maruff

, Enayetallah

, Wang

, Chen

, Lerner

, Tatsuoka

, Australian Imaging Biomarkers and Lifestyle Flagship Study of Ageing (2019) Associating cognition with amyloid status using partially ordered set analysis. Front Neurol 10, 976.

40.

Verberk

IMW

, Slot

, Verfaillie

SCJ

, Heijst

, Prins

, van Berckel

BNM

, Scheltens

, Teunissen

, van der Flier

(2018) Plasma amyloid as prescreener for the earliest Alzheimer pathological changes. Ann Neurol 84, 648–658.

41.

Knopman

, Caselli

(2012) Appraisal of cognition in preclinical Alzheimer’s disease: A conceptual review. Neurodegener Dis Manag 2, 183–195.

42.

Kandel

, Avants

, Gee

, Arnold

, Wolk

, Alzheimer’s Disease Neuroimaging Initiative (2015) Neuropsychological testing predicts cerebrospinal fluid amyloid-β in mild cognitive impairment. J Alzheimers Dis 46, 901–912.

43.

Townley

, Syrjanen

, Botha

, Kremers

, Aakre

, Fields

, Machulda

, Graff-Radford

, Savica

, Jones

, Knopman

, Petersen

, Boeve

(2019) Comparison of the short test of mental status and the Montreal Cognitive Assessment across the cognitive spectrum. Mayo Clin Proc 94, 1516–1523.

44.

Rizk-Jackson

, Insel

, Petersen

, Aisen

, Jack

, Weiner

(2013) Early indications of future cognitive decline: Stable versus declining controls. PLoS One 8, e74062.

45.

Lin

, Gong

, Yang

, Ye

, Albin

, Dodge

(2018) Big data analytical approaches to the NACC dataset: Aiding preclinical trial enrichment. Alzheimer Dis Assoc Disord 32, 18–27.

46.

Machulda

, Pankratz

, Christianson

, Ivnik

, Mielke

, Roberts

, Knopman

, Boeve

, Petersen

(2013) Practice effects and longitudinal cognitive change in normal aging vs. incident mild cognitive impairment and dementia in the Mayo Clinic Study of Aging. Clin Neuropsychol 27, 1247–1264.

47.

Elman

, Jak

, Panizzon

, Tu

, Chen

, Reynolds

, Gustavson

, Franz

, Hatton

, Jacobson

, Toomey

, McKenzie

, Xian

, Lyons

, Kremen

(2018) Underdiagnosis of mild cognitive impairment: A consequence of ignoring practice effects. Alzheimers Dement (Amst) 10, 372–381.

48.

Machulda

, Hagen

, Wiste

, Mielke

, Knopman

, Roberts

, Vemuri

, Lowe

, Jack

Jr. , Petersen

(2017) Practice effects and longitudinal cognitive change in clinically normal older adults differ by Alzheimer imaging biomarker status. Clin Neuropsychol 31, 99–117.

49.

Buckley

, Sparks

, Papp

, Dekhtyar

, Martin

, Burnham

, Sperling

, Rentz

(2017) Computerized cognitive testing for use in clinical trials: A comparison of the NIH Toolbox and Cogstate C3 batteries. J Prev Alzheimers Dis 4, 3–11.

50.

Machulda

, Lundt

, Albertson

, Kremers

, Mielke

, Knopman

, Bondi

, Petersen

(2019) Neuropsychological subtypes of incident mild cognitive impairment in the Mayo Clinic Study of Aging. Alzheimers Dement 15, 878–887.

51.

Knopman

, Beiser

, Machulda

, Fields

, Roberts

, Pankratz

, Aakre

, Cha

, Rocca

, Mielke

(2015) Spectrum of cognition short of dementia: Framingham heart study and Mayo Clinic study of aging. Neurology 85, 1712–1721.