In-Out-Test : A New Paradigm for Sorting the Wheat from the Chaff in Prodromal Alzheimer’s Disease

Abstract

Background:

Assessment of hippocampal amnesia is helpful to distinguish between normal cognition and mild cognitive impairment (MCI), but not for identifying converters to dementia. Here biomarkers are useful but novel neuropsychological approaches are needed in their absence. The In-out-test assesses episodic memory using a new paradigm hypothesized to avoid reliance on executive function, which may compensate for damaged memory networks.

Objective:

To assess the validity of the In-out-test in identifying prodromal Alzheimer’s disease (PAD) in a clinical setting, by comparing this to the Free and Cued Selective Reminding Test (FCSRT) and cerebrospinal fluid biomarkers.

Methods:

A cross-sectional study of 32 cognitively healthy, 32 MCI, and 30 progressive dementia subjects. All participants were given both the In-out-test and the FCSRT; 40 of them also received a lumbar puncture.

Results:

Internal consistency was demonstrated using Cronbach Alpha (r = 0.81) and Inter-rater reliability with Kappa (k = 0.94). Intraclass correlation (ICC) for test-retest reliability: r = 0.57 (p = 0.57). ICC between the In-out-test and FCSRT r = 0.87 (p = 0.001). ICC between the In-out-test and Aβ₄₂ and P-tau/Aβ₄₂ for controls: 0.73 and 0.75, respectively; P-tau for MCI: 0.77 and total sample: 0.70; Aβ₄₂ for dementia: 0.71. All ICC measures between FCSRT and biomarkers were ≤0.264. AD diagnosis: In-out-test k = 0.71; FCSRT k = 0.49. PAD diagnosis (N = 35): In-out-test k = 0.69; FCSRT k = 0.44.

Conclusions:

The In-out-test detected prodromal AD with a higher degree of accuracy than a conventional hippocampal-based memory test. These results suggest that this new paradigm could be of value in clinical settings, predicting which patients with MCI will go on to develop AD.

Keywords

Alzheimer’s disease amyloid biomarkers dementia early diagnosis episodic memory mild cognitive impairment neuropsychological tests tau proteins

INTRODUCTION

Mild cognitive impairment (MCI) is defined as a decline in cognitive functioning that is greater than would be expected for a patient’s age and educational background and that goes beyond normal changes seen in aging [1]. Various cognitive domains can be affected including learning and memory, attention, executive function, language, and visuospatial skills. MCI is viewed as a transitional state between normal aging and dementia; the annual progression rate to dementia is 10% in clinical settings and 5% in community settings [2]. Conversely, reversion rates of 23% and 10% are seen from population and clinical studies, respectively, implying this is a dynamic diagnosis.

There is some evidence to suggest a nutritional intervention may be beneficial in this early stage of disease [3]. Furthermore, with other disease-modifying therapies likely to become available in the next years, it is important to be able to identify which patients with MCI will go on to develop Alzheimer’s disease (AD). Currently cerebrospinal fluid (CSF) biomarkers are relied upon, but patients may be unwilling to undergo a lumbar puncture or facilities for analyzing samples may not be available. Thus, there is a great need for a neuropsychological assessment that could detect such patients with accuracy.

Characteristic of AD and prodromal AD (MCI due to AD) is amnestic syndrome of hippocampal type (HA) which is a recall deficit that does not significantly improve with cueing or recognition procedures, after effective encoding of information [4]. Other neuropsychological deficits and subtle changes in the activities of daily living (ADL). However, it is widely accepted that AD is characterized by the presence of HA.

The incorporation of biomarkers has been instrumental in the early diagnosis of AD posing a challenge to the Neuropsychologist to design sensitive and accurate tests to predict whether a patient consulting with memory complaints will go on to develop AD and subsequent dementia [5]. Many investigations have looked at how neuroimaging, changes in the CSF biomarkers, and serum and blood markers may predict the risk of conversion from MCI to dementia. Thus, cerebral atrophy, hypometabolism, and amyloid burden have been suggested as neuroimaging hallmarks, and decreased levels of amyloid-β₄₂ (Aβ₄₂) in CSF, in parallel with changes in the ratio total tau (t-tau) to Aβ₄₂ and phosphorylated tau protein (p-tau) to Aβ₄₂ as biochemical indicators. However, these values are not sufficiently accurate, and there is still high controversy regarding the criteria to establish in terms of CSF biochemistry, the onset and longitudinal changes of CSF biomarkers [6 –10].

Obviously, further characterization and deeper studies are required. For instance, it has been thought for a long time that changes in CSF biomarkers precede the symptoms of MCI [11]. Thus, it seems that classical symptoms-based diagnostic criteria of AD lack of sufficient sensitivity for detecting “hiding clinical signs” at prodromal and preclinical stages of the disease. In addition, current data show a wide variation on the cut-off of the levels of Aβ₄₂, t-tau, and p-tau making it difficult to determine the reliability of these biomarkers.

Recently, the complexity of standard clinical diagnostic tests [12 –14] have been increased in the attempt to increase sensitivity and specificity in the early diagnosis of AD. In line with this, the results of the SOMI (Stages of Objective Memory Impairment) are promising, by showing the ability of the Free and Cued Selective Reminding Test (FCSRT) to retrospectively describe the temporal unfolding of declining episodic verbal memory in predementia and AD [15].

Other paradigms have been proposed to evaluate episodic memory. Rentz et al. designed an associative encoding task that comprised faces paired with fictional first names. The authors describe a pattern on functional MRI activation during the encoding which is differentially altered in the early stages of AD compared with normal aging. Recently, Rentz et al. found that performance on face-name retrieval was associated with Aβ burden in cognitively normal older individuals [16]. These results open a promising line of investigation in individuals with normal cognition. However, the studies involved only individuals with higher education and this can limit the external validity of the results. In addition, as AD progresses, tests that are sensitive at the preclinical stage may be challenging to the subject at the MCI stage. For this reason, shorter versions of these tests have been developed by others [17].

Different mechanisms might interfere with setting the threshold for detecting amnesia and other cognitive deficits by conventional neuropsychological tests in the early stages of AD. Those mechanisms show paradoxical features for degenerative processes, such as hypertrophy and hyperactivity, which may reflect compensated cognitive performances. In degenerative processes, the loss of both neurons and neural connections is preceded by different aspects, including nuclear and body cell hypertrophy, higher amount of dendrites, and increased dendritic size and length in both the hippocampus and neocortex due to the action of neurotrophins binding to receptor p75 [18]. Paradoxical changes are also found in functional neuroimaging studies [5] and Aβ burden measured by PET [7]. In patients with MCI, who later converted to dementia, there was an increased neuronal activity in both, functional MRI [19 –25] and magneto-encephalography in areas other than the medial temporal lobe (MTL), particularly in frontal and parietal lobes [26 –28]. The increased activity might start in the hippocampus of the same hemisphere in very early stages of MCI, when these structures have only partial lesions [10 , 30]. A recent meta-analysis study has reported that both patients with AD and those at significant risk showed statistically significant consistent activation differences in the MTL relative to controls during the episodic memory [31]. Furthermore, it has been observed that cerebral hyperactivation acts as a compensatory mechanism activated in response to neuronal damage at the onset of AD. In this order of ideas, a longitudinal study comparing individuals carrying the apolipoprotein E4 isoforms gene (APOE ɛ4) to non-carriers, demonstrated that the subjects carrying the APOE ɛ4 gene had higher cerebral activity that decreased within the following five years, a phenomenon that was accompanied by medial temporal atrophy and decreased cognition [32].

The fact that most available tests of memory and executive function, taken individually, do not accurately distinguish patients with prodromal AD from patients with subjective cognitive impairment (SCI) or non-AD MCI, supports the idea that early MTL lesions may be largely overcome by other brain areas in an exceptional manner. Indeed, these other brain areas may play a significant role in non-mnemonic cognitive functions, independent of MTL activities in healthy people. Taking into account these observations, it could be suggested that networks involved in executive functions may take the command of memory during prodromal AD so consequently, dual tasks might be good candidate tests to distinguish converters from non-converters. With this in mind, it was hypothesized that memory deficits underlying prodromal AD might be unveiled if simultaneous executive tasks are used to engage such neuronal networks which support memory encoding in MTL. Despite many investigations of memory performance in a dual task context, to the best of our knowledge, no reliable strategy has been successfully established so far. Most attempts have included immediate and working memory in dual tasks [33] with the absence of significant differences between MCI and healthy elderly people when performance is adjusted to individual abilities [34]. Considering that short- and long-term episodic memory are more relevant than immediate and working memory in AD, both for diagnosing dementia and predicting conversion to dementia, a new paradigm has been developed. The In-out-test was designed to assess episodic memory along with a simultaneous executive task thereby interfering encoding by non-mnemonic cognitive functions [35]. In contrast to tests that evaluate memory after effective encoding of information (FSCRT), using the HA paradigm, the In-out-test makes an inference at the moment of encoding through simultaneous performance of an executive task.

A pilot study was conducted investigating 110 subjects using successive assessments over three years. Conversion to dementia based on clinical signs, impaired ADLs, and Mini-Mental State Examination (MMSE) score were measured. Conversion to dementia from MCI or SCI was predicted using the In-out-test with a sensitivity and specificity of 0.90, and 0.94, respectively, and with a positive predictive value (PPV) of 0.90 and a negative predictive value (NPV) of 0.94 [35].

The objective of the present study is to evaluate the validity of the In-out-test in identifying which patients with MCI have prodromal AD and will likely go to develop a dementia. To test this, the study compared the outcomes of the In-out-test with the outcomes of the FCSRT and also evaluated the degree of correlation between the In-out-test and the FCSRT with the CSF-biomarkers of AD.

METHODS

Setting and participants

The participants were consecutively recruited from the ongoing Prodromal Alzheimer Project (PROA) at Hospital Universitario de Gran Canaria Dr. Negrin (HUGCD).

The inclusion criteria were consecutive outpatients at the Neurology service, older than 59 years, able to read and write, consulting for memory complaints. Healthy controls were recruited from the community and the orthopedic surgery scheduled for a CSF study.

The exclusion criteria were suspected focal or diffuse brain damage due to clinical conditions different to AD; uncontrolled systemic diseases or delirium in the last 30 days; history of drug addiction or alcoholism, being under treatment for AD; history of major depression or being under treatment with two or more antidepressants; more than one dose per day of benzodiazepines; severe perceptive or motor disorders.

One hundred participants were recruited consecutively from the Dementia Unit. Six participants dropped out: two were diagnosed with frontal dementia, four did not complete the protocol. The remaining 94 participants, aged 60 to 92 years, had formal education from 0 to 20 years. Out of them, 32 showed normal psychometric performance (NPsych). The latter group was formed by 16 cognitively healthy participants without cognitive complaints (CH), and 16 with SCI who had no evidence of either cognitive impairment on the neuropsychological tests, nor impact on ADL.

32 participants were classified as MCI (24 amnestic and 8 non-amnestic), and 30 were mild to moderate progressive dementia. All subjects underwent clinical and neuropsychological assessment and 40 had lumbar puncture. The Medical Ethics Committee of our center rated the study, which was undertaken in accordance with the Helsinki Declaration of 1975, as revised in 1983. All subjects gave written informed consent to participate. The consent of dementia and MCI patients was also signed by a close relative.

Criteria of the National Institute of Neurological and Communicative Disorders and Stroke and Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA), and the Diagnostic and Statistical Manual of Mental Disorders (DSM IV-TR) were applied for probable AD diagnosis, taking into account clinical information, objective measures derived from the neuropsychological results and ADL, as measured by the Blessed Dementia Rating Scale (BDRS) and Instrumental Activities for Daily Living (IADL) (BDRS part A >1.5 and IADL <6 for women and <5 for men) [36 –38]. The criteria from the report of the MCI working group of the European Consortium on Alzheimer’s Disease (EADC) were applied for MCI [39]. Although SCI can occur in both apparently healthy persons and specific conditions (MCI, dementia, depression among others) [40], in this study the term SCI is used for patients who had cognitive complaints but normal performance in a full neuropsychological evaluation without impact to their ADL.

All patients with dementia had a Global Deterioration Scale (GDS) score = 4. Cognitive cut-off score definitions took into account the age and education of the participants, and were considered impaired if they were more than 1.5 standard deviations below the mean, according to the Spanish data from NORMACODEM and NEURONORMA Studies [38 , 41–44].

Neuropsychological assessment

A neuropsychological battery made following the recommendations of the Development of screening guidelines and criteria for predementia Alzheimer’s disease study (DESCRIPA) [45] was used. Other tests were included to further investigate some particular areas of interest, such as memory and executive functions. The battery included an assessment of the following cognitive domains: memory, language, praxis, visual perception, and executive function. MMSE was used as a global test of cognition [46, 47]. Episodic verbal memory was assessed by means of the word list Spanish version of the FCSRT [43 , 49], and the “Test Episódico” which is a Spanish test of ecological memory [50]. Visual memory, constructive praxis, and visuoperceptual function were assessed through the Rey-Osterrieth Complex Figure [43, 51]. A short-15 words version of The Boston Naming Test [52], semantic fluency with the “animals” category [53], and Token test [54] were used to assess language. Executive functions were assessed using the Color and Word Stroop test [55], the subtests of the WAIS battery inverse Digit Span and the Digit-Symbol [56], and subtests of de Scopa-cog scale Figure Completion and squares [57]. Short-term memory was evaluated by means of the Direct Digit Span from the WAIS battery [56]. GDS [58] was used to classify the severity of the global deterioration and functional state by means of the BDRS and IADL [37]. Anxiety and depression symptoms were assessed by the Hospital Anxiety and Depression Scale [59].

In-out-test

This is a new paradigm that assesses episodic memory after the patient performs a categorization task and memorizes 6 words simultaneously. The time taken to complete this test was 5 to 12 minutes. Three parts compose the In-out-test: Learning of the Organized Series (LOS), Memory of the Organized Series (MOS), and Random Memory (RAM). A participant performing the In-out-test views an image of six rows each made of two squares. One square of the row contains a drawing of a house and the other is black (Fig. 1). Every time the evaluator says the name of an object the participant repeats the name while pointing to either the house or the black square to indicate the usual location of the object, inside or outside the home. The task is repeated for each object from the bottom to the top until the participant gets the 6 names and locations correctly. The participant is then asked to remember and say the names of the objects in the given sequence while pointing to the corresponding square. This learning task is repeated up to ten times, or until the participant reaches 3 total consecutive ordered series, or after 5 attempts if the participant does not remember any word. After a distraction period (2.5 to 3 minutes) by a non-related task, the participant is asked to resume the In-out-test learning part by saying in the right sequence the name of the objects. In the case that the participant is not able to reproduce the complete series, he/she is asked to recall as much words as possible without taking into account the order. Scoring: the total items remembered in LOS is multiplied per 0.20 (maximum possible punctuation: 12); MOS is multiplied per 0.5 (maximum possible punctuation: 3); RAM is multiplied per 0.5 (maximum possible punctuation: 3); one extra-point is given when the participant reaches at least one complete series in LOS and another extra-point if he/she reaches three consecutive series. The maximal score is 20 points resulting of the following sum: 12+3+3+1+1.

Fig.1

Work template of In-out-test.

Collection of CSF biomarkers

40 of the participants underwent morning lumbar punctures. The samples were centrifuged and stored in polypropylene tubes at –80°C within the first hour after extraction. CSF concentrations of Aβ_1 - 42 peptide, total-tau, and phospho-tau (p-tau_181p) were determined using appropriate ELISA-kits (INNOTEST, Fujirebio, Ghent, Belgium)

Statistical analysis

The number of participants per group (healthy, MCI, and dementia) was determined based on our preliminary data that revealed the need of only 11 participants per group. However, the number of participants was increased to 30 per group to ensure an adequate statistical sample. The size of the sample was calculated based on the performance of the In-out-test in healthy individuals (16.6±3) and converters (9.23±5.9). The potency was determined as 90% and 0% loss, with bilateral contrast, and α= 5%.

In a first level of analysis, descriptive statistics was performed on participant data as well as on scores from the In-out-test. Afterwards, in order to establish the construct validity, the internal consistency of the In-out-test was analyzed through the establishment of the Cronbach alpha parameter r (both for whole scales and subscales). Interclass correlation (ICC) was established between the In-out-test and the FCSRT to establish the concurrent validity. To determine the reliability between valuators, the Kappa index (k) was calculated in 20 participants valued by two examiners (one of which led the session) who would later score the In-out-test. Finally, the test-retest reliability was analyzed using an ICC between two different time/same participant assessment (rank time: 5–25 days) in 20 patients.

Non-parametric analysis was used both for those variables that did not follow a Gaussian distribution or when the number of participants was lower than 30, in order to compare the scores in the different tests (Mann-Whitney U-test), scales and other sociodemographic variables. The GLM/ANOVA model and the post hoc Least Significant Difference test (LSD) were used in normal distributions. A two-sided significance level of 5% was used for all analyses.

To establish the concordance (not only relationship) between the In-out-test and the CFS biomarkers, ICC was determined in NPsych, MCI, dementia groups as well as in the total sample, using two way-Random effect model, absolute agreement, multiple raters/measurements as defined McGraw & Wong and Shrout & Fleiss classification [60]. To do this, the z-value was calculated as a standardized measure for the difference between the mean value and each specific subject, both for the In-out-test scores and for the CSF biomarkers values.

RESULTS

The sociodemographic characteristics and functional features of the total sample are shown in Table 1. There were no significant differences in age (U = 29.0; p = 0.10), education (U = 37.5; p = 0.30), anxiety symptoms (U = 49.5 p = 0.86), or depressive symptoms (U = 32.00; p = 0.16). Neuropsychological performances in CH (n = 16) and SCI (n = 16) were statistically similar and thus were joined into the NPsych group (n = 32). No statistically significant differences between the three groups were observed for formal education (F = 1.84; p = 0.1700), anxiety symptoms (F = 0.16; p = 0.8470), and depression symptoms (F = 0.20; p = 0.8180). Conversely, significant differences were detected for age (F = 6.46; p = 0.0040). Post-hoc LSD test revealed significant differences between NPsych and MCI (p = 0.001) and between NPsych and dementia group (p = 0.001).

Table 1

Demographic characteristics for cognitively healthy, mild cognitive impairment, and dementia groups

	Total N = 94	NPsych N = 32	MCI N = 32	Dementia N = 30	F (sig.)
Male	30	8	15	7
Female	64	24	17	23	^*
Age	60–92	60–92	65–90	68–88
(Mean/SD)	(76.17/7.58)	(71.90/8.52)	(77.25/5.7)	(79.57/6.18)	6.46 (0.0040)
Education	0–20	0–20	0–15	1–13
(Mean/SD)	(7.34/3.90)	(8.22/4.62)	(7.05/3.88)	(6.70/2.93)	1.84 (0.1700)
MMSEadj	15–32	25–32	18–31	15–29
(Mean/SD)	(25.60/4.43)	(29.16/1.80)	(26.38/2.46)	(20.96/4.03)	61.50 (0.0001)
HADS	0–26	1–26	0–20	0–18
(Mean/SD)	(8.09/5.57)	(9.13/6.29)	(7.81/5.47)	(7.27/4.81)	0.11 (0.8960)
HADS-A	0–26	1–16	0–16	0–11
(Mean/SD)	(8.09/5.57)	(5.25/4.2)	(3.74/3.23)	(4.23/3.51)	0.16 (0.8470)
HADS-D	0–26	0–11	0–12	0–8
(Mean/SD)	(8.09/5.57)	(3.88/2.9)	(4.06/2.87)	(3.03/1.97)	0.20 (0.8180)
BDRS-A	0–7	0–2.5	0–3.5	2–7
(Mean/SD)	(1.93/1.77)	(0.50/0.75)	(1.42/1.01)	(3.98/1.20)	35.9850 (0.0001)
IADL	1–8	6–8	5–8	1–5
(Mean/SD)	(6.18/1.97)	(7.78/0.49)	(6.90/1.00)	(3.7/1.15)	39.5100 (0.0001)

^*χ² = 35.230(0.0060); NPsych, normal psychometric performance; MCI, mild cognitive impairment; MMSEadj, Mini-Mental State Examination adjusted by age and education; HADS, Hospital Anxiety and Depression Scale; BDRS-A, Blessed Dementia Rating Scale part A; IADL, Instrumental Activity of Daily Living scale.

The internal consistency of the In-out-test showed a Cronbach Alpha of r = 0.81 for the total scale, the index being smaller if the total score was excluded (α= 0.61). The reliability between evaluators was k = 0.94. ICC for the test-retest reliability was r = 0.57 (p = 0.575). The ICC for the evaluation of the concurrent validity between the In-out-test and the FCSRT was r = –0.87 (p = 0.001).

The three groups exhibited varying performances on the three different parts of the In-out-test. Thus, ANOVA analyses rendered significant differences for total score (F = 107.192; p = 0.01), LOS (F = 94.227; p = 0.01), RAM (F = 76.612; p = 0.01), and MOS (F = 47.097; p = 0.01).

The concordance between the In-out-test and levels of CSF biomarkers in NPsych, MCI, dementia, and total sample are shown in Tables 2 and 3. Also, the concordances between the p-tau/Aβ₄₂ ratio for different groups and the different cut off values for the In-out-test were calculated. Thus, a high degree of correlation between In-out-test total score with CSF biomarkers (Table 2) for healthy (Aβ₄₂ r = 0.73, p = 0.01 and p-tau/ Aβ₄₂ r = 0.75, p = 0.01); MCI (Aβ₄₂ r = 0.64, p = 0.01 and p-tau, r = 0.77, p = 0.01); DAT (Aβ₄₂ r = –0.71, p = 0.01), as well as for the total sample (Aβ₄₂ r = 0.67, p = 0.01 and p-tau r = 0.70, p = 0.01). In addition, we could detect significant relationships between learning tasks and p-tau (Table 3). Table 4 shows the correlation between FCSRT with CSF biomarkers. Further, we observed that these relationships were not linear, but rather organized in ranges (Fig. 2). Indeed, a high correlation with p-tau up to 50 pg/ml (r = 0.762, p = 0.01) and above 80 pg/ml (r = –0.740, p = 0.01), was observed, but not in the range 50 to 80 pg/ml (r = 0.30, p = 0.01).

Finally, the diagnosis of AD using the 6 words recalled (RAM6) together with the p-tau/ Aβ₄₂, taking into account the predetermined cut-off (0.12) by our laboratory (Table 5), was compared. Individual performance on the six words in a random way (RAM6) showed the highest concordance (k = 0.71) (Table 5). Those using the FCSRT had a concordance of k = 0.49. When diagnosing prodromal AD, the concordance for the In-out-test was a k = 0.69 (Table 6) and for the FCSRT a k = 0.44. In addition, sensitivity, specificity, predictive values and likelihood ratios were also calculated from the tables of contingence (Tables 5 and 6).

Table 2

Intraclass correlation between total score of In-out-test and CSF biomarkers

Group	N	Aβ₄₂	T-tau	p-tau	p-tau/ Aβ₄₂
Npsych	16	0.727	0.576	0.654	0.746
MCI	19	0.636	0.485	0.769	0.194
DEM	5	0.709	0.462	0.602	0.671
All groups	40	0.666	0.445	0.703	0.373

Aβ₄₂, amyloid β₄₂; T-tau, Total tau; p-tau, phosphorylated tau; NPsych: normal psychometric performance; MCI, mild cognitive impairment; DEM, dementia.

Table 3

Intraclass correlation between learning of In-out-test and CSF biomarkers

Group	N	Aβ₄₂	T-tau	p-tau	p-tau/ Aβ₄₂
NPsych	16	0.706	0.555	0.559	0.747
MCI	19	0.629	0.480	0.722	0.436
DEM	5	0.697	0.461	0.575	0.613
All groups	40	0.646	0.444	0.681	0.414

Aβ₄₂, amyloid β₄₂; T-tau, Total tau; p-tau, phosphorylated tau; NPsych, normal psychometric performance; MCI, mild cognitive impairment; DEM, dementia.

Table 4

Intraclass correlation between FCSRT total delayed recall and CSF biomarkers

Group	N	Aβ₄₂	T-tau	p-tau	p-tau/ Aβ₄₂
NPsych	16	0.010	0.030	0.201	0.141
MCI	19	0.002	0.004	0.068	0.162
DEM	5	0.112	0.081	0.189	0.264
All groups	40	0.102	0.036	0.162	0.089

Aβ₄₂, amyloid β₄₂; T-tau, Total tau; p-tau, phosphorylated tau; NPsych, normal psychometric performance; MCI, mild cognitive impairment; DEM, dementia.

Fig.2

Relationship between the learning task of In-out-test and p-tau levels in CSF.

DISCUSSION

The present study introduces a novel paradigm to assess the risk of conversion to dementia, the In-out-test. Unlike other tests using the HA paradigm, such as the FCSRT, which assesses memory after effective encoding of information, the In-out-test provokes an interference during encoding by means of a simultaneous executive task. This paradigm is proposed based on the compensatory functional networks which are believed to be widespread in early stages of AD. If parts of those networks that are usually involved in other functions take the command of memory during the earliest stages of AD, as proposed in the new compensatory brain theories, it can be predicted that these networks would be less efficient in these persons than those with the same memory complaints, but not suffering the disease and, therefore, do not need these auxiliary networks. According to this model the memory deficit in prodromal AD can be unmasked when those compensatory networks are saturated by an executive activity without being able to compensate the already damaged mnemonic activity.

Even though neuropsychological tests oriented to detect HA can distinguish cognitively healthy, MCI, and dementia groups these tests lack the ability to separate early AD from non-progressive MCI consistently. To overcome the limitations for diagnosing prodromal AD, some authors have increased the complexity of the original tests [13]. Other studies have used different models to quantify amnesia. Grober et al. propose that the FCSRT free recall is the best measure for detecting prevalent dementia and predicting future dementia, and that the “total recall impairment” component of the test discriminates between AD and non-AD dementia [61]. In a recent study, researchers from this same group used the sum of both scores to retrospectively track the progression of patients already diagnosed with dementia [15]. In this and other previous studies that use the hippocampal memory paradigm [62], it is shown that free recall can be an early marker of later impairment. These data can be useful in clinical trials where memory performance is assessed by means of such a paradigm.

The authors of the Memory Binding Test propose a model that amplifies the numbers of items to memorize as they consider the FCSRT is limited by the presence of a “ceiling effect” [12, 13]. Despite the fact that the conventional tests have acceptable sensitivity and specificity, their low predictive ability makes them unsuitable for clinical settings. For example, in a study assessing conversion to dementia, Auriacombe et al. report a PPV of 7 for FCSRT [63]. Clinically, it means that only 7% of the patients with positive results will have dementia in the future and more than 90% with positive results will not have dementia. In a recent study, Derby et al. report a PPV of 30 [64].

Table 5

Contingence table comparing AD diagnosis among 6 words evocated in In-out-test to CSF biomarkers (p-tau/Aβ₄₂ ratio) in the total sample

RAM	AD(+)	AD(–)	Total
<6 words	25	3	28
6 words	2	10	12
Total	27	13	40

Agreement between biomarkers diagnosis and evocation of the 6 words in In-out-test (RAM6) by kappa index (k) = 0.709; sensitivity = 93%; specificity = 77%; predictive positive value (PPV) = 0.89; predictive negative value (PNV) = 0.83; positive likelihood ratio (+LR) = 4.04; negative likelihood ratio (–LR) = 0.09. RAM, random memory. AD(+) and AD(–): positive and negative diagnosis for Alzheimer’s disease, respectively, for CSF p-tau/Aβ₄₂ ratio (cut-off 0.12).

Table 6

Contingence table comparing prodromal AD diagnosis among 6 words evocated in In-out-test to CSF biomarkers (p-tau/Aβ₄₂ ratio) in the no dementia sample

RAM	AD (+)	AD (–)	Total
<6 words	20	3	23
6 words	2	10	12
Total	22	13	35

Agreement between biomarkers diagnosis and evocation of the 6 words in In-out-test (RAM6) by kappa index (k) = 0.727; sensitivity = 91%; specificity = 77%; predictive positive value (PPV) = 0.87; predictive negative value (PNV) = 0.83; positive likelihood ratio (+LR) = 3.96; negative likelihood ratio (–LR) = 0.12. RAM, random memory. AD(+) and AD(–): positive and negative diagnosis for Alzheimer’s disease, respectively, for CSF p-tau/Aβ₄₂ ratio (cut-off 0.12).

The objective of this study was to evaluate the validity of the In-out-test for identifying prodromal AD. First, performance on the In-out-test and that on the FCSRT were compared. FCSRT was chosen as several studies have demonstrated its reliability for the diagnose of AD in addition to biomarkers [4]. It was of particular interest to administer the In-out test with the approach of naturalistic studies to increase the possibility to extrapolate results to future evaluations in clinical setting.

Preliminary results suggest that the In-out-test is not affected by the formal education and does not require reading and writing skill for its proper execution. However, the present study included reading and writing skills as participation criteria.

Fig.3

Relationship between the learning task of In-out-test and p-tau levels in CSF.

The In-out-test accomplishes the basic principles for construct, criterion validity, and classic psychometrics, as well as reliability, with high correlation indexes. Based on such results, the In-out-test can be considered as a valid, reliable, and easily applicable tool for the diagnosis of AD. The results showed high correlations between the three parts of the In-out-test (LOS, MOS, and RAM) with FCSRT across the total sample. There were significant differences between cognitively healthy, MCI, and patients with dementia. Furthermore, there was no influence of age or formal education in NPsych, although differences were detected in the total sample. This result might be related to the increased age of the participants in the dementia group and this, in turn, may be related to fewer years of compulsory education in that group. Participants were recruited consecutively in the ongoing PROA project, which is working on homogenizing the ratio of age and years of education in the future. In agreement, no differences were found in the pilot study [35].

In the second part of this study the relationship of the In-out-test performance with levels of CSF biomarker was analyzed. ICC showed a high concordance between two different measures. Thus, the results obtained in the In-out-test were in good agreement with the standardized values of CSF biomarkers, specifically in p-tau for the MCI and Aβ₄₂ for NPsych and dementia. Therefore p-tau value can be inferred with high reliability in MCI patients from performance on the In-out test. Likewise, the Aβ₄₂ value can be inferred in cognitively healthy and dementia groups (Tables 2 and 3). Conversely, concordance of FCSRT performance and CSF biomarkers was very poor (ICC <0.3) both for the total sample and within the different groups (Table 4). Cohen’s Kappa was used to diagnose AD and prodromal AD. Furthermore, the In-out-test was a better predictor than FCSRT, with good agreement in the first and moderate for FCSRT both for the diagnosis of AD and for prodromal AD.

There was a significant coefficient correlation between the In-out-test performance and p-tau levels (Fig. 2) and, notably it was also observed that correlations varied depending on p-tau ranges (Fig. 3). Thus, a statistically significant positive correlation was found between the In-out-test to p-tau in NPsych being 50 pg/ml the cut off level. There was a gap between 50 pg/ml to 80 pg/ml without any significant correlation, which is followed by a strong negative correlation beyond 80 pg/ml (Fig. 3). This is particularly interesting since the cut off level of 50 pg/ml is in agreement with some previous reports. Thus, both Mogekar [11] and, independently, Höltta [65] reported that patients who converted to MCI or dementia exhibited a mean p-tau level of 49 pg/ml. The positive relationship between p-tau and the In-out-test up to 50 pg/ml seems paradoxical, as the performance improved in parallel with a rise of p-tau levels as a marker of neurodegeneration (p-tau) rises as well. In agreement, in a longitudinal study by Seppälä and colleagues [66], a slight increase of p-tau in healthy and stable MCI participants along a three years follow-up was reported. They also showed that changes in biomarkers occur at different paces in each stage of the AD continuum. The increase of p-tau reaches its maximum in dementia patients, to a lesser degree in MCI, and are lowest in healthy controls [66]. The authors found a positive correlation between the results in the In-out-test and p-tau up to 50 pg/ml. Up to 50 pg/ml the low speed in neuronal loss in healthy individuals, might allow the activation of compensatory mechanisms as shown by Rao et al. [32]. The limit of 50 pg/ml is also in agreement with the upper limit in p-tau levels in the control group in a study on TREM2 and AD (49 pg/ml) [67]. The low correlation in the 50–80 pg/ml gap might be a consequence of the accelerated neuronal loss in MCI and AD patients (intermingled in this range) with a high interindividual variability in the remaining compensatory mechanisms typical of aging and others due to AD. Finally, it can be speculated that the total loss of compensatory mechanisms makes performance on the In-out-test only dependent on the neuronal loss as measured by p-tau levels (>80 pg/ml).

Although the results of this study suggest that the In-out-test might be useful for accurate clinical decisions, there were of course some limitations worthy of discussion.

Firstly, the sample size was rather small in particular with respect to the CSF biomarkers. Although a sample size calculation informed that there were sufficient participants in each group for the In-out test. Secondly, there were some variations in the composition of the groups: there were more women than men, the individuals in the dementia group were older (but this is to be expected), and the level of education of participants was below. Finally, some participants were taking antidepressants and anxiolytics although they did not have severe depression.

Conclusions

The present study demonstrates that the In-out-test can be considered as a valid, reliable and easily applicable tool for the assessment of episodic memory in cognitively healthy, MCI, and demented subjects, but also for the identification of prodromal AD. The data presented here support the hypothesis that performing an executive task during the encoding can unmask the underlying memory deficits in early stages of AD.

The In-out-test allows detection of individuals at risk of AD in good agreement with the comparative study of CSF biomarkers. Further, the results of this study suggest that a value of p-tau of 50 pg/ml may be a critical threshold for the diagnosis of early AD. Due to the continuum of the AD process, persons with high scoring in the In-out-test are likely to have a high probability of being negative for biomarkers.

The In-out-test may be useful for accurate clinical decisions, including the evaluation of the pertinence of lumbar puncture, starting earlier treatments, and correctly selecting true controls and very early AD patients in clinical trials.

Future studies are necessary to corroborate the clinical usage of the In-out-test in the identification of prodromal AD and its ability to predict conversion to dementia.

Footnotes

ACKNOWLEDGMENTS

Supported by grants from Fundación Canaria del Colegio de Médicos de Las Palmas (Spain), and SAF2010-22114-C02-01/02 and SAF2014-52582-R from Ministerio de Economía y Competitividad (Spain). We would like to thank to: Dr. Jazmir M. Hernández; Dr. Abdón Rodríguez; Dr. Jesús Villar; Ángel Aranda; Dr. Marcelo Berthier; Miguel Ángel García Bello and Jesús González Martín for their advice in statistical analysis; Lidia Santana; Laura Torres; Rita Sosa; Belén Sánchez; Franca Palombini, and Esther Cabrera, Cograf Comunicaciones as well as to the personnel from Traumatology, Immunology and Anesthesia Services at HUGCDN for their collaboration during the development of the present study. Finally, we would also like to thank Dr. Eduardo Castro and Dr. Sasha Bozeat (Nutricia) for advice on manuscript development.

Authors’ disclosures available online ().

References

Petersen

, Smith

, Waring

, Ivnik

, Tangalos

, Kokmen

(1999) Mild cognitive impairment: Clinical characterization and outcome. Arch Neurol 56, 303–308.

Mitchell

, Shiri-Feshki

(2009) Rate of progression of mild cognitive impairment to dementia–meta-analysis of 41 robust inception cohort studies. Acta Psychiatr Scand 119, 252–265.

Soininen

, Solomon

, Visser

, Hendrix

, Blennow

, Kivipelto

, Hartmann

, Hallikainen

, Helisalmi

, Lappalainen

, Liu

, Paajanen

, Wahlund

L-O

, Freund-Levi

, Andreasen

, Hagman

, Lindblom

, Fassbender

, Riemenschneider

, Grimm

, Klees-Rollmann

, Luley

, Lyros

, Schomburg

, Kennel

, Ramelli

, Frölich

, Hausner

, Laske

, Leyhe

, Mychajliw

, Koehler

, Schiekofer

, Klünemann

, Schröder

, Lütjohann

, Scheltens

, van Rossum

, Scheltens

, Bertens

, ten Kate

, Barkhof

, Henselmans

, Roks

, van Hees

, Ellison

(2017) 24-month intervention with a specific multinutrient in people with prodromal Alzheimer’s disease (LipiDiDiet): A randomised, double-blind, controlled trial. Lancet Neurol 16, 965–975.

Dubois

, Feldman

, Jacova

, Hampel

, Molinuevo

, Blennow

, DeKosky

, Gauthier

, Selkoe

, Bateman

, Cappa

, Crutch

, Engelborghs

, Frisoni

, Fox

, Galasko

, Habert

M-O

, Jicha

, Nordberg

, Pasquier

, Rabinovici

, Robert

, Rowe

, Salloway

, Sarazin

, Epelbaum

, de Souza

, Vellas

, Visser

, Schneider

, Stern

, Scheltens

, Cummings

(2014) Advancing research diagnostic criteria for Alzheimer’s disease: The IWG-2 criteria. Lancet Neurol 13, 614–629.

Rentz

, Parra Rodriguez

, Amariglio

, Stern

, Sperling

, Ferris

(2013) Promising developments in neuropsychological approaches for the detection of preclinical Alzheimer’s disease: A selective review. Alzheimers Res Ther 5, 58.

Bobinski

, Wegiel

, Wisniewski

, Tarnawski

, Reisberg

, Mlodzik

, de Leon

, Miller

(1995) Atrophy of hippocampal formation subdivisions correlates with stage and duration of Alzheimer disease. Dementia 6, 205–210.

Engler

, Forsberg

, Almkvist

, Blomquist

, Larsson

, Savitcheva

, Wall

, Ringheim

, Långström

, Nordberg

(2006) Two-year follow-up of amyloid deposition in patients with Alzheimer’s disease. Brain 129, 2856–2866.

Larrieu

, Letenneur

, Orgogozo

, Fabrigoule

, Amieva

, Le Carret

, Barberger-Gateau

, Dartigues

(2002) Incidence and outcome of mild cognitive impairment in a population-based prospective cohort. Neurology 59, 1594–1599.

Albert

, DeKosky

, Dickson

, Dubois

, Feldman

, Fox

, Gamst

, Holtzman

, Jagust

, Petersen

, Snyder

, Carrillo

, Thies

, Phelps

(2011) The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 7, 270–279.

10.

Putcha

, Brickhouse

, O’Keefe

, Sullivan

, Rentz

, Marshall

, Dickerson

, Sperling

(2011) Hippocampal hyperactivation associated with cortical thinning in Alzheimer’s disease signature regions in non-demented elderly adults. J Neurosci 31, 17680–17688.

11.

Moghekar

, Li

, Lu

, Li

, Wang

M-C

, Albert

, O’Brien

, BIOCARD Research Team (2013) CSF biomarker changes precede symptom onset of mild cognitive impairment. Neurology 81, 1753–1758.

12.

Rentz

, Locascio

, Becker

, Moran

, Eng

, Buckner

, Sperling

, Johnson

(2010) Cognition, reserve, and amyloid deposition in normal aging. Ann Neurol 67, 353–364.

13.

Gramunt

, Sánchez-Benavides

, Buschke

, Lipton

, Masramon

, Gispert

, Peña-Casanova

, Fauria

, Molinuevo

(2016) Psychometric properties of the memory binding test: Test-retest reliability and convergent validity. J Alzheimers Dis 50, 999–1010.

14.

Buschke

, Mowrey

, Ramratan

, Zimmerman

, Loewenstein

, Katz

, Lipton

, Mowrey

(2017) Memory binding test distinguishes amnestic mild cognitive impairment and dementia from cognitively normal elderly. Arch Clin Neuropsychol 32, 29–39.

15.

Grober

, Veroff

, Lipton

(2018) Temporal unfolding of declining episodic memory on the Free and Cued Selective Reminding Test in the predementia phase of Alzheimer’s disease: Implications for clinical trials. Alzheimers Dement (Amst) 10, 161–171.

16.

Rentz

, Amariglio

, Becker

, Frey

, Olson

, Frishe

, Carmasin

, Maye

, Johnson

, Sperling

(2011) Face-name associative memory performance is related to amyloid burden in normal elderly. Neuropsychologia 49, 2776–2783.

17.

Papp

K V.

, Amariglio

, Dekhtyar

, Roy

, Wigman

, Bamfo

, Sherman

, Sperling

, Rentz

(2014) Development of a psychometrically equivalent short form of the face-name associative memory exam for use along the early alzheimers disease trajectory. Clin Neuropsychol 28, 771–785.

18.

Frade

, López-Sánchez

(2010) A novel hypothesis for Alzheimer disease based on neuronal tetraploidy induced by p75 (NTR). Cell Cycle 9, 1934–1941.

19.

Cohen

, Price

, Weissfeld

, James

, Rosario

, Bi

, Nebes

, Saxton

, Snitz

, Aizenstein

, Wolk

, Dekosky

, Mathis

, Klunk

(2009) Basal cerebral metabolism may modulate the cognitive effects of Abeta in mild cognitive impairment: An example of brain reserve. J Neurosci 29, 14770–14778.

20.

Belleville

, Clément

, Mellah

, Gilbert

, Fontaine

, Gauthier

(2011) Training-related brain plasticity in subjects at risk of developing Alzheimer’s disease. Brain 134, 1623–1634.

21.

Bondi

, Houston

, Eyler

, Brown

(2005) fMRI evidence of compensatory mechanisms in older adults at genetic risk for Alzheimer disease. Neurology 64, 501–508.

22.

Clément

, Belleville

(2010) Compensation and disease severity on the memory-related activations in mild cognitive impairment. Biol Psychiatry 68, 894–902.

23.

Grady

, McIntosh

, Beig

, Keightley

, Burian

, Black

(2003) Evidence from functional neuroimaging of a compensatory prefrontal network in Alzheimer’s disease. J Neurosci 23, 986–993.

24.

Peters

, Collette

, Degueldre

, Sterpenich

, Majerus

, Salmon

(2009) The neural correlates of verbal short-term memory in Alzheimer’s disease: An fMRI study. Brain 132, 1833–1846.

25.

Sperling

, Dickerson

, Pihlajamaki

, Vannini

, LaViolette

, Vitolo

O V

, Hedden

, Becker

, Rentz

, Selkoe

, Johnson

(2010) Functional alterations in memory networks in early Alzheimer’s disease. Neuromolecular Med 12, 27–43.

26.

Buldú

, Bajo

, Maestú

, Castellanos

, Leyva

, Gil

, et

al.

(2011) Reorganization of functional networks in mild cognitive impairment. PLoS One 6, e19584.

27.

Maestú

, Fernández

, Simos

, Gil-Gregorio

, Amo

, Rodriguez

, Arrazola

, Ortiz

(2001) Spatio-temporal patterns of brain magnetic activity during a memory task in Alzheimer’s disease. Neuroreport 12, 3917–3922.

28.

Malpass

(2011) Neurodegenerative disease: Functional networks are altered in mild cognitive impairment. Nat Rev Neurol 7, 356.

29.

Celone

, Calhoun

, Dickerson

, Atri

, Chua

, Miller

, DePeau

, Rentz

, Selkoe

, Blacker

, Albert

, Sperling

(2006) Alterations in memory networks in mild cognitive impairment and Alzheimer’s disease: An independent component analysis. J Neurosci 26, 10222–10231.

30.

Dickerson

, Salat

, Greve

, Chua

, Rand-Giovannetti

, Rentz

, Bertram

, Mullin

, Tanzi

, Blacker

, Albert

, Sperling

(2005) Increased hippocampal activation in mild cognitive impairment compared to normal aging and AD. Neurology 65, 404–411.

31.

Browndyke

, Giovanello

, Petrella

, Hayden

, Chiba-Falek

, Tucker

, Burke

, Welsh-Bohmer

(2013) Phenotypic regional functional imaging patterns during memory encoding in mild cognitive impairment and Alzheimer’s disease. Alzheimers Dement 9, 284–294.

32.

Rao

, Bonner-Jackson

, Nielson

, Seidenberg

, Smith

, Woodard

, Durgerian

(2015) Genetic risk for Alzheimer’s disease alters the five-year trajectory of semantic memory activation in cognitively intact elders. Neuroimage 111, 136–146.

33.

MacPherson

, Della Sala

, Logie

(2004) Dual-task interference of encoding and retrieval processes in healthy and impaired working memory. Cortex 40, 183–184.

34.

Foley

, Kaschel

, Logie

, Della Sala

(2011) Dual-task performance in Alzheimer’s disease, mild cognitive impairment, and normal ageing. Arch Clin Neuropsychol 26, 340–348.

35.

Torrealba

, Garcia-Morales

, Cejudo

, Gramunt

, Sánchez-García

, Fabre-Pi

(2012) Inout test: Diagnosis of potential converters from mild cognitive impairment and subjective complaints to dementia by a new paradigm. In 12th International Stockholm/Springfield Symposium on Advances in Alzheimer Therapy, Stockholm, p. 261.

36.

Blessed

, Tomlinson

, Roth

(1968) The association between quantitative measures of dementia and of senile change in the cerebral grey matter of elderly subjects. Br J Psychiatry 114, 797–811.

37.

Lawton

, Brody

(1969) Assessment of older people: Self-maintaining and instrumental activities of daily living. Gerontologist 9, 179–186.

38.

Peña-Casanova

, Monllau

, Böhm

, Aguilar

, Sol

, Hernández

, Blesa

, Grupo NORMACODEM (2005) Diagnostic value and test-retest reliability of the Blessed Dementia Rating Scale for Alzheimer’s disease: Data from the NORMACODEM project. Neurologia 20, 349–355.

39.

Portet

, Ousset

, Visser

, Frisoni

, Nobili

, Scheltens

, Vellas

, Touchon

, MCI Working Group of the European Consortium on Alzheimer’s Disease (EADC) (2006) Mild cognitive impairment (MCI) in medical practice: A critical review of the concept and new diagnostic procedure. Report of the MCI Working Group of the European Consortium on Alzheimer’s Disease. J Neurol Neurosurg Psychiatry 77, 714–718.

40.

Reisberg

, Shulman

, Torossian

, Leng

, Zhu

(2010) Outcome over seven years of healthy adults with and without subjective cognitive impairment. Alzheimers Dement 6, 11–24.

41.

Peña-Casanova

, Quiñones-Ubeda

, Quintana-Aparicio

, Aguilar

, Badenes

, Molinuevo

, Torner

, Robles

, Barquero

, Villanueva

, Antúnez

, Martínez-Parra

, Frank-García

, Sanz

, Fernández

, Alfonso

, Sol

, Blesa

, NEURONORMA Study Team (2009) Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for verbal span, visuospatial span, letter and number sequencing, trail making test, and symbol digit modalities test. Arch Clin Neuropsychol 24, 321–341.

42.

Peña-Casanova

, Quiñones-Ubeda

, Gramunt-Fombuena

, Quintana

, Aguilar

, Molinuevo

, Serradell

, Robles

, Barquero

, Payno

, Antúnez

, Martínez-Parra

, Frank-García

, Fernández

, Alfonso

, Sol

, Blesa

, NEURONORMA Study Team (2009) Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for the Stroop color-word interference test and the Tower of London-Drexel. Arch Clin Neuropsychol 24, 413–429.

43.

Peña-Casanova

, Gramunt-Fombuena

, Quiñones-Ubeda

, Sánchez-Benavides

, Aguilar

, Badenes

, Molinuevo

, Robles

, Barquero

, Payno

, Antúnez

, Martínez-Parra

, Frank-García

, Fernández

, Alfonso

, Sol

, Blesa

, NEURONORMA Study Team (2009) Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for the Rey-Osterrieth complex figure (copy and memory), and free and cued selective reminding test. Arch Clin Neuropsychol 24, 371–393.

44.

Peña-Casanova

, Blesa

, Aguilar

, Gramunt-Fombuena

, Gómez-Ansón

, Oliva

, Molinuevo

, Robles

, Barquero

, Antúnez

, Martínez-Parra

, Frank-García

, Fernández

, Alfonso

, Sol

, NEURONORMA Study Team (2009) Spanish Multicenter Normative Studies (NEURONORMA Project): Methods and sample characteristics. Arch Clin Neuropsychol 24, 307–319.

45.

Visser

, Verhey

FRJ

, Boada

, Bullock

, De Deyn

, Frisoni

, Frolich

, Hampel

, Jolles

, Jones

, Minthon

, Nobili

, Olde Rikkert

, Ousset

P-J

, Rigaud

A-S

, Scheltens

, Soininen

, Spiru

, Touchon

, Tsolaki

, Vellas

, Wahlund

L-O

, Wilcock

, Winblad

(2008) Development of screening guidelines and clinical criteria for predementia Alzheimer’s disease. The DESCRIPA Study. Neuroepidemiology 30, 254–265.

46.

Blesa

, Pujol

, Aguilar

, Santacruz

, Bertran-Serra

, Hernández

, Sol

, Peña-Casanova

, Soler

, Zabay

, Riera

, Castellví

, Torner

, Charques

, Toirán

, Manero

, Peter Böhm

, Martí

, Meza

, Crespo

(2001) Clinical validity of the “mini-mental state” for Spanish speaking communities. Neuropsychologia 39, 1150–1157.

47.

Folstein

, Folstein

, McHugh

(1975) “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12, 189–198.

48.

Buschke

(1984) Cued recall in amnesia. J Clin Neuropsychol 6, 433–440.

49.

Gramunt Fombuena

(2008) Normalización y validación de un test de memoria en envejecimiento normal, deterioro cognitivo leve y enfermedad de Alzheimer (thesis). Universitat Ramon Llull. FPCEEB - Psicologia.

50.

Ojea Ortega

, González Álvarez de Sotomayor

, Pérez González

, Fernández Fernández

(2013) A new assessment for episodic memory. Episodic memory test and caregiver’s episodic memory test. Neurologia 28, 488–496.

51.

Rey

(1980) Test de copia de una figura compleja Manual Adaptacion espanola, TEA ediciones, Madrid.

52.

Casals-Coll

, Sánchez-Benavides

, Meza-Cavazos

, Manero

, Aguilar

, Badenes

, Molinuevo

, Robles

, Barquero

, Antúnez

, Martínez-Parra

, Frank-García

, Fernández

, Blesa

, Peña-Casanova

, NEURONORMA Study Team (2014) Spanish multicenter normative studies (NEURONORMA project): Normative data and equivalence of four BNT short-form versions. Arch Clin Neuropsychol 29, 60–74.

53.

Peña-Casanova

, Quiñones-Ubeda

, Gramunt-Fombuena

, Quintana-Aparicio

, Aguilar

, Badenes

, Cerulla

, Molinuevo

, Ruiz

, Robles

, Barquero

, Antúnez

, Martínez-Parra

, Frank-García

, Fernández

, Alfonso

, Sol

, Blesa

, NEURONORMA Study Team (2009) Spanish Multicenter Normative Studies (NEURONORMA Project): Norms for verbal fluency tests. Arch Clin Neuropsychol 24, 395–411.

54.

De Renzi

, Faglioni

(1978) Normative data and screening power of a shortened version of the Token Test. Cortex 14, 41–49.

55.

Golden

(1994), Stroop Test de colores y palabras. Manual, Madrid.

56.

Wechsler

(1999) WAIS III. Escala de inteligencia Wechler para adultos III. Manual tecnico, TEA ediciones, Madrid.

57.

Martínez-Martín

, Prieto-Jurczynska

, Frades-Payo

(2014) Psychometric attributes of the Parkinson’s Disease-Cognitive Rating Scale. An independent validation study. Rev Neurol 49, 393–398.

58.

Reisberg

, Ferris

, de Leon

, Crook

(1988) Global Deterioration Scale (GDS). Psychopharmacol Bull 24, 661–663.

59.

Snaith

, Zigmond

(1986) The hospital anxiety and depression scale. Br Med J (Clin Res Ed) 292, 344.

60.

Shrout

, Fleiss

(1979) Intraclass correlations: Uses in assessing rater reliability. Psychol Bull 86, 420–428.

61.

Grober

, Sanders

, Hall

, Lipton

(2010) Free and cued selective reminding identifies very mild dementia in primary care. Alzheimer Dis Assoc Disord 24, 284–290.

62.

Cejudo-Bolivar

(2016) Redefining amnestic mild cognitive impairment as an early form of Alzheimer’s disease based on assessment of memory systems. J Alzheimers Dis 53, 705–712.

63.

Auriacombe

, Helmer

, Amieva

, Berr

, Dubois

, Dartigues

(2010) Validity of the free and cued selective reminding test in predicting dementia: The 3C study. Neurology 74, 1760–1767.

64.

Derby

, Burns

, Wang

, Katz

, Zimmerman

, L’italien

, Guo

, Berman

, Lipton

(2013) Screening for predementia AD: Time-dependent operating characteristics of episodic memory tests. Neurology 80, 1307–1314.

65.

Hölttä

, Hansson

, Andreasson

, Hertze

, Minthon

, Nägga

, Andreasen

, Zetterberg

, Blennow

(2013) Evaluating amyloid-β oligomers in cerebrospinal fluid as a biomarker for Alzheimer’s disease. PLoS One 8, e66381.

66.

Seppälä

, Koivisto

, Hartikainen

, Helisalmi

, Soininen

, Herukka

S-K

(2011) Longitudinal changes of CSF biomarkers in Alzheimer’s disease. J Alzheimers Dis 25, 583–594.

67.

Kleinberger

, Yamanishi

, Suárez-Calvet

, Czirr

, Lohmann

, Cuyvers

, Struyfs

, Pettkus

, Wenninger-Weinzierl

, Mazaheri

, Tahirovic

, Lleó

, Alcolea

, Fortea

, Willem

, Lammich

, Molinuevo

, Sánchez-Valle

, Antonell

, Ramirez

, Heneka

, Sleegers

, van der Zee

, Martin

J-J

, Engelborghs

, Demirtas-Tatlidede

, Zetterberg

, Van Broeckhoven

, Gurvit

, Wyss-Coray

, Hardy

, Colonna

, Haass

(2014) TREM2 mutations implicated in neurodegeneration impair cell surface transport and phagocytosis. Sci Transl Med 6, 243ra86.