The Fragility of Temporal Memory in Alzheimer’s Disease

Abstract

Background:

Although episodic memory impairment is one of the hallmarks of Alzheimer’s disease (AD), the relative decline in the components of episodic memory (What, Where, and When) and the effects of cognitive training on each of them are still unknown.

Objective:

We aimed to independently assess the impairment in each component of episodic memory in early to moderate AD and address whether it can be enhanced through active, spatiotemporal episodic training.

Methods:

A non-verbal scene-based episodic memory task was developed to assess the ability to remember What, Where, and When information. Experiment 1 tested whether this task can differentiate AD subjects (N = 16) from healthy controls (N = 16). In Experiment 2, 13 AD subjects underwent 16 training sessions, followed by a re-administration of the scene-based memory task. Experiment 3 tested 42 healthy older adults and 51 younger adults on the same task to investigate the effects of normal aging.

Results:

Of the three components, When memory had the highest predictive power in distinguishing AD from normal aging. Following training of AD subjects, only Where memory improved. Only What memory revealed a significant decline in healthy subjects from 65–85 years of age.

Conclusion:

These findings shed new light on the importance of the temporal component of episodic memory as a behavioral marker of AD. The selective improvement of spatial but not temporal memory through training further demonstrates the fragility of temporal memory even in early AD. Neuroscientific research is needed to distinguish whether the Where component was enhanced by improvements in hippocampal spatial representation or by other compensatory mechanisms.

Keywords

Alzheimer’s disease cognitive training episodic memory temporal order memory visual scene memory

INTRODUCTION

Alzheimer’s disease (AD) is associated with the degeneration of the medial temporal lobe and, in particular, the hippocampal formation [1, 2]. Hippocampal dysfunction differentiates AD from normal aging [3, 4], and the hippocampal locus of early AD pathology causes cognitive impairments in everyday activities [5]. According to recent diagnostic criteria [6], episodic memory impairment is regarded as a core symptom of early AD. Thus, accurate behavioral indicators of hippocampal episodic memory performance are crucial to the detection of AD in its early stages, as delays in clinical and cognitive intervention significantly reduce the effectiveness of treatment.

Unfortunately, conventional neuropsychological assessments of episodic memory used in clinical settings have been reported to be unsatisfactory in their sensitivity and specificity for screening AD [7, 8]. Most existing tests assess episodic memory by using verbal tasks, despite the fact that a large part of what individuals remember in their daily life, and what is damaged in AD, consists also of spatiotemporal information and imagery [4 , 10]. Verbal, often autobiographical, assessments of episodic memory may be easy to administer, but they do not optimally model hippocampal memory function in the way cognitive neuroscientists have defined it [11].

Episodic memories are temporally dated memories of personally-experienced events, organized in such a way that allows us to replay or mentally re-experience those events [12]. Therefore, episodic memory is defined by what happened, where it happened, and when it happened (in what temporal sequence) [14]. Of these three components (What, Where, and When), spatial memory (Where) has been shown to be heavily dependent on hippocampal function [11 , 15], with the Nobel Prize-winning characterization of the hippocampus as the brain’s “GPS system” [16]. Thus, unsurprisingly, numerous behavioral studies in the past decade on the early cognitive effects of AD have focused on spatial memory deficits among the early cognitive markers of AD [4 , 17–19].

However, both spatial navigation (e.g., following a route from one place to another) and episodic memory involve the binding of What and Where components into a continuous temporal framework, which has been more recently discovered to be represented in the lateral entorhinal cortex as well as hippocampus proper, not only in rodents but also in humans [20 –24]. Previous studies reported that AD patients are impaired in the spatiotemporal components of memory (e.g., in contrast to object memory), suggesting that testing these components could be particularly useful in discriminating AD from normal aging [4 , 25]. Nevertheless, there is still uncertainty over whether temporal memory and spatial memory differ as cognitive-behavioral markers of AD [4, 26]. This distinction is crucial, not only in diagnosing AD in its earliest stages but also in designing effective training programs that specifically target these essential episodic components to enhance or rescue cognitive function in AD patients.

To date, there is ongoing controversy over the effectiveness of conventional cognitive training in dementia patients. A recent meta-analysis on various cognitive training studies conducted in clinical settings, for instance, found no evidence that participating in cognitive training was any more helpful than engaging in non-cognitive activities [7]. Moreover, there are also limitations in cognitive training intended specifically to improve episodic memory, such as reminiscence therapy, errorless learning, or spaced retrieval training [11, 27], in that most of them involve object or semantic memory, rather than the spatial or temporal aspects of memory.

In this study, our first goal was to develop an objective, sensitive, and specific behavioral (non-verbal) marker of AD by separately assessing the What, Where, and When components of episodic memory (Experiment 1). Our second goal was to identify the effects of a new 8-week (16 sessions) cognitive training paradigm, based on an active episodic navigation task, involving the placement of objects in various locations around the room in a particular temporal order, and testing its effectiveness in improving episodic memory in AD patients (Experiment 2). Finally, as a control, we tested a larger group of healthy older subjects (ages 65 to 85) and a separate group of younger subjects in their 20 s to observe the effects of normal aging on the What, Where, and When components of episodic memory (Experiment 3).

EXPERIMENT 1

Experiment 1 investigated differences in episodic memory between older adults with AD and age-matched healthy subjects, particularly in their memory profile on What, Where, and When. To do so, we developed a scene-based episodic memory task, based on the scene construction theory [9], which proposes that the hippocampus facilitates the binding together of memory elements into a coherent episode through the reconstruction of temporally organized scenes. By assessing subjects’ memory of the identity of objects in a scene, their location within the scene, and the temporal order in which the scenes were presented, we aimed to characterize the memory profile of AD patients with respect to the components of episodic memory and, in particular, to compare and contrast their spatial and temporal memory impairment.

METHODS

Subjects

16 patients (12 female) with AD were recruited from senior daycare centers in Daejeon, Korea. The imbalance in the number of male and female subjects was due to a higher tendency of females to use senior centers in Korea.

Inclusion criteria were as follows: 1) over 65 years old; 2) no history of other neurological or psychiatric disease; 3) an ability to understand verbal/written instructions; and 4) a diagnosis of AD confirmed by a physician according to the Diagnostic and Statistical Manual of Mental Disorders (DSM)-IV. The severity of dementia was defined in terms of the Global Deterioration Scale (GDS) (Table 1). The GDS provides a measure of cognitive function (1: no cognitive decline, 2: very mild cognitive decline, 3: mild cognitive impairment, 4: mild dementia, 5: moderate dementia, 6: moderately severe dementia, and 7: severe dementia) for those suffering from a primary degenerative dementia such as AD [28]. Written consent was obtained from the subjects’ caregivers, and verbal consent was given by the patient.

Table 1
Clinical characteristics of subjects

Characteristics Age-matched healthy group (n = 16) AD (n = 16) Mann-Whitney U /χ² 65–75 y (n = 19) 76–85 y (n = 23) Mann-Whitney U /χ² 20–28 y

Male (n = 24) Female (n = 27)

Age (y) 82.31 (1.70) 83.81 (2.66) 86.500 66.14 (1.34) 74.89 (2.53) – 22.70 (2.05) 21.33 (1.44)

Sex (M/F) 5/11 4/12 0.694 6/13 7/16 0.006 –

Education (y) 8.18 (4.66) 5.31 (5.54) 87.500 9.85 (2.85) 7.73 (3.78) 1.194 12.00 (0.0) 12.00 (0.0)

MMSE-K 26.50 (3.28) 21.31 (3.87) 39.500^†† 27.35 (2.14) 26.86 (2.86) 0.235 – –

SVLT 44.38 (7.18) 33.25 (8.42) 40.500^†† 49.05 (7.78) 44.95 (7.16) 1.683 – –

K-BNT 48.12 (7.06) 42.31 (7.29) 61.000^† 50.29 (6.41) 49.47 (6.65) 0.385 – –

GDS – 4.80 (0.75) – – – – – –

Characteristics	Age-matched healthy group (n = 16)	AD (n = 16)	Mann-Whitney U /χ²	65–75 y (n = 19)	76–85 y (n = 23)	Mann-Whitney U /χ²	20–28 y
Age (y)	82.31 (1.70)	83.81 (2.66)	86.500	66.14 (1.34)	74.89 (2.53)	–	22.70 (2.05)	21.33 (1.44)
Sex (M/F)	5/11	4/12	0.694	6/13	7/16	0.006	–
Education (y)	8.18 (4.66)	5.31 (5.54)	87.500	9.85 (2.85)	7.73 (3.78)	1.194	12.00 (0.0)	12.00 (0.0)
MMSE-K	26.50 (3.28)	21.31 (3.87)	39.500^††	27.35 (2.14)	26.86 (2.86)	0.235	–	–
SVLT	44.38 (7.18)	33.25 (8.42)	40.500^††	49.05 (7.78)	44.95 (7.16)	1.683	–	–
K-BNT	48.12 (7.06)	42.31 (7.29)	61.000^†	50.29 (6.41)	49.47 (6.65)	0.385	–	–
GDS	–	4.80 (0.75)	–	–	–	–	–	–

Shown are mean values (standard deviation). Due to a ceiling effect, healthy younger adults (20–28 y) did not perform the MMSE-K, SVLT, K-BNT, and GDS. ^†comparison of the age-matched healthy group and the AD group, ^†p < 0.05, ^††p < 0.01. SVLT, Seoul Verbal Learning Test; K-BNT, Korean version of Boston Naming Test; GDS, Global Deterioration Scale.

The age-matched healthy older adults (11 female) were recruited from the local senior center, and the inclusion criteria were as listed above (1–3), with the exception of AD diagnosis (4), including a medical questionnaire to verify that they had no previous neurological or psychiatric disorders and had not undergone any kind of treatment that would affect their cognition. Written consent was obtained from all healthy subjects.

This study received the approval of the institutional review board of Korea Advanced Institution of Science and Technology (KAIST) (KH2017–91) and was conducted in accordance with ethical guidelines for research on human subjects.

Design

16 patients with AD and their 16 age-matched control subjects performed the scene-based episodic memory task (Fig. 1). After 10 min of rest, all subjects performed the neuropsychological assessments, which consisted of the Korean version of the Mini-Mental State Exam (MMSE-K) (Supplementary Material 1) developed to assess global cognitive function, Seoul Verbal Learning Test (SVLT) for verbal learning and memory (Supplementary Material 2), and the Korean version of the Boston Naming Test (K-BNT) for verbal fluency (Supplementary Material 3). The MMSE-K and the K-BNT are standardized Korean versions of the original tests, with some visual stimuli modified to suit the Korean culture. In the SVLT, a Korean standardized test of verbal memory [29], subjects are given three learning trials from a list of 12 nouns (4 words in each of three semantic categories) and then asked to recall them after 20 min.

Fig. 1

Ten visual scene images were presented in each block. Each image was displayed for 3 seconds. A) Example of the What condition for object memory. B) Example of the Where condition for spatial memory. C) Example of the When condition for temporal memory (instructions in the figure have been translated from Korean to English).

Scene-based episodic memory task

The scene-based componential episodic memory task was executed using E-Prime software (Psychology Software Tools, PA). The visual images used in the test were taken at a resolution of 4032×3024 pixels using a digital camera on a tripod to keep the image in the same viewpoint when an object was removed from the scene or moved to a new spatial position. Subjects were randomly given three blocks of 10 trials each, consisting of three different conditions (What, Where, and When) in which subjects were shown 10 different scenes for 3 s each (Fig. 1). Subjects did not know during the encoding (picture presentation) phase which test condition they were in because the conditions differed only in the retrieval phase.

In the What condition, subjects were shown a scene that they had seen but with an object removed from the original scene. Subjects were asked to choose between pictures of two objects: the target object that they had seen in the original scene and a lure object that they had never seen (Fig. 1A) and asked to choose the object that was missing from the scene (“Z” key for the object on the left, “M” key for the object on the right).

In the Where condition, subjects were shown two pictures side by side: the scene that they had seen before, and a similar scene in which an object was moved to a new spatial position on the other side of the scene (Fig. 1B). Subjects were instructed to choose the original scene that they had seen by pressing the “Z” or the “M” key.

In the When condition, subjects were shown two of the previously presented scenes and asked to judge which scene they had seen earlier in the sequence of images (Fig. 1C) by pressing “Z” or “M.” The entire experiment had a total of 30 trials. Subjects’ accuracy and reaction time were automatically recorded on all trials.

Analysis

Demographic characteristics of the participants were analyzed using descriptive statistics. A comparison between males and females was precluded due to the small number of male subjects. The AD group was compared with the age-matched group of healthy adults by using the Mann-Whitney test and the χ² test.

The Wilcoxon test was used to compare accuracy on each condition with a chance level of 50%within each group. To calculate the discriminatory power of each component of episodic memory between the AD group and the healthy controls, a receiver operating characteristic (ROC) curve analysis was performed. Optimal cutoff scores that maximized the sum of sensitivity and specificity were determined. In order to assess differences in task difficulty, the Rasch analysis was conducted across the various tests. In this analysis, outcome variables are converted into logit measurements using a logistic function and item difficulty is arranged linearly, so that it is possible to compare the order of difficulty across tasks. A Spearman correlation was conducted to assess the relation between severity of cognitive impairment in AD and cognitive performance. Statistical significance was set at p < 0.05.

RESULTS

Subject characteristics

AD participants did not differ significantly from the age-matched healthy group in age, sex ratio, and education level. In the standardized cognitive tests, however, significant differences were found in performance in the MMSE-K, the SVLT, and the K-BNT (U_(1,30) = 39.500, p < 0.001; U_(1,30) = 40.500, p = 0.001; U_(1,30) = 61.000, p = 0.011, respectively) (Table 1). These results demonstrate that the AD group showed deficits in global cognition, verbal episodic memory, and semantic memory compared to the age-matched healthy group.

Comparison between episodic memory in AD patients and age-matched healthy controls

There were significant differences in accuracy on the Where and the When conditions (U_(1,30) = 57.000, p = 0.005; U_(1,30) = 33.500, p < 0.001, respectively) but not on the What condition between the age-matched healthy group and the AD group (U_(1,30)= 109.000, p = 0.455) (Table 2, Fig. 2).

Table 2

Effect of normal aging and AD on accuracy on the three conditions of the memory task

Episodic scene task		Age-matched healthy group (n = 16)	AD (n = 16)	Mann-Whitney U	65–75 y (n = 19)	76–85 y (n = 23)	Mann-Whitney U	20–28 y
Accuracy (%)	What	64.37 (15.47)	62.50 (11.83)	109.000	74.21 (9.61)	66.08 (14.69)	2.005^*	82.94 (14.60)
	Where	64.37 (6.29)	53.12 (13.52)	57.000^††	69.47 (13.11)	64.34 (8.95)	0.927	78.23 (13.81)
	When	57.50 (10.64)	41.87 (6.55)	33.500^†††	62.10 (11.82)	56.08 (14.37)	1.410	67.05 (12.37)

Shown are mean values (standard deviation). ^†comparison of the age-matched healthy group and the AD group, ^††p < 0.01, ^†††p < 0.001, ^*comparison between the older groups, p < 0.05.

Fig. 2

Comparison of accuracy between the AD group and the age-matched healthy group (Scene-based Episodic Memory Task). AD subjects were significantly lower in accuracy on the Where and the When conditions compared to the healthy group (Mann-Whitney test: see the asterisks between the bars). While healthy subjects were above chance in all three conditions, AD subjects were above chance in the What condition, no different from chance in the Where condition, and below chance in the When condition (based on the Scene-based Episodic Memory Task) (Wilcoxon test: see the asterisks above the bars). Below chance performance in the When condition may be attributed to the AD subjects’ tendency to remember, if at all, the images that appeared later in the block (i.e., the more recent images).

When separately compared against a chance level of 50%, the AD group performed significantly above chance on the What condition (Z₍₁₅₎ = 2.844, p =0.004) but not on the Where and the When conditions (Z₍₁₅₎ = 0.812, p = 0.417; Z₍₁₅₎ = 3.127, p = 0.002, respectively). In fact, AD subjects significantly preferred the incorrect response in the When condition. This could be explained by the fact that the more recently seen pictures were more likely to be recognized at all for the AD patients.

In contrast, the healthy group performed above chance in all three conditions (What: Z₍₁₅₎ = 2.574, p = 0.010; Where: Z₍₁₅₎ = 3.508, p < 0.001; When: Z₍₁₅₎ = 2.364, p = 0.018). These findings indicate that the AD group was impaired in spatial and temporal order memory compared to healthy subjects.

Sensitivity, specificity, and discriminant power

For dissociating AD from the age-matched healthy subjects, a cutoff score of 55.0/100 for temporal order memory yielded maximum sensitivity (68.8%) and specificity (100.0%) (Table 3, Fig. 3), suggesting that this novel visual scene-based task involving the What, Where, and When components of episodic memory can better discriminate AD compared to conventional neuropsychological tests. With high specificity, temporal order memory is more precise in identifying AD and designating older adults who do not have AD as a negative. On the other hand, with high sensitivity, spatial location memory is more inclusive and less likely to miss those with AD. Consistent with the view that spatial and temporal memory play complementary predictive roles, we found that the combination of Where+When shows the highest predictive power for detecting AD subjects, compared to What+Where and What+When (see Table 3).

Table 3

Sensitivity and specificity for detection of AD, for performance on each of the three memory conditions, their combined pairs, and standardized neuropsychological tests

Episodic scene task		AUC	SEM	p	Cutoff	Se	Sp	Youden index
What	ACC	0.555	0.105	0.598	75.0	25.0	87.5	0.125
Where	ACC	0.777	0.085	0.007	55.0	93.8	50.0	0.437
When	ACC	0.869	0.066	<0.001	55.0	68.8	100.0	0.687
What + Where	ACC	0.721	0.092	0.033	62.5	62.5	37.5	0.375
Where + When	ACC	0.895	0.057	<0.001	57.5	68.8	93.8	0.625
What + When	ACC	0.801	0.080	0.080	57.5	75.0	75.0	0.500
MMSE-K		0.846	0.068	0.001	25.5	62.5	93.7	0.562
SVLT		0.842	0.073	0.001	37.5	87.5	75.0	0.625
K-BNT		0.767	0.091	0.012	50.5	50.0	93.7	0.437

AUC, Area under the curve; K-BNT, Korean version of Boston Naming Test; MMSE-K, Korean version of Mini-Mental Status Examination; SEM, Standard Error of the Mean; Se, Sensitivity; Sp, Specificity; SVLT, Seoul Verbal Learning Test.

Fig. 3

ROC curves of significant five predictors. Greater AUC values indicate higher power in discriminating AD from healthy subjects.

To rule out the alternative hypothesis that the predictive power of temporal memory was due to the fact that AD patients are simply more impaired on difficult tasks (in general, rather than on specific memory conditions), we performed the Rasch test (see Methods for explanation). The results showed that the MMSE and the SVLT are harder than temporal memory (Fig. 4); given that the predictive power of temporal memory is higher than that of MMSE and SVLT, we can conclude that the differences in predictive power cannot be attributed to the differences in task difficulty (Figs. 3, 4). Taken together, our results suggest that this multi-component (temporal and spatial) memory task could be a useful tool in diagnosing AD.

Fig. 4

Representation of item difficulty using the Rasch analysis. Items are located according to their difficulty. Less difficult items are located on the bottom, and more difficult items are on the top. For instance, the MMSE presents the most difficult item.

Correlations in outcome measures with severity of AD

GDS was found to be significantly correlated with all the outcome measurements (Where: r = –0.482, p = 0.005; When: r = –0.696, p < 0.001; MMSE-K: r = –0.702, p < 0.001; SVLT: r = –0.650, p < 0.001; K-BNT: r = –0.564, p = 0.001), except for accuracy in the What condition (r = –0.211, p = 0.245) (see Table 4).

Table 4

Correlation of GDS with accuracy on the three memory conditions and standardized neuropsychological tests

	Accuracy			MMSE-K	SVLT	K-BNT
	What	Where	When
GDS	–0.211	–0.482^**	–0.696^***	–0.702^***	–0.269^***	–0.365^**

^**p < 0.01, ^***p < 0.001. GDS, Global Deterioration Scale.

EXPERIMENT 2

The main goal of Experiment 2 was to investigate the effects of training on episodic memory in older adults with AD. The training task was motivated from studies reporting that extensive navigation experience enhances memory mechanisms due to the plasticity of hippocampal function [14 , 30]. However, we aimed to identify which episodic memory component can be enhanced through a cognitive training program which required the participants to actively navigate around a room and remember what objects they had previously placed in which location and in what order. 13 AD subjects individually underwent 16 trainings sessions across 8 weeks, and then were tested again on the scene-based episodic memory task, as well as the standardized tests.

METHODS

13 out of the 16 AD patients (11 female) from Experiment 1, who expressed a willingness to participate in additional sessions, received a total of 16 training sessions, two days a week, for 8 weeks. During each session, subjects completed an active navigation-like episodic memory training task involving the temporally sequential placement of objects in various locations around the room. The scene memory task and neuropsychological assessments which were identical to those used in the Experiment 1 were re-administered as post-tests (after the sixteenth training session was completed) to evaluate the effects of training.

Episodic memory training

The training task was designed to mimic episodic memory in real life situations. In this task, modeled after Mastrogiuseppe et al., 2019 [31], participants were asked to observe the experimenter navigating around the room hiding five out of seven objects of different shape, one-by-one into five of seven different boxes (demonstration phase). They were then asked to walk around the room and imitate the experimenter’s actions (encoding phase) as closely as possible, in sequentially hiding the five objects. Then, after a mental arithmetic interference task composing of sequential addition or subtraction for 3 min (interference phase), the patients were instructed to re-enact their own actions (retrieval phase) (Fig. 5). The logic behind this design was to make the memory autobiographical and navigational, in order to fully engage the hippocampus. During the retrieval phase, the experimenter promptly provided feedback on any errors made and allowed subjects to correct them. However, if the subjects continued to make mistakes, the experimenter gave them the correct answer.

Fig. 5

Active navigation-like episodic memory training in a real-world environment. A) Observation of the experimenter hiding objects one-by-one into boxes. B) Imitation of the experimenter’s actions. C) A mental arithmetic task as an interference phase for 3 min. D) Re-enactment of their own hiding sequence from the encoding phase.

The training took about 20 min and the subjects performed three rounds of training in a session with 5 min intervals in between. For each session, the experimenter randomized the target objects and locations, as well as their temporal order. We calculated the mean accuracy across the three rounds on whether the subjects selected the correct objects and put them in the correct locations, in the correct order.

6.2 Analysis

The Wilcoxon test was used to investigate changes in accuracies on the three conditions of the scene-based episodic memory task after 16 training sessions. The Friedman test was carried out to test for changes in performance in the training task during 16 training sessions, grouped into quartiles. Statistical significance was set at p < 0.05.

RESULTS

Subject characteristics

The number of subjects with AD in the GDS 4, 5, and 6 was 5 (3 female), 5 (5 female), and 3 (3 female), respectively (Table 5). The average of the GDS level of the AD group was 4.85, indicating that the AD patients, as a group, suffered from mild dementia to moderate dementia.

Table 5

Detailed characteristics of older adults with AD, according to GDS levels

Characteristics	GDS 4 (n = 5)	GDS 5 (n = 5)	GDS 6 (n = 3)
Age (y)	84.00 (3.36)	84.33 (2.33)	82.33 (2.32)
Sex (M/F)	2/3	0/5	0/3
Education (y)	7.85 (6.17)	4.00 (4.89)	2.00 (3.46)

Average education level of subjects in GDS 5 and 6 groups was quite low since majority of them had no education experience. GDS, Global Deterioration Scale.

Effects of episodic memory training

After the 16 training sessions, there was a significant improvement in accuracy on the Where condition (Z₍₁₂₎ = 2.179, p = 0.029), to performance above chance, but no improvement in the What (Z₍₁₂₎ =0.690, p = 0.490) and the When conditions (Z₍₁₂₎ =0.707, p = 0.480) (Fig. 6). Specifically, there was a significant improvement in accuracy on the Where condition in the AD patients with GDS 4 but not in patients with GDS 5 or 6 (Table 6), suggesting that spatial location memory could be enhanced by our active navigation-like episodic memory training in AD patients but only at the early stages of its progression.

Fig. 6

Effects of episodic memory training on the scene-based cognitive task, as shown by performance on each condition before and after the training (Wilcoxon test: see the asterisk between the bars). Only spatial memory (Where) performance improved (Scene-based Episodic Memory Task) and rose to above chance level (Scene-based Episodic Memory Task), while performance on the object (What) and temporal (When) memory remained the same (Wilcoxon test: see the asterisks above the bars).

Table 6

Effects of active episodic memory training on the three memory conditions in AD subjects (N = 13)

Episodic scene task		GDS	Pre test	Post test	Z
Accuracy	What	4 (n = 5)	66.00 (8.964)	70.00 (12.24)	0.552
		5 (n = 5)	68.00 (8.36)	70.00 (7.07)	0.379
		6 (n = 3)	46.67 (15.27)	46.67 (11.54)	1.000
	Where	4 (n = 5)	50.00 (10.00)	62.00 (8.36)	2.121^*
		5 (n = 5)	50.00 (12.24)	52.00 (8.36)	0.577
		6 (n = 3)	53.33 (20.81)	60.00 (10.00)	0.414
	When	4 (n = 5)	44.00 (5.47)	42.00 (4.47)	0.577
		5 (n = 5)	40.00 (7.07)	44.00 (5.47)	1.414
		6 (n = 3)	40.00 (0.00)	43.33 (11.54)	0.564
Neuropsychological assessment
MMSE-K			21.15 (4.29)	21.31 (3.37)	0.443
SVLT			32.46 (8.68)	34.15 (6.96)	2.109^*
K-BNT			42.46 (8.12)	42.38 (7.81)	0.182

Shown are mean values (standard deviation). ^*p < 0.05. GDS, Global Deterioration Scale.

On the other hand, in the neuropsychological assessments, there was a significant improvement in the SVLT (Z₍₁₂₎ = 2.109, p = 0.035) but not in the MMSE-K, the K-BNT (Z₍₁₂₎ = 0.443, p > 0.05; Z₍₁₂₎ = 0.182, p > 0.05, respectively) (Table 6). Given that repeated verbal learning tests like the SVLT with same forms often show practice effects [32 –34], we could not determine if our training is effective in verbal episodic memory.

We also evaluated performance on the training task itself, with the 16 sessions divided up into quarters: Although accuracy on identifying the objects and their locations tended to improve when not considering their temporal order, each quarter of the training period nevertheless showed a gradual increase in overall episodic memory accuracy (including temporal order). In particular, there was a significant statistical difference in overall accuracies between the 2^nd and the 3^rd quarters (Z₍₁₂₎ = 2.224, p = 0.026) and between the 3^rd and the 4^th quarters (Z₍₁₂₎ = 2.056, p = 0.040) (Fig. 7).

Fig. 7

Results of episodic memory training by sessions. A, B) Relatively high accuracy on remembering object identity and location compared to their temporal order. C) Significant improvement in overall performance between the 2nd and the 3rd quarters and between the 3rd and the 4th quarters (based on performance of episodic memory training, ^*p < 0.05).

EXPERIMENT 3

Given the novelty of our scene-based episodic memory task, it is important to understand it in further detail, especially regarding how younger and older healthy adults perform in the task. Experiment 3a was designed to identify how normal aging affects each component of episodic memory and to see if there is a difference in performance among the components (What, Where, and When) in normal aging. In Experiment 3b, we tested younger adults in their twenties.

METHODS

Subjects

Experiment 3a

42 healthy older adults (29 female) including the age-matched subjects in the Experiment 1 participated in Experiment 3a. Subjects were recruited from the local senior center, and they completed a medical questionnaire to verify that they had no previous neurological or psychiatric disorders and had not undergone any kind of treatments that would affect their cognition. They were divided into two age groups (65–75 years old group, N = 19; 76–85 years old group, N = 23) (Table 1).

Experiment 3b

51 healthy younger (27 female) adults in their twenties participated in Experiment 3b; they were recruited from the university campus (Korea Advanced Institute of Science and Technology, Daejeon, Korea) (Table 1).

Design

Healthy aging subjects (Experiment 3a) performed the scene-based episodic memory task and the neuropsychological assessment as did the patients with AD. Due to a ceiling effect, healthy younger adults (Experiment 3b) did not perform the neuropsychological assessments and only completed the scene-based episodic memory task.

Analysis

The Friedman test was conducted to test for differences in accuracy among the conditions (What, Where, and When), and the Wilcoxon test was conducted for pairwise comparisons across the conditions. We compared accuracy on each condition with a chance level of 50%(Wilcoxon test) and tested for sex differences in younger adults in their twenties using the Mann-Whitney test. Statistical significance was set to p < 0.05. The only group comparisons performed here was in Experiment 3a, in which the elderly subjects were divided into two age groups: 65–75 years old and 76–85 years old. We chose this division in age group both in accordance with the fact that many cognitive functions have been found to be well-preserved prior to the mid-seventies [33] and to balance the number of the subjects across groups.

RESULTS

Subject characteristics

Experiment 3a

There was no significant difference in education levels between 65–75-year-old and 76–85-year-old groups (U_(1,40) = 1.194, p > 0.05). There was also no significant difference in most of the standardized scores between the 65–75-year-old group and the 75–86-year-old group (MMSE-K: U_(1,40) = 0.235, p > 0.05; SVLT: U_(1,40) = 1.683, p > 0.05; K-BNT: U_(1,40) = 0.385, p > 0.05) (Table 1). However, age did significantly correlate with only the SVLT scores (r = –0.402, p < 0.01). These findings indicate that standardized test scores (other than perhaps the SVLT) might not be sensitive enough to pick up cognitive decline in the age groups that we tested.

Experiment 3b

Age and education levels of the 20–28-year-old group did not significantly differ between males and females (Table 2).

Characteristics of episodic memory in healthy subjects

Experiment 3a

As in Experiment 1, healthy aging subjects in both age groups performed above chance in all three conditions: What condition (65–75 years: Z₍₁₈₎ = 3.787, p < 0.001; 76–85 years: Z₍₂₂₎ = 3.345, p = 0.001), the Where condition (65–75 years: Z₍₁₈₎ = 3.771, p <0.001; 76–85 years old: Z₍₂₂₎ = 4.003, p < 0.001), and the When condition (65–75 years: Z₍₁₈₎ = 3.005, p = 0.003 76–85 years: Z₍₂₂₎ = 1.844, p = 0.045) (Fig. 8).

Fig. 8

Changes in accuracy on the three memory conditions across all four groups (younger adults in their 20 s, 65–75 years, 76–85 years, and AD).

In the episodic memory task, there were significant differences in accuracy among the memory components in both older age groups (65–75 years: χ²₍₁₈₎ = 10.233, p < 0.01; 76–85 years: χ²₍₂₂₎ = 5.040, p < 0.05). Pairwise comparisons revealed that accuracy on the When condition was lower than that on the What (65–75 years: Z₍₁₈₎ = 3.097, p < 0.01; 76–85 years: Z₍₂₂₎ = 2.093, p < 0.05) and the Where conditions (65–75 years: Z₍₁₈₎ = 1.981, p < 0.05, 76–85 years: Z₍₂₂₎ = 2.313, p < 0.05), but that there were no differences between the What and the Where condition (65–75 years: Z₍₁₈₎ = 1.331, p > 0.05, 76–85 years: Z₍₂₂₎ = 0.690, p > 0.05) (Table 7).

Table 7

Comparisons of accuracy among the three memory conditions in healthy subjects

Age range	What ^a	Where ^b	When ^c	H	Pairwise comparisons
66–75	74.21 (9.61)	69.47 (13.11)	62.10 (11.85)	10.223^**	a, b > c
76–85	66.08 (14.69)	64.34 (8.95)	56.08 (14.37)	5.040^*	a, b > c
20–28	82.94 (14.60)	78.23 (13.81)	67.05 (12.37)	37.789^***	a,b > c

^*p < 0.05, ^**p < 0.01, ^***p < 0.001.

Comparing the two age groups, there was a significant difference in accuracy on the What condition (U_(1,40) = 2.005, p < 0.05) but not on the Where (U_(1,40) = 0.927, p > 0.05) and the When conditions (U_(1,40) = 1.410, p > 0.05) (Table 2), implying that although memory performance does decline in normal aging, only performance in the What condition was significantly different between subjects 65–75 years old and 75–85 years old. These findings indicate that while temporal order memory is the hardest for the subjects in general, in contrast with the impairments seen in AD, normal aging affects object memory significantly more in the age groups tested. Finally, sex differences were not observed across all three conditions (Table 8).

Table 8

Sex differences in accuracy across the memory conditions in healthy subjects

Condition	Sex	66–75	76–85	20–28
What	Male	73.33 (10.32)	58.57 (19.51)	85.83 (80.37)
	Female	74.61 (9.6)	69.37 (11.23)	80.37 (14.80)
	U	33.000	37.000	248.500
Where	Male	70.00 (15.49)	65.71 (5.34)	73.75 (13.45)
	Female	69.23 (12.55)	63.75 (10.24)	82.22 (13.10)
	U	38.500	49.500	201.500^*
When	Male	63.33 (5.16)	55.71 (15.11)	67.08 (13.01)
	Female	61.53 (14.05)	56.20 (14.54)	67.03 (12.03)
	U	38.000	54.500	316.500

^*p < 0.05. 20–28-year-old females outperformed males in spatial location memory.

Experiment 3b

The pattern of performance across conditions in the older adults was found to be consistent in subjects in their twenties. There was a significant difference in accuracy among the conditions (χ²₍₅₀₎ =37.789, p < 0.001), with a significantly lower score on the When condition than the What (Z₍₅₀₎ = 5.176, p < 0.001) and the Where conditions (Z₍₅₀₎ = 3.848, p < 0.001), but no significant difference between the What and the Where condition (Z₍₅₀₎ = 1.862, p >0.05) (Table 7). Although younger adults in their twenties also found temporal order memory to be the most difficult of the three conditions, they were well over chance in all three conditions (What: Z₍₅₀₎ =6.170, p < 0.001; Where: Z₍₅₀₎ = 6.155, p < 0.001; and When: Z₍₅₀₎ = 5.857, p < 0.001) (Fig. 8). Interestingly, we found a significant sex difference in accuracy on the Where condition (U_(1,50) = 201.500, p = 0.017), with females outperforming males in spatial memory, but not on the What (U_(1,50) = 248,500, p = 0.142) and When conditions (U_(1,50) = 316.500, p = 0.883).

DISCUSSION

Scene-based episodic memory in Alzheimer’s disease and in healthy aging

Consistent with past studies showing that hippocampal atrophy in AD causes spatiotemporal memory decline [8, 27], our non-verbal scene-based episodic memory task revealed a significant impairment in the Where and When conditions in the AD patients, compared to the age-matched healthy controls. In contrast, we observed that the AD group showed similar accuracy on the What condition as the control group. This result can be interpreted based on past research indicating that the hippocampus is generally more correlated with spatiotemporal binding of memory than object or item memory [8, 35]; for instance, medial temporal lobectomy patients have been reported to have impaired spatial memory but intact object memory [27]. Other explanations may be that object memory can be represented in multiple ways—visually, semantically, verbally [36]—or, alternatively, that the What condition can be solved based on familiarity even with difficulties in hippocampal episodic recall.

Interestingly, in Experiment 1 and 2, AD patients showed worse performance in the When condition than chance (in both the pre- and post-test in Experiment 2). Our scene-based memory task was a forced-choice paradigm in that subjects were required to choose from two scenes the one that they thought they had seen first. This leaves open the possibility that familiarity-based judgement could have influenced temporal order memory assessment in our task, particularly in the AD group, who may have been inclined to choose the more recent scene that they recognized based on familiarity, even in the absence of episodic recall. In previous studies, spatiotemporal order memory was assessed by asking AD patients to remember the sequence of body turns or to trace a route for navigation in a virtual space, which may offer alternative ways to assess temporal order memory while minimizing familiarity effects [4, 25]. Nevertheless, while our scene-based episodic memory task may seem less ecologically valid compared to navigation tasks, it has practical advantages in assessing episodic memory in older subjects and in patient populations. Similar to tasks such as the 4-Mountains-Test of topological scenes [37], our simple scene-based task can be useful in terms of its implementation with higher numbers of trials, over repeated testing sessions on elderly subjects or patients, and in combination with neuro-imaging methods. Moreover, our scene-based task has the additional advantage that spatial and temporal memory could be separately evaluated in order to more closely and accurately track memory decline in AD.

Tests of healthy subjects in Experiment 3 showed that, while all memory components were better remembered by the younger adults in their twenties, they also found the When condition to be the most difficult. On the other hand, a comparison between the 65–75-year-old group and the 75–85-year-old group showed that only object memory was affected by normal aging in this age range. Interestingly, we found effects of normal aging on episodic memory using the scene-based task but not on the episodic memory test using verbal tasks (e.g., SVLT) and other neuropsychological assessments, indicating that our task was more sensitive to normal aging effects. This can be attributed to the fact that our task based on the scene construction theory is more linked to age-related non-verbal episodic memory decline than verbal memory [9]. Nonetheless, several studies have found that verbal episodic memory declines steadily in normal aging, in contrast with global cognition and verbal fluency from mid 60 s to mid 80 s [38]. This finding suggests that, despite the results in the present study, verbal episodic memory assessments which are sensitive enough to detect subtle changes may still be useful in monitoring memory decline in normal aging.

Although we did not see any sex differences in our healthy aging subjects, we did find a significant difference in spatial memory accuracy between sexes in young adults in their twenties, with females outperforming males. This was contrary to previous studies which report that males typically perform better than females on tasks of spatial abilities (e.g., [39]) even in older subjects [40]. However, the literature on sex differences in spatial memory is quite mixed, with some studies showing females showing higher performance than males in tasks requiring recall of object location [41 –43].

Complementary roles of spatial and temporal memory in AD classification

Temporal order memory, as measured by accuracy on the When condition, showed the highest Youden index of all the outcome measures, indicating that temporal order memory is the best factor to discriminate AD from normal aging. On the other hand, spatial memory showed the highest sensitivity, similar to the recall in the standardized verbal memory test (i.e., SVLT). Therefore, the high specificity for the temporal memory component and the high sensitivity for the spatial memory component may provide the optimal combination in differentiating AD patients from healthy elderly by minimizing false positives and false negatives respectively.

What could be the neural basis for such complementary roles of spatial and temporal representations? Associating temporally sequenced elements is crucial for the formation of episodic memory, and this process may depend on entorhinal inputs to the hippocampus [20, 44]. Neuropathological studies have shown that the entorhinal cortex is one of the earliest regions to be affected in early AD [22, 23]. One study indicated that AD patients were severely impaired in sequential memory in a task requiring them to encode locations in a specific temporal order, but were similar to control subjects when they were instructed to recall only spatial information [25]. Specifically, our results replicate and extend past findings by showing that assessing the temporal ordering of pairs of visual scenes (out of a sequence of images) may be sufficient in dissociating AD from normal aging [25].

Deficits in spatial memory, compared to temporal memory, have traditionally been more widely explored as useful early indicators of AD [4, 45]. This is supported by the present findings that spatial memory has highest sensitivity in our task. Meanwhile, despite past attempts, there are still no consistent results that one memory could be superior to the other in AD classification [4 , 46]. However, considering the potential importance of entorhinal cortex dysfunction in AD (as accompanying or even preceding hippocampal function), our study suggests that temporal memory relying on entorhinal cortex could be implemented as a useful behavioral marker of AD, without requiring an inconvenient and expensive brain-imaging procedure each time.

The effect of spatiotemporal episodic memory training in AD

Given that Where and When memory components of episodic memory are significantly impaired in AD, we explored a training paradigm that required subjects to walk around placing various objects in various locations around the room, in a particular order, in order to engage the hippocampus. Based on our data, the improvement was observed only in the Where condition in the mild AD patients. From these findings we may draw two conclusions: First, the results suggest that spatial memory performance can improve with training, which is consistent with previous studies reporting the effects of spatial navigation training [20, 27]. During the training, patients with AD had to navigate around the room to place objects in various spatial locations, which is a navigation task involving the hippocampus, particularly in the right hemisphere [14 , 27]. Therefore, although it is not clear whether the training improves hippocampal function specifically or a different compensatory mechanism (e.g., visual scene recognition), it is still important to note the significant enhancement in cognitive performance that comes with training in the early stages of AD. Prior studies have shown that cognitive training is more effective in improving cognitive function in the early stages of AD than in the later stages of AD due to the loss of neuroplasticity that comes with progression of the disease [47]. In fact, some studies have shown that severe AD patients show little improvement after cognitive intervention [48]. Nevertheless, developing effective intervention methods for moderate to severe AD patients is just as important as it is for the earlier stages, in order to modify their maladaptive behaviors and preserve their remaining capacities [48].

In contrast to spatial memory, our findings suggest that the temporal component of episodic memory is either not effectively improved using our specific training task and paradigm (e.g., frequency/length) or, alternatively, may be past the point of rescue in our sample of AD patients. For instance, given that entorhinal cortex could begin to dysfunction from the early stages of AD, the effects of the training may have been inadequate for patients with moderate to severe stage of AD in this study. Finally, our training also resulted in an improvement in verbal episodic memory (but not global cognitive function and verbal fluency), which may be brought about by either enhanced semantic memory or hippocampal function. Nevertheless, the neural correlates of our task and our training would need to be verified with neuroimaging data.

Conclusion

To conclude, the present study showed that temporal componential episodic memory test might be a useful tool in discriminating AD patients from age-matched subjects. Even outside of the clinic, temporal order memory of personal events can be self-monitored in one’s daily life instead of looking out for signs of spatial disorientation or taking verbal memory tests that take a lot of time and are often not very sensitive to the early effects of AD. Nevertheless, because spatial memory (Where condition) showed the highest sensitivity in this study, screening tools that include both spatial and temporal memory assessment may be optimal—both highly effective and accurate, while avoiding misdiagnosis of AD patients. We hope that these results will help guide the design of simple, scene-based diagnostic tests and cognitive training programs in the future that target hippocampal (and entorhinal) function from the earliest, perhaps even in the preclinical, stages of AD.

Footnotes

ACKNOWLEDGMENTS

We would like to thank the participants and their family members who made this research possible. This work has been supported by grants to S.A.L. funded by the National Research Foundation of Korea (NRF-2019R1F1A1062801, NRF-2020R1F1A1076033).

Authors’ disclosures available online ()

The supplementary material is available in the electronic version of this article: .

References

Scahill

, Schott

, Stevens

, Rossor

, Fox

(2002) Mapping the evolution of regional atrophy in Alzheimer’s disease: Unbiased analysis of fluid-registered serial MRI. Proc Natl Acad Sci U S A 99, 4703–4707.

Thompson

, Hayashi

, de Zubicaray

, Janke

, Rose

, Semple

, Herman

, Hong

, Dittmer

, Doddrell

, Toga

(2003) Dynamics of gray matter loss in Alzheimer’s disease. J Neurosci 23, 994–1005.

Rabinovici

, Furst

, O’Neil JP Racine

, Mormino

, Baker

, Chetty

, Patel

, Pagliaro

, Klunk

, Mathis

, Rosen

, Miller

, Jagust

(2007) 11C-PIB PET imaging in Alzheimer disease and frontotemporal lobar degeneration. Neurology 68, 1205–1212.

Plancher

, Tirad

, Gyselinck

, Nicolas

, Piolino

(2012) Using virtual reality to characterize episodic memory profiles in amnestic mild cognitive impairment and Alzheimer’s disease: Influence of active and passive encoding. Neuropsychologia 50, 592–602.

Pai

, Jacobs

(2004) Topographical disorientation in community- residing patients with Alzheimer’s disease. Int J Geriatr Psychiatry 19, 250–255.

Dubois

, Feldman

, Jacova

, Cummings

, Dekosky

, Barberger-Gateau

, Delacourte

, Frisoni

, Fox

, Galasko

, Gauthier

, Hampel

, Jicha

, Meguro

, O’Brien

, Pasquier

, Robert

, Rossor

, Salloway

, Sarazin

, De Souza

, Stern

, Visser

, Scheltens

(2010) Revising the definition of Alzheimer’s disease: A new lexicon. Lancet Neurol 9, 1118–1127.

Thompson

, Stopford

, Snowden

, Neary

(2005) Qualitative neuropsychological performance characteristics in frontotemporal dementia and Alzheimer’s disease. J Neurol Neurosurg Psychiatry 76, 920–927.

Tierney

, Yao

, Kiss

, McDowell

(2005) Neuropsychological tests accurately predict incident Alzheimer disease after 5 and 10 years. Neurology 64, 1853–1859.

Mullally

, Maguire

(2014) Memory, imagination, and predicting the future: A common brain mechanism. Neuroscientist 20, 220–234.

10.

Hassabis

, Kumaran

, Vann

, Maguire

(2007) Patients with hippocampal amnesia cannot imagine new experiences. Proc Natl Acad Sci USA 104, 1726–1731.

11.

Burgess

, Maguire

, O’Keefe

(2002) The human hippocampus and spatial and episodic memory. Neuron 35, 625–641.

12.

Tulving

(1993) What is episodic memory? Curr Dir Psychol Sci 2, 67–70.

13.

Tulving

(1972) Episodic and semantic memory. In Organization of memory, Tulving E, Donaldson W, eds. Academic Press, New York, pp. 381–403.

14.

Maguire

, Woollett

, Spiers

(2006) London taxi drivers and bus drivers: A structural MRI and neuropsychological analysis. Hippocampus 16, 1091–1101.

15.

Iglói

, Doeller

, Berthoz

, Rondi-Reig

, Burgess

(2010) Lateralized human hippocampal activity predicts navigation based on sequence or place memory. Proc Natl Acad Sci USA 107, 14466–14471.

16.

O’Keefe

(2014) Spatial cells in the hippocampal formation. Nobel Lecture on 7 December 2014 at Aula Medica, Karolinska Institutet in Stockholm.

17.

Wood

, Moodley

, Lever

, Minati

, Chan

(2016) Allocentric spatial memory testing predicts conversion from mild cognitive impairment to dementia: An initial proof-of-concept study. Front Neurol 7, 215.

18.

Lövdén

, Schaefer

, Noack

, Bodammer

, Kühn

, Heinze

, Düzel

, Bäckman

, Lindenberger

(2012) Spatial navigation training protects the hippocampus against age-related changes during early and late adulthood. Neurobiol Aging 33, 620.

19.

Serino

, Cipresso

, Morganti

, Riva

(2014) The role of egocentric and allocentric abilities in Alzheimer’s disease: A systematic review. Ageing Res Rev 16, 32–44.

20.

Suh

, Rivest

, Nakashiba

, Tominaga

, Tonegawa

(2011) Entorhinal cortex layer III input to the hippocampus is crucial for temporal association memory. Science 6061, 1415–1420.

21.

Velayudhan

, Proitsi

, Westman

, Muehlboeck

, Mecocci

, Vellas

, Tsolaki

, Kłoszewska

, Soininen

, Spenger

, Hodges

, Powell

, Lovestone

, Simmons

, dNeuroMed Consortium (2013) Entorhinal cortex thickness predicts cognitive decline in Alzheimer’s disease. J Alzheimers Dis 33, 755–766.

22.

Maass

, Lockhart

, Harrison

, Bell

, Mellinger

, Swinnerton

, Baker

, Rabinovici

, Jagust

(2018) Entorhinal tau pathology, episodic memory decline, and neurodegeneration in aging. J Neurosci 38, 530–543.

23.

Slotnick

(2010) Does the hippocampus mediate objective binding or subjective remembering? Neuroimage 49, 1769–1776.

24.

Scharfman

, Chao

(2013) The entorhinal cortex and neurotrophin signaling in Alzheimer’s disease and other disorders. Cogn Neurosci 4, 123–135.

25.

Bellassen

, Iglói

, de Souza

, Dubois

, Rondi-Reig

(2012) Temporal order memory assessed during spatiotemporal navigation as a behavioral cognitive marker for differential Alzheimer’s disease diagnosis. J Neurosci 32, 1942–1952.

26.

Fabiani

, Friedman

(1997) Dissociations between memory for temporal order and recognition memory in aging. Neuropsychologia 35, 129–141.

27.

Kyle

, Smuda

, Hassan

(2015) Roles of human hippocampal subfields in retrieval of spatial and temporal context. Behav Brain Res 278, 549–558.

28.

Mokhtari

, Aloulou

, Tiberghien

, Biswas

, Racoceanu

, Yap

(2012) New trends to support independence in persons with mild dementia–a mini-review. Gerontology 58, 554–563.

29.

Park

, Park

, Sohn

, Kim

, Park

(2017) Memory performance on the story recall test and prediction of cognitive dysfunction progression in mild cognitive impairment and Alzheimer’s dementia. Geriatr Gerontol Int 17, 1603–1609.

30.

Caglio

, Latini-Corazzini

, D’Agata

, Cauda

, Sacco

, Monteverdi

, Zettinm

, Duca

, Geminiani

(2012) . Virtual navigation for memory rehabilitation in a traumatic brain injured patient. Neurocase 18, 123–131.

31.

Mastrogiuseppe

, Bertelsen

, Bedeschi

, Lee

(2019) The spatiotemporal organization of episodic memory and its disruption in a neurodevelopmental disorder. Sci Rep 9, 18447.

32.

Woods

, Delis

, Scott

, Kramer

, Holdnack

(2006) The California Verbal Learning Test–second edition: Test-retest reliability, practice effects, and reliable change indices for the standard and alternate forms. Arch Clin Neuropsychol 21, 413–420.

33.

Benedict

(2005) Effect of using same-versus alternate-form memory tests during short-interval repeated assessments in multiple sclerosis. J Int Neuropsychol Soc 11, 727–736.

34.

Bird

, Papadopoulou

, Ricciardelli

, Rossor

, Cipolotti

(2003) Test-retest reliability, practice effects and reliable change indices for the recognition memory test. Br J Clin Psychol 42, 407–425.

35.

Small

, Dixon

, McArdle

(2011) Tracking cognition-health changes from 55 to 95 years of age. J Gerontol B Psychol Sci Soc Sci 66, 153–161.

36.

Pigott

, Milner

(1993) Memory for different aspects of complex visual scenes after unilateral temporal- or frontal-lobe resection. Neuropsychologia 31, 1–15.

37.

Barton

, Brewer

(2013) Visual working memory in human cortex. Psychology (Irvine) 4, 655–662.

38.

Chan

, Gallaher

, Moodley

, Minati

, Burgess

, Hartley

(2016) The 4 mountains test: A short test of spatial memory with high sensitivity for the diagnosis of pre-dementia Alzheimer’s disease. J Vis Exp 116, e54454.

39.

Tromp

, Dufour

, Lithfous

, Pebayle

, Després

(2015) Episodic memory in normal aging and Alzheimer disease: Insights from imaging and behavioral studies. Ageing Res Rev 24, 232–262.

40.

Yuan

, Kong

, Luo

, Zeng

, Lan

, You

(2019) Gender differences in large-scale and small-scale spatial ability: A systematic review based on behavioral and neuroimaging research. Front Behav Neurosci 13, 128.

41.

León

, Tascón

, Cimadevilla

(2016) Age and gender-related differences in a spatial memory task in humans. Behav Brain Res 306, 8–12.

42.

Silverman

, Eals

(1992) Sex differences in spatial abilities: Evolutionary theory and data. In The adapted mind: Evolutionary psychology and the generation of culture, Barkow JH, Cosmides L, Tooby J, eds. Oxford University Press, pp. 533–549.

43.

Barnfield

(1999) Development of sex differences in spatial memory. Percept Mot Skills 89, 339–350.

44.

(2014) Why women see differently from the way men see? A review of sex differences in cognition and sports. J Sport Health Sci 3, 155–162.

45.

Chao

, Huston

, Li

J-S

, Wang

A-L

, de Souza Silva

(2015) The medial prefrontal cortex—lateral entorhinal cortex circuit is essential for episodic-like memory and associative object-recognition. Hippocampus 26, 633–645.

46.

Ohno

, Chang

, Tseng

, Oakley

, Citron

, Klein

, Vassar

, Disterhoft

(2006) Temporal memory deficits in Alzheimer’s mouse models: Rescue by genetic deletion of BACE1. Eur J Neurosci 23, 251–260.

47.

Howett

, Castegnaro

, Krzywicka

, Hagman

, Marchment

, Henson

, Rio

, King

, Burgess

, Chan

(2019) Differentiation of mild cognitive impairment using an entorhinal cortex-based test of virtual reality navigation. Brain 142, 1751–1766.

48.

Herholz

, Herholz

(2013) Non-pharmacological interventions and neuroplasticity in early stage Alzheimer’s disease. Expert Rev Neurother 13, 1235–1245.

49.

Voisin

, Vellas

(2009) Diagnosis and treatment of patients with severe Alzheimer’s disease. Drugs Aging 26, 135–144.