Computerized Cognitive Screen (CoCoSc): A Self-Administered Computerized Test for Screening for Cognitive Impairment in Community Social Centers

Abstract

Background:

Computerized cognitive tests may serve as a preliminary, low-cost method to identify individuals with suspected cognitive impairment in the community.

Objective:

To develop a self-administered computerized test, namely the “Computerized Cognitive Screen (CoCoSc), Hong Kong version”, for screening of individuals with cognitive impairment (CI) in community settings.

Methods:

The CoCoSc is a 15-min computerized cognitive screen covering memory, executive functions, orientation, attention and working memory, and prospective memory administered on a touchscreen computer. Individuals with CI and cognitively normal controls were administered the CoCoSc and the Montreal Cognitive Assessment (MoCA). Validity of the CoCoSc was assessed based on the relationship with the MoCA using Pearson correlation. Receiver operating characteristic curve (ROC) was used to examine the ability of the CoCoSc to differentiate CI from controls.

Results:

Fifty-nine individuals with CI and 101 controls were recruited. Seventy-five (46.9%) participants had ≤6 years of education. Performance on the CoCoSc differed between normal and CI groups in both low and high education subgroups. Total scores of the CoCoSc and MoCA were significantly correlated (r = 0.71, p < 0.001). The area under ROC was 0.78, p < 0.001 for the CoCoSc total score in differentiating the CI group from the cognitively normal group. A cut-off of ≤30 on the CoCoSc was associated with a sensitivity of 0.78 and specificity of 0.69. The CoCoSc was well accepted by attendees of community social centers.

Conclusion:

The CoCoSc is a promising computerized cognitive screen for self-administration in community social centers. It is feasible for testing individuals with high or low education levels.

Keywords

Cognitive function mass screening mild cognitive impairment neuropsychology self-assessment

INTRODUCTION

Objective cognitive testing is an indispensable component for early identification of dementia. Computerized cognitive tests have been developed for cognitive screening or detailed cognitive profiling for over two decades [1]. To date, a number of computerized cognitive tests have been developed to serve different scopes of assessment. For example, the Cambridge Neuropsychological Test Automated Battery (CANTAB; http://www.cambridgecognition.com) [2], the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI; http://screen-inc.com) [3], the Automated Neuropsychological Assessment (ANAM; http://vistalifesciences.com/anam-intro) [4, 5], and the CNS Vital Signs (http://www.cnsvs.com) [6] are more lengthy (>30 min) test batteries capable of deriving a detailed cognitive profile; whereas some tests, such as the CANS-MCI, Cognifit Cognitive Assessment Battery (https://www.cognifit.com) [7], the Cognitive Stability Index (CSI) [8], and the Cogstate (http://cogstate.com) contain brief modules (∼15 min) for cognitive screening [9]. Computerized cognitive tests assess a multitude of cognitive domains including executive functions, attention, processing speed, memory, vigilance, visuospatial functions, learning and even social cognition. While it is unlikely that a computerized cognitive test will generate the rich information gained from the qualitative observations in a formal neuropsychological assessment performed by trained professionals, there are nevertheless a number of benefits associated with computerized cognitive testing. Psychometrically, computerized cognitive tests have improved reliability due to highly standardized administration and scoring. Moreover, some elderly persons may refuse formal cognitive testing due to anxiety toward healthcare professionals and these individuals may be more receptive to computer testing. Pertinent to the benefits of cognitive screening in the community, computerized cognitive tests can be made widely and readily accessible in non-medical settings such as community social centers. From the perspective of healthcare organizations, computerized cognitive tests that require a minimal demand for professional administration may save healthcare costs in the long term. In community settings, computerized cognitive testing may serve as a preliminary, first-tier, low-cost method to identify people with suspected cognitive impairment who can then be referred for further assessment by professionals. However, the feasibility of self-administered computerized cognitive testing in community settings has not been adequately addressed, especially for older adults with low education level and little prior experience with computers. This is an issue particularly relevant in populations where the education level is highly varied. The objective of this study is to develop a brief, self-administered computerized test, namely the “Computerized Cognitive Screen (CoCoSc), Hong Kong version”, for screening of cognitive impairment (CI) in individuals with varied levels of education attending community social centers.

METHODS

Participants

Participants were community-dwelling middle-aged and older adults. Inclusion criteria were 1) age ≥55 y; 2) Chinese ethnicity; and 3) adequate perceptual-motor ability to complete cognitive testing. Persons with uncontrolled psychiatric illnesses were excluded. Participants were classified as having CI if they 1) had subjective memory complaints, as defined as a positive response to the question “Are you concerned with your memory performance?” asked at the time of cognitive testing; and 2) scored ≤1.5 standard deviations below (i.e., 7th percentile) age- and education corrected cutoff on the Hong Kong version of the Montreal Cognitive Assessment (MoCA) [10, 11]. Participants aged below 65 y were classified using the 65–69 age bracket norms correcting for education. Participants who did not meet the criteria for CI were classified as cognitively normal.

Description of the CoCoSc

The CoCoSc was developed by an expert panel consisting of a clinical psychologist (A.W.), a team of biomedical and computer engineers (R.K.T and C.F.), and a cognitive neurologist (V.M.), as a collaborative project between the Division of Neurology, Department of Medicine and Therapeutics at the Chinese University of Hong Kong and the Division of Biomedical Engineering, Department of Health Technology and Informatics at the Hong Kong Polytechnic University. The CoCoSc is a fully automated, self-administered test which consists of six subtests covering five cognitive domains including learning and memory, executive functions, orientation, attention and working memory and time- and event-based prospective memory (PM). A Microsoft Windows-based PC with a touchscreen panel captures all responses on a forced-choice basis. No verbal or graphical inputs are required for qualitative analysis. Test instructions are presented as voice recordings in Chinese Cantonese language. Subtests are presented in a fixed order. Within each subtest, demonstration and practice items are presented prior to the test items to ensure that the examinee understands the instructions properly. The total scores of the CoCoSc range between 0 and 49. A minimal set of cognitive domains was selected based on reported associations with mild cognitive impairment (MCI) or ability to predict progression in preclinical AD [12 –18]. Test paradigms were chosen to enable scoring without the need for qualitative ratings or voice analysis. A brief description of the subtests is given asfollows:

1. Time-Based PM [12]: examinees are asked to perform a designated, un-cued target action (i.e. tapping a gas stove icon positioned on the left upper quadrant of the screen) after three minutes. A digital clock face showing the real time is displayed on the left side of the screen for participants to monitor the time. The clock disappears when the examinee successfully performs the target action or after 30 seconds if no action was executed.

2. Conflict Inhibition [13]: examinees are requested to tap on the screen twice in response to a single chime from the system and tap once in response to a double chime. Six practice trials are administered before the presentation of the actual test items.

3. Word List Learning [14, 15]: a list of six two-syllable concrete words are presented at a rate of one second per word for two times. After each presentation of the list, examinees are requested to verbally recall the words aloud in any order to facilitate memory encoding. Performance on this item is not scored.

4a. Orientation to Year, Month, Day and Day of the Week (4 items) [16]: for each question, examinees are presented with a set of selections that include one correct answer and one ‘Don’t know’ response intermixed with incorrect responses.

4b. Orientation to Place (5 items) [16]: examinees are to choose one correct answer among a choice of 5 for region, 7 for district (among 18 districts in Hong Kong), 7 for residential area, 7 for name of the social center where the test is administered, and 7 for floor of the location or the setting where the test is conducted. Each set of response options includes a choice of ‘Don’t know’ and ‘None of the above’ responses among the incorrect responses.

5. Attention and Working Memory (8 items) [17]: analogous to the spatial span paradigm from the Wechsler Memory Scale [19], examinees are to reproduce the order of the sequence in forward or backward fashion of where the bunnies appear among the 8 locations randomly dispersed on the screen.

6. Delayed Word List Memory [14, 15]: a list of 12 two-syllable words containing 6 target words and 6 distractors are presented. Examinees are asked to indicate whether the word presented is a target or distractor.

7. Event-Based PM [12]: examinees are requested to perform a target action of clicking on a telephone icon on the screen immediately upon the completion of the Delayed Word List Memory subtest.

Sample screen shots of the CoCoSc subtests are shown in Fig. 1.

Fig.1

Sample screen shots of CoCoSc subtests.

Study procedure

Participants were administered the CoCoSc in 12 local community social centers operated under the Hong Kong Housing Society. Volunteers, who were also regular visitors of the centers, were trained individually by professional staff members (e.g., social workers, occupational therapists, case managers) of the centers on preparing the participants to complete the CoCoSc. Prior to the administration of the CoCoSc, volunteers scanned the participant’s membership card using the built-in camera of the CoCoSc terminal computer to initiate the test. Each CoCoSc terminal was connected to a centralized computer server containing the personal and demographic data of the participants. The volunteers then explained to the participants how the CoCoSc would be administered, i.e., that instructions would be presented visually on the screen and verbally via a pair of headphones and answers are captured via the touchscreen panel. Volunteers instructed the participants to follow the test instructions and perform the tasks to the best of their ability. Special emphasis was given to the volunteers to refrain from helping the examinees (e.g. giving hints) or to give any feedback regarding the accuracy of participants’ responses on the test. Professional staff members initially supervised the volunteers and made necessary guidance until the volunteers demonstrated the expected standard. They were then observed and supervised regularly over the duration of the study. The MoCA was administered as part of a health check program in the community social centers. The order for the administration of the MoCA and CoCoSc was not fixed. The center professional staff explained to the participants the research purpose of community pilot testing with the CoCoSc and obtained verbal informed consent from the participants. The study was approved by the Hong Kong Housing Society and carried out in accordance with the principles stated in the Declaration of Helsinki.

Statistical analysis

Demographic data were compared using independent t test or χ² test as appropriate. Pearson correlation was used to examine the effects of age and education upon the CoCoSc total score. The effect size of the CoCoSc total and subtest scores were indicated by the Cohen’s d statistic. To examine the performance of the CoCoSc in participants with different education levels, separate group comparisons on the CoCoSc total and subtest scores were performed among participants with high (>6 y) or low (≤6 y) education level. To examine the concurrent validity of the CoCoSc, association between the total score of the CoCoSc and the MoCA were tested using Pearson correlation. Inter-scale agreement between the CoCoSc and MoCA total scores was examined using the Bland-Altman Method [20], which calculates the mean difference (i.e. bias) as well as the 95% limits of agreement (LoA) between the CoCoSc and MoCA total scores. The mean difference and width of 95% LoA were also calculated. To test the criterion validity of the CoCoSc, receiver operating characteristics curve (ROC) analysis was used to test the ability of the CoCoSc total score to differentiate the CI from the cognitively normal group. A cut-off at an optimal combination of sensitivity and specificity was derived for the CoCoSc total score.

RESULTS

One hundred and sixty participants, including 101 cognitively normal and 59 individuals with CI were recruited. The CoCoSc took approximately 15 min to complete. Group comparison of the demographic characteristics is shown in Table 1. Note that the overall education level was highly varied with 75 participants (46.9%) having ≤6 y of formal education. In terms of group difference, the CI group was older and less educated than the cognitively normal group. Compared to the cognitively normal group, the CI group scored lower in both MoCA and CoCoSc total scores as well as in the CoCoSc subtests in the whole sample and in both low and high education subgroups (Table 2).

Table 1

Group comparison on demographic characteristics

	Normal	CI
N	101	59
Age	70.5 (8.6)	78.2 (8.1)^**
Female	79 (78.2%)	45 (76.3%)
Low education (≤6 y)	30 (29.7%)	45 (76.3%)^**

^*p < 0.05; ^**p < 0.01.

Table 2

Mean (Standard deviations) and effect size measures for CoCoSc and MoCA performance in the whole sample and education subgroups

	Whole sample			High education		Low education
				(>6 y)		(≤6 y)
	Normal	CI	d	Normal	CI	Normal	CI
Time-based PM	1.5 (1.4)	0.8 (1.2)^**	0.51	1.5 (1.4)	1.2 (1.5)	1.3 (1.4)	0.6 (1.1)^*
Conflict Inhibition	6.4 (4.3)	3.1 (3.9)^**	0.8	6.6 (4.1)	3.7 (3.7)^*	6.0 (4.6)	3.0 (3.9)^**
Orientation	8.4 (1.0)	7.4 (2.0)^**	0.69	8.5 (0.9)	7.1 (2.7)	8.0 (1.2)	7.4 (1.8)
Attention and Working Memory	5.4 (2.9)	4.0 (2.9)^**	0.5	5.8 (2.7)	4.2 (2.6)^*	4.5 (3.3)	3.9 (3.1)
Delayed Word List Memory	8.0 (3.1)	4.9 (3.5)^**	0.94	8.3 (3.0)	4.1 (2.9)^**	7.2 (3.4)	5.2 (3.6)^*
Event-based PM	4.0 (1.7)	2.5 (2.2)^**	0.81	4.0 (1.6)	2.4 (2.4)^*	3.8 (1.9)	2.5 (2.1)^**
CoCoSc Total	33.7 (9.5)	22.6 (10.4)^**	1.12	34.9 (8.5)	22.8 (11.9)^**	30.9 (11.4)	22.6 (10.1)^**
MoCA Total	23.8 (4.1)	15.8 (4.0)^**	1.97	24.9 (2.8)	17.1 (3.6)^**	21.3 (5.4)	15.4 (4.1)^**

^*p < 0.05; ^**p < 0.01; d = Cohen’s d. CoCoSc, Computerized Cognitive Screening; MoCA, Hong Kong version of Montreal Cognitive Assessment; PM, prospective memory.

In the whole sample, the effect size for CoCoSc total and subtest scores, expressed in Cohen’s d, ranged between 0.50 and 1.12. The largest effect size was observed for the CoCoSc total score (Cohen’s d = 1.12), followed by Delayed Memory (0.94) and Event-based PM (0.81). Total score of the CoCoSc significantly correlated with age (r = –0.55, p < 0.001) and education (r = 0.43, p < 0.001). Total scores of the CoCoSc and MoCA were significantly correlated (r = 0.71, p < 0.001; Fig. 2). The area under ROC was 0.78, p < 0.001 for the CoCoSc total score (Fig. 3) in differentiating the CI from the cognitively normal participants. A cut-off score of ≤30 on the CoCoSc was associated with a sensitivity of 0.78 and specificity of 0.69.

Fig.2

Scatterplot showing relationship between total scores of CoCoSc and MoCA in the whole sample (r = 0.71, p < 0.001).

Fig.3

ROC curves analysis for total scores of CoCoSc in differentiating CI from cognitively normal participants. Area under ROC curve: CoCoSc: 0.78, p < 0.001.

Bland-Altman analysis for agreement between total scores of CoCoSc and MoCA showed a mean difference (calculated as CoCoSc total score minus MoCA total score) of 8.7 (7.4 to 10.0) points with 95% LoA between –7.4 (–9.7 to –5.2) to 24.9 (22.7 to 27.1) points. There appeared to be a slight proportional bias of the CoCoSc as the inter-scale difference increased with higher average score between CoCoSc and the MoCA (Fig. 4).

Fig.4

Bland-Altman Plot showing mean difference between CoCoSc and MoCA total scores as a function of the average of CoCoSc and total Scores. The mean inter-scale difference was 8.7 points with 95% LoA between –7.4 to 24.9 points (solid lines).

DISCUSSION

In this study, we showed that a computerized cognitive screen, namely the CoCoSc, is a promising tool for the screening of cognitively impaired individuals in community settings and is feasible for self-administration for individuals with different education levels.

We observed good effect size between the CI and the cognitively normal groups for the CoCoSc subtests and total scores. According to the interpretation of Cohen’s d statistic [21], the CoCoSc subtests exhibited medium (Time-based PM, Orientation, and Attention & Working Memory subtests) to large (Delayed Word List Memory, Event-based PM subtests) effect size. The CoCoSc total score showed a large effect size as well (d = 1.12). Importantly, significant group differences were found for the total and domain scores in both low and high education subgroups. It is interesting to note that the pattern of group differences on cognitive domains varied between the low and high education subgroups. This difference could be due to small subgroup sample size, or possibly, the effects of formal education upon cognitive processing and task strategies [22]. For example, formal education may influence the capacity of working memory [23]. Given the already lower performance in working memory in even normal individuals having lower education, group difference on tests of working memory, such as the digit span paradigm in the CoCoSc, might be rendered statistically non-significant. Nevertheless, we showed that the CoCoSc total score, which is the key index interpreted for the purpose of screening, remained significantly different between the normal and CI groups in both education subgroups.

In the ROC analysis, the AUC of the CoCoSc in the detection of CI was 0.78, which is comparable to that of the MoCA in differentiating patients with cerebral small vessel disease having similar demographic background (AUC 0.81) [10]. The sensitivity of 0.78 of the CoCoSc indicated that a cut-off score of ≤30 identified 78% of individuals with CI. On the other hand, this cut-off only had a moderate specificity of 0.69. As a screening test, the selection of cut-off score should favor sensitivity over specificity in order to minimize missing cognitively impaired individuals. The false positive cases can then be screened out at later stages with detailed neuropsychological assessment. Therefore, individuals screened positive on the CoCoSc should be considered for referral to formal assessment by professionals. Given the psychometric profile of the CoCoSc, it is best positioned as an initial screen for identifying individuals with possible cognitive impairment for detailed neuropsychological examination by a professional at a later stage.

We showed that the agreement between the CoCoSc and the MoCA differed as a function of cognitive abilities as the Bland-Altman analysis revealed that the difference between CoCoSc and MoCA became larger and more positive in individuals with better cognitive abilities, as measured as the average between the total scores on the CoCoSc and MoCA. Clinically, although it may not be a significant issue for cross-sectional measurements, the intra-scale difference in the CoCoSc and MoCA, such as that obtained in repeated testing, should not be interpreted as equivalent across the two scales.

A number of advantages are associated with computerized cognitive screening in community setting. First and most importantly, it makes available to the general public a cognitive screen that is easily and readily accessed. Second, for some individuals, interacting with a computer feels less threatening than a formal clinical assessment. According to the ‘foot-in-the-door’ technique in social psychology, individuals may actually become more receptive to formal clinical assessment and diagnostic procedures after having the experience of cognitive testing with a computer panel [24]. Moreover, despite being a computerized test, our study showed that the CoCoSc was well accepted by individuals with low education level having no or little prior experience with computers. According to a survey published in 2015, only 18% of older adults in Hong Kong used computers [25] and computer literacy would be expected to be even lower among those with less education. We showed that the CoCoSc is feasible for even illiterate individuals when clear instructions are given at the outset of testing. Note that trained volunteers who are themselves visitors in the community centers are adequate for preparing examinees for the CoCoSc, and, therefore, there is minimal demand for in-house professional staff to administer the CoCoSc. All in all, it is expected that these benefits add up to improve the identification of cognitively impaired persons in the community.

There are strengths and limitations of the CoCoSc over other computerized cognitive tests. Most importantly, the CoCoSc is currently shown to be feasible for individuals with low education level. This benefit is important given the overall low education level among older adults in many parts of the world. Also, as a cognitive test for use in community settings, the CoCoSc can be self-administered and it takes only 15 minutes to complete. Moreover, a unique feature of the CoCoSc that is not shared by other common cognitive screening tests is the inclusion of PM subtests. PM refers to the cognitive process of forming intentions and accurately realizing such intentions in some designated times in the future and is commonly referred to as the ability of “remembering to remember” [26]. Failure of PM may be attributed to dysfunction in one or more cognitive domains, including attention, episodic memory, and executive monitoring [26]. Decline in PM has been observed in MCI and it predicts impairment in daily functioning such as poor financial and medication management as well as a poor self-perceived quality of life in various medical conditions [27 –29]. A limitation of the CoCoSc is that as a brief cognitive screen, it is not designed for deriving a detailed cognitive profile which is possible with more extensive batteries. Another limitation of the CoCoSc is that the delayed memory performance is assessed using a multiple-choice recognition paradigm rather than a free recall paradigm. However, it can be argued that poor performance on a delayed recognition test may indeed reflect deficits in memory encoding and consolidation, which are early cognitive markers of Alzheimer’s Disease [30, 31]. Moreover, cognitive domains included in the CoCoSc are by no means exhaustive, for example, it lacks items to measure reaction time, problem-solving, and social cognition that are included in other computerized cognitive tests. Nonetheless, as a brief cognitive screen, the CoCoSc performed well in differentiating individuals with cognitive impairment from normal controls with a minimal set ofitems.

There are several limitations of this study. First of all, the sample size of the CI group was small. However, even with this small sample size, the CoCoSc differentiated the CI from the cognitively normal group with large effect size and good sensitivity. Second, test-retest reliability of the CoCoSc was not examined in this study.

In conclusion, we showed the CoCoSc is a promising tool for screening of cognitively impaired individuals in community settings and is feasible for self-administration for individuals with different education levels.

Footnotes

ACKNOWLEDGMENTS

We express gratitude for the Hong Kong Housing Society Elderly Resources Center for providing funds to support the initial development of the CoCoSc software. We also thank Dr. Bonnie Lam, Post-Doctoral Fellow, Department of Medicine & Therapeutics, The Chinese University of Hong Kong, for proofreading the manuscript.

Authors’ disclosures available online ().

References

Wild

, Howieson

, Webbe

, Seelye

, Kaye

(2008) Status of computerized cognitive testing in aging: A systematic review. Alzheimers Dement 4, 428–437.

Robbins

, James

, Owen

, Sahakian

, McInnes

, Rabbitt

(1994) Cambridge Neuropsychological Test Automated Battery (CANTAB): A factor analytic study of a large sample of normal elderly volunteers. Dementia 5, 266–281.

Tornatore

, Hill

, Laboff

, McGann

(2005) Self-administered screening for mild cognitive impairment: Initial validation of a computerized test battery. J Neuropsychiatry Clin Neurosci 17, 98–105.

Levinson

, Reeves

, Watson

, Harrison

(2005) Automated neuropsychological assessment metrics (ANAM) measures of cognitive effects of Alzheimer’s disease. Arch Clin Neuropsychol 20, 403–408.

Kabat

, Kane

, Jefferson

, DiPino

(2001) Construct validity of selected Automated Neuropsychological Assessment Metrics (ANAM) battery measures. Clin Neuropsychol 15, 498–507.

Gualtieri

, Johnson

(2005) Neurocognitive testing supports a broader concept of mild cognitive impairment. Am J Alzheimers Dis Other Demen 20, 359–366.

Peretz

, Korczyn

, Shatil

, Aharonson

, Birnboim

, Giladi

(2011) Computer-based, personalized cognitive training versus classical computer games: A randomized double-blind prospective trial of cognitive stimulation. Neuroepidemiology 36, 91–99.

Erlanger

, Kaushik

, Broshek

, Freeman

, Feldman

, Festa

(2002) Development and validation of a web-based screening tool for monitoring cognitive status. J Head Trauma Rehabil 17, 458–476.

Hammers

, Spurgeon

, Ryan

, Persad

, Barbas

, Heidebrink

, Darby

, Giordani

(2012) Validity of a brief computerized cognitive screening test in dementia. J Geriatr Psychiatry Neurol 25, 89–99.

10.

Wong

, Xiong

, Kwan

, Chan

, Lam

, Wang

, Chu

, Nyenhuis

, Nasreddine

, Wong

, Mok

(2009) The validity, reliability and clinical utility of the Hong Kong Montreal Cognitive Assessment (HK-MoCA) in patients with cerebral small vessel disease. Dement Geriatr Cogn Disord 28, 81–87.

11.

Wong

, Law

, Liu

, Wang

, Lo

, Lau

, Wong

, Mok

(2015) Montreal Cognitive Assessment: One cutoff never fits all. Stroke 46, 3547–3550.

12.

Costa

, Perri

, Serra

, Barban

, Gatto

, Zabberoni

, Caltagirone

, Carlesimo

(2010) Prospective memory functioning in mild cognitive impairment. Neuropsychology 24, 327–335.

13.

Wylie

, Ridderinkhof

, Eckerle

, Manning

(2007) Inefficient response inhibition in individuals with mild cognitive impairment. Neuropsychologia 45, 1408–1419.

14.

Greenaway

, Lacritz

, Binegar

, Weiner

, Lipton

, Munro Cullum

(2006) Patterns of verbal memory performance in mild cognitive impairment, Alzheimer disease, and normal aging. Cogn Behav Neurol 19, 79–84.

15.

Ribeiro

, Guerreiro

, De Mendonca

(2007) Verbal learning and memory deficits in Mild Cognitive Impairment. J Clin Exp Neuropsychol 29, 187–197.

16.

Nasreddine

, Phillips

, Bedirian

, Charbonneau

, Whitehead

, Collin

, Cummings

, Chertkow

(2005) The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53, 695–699.

17.

Saunders

, Summers

(2010) Attention and working memory deficits in mild cognitive impairment. J Clin Exp Neuropsychol 32, 350–357.

18.

Ritchie

, Ropacki

, Albala

, Harrison

, Kaye

, Kramer

, Randolph

, Ritchie

(2017) Recommended cognitive outcomes in preclinical Alzheimer’s disease: Consensus statement from the European Prevention of Alzheimer’s Dementia project. Alzheimers Dement 13, 186–195.

19.

Wechsler

(1997) Wechsler Memory Scale: Manual, Psychological Corporation, San Antonio, Texas.

20.

Bland

, Altman

(1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1, 307–310.

21.

Cohen

(1988) Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum Associates, Hillsdale, N.J.

22.

Ardila

, Bertolucci

, Braga

, Castro-Caldas

, Judd

, Kosmidis

, Matute

, Nitrini

, Ostrosky-Solis

, Rosselli

(2010) Illiteracy: The neuropsychology of cognition without reading. Arch Clin Neuropsychol 25, 689–712.

23.

Gathercole

, Pickering

, Knight

, Stegmann

(2004) Working memory skills and educational attainment: Evidence from national curriculum assessments at 7 and 14 years of age. Appl Cogn Psychol 18, 1–16.

24.

Freedman

, Fraser

(1966) Compliance without pressure: The foot-in-the-door technique. J Pers Soc Psychol 4, 195–202.

25.

Kwong

(2015) Digital divide: Computer and Internet use by elderly people in Hong Kong. Asian J Gerontol Geriatr 10, 5–9.

26.

Ellis

(1996) Prospective memory or the realisation of delayed intentions: A conceptual framework for research. Prospective memory: Theory and applications, Brandimonte

, Einstein

, McDaniel

, eds. Lawrence Erlbaum Associates, New Jersey, pp. 1–22.

27.

Thompson

, Henry

, Rendell

, Withall

, Brodaty

(2010) Prospective memory function in mild cognitive impairment and early dementia. J Int Neuropsychol Soc 16, 318–325.

28.

Pirogovsky

, Woods

, Vincent Filoteo

, Gilbert

(2012) Prospective memory deficits are associated with poorer everyday functioning in Parkinson’s disease. J Int Neuropsychol Soc 18, 986–995.

29.

Woods

, Moran

, Carey

, Dawson

, Iudicello

, Gibson

, Grant

, Atkinson

(2008) Prospective memory in HIV infection: Is “remembering to remember” a unique predictor of self-reported medication management? Arch Clin Neuropsychol 23, 257–270.

30.

Sperling

, Aisen

, Beckett

, Bennett

, Craft

, Fagan

, Iwatsubo

, Jack

Jr. , Kaye

, Montine

, Park

, Reiman

, Rowe

, Siemers

, Stern

, Yaffe

, Carrillo

, Thies

, Morrison-Bogorad

, Wagster

, Phelps

(2011) Toward defining the preclinical stages of Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 7, 280–292.

31.

Jack

Jr , Knopman

, Jagust

, Shaw

, Aisen

, Weiner

, Petersen

, Trojanowski

(2010) Hypothetical model of dynamic biomarkers of the Alzheimer’s pathological cascade. Lancet Neurol 9, 119–128.