Detection of Mild Cognitive Impairment in an At-Risk Group of Older Adults: Can a Novel Self-Administered Serious Game-Based Screening Test Improve Diagnostic Accuracy?

Abstract

Background:

Literature supports the use of serious games and virtual environments to assess cognitive functions and detect cognitive decline. This promising assessment method, however, has not yet been translated into self-administered screening instruments for pre-clinical dementia.

Objective:

The aim of this study is to assess the performance of a novel self-administered serious game-based test, namely the Virtual Supermarket Test (VST), in detecting mild cognitive impairment (MCI) in a sample of older adults with subjective memory complaints (SMC), in comparison with two well-established screening instruments, the Montreal Cognitive Assessment (MoCA) and the Mini-Mental State Examination (MMSE).

Methods:

Two groups, one of healthy older adults with SMC (N = 48) and one of MCI patients (N = 47) were recruited from day centers for cognitive disorders and administered the VST, the MoCA, the MMSE, and an extended pencil and paper neuropsychological test battery.

Results:

The VST displayed a correct classification rate (CCR) of 81.91% when differentiating between MCI patients and older adults with SMC, while the MoCA displayed of CCR of 72.04% and the MMSE displayed a CCR of 64.89%.

Conclusion:

The three instruments assessed in this study displayed significantly different performances in differentiating between healthy older adults with SMC and MCI patients. The VST displayed a good CCR, while the MoCA displayed an average CCR and the MMSE displayed a poor CCR. The VST appears to be a robust tool for detecting MCI in a population of older adults with SMC.

Keywords

Aging Alzheimer’s disease computers dementia diagnosis memory disorders mild cognitive impairment serious games subjective cognitive decline user-computer interface

INTRODUCTION

Dementia has been characterized as a global public health priority and its impact in individuals, families, and societies has been enormous [1]. Despite a marked reduction in the prevalence of dementia, mainly in high-income countries, the number of people with dementia is set to triple by 2050 [2]. Still dementia remains underdiagnosed with more than half of dementia cases being undiagnosed [3, 4]. Older adults recognize the importance of having their cognition checked but even in high income countries such as the United States, only a small percentage of older adults receive regular cognitive assessments [5].

The detection of cognitive impairment at its earliest stages is crucial as it allows for management of reversible causes and better disease management if the cause of impairment is a neurodegenerative disease. Furthermore, it allows for care planning and lifestyle changes that can enhance the quality of life of the patient and their family [6]. Mild cognitive impairment (MCI) represents a diagnostic entity that is often a precursor of dementia. MCI patients exhibit cognitive deficits as measured by neuropsychological tests in comparison to age and education matched cognitively healthy older adults. At the same time, they retain their ability to perform instrumental abilities of daily living and remain high-functioning and able to live autonomously [7, 8]. The detection of cognitive impairment at the MCI stage is clinically useful and allows for better communication between doctors, patients, and caregivers as it often acts as a starting point for a care and treatment plan [9]. It has been shown that non-pharmacological interventions at this stage can stabilize or even improve patients’ cognitive functioning [10].

There is no consensus concerning the effectiveness of population-wide screening for cognitive disorders particularly when taking into account the high costs associated with such endeavors [11, 12]. It has been shown, however, that early testing with computerized brief cognitive assessment tests can help in predicting future dementia [13]. At the same time, general practitioners (GPs), who are often the first point of contact between older adults and healthcare services, often fail to diagnose dementia and provide appropriate follow-up and referrals. Lack of knowledge about dementia services, limited consultation time, uncertainty about the procedure of diagnosis and disclosure, as well as pessimistic attitudes toward aging and cognition are some of the causes of dementia underdetection in general practices [14 –17]. Self-administered computerized screening tests have been proposed as a solution to this issue and implementation studies in primary care have assessed their feasibility and barriers to their implementation [18 –20]. The results of these studies are promising; however, this screening model still relies on administration in a healthcare setting and older adults have to be willing to visit such a setting and undergo what is essentially a standardized medical questionnaire/ testing procedure [21, 22].

Furthermore, brief cognitive tests are designed to differentiate between healthy older adults and MCI patients. The majority of studies validating those instruments and assessing their diagnostic utility are conducted in samples of healthy older adults with no cognitive concerns, and MCI or dementia patients. At the same time, people who present to memory clinics for cognitive testing often have subjective cognitive complaints. The term “subjective cognitive decline” (SCD) has been proposed to describe this stage where the person may be experiencing subjective cognitive decline which is not reflected in objective testing [23]. Older adults with SCD are at risk of progressing to MCI and dementia and they display a higher prevalence of positive biomarkers for amyloidosis and neurodegeneration compared to older adults without SCD [24 –26]. The definition of objective impairment is based on the deviation of a person’s performance from the norm of age and education-matched controls in standardized neuropsychological testing [7 , 27]. Thus, it is possible that a person’s cognitive functioning has declined but decline is subtle enough so that the person still scores within the normal range in neuropsychological testing [23]. Therefore older adults with SCD may score in the normal range; however, they may score lower than healthy older adults without SCD and therefore their difference in performance with MCI patients can be smaller than the difference in performance between MCI patients and healthy older adults without SCD [25]. It is imperative to understand if instruments designed to detect MCI among healthy older adults can still detect that condition in older adults who present with SCD as they represent the majority of people who visit memory clinics.

Lately serious games and virtual environments (with varying degrees of realism) have been used to assess cognitive performance and detect cognitive decline [28, 29]. They allow for ecologically valid assessment through complex metrics and can be configured to detect subtle changes in various facets of the user’s performance including spatial navigation which often declines with the onset of preclinical Alzheimer disease [29, 30]. Using serious games as screening tools can lead to the emergence of a new cognitive screening paradigm where screening becomes de-medicalized, is self-administered by older adults in their preferred setting, and is tied to an enjoyable activity. Indeed, efforts have been made to analyze longitudinal performance in a self-administered serious game that older adults used at home, over a period of time, to detect MCI [31].

The present study aims to assess the diagnostic utility of a fully self-administered, automated, serious game-based MCI screening routine, that can be completed in 30 minutes and does not rely on longitudinal assessment. Furthermore, this study aims to compare this novel instrument to two established brief cognitive tests that are often used for initial assessment, the Mini-Mental State Examination (MMSE) and the Montreal Cognitive Assessment (MoCA).

MATERIALS AND METHODS

Participants

Participants were recruited from a cohort of older adults from various socioeconomic backgrounds with subjective memory complaints (SMC) visiting the day centers of the Greek Association of Alzheimer’s Disease and Related Disorders (GAADRD), in Thessaloniki, Greece. Participants were recruited between April 2018 and April 2019. The study was approved by the GAADRD ethical committee and all participants provided informed consent. Diagnosis was conferred by a neurologist after a full neurological, neuropsychological and laboratory assessment. MCI diagnosis was based on Petersen criteria [7, 27]. Exclusion criteria were: diagnosis of dementia or another major neurological or psychiatric disorder, illiteracy, health issues such as motor, hearing and vision difficulties that could interfere with the use of the exercise, treatment with cholinesterase inhibitors or other drugs that could affect cognitive performance, alcoholism or drug abuse.

Demographic characteristics of participants are presented in Table 1. Mean age was 66.92 years ranging from 54 to 75 years. Subjects had a mean of 13.27 years of formal education ranging from 6 to 21 years of formal education. The majority of the participants were female (73 persons). The sample included 48 healthy older adults with SCD and 47 MCI patients. No statistically significant differences were observed between the SCD and MCI groups in age and education, while statistically significant differences in MMSE and MoCA scores were observed as expected.

Table 1

Demographic characteristics of participants

	SCD	MCI
Male/Female	10/38	12/35
Mean Age, y (Std. Err.)	65.96 (0.652)	67.89 (0.759)
Mean Education, y (Std. Err.)	13.67 (0.430)	12.87 (0.430)
Mean MMSE (Std. Err.)	28.88 (0.154)	28.04 (0.245)
Mean MoCA (Std. Err.)	27.98 (0.266)	25.34 (0.398)

SCD, subjective cognitive decline; MCI, mild cognitive impairment; Std. Err., standard error of mean.

Neuropsychological assessment

Participants were administered an extended neuropsychological test battery including the following cognitive tests: Ray Auditory Verbal Learning Test [32], a Greek version of the “FAS” verbal fluency test [33], Rey-Osterrieth Complex Figure Test [34], Rivermead Behavioural Memory Test [35], Test of Everyday Attention items 1, 4 & 6 [36], and Trail Making Test part B [37]. The battery also included the following functional scales: Functional Rating Scale for Symptoms of Dementia [38] and Functional Cognitive Assessment Scale [39]. Furthermore, the battery included the following measures of depression and anxiety: Geriatric Depression Scale [40] and Short Anxiety Screening Test [41]. This battery was used as part of the diagnostic procedure. Participants were also administered the MoCA [42] and MMSE [43] in order to compare the diagnostic performance of these tests to the diagnostic performance of the Virtual Supermarket Test (VST).

Serious game-based cognitive screening test

The VST has been developed through a collaboration of the Centre for Research and Technology Hellas/ Information Technologies Institute and the GAADRD. It is based on a shopping exercise which is essentially a simple virtual reality task with a low degree of immersion. It runs on tablet devices with Android operating system, and a 10-inch tablet is recommended for its administration. The application has been described in detail in previous studies that have established the diagnostic accuracy of its examiner-administered version and its concurrent validity with established pencil and paper neuropsychological tests [31 , 45].

The latest version of the VST comprises a fully automated, self-administered screening routine that can be completed in the span of 30 minutes. An interactive training session ensures that users are familiarized with the operation of the tablet and the various actions they would have to perform during testing. After completion of training, a test session is administered three times through an automated administration routine. The VST is designed to mimic one of the most common activities of daily living, daily shopping in a super market. During the test sessions, a shopping list is provided to the user who is allowed to navigate freely, buy the products they are instructed to buy, and proceed to pay at the till, by entering the correct amount. The application is aimed at activating a multitude of cognitive processes namely visual and verbal memory, executive function, attention, and spatial navigation with the emphasis placed on executive function. The need of simultaneous activation of different cognitive processes makes the program challenging enough to correspond to the ability of the target population while reducing ceiling effects. The latest version of the VST includes advanced navigation metrics with the virtual space divided into three zones (green, yellow, and red). Different zones represent different deviations from a pattern of optimal navigation for task completion.

Administration of the VST

Each participant was administered the VST in a quiet room of a day center of the GAADRD. A research assistant presented the participant with a tablet and entered their demographic data in the VST application. Participants were informed that they will be completing the screening routine on their own and that the research assistant will remain in the room to observe the administration. The research assistant did not provide any further assistance or instructions to the participants.

RESULTS

Data preparation and classification methodology

During the data analysis, we investigated a series of variables, which were calculated from the metrics that were recorded during the VST sessions. Several variables were calculated as the ratios or differences between the same type of variables from different trials, e.g., a variable expressing time spend in virtual space locations that are indicative of deviation from optimal trajectory as a fraction of time spent in all virtual space locations in a certain trial. Also, variables expressing average performance across trials were calculated, e.g., a variable expressing average time needed to complete a single test trial.

For optimal feature selection to solve our binary classification problem (SCD versus MCI) we applied the following methodology:

We performed Univariate feature selection, which works by selecting the best features based on univariate statistical tests. We kept the k highest scoring features using the ANOVA F-value metric.

We used Sequential Backward Floating Selection (SBFS) for the purpose of optimal feature selection [46, 47]. In SBFS, a subset of optimal features is selected among the k number of total features in such a way that, the classifier should give maximum accuracy on the selected features. We used Linear Discriminant Analysis (LDA) as classifier, one of the most widely used classification algorithms, which operates through finding linear combination of features to classify two or more classes [48]. This process has k iterations, each one reducing the number of features until the last iteration selects only one feature. The results and accuracy scores of LDA on all iterations of the SBFS method were stored.

We selected the first 10 subsets of variables sorted by their accuracy scores, in descending order, from step 2 and run LDA n times (n being the number of subjects), for each of the 10 subsets, by leaving one subject out in each iteration to validate the selection of variables. This process is called Leave One Out cross-validation. We measured the overall classification accuracy for each of the 10 subsets, that is the ratio of all correctly classified outcomes to the number of subjects.

Steps 1–3 were repeated starting with k = number of total variables to k = 2 with step = –1. We always stored the subset of variables scoring the highest accuracy in step 3.

The final set of features comprised variables expressing average time needed to complete the exercise, accuracy of navigation inside the virtual space – deviation from optimal trajectory, attention – mistakes conducted while completing the exercise, and learning – difference in performance between trials. A description of VST metrics and the 12 variables used for classification can be found in the Supplementary Material. Classification analysis was also conducted using the scores of the MMSE and MoCA tests and a combination of MMSE and MoCA.

Classification accuracy of the VST, MoCA, and MMSE

The VST displayed a correct classification rate (CCR) of 81.91% with a sensitivity of 76.27% and a specificity of 91.43% for MCI detection. The MoCA displayed a CCR of 72.04% with a sensitivity of 69.81% and a specificity of 75%. The MMSE displayed a CCR of 64.89% with a sensitivity of 63.64% and a specificity of 66.67%. A classifier utilizing both the MoCA and the MMSE was trialed but failed to improve the CCR of the MoCA. The CCR, sensitivity, and specificity of the VST, MoCA, and MMSE are presented in Table 2.

Table 2

Classification accuracy of the VST, the MoCA and the MMSE

	CCR	Sensitivity	Specificity
VST	81.91%	76.27%	91.43%
MoCA	72.04%	69.81%	75%
MMSE	64.89%	63.64%	66.67%

CCR, correct classification rate; VST, Virtual Supermarket Test; MoCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination.

DISCUSSION

There is a body of literature on the use of virtual shopping tasks/serious games for the assessment of cognitive functions and everyday functionality, and for the detection of cognitive decline [31 , 49–51]. Nevertheless, this innovative and engaging paradigm of cognitive assessment has not moved beyond research or clinical settings. The need for the presence of an examiner during administration is a barrier to its translation into applications that could allow at-risk groups to self-assess their functionality and be active participants in the management and monitoring of their brain health. The basic aim of this study was to assess whether a self-administered serious game-based paradigm has potential as a robust cognitive screening method aimed at those adults that are at risk for cognitive decline, namely older adults with SMC.

The VST displayed a good CCR, while the MoCA displayed an average CCR and the MMSE displayed a poor CCR. The VST differentiated between SCD and MCI with a CCR of 81.91%. Its CCR is significantly higher than that of the MoCA (which is often considered a gold standard screening test for MCI) in this sample, indicating that it can be a robust screening test [42 , 53]. At the same time, its CCR was also much higher than that of the MMSE which is known to perform worse than the MoCA and is not considered ideal for MCI detection [52, 53]. The CCR of the VST is in line with findings of previous VST studies [31 , 45]. The self-administered nature of the latest version of the VST has led to only a slight decrease of its CCR in comparison to previous examiner-administered versions.

It is worth noting that all tests in the present study were tasked with detecting MCI in an at-risk sample avoiding the usual comparison of MCI patients with healthy counterparts without SMCs. The need to focus on at-risk populations instead of “supernormal” healthy participants with no complaints has been highlighted in the assessment of the effectiveness of cognitive training interventions [54]. A similar approach could also benefit the field of cognitive screening as healthy older adults with no cognitive concerns are unlikely to seek screening. Memory clinics and primary care services are usually tasked with detecting objective cognitive decline among people who present with subjective cognitive concerns. It can be argued that the same group of people would be interested in using a self-administered screening test (like the VST), at home to assess their cognitive status. Furthermore, at the stage of SCD or MCI older adults retain the ability to operate a self-administered test and navigate inside a virtual environment. Both of these abilities can decline at the onset of dementia; therefore, it is imperative that self-administered screening tests are aimed at older adults with SCD or MCI.

At the same time there is uncertainty about how the screening accuracy of a self-administered test in a research setting translates into “real-world” screening accuracy [18 –20]. The absence of a research assistant who can verify that the examinee is focused and engaged with the task, and technical variability among tablet PC software and hardware (screen size and aspect radio, image quality, touch device sensitivity, computing power) can affect test results. Furthermore, disclosure of results in a meaningful way that is non-threatening to the examinee and promotes necessary follow-up and actions that are beneficial to their brain health is a huge and very delicate issue that must be dealt with appropriately [55 –57]. Even if all these issues are resolved, it is still unclear if receiving a first indication through this screening method will actually lead to follow-up, further testing, and seeking of appropriate care and interventions. Still, the results of this study provide the necessary support for implementation studies in “real world” environments which could answer these questions and assess the benefits of such a screening method for the individual and the healthcare system.

The strengths of this study include the thorough assessment of all participants prior to assignment to one of the two study groups. A full neurological, neuropsychological, and laboratory assessment was conducted and diagnosis was conferred by a neurologist. Furthermore, the VST has been used in previous studies and is known to be usable and accepted by both healthy older adults and MCI patients. Limitations of the study comprise the inclusion of significantly more female participants and the lack of test-retest reliability data for the VST. Future studies should focus on assessing the “real-world” utility of serious game-based self-screening applications. This should include assessment of their usability and the characteristics that make them easy to use and desirable. Furthermore, studies should pilot and test the integration of such applications in existing healthcare models and their effect in increasing timely diagnosis of cognitive disorders and decreasing the burden of healthcare services.

Conclusion

The VST displayed excellent performance in the detection of MCI in a sample of older adults with SMC. The MoCA displayed average classification performance and the MMSE displayed poor classification performance in the same sample. The self-administered format and excellent classification performance of the VST can enable older adults to self-screen for cognitive decline. Further studies are needed in order to assess the integration of the VST in healthcare systems and practices and its impact in promoting timely detection of cognitive disorders and appropriate follow-up.

Footnotes

ACKNOWLEDGMENTS

Stelios Zygouris is an Atlantic Fellow for Equity in Brain Health at the Global Brain Health Institute (GBHI) and is supported with funding from GBHI and the Alzheimer’s Association (GBHI_ALZ-18-541600). Stelios Zygouris also received a scholarship by the Robert Bosch Foundation Stuttgart within the Graduate Program People with Dementia in General Hospitals, located at the Network Aging Research (NAR), Heidelberg University, Germany. The authors would like to express their gratitude to all the study participants who gave generously of their time.

Authors’ disclosures available online ().

The supplementary material is available in the electronic version of this article: .

References

World Health Organization (2017), Global action plan on the public health response to dementia 2017-2025.

Alzheimer Europe (2019), Dementia in Europe Yearbook 2019.

Lang

, Clifford

, Wei

, Zhang

, Leung

, Augustine

, Danat

, Zhou

, Copeland

, Anstey

, Chen

(2017) Prevalence and determinants of undetected dementia in the community: A systematic literature review and a meta-analysis. BMJ Open 7, e011146.

Amjad

, Roth

, Sheehan

, Lyketsos

, Wolff

, Samus

(2018) Underdiagnosis of dementia: An observational study of patterns in diagnosis and awareness in US older adults. J Gen Intern Med 33, 1131–1138.

Alzheimer’s Association (2019) 2019 Alzheimer’s disease facts and figures. Alzheimers Dement 15, 321–387.

Morley

, Morris

, Berg-Weger

, Borson

, Carpenter

, del Campo

, Dubois

, Fargo

, Fitten

, Flaherty

, Ganguli

, Grossberg

, Malmstrom

, Petersen

, Rodriguez

, Saykin

, Scheltens

, Tangalos

, Verghese

, Wilcock

, Winblad

, Woo

, Vellas

(2015) Brain health: The importance of recognizing cognitive impairment: An IAGG Consensus Conference. J Am Med Dir Assoc 16, 731–739.

Petersen

, Roberts

, Knopman

, Boeve

, Geda

, Ivnik

, Smith

, Jack

Jr (2009) Mild cognitive impairment: Ten years later. Arch Neurol 66, 1447–1455.

Petersen

(2004) Mild cognitive impairment as a diagnostic entity. J Intern Med 256, 183–194.

Bertens

, Vos

, Kehoe

, Wolf

, Nobili

, Mendonça

, van Rossum

, Hort

, Molinuevo

, Heneka

, Petersen

, Scheltens

, Visser

(2019) Use of mild cognitive impairment and prodromal AD/MCI due to AD in clinical care: A European survey. Alzheimers Res Ther 11, 74.

10.

Rodakowski

, Saghafi

, Butters

, Skidmore

(2015) Non-pharmacological interventions for adults with mild cognitive impairment and early stage dementia: An updated scoping review. Mol Aspects Med 43–44, 38–53.

11.

Martin

, Kelly

, Khan

, Cullum

, Dening

, Rait

, Fox

, Katona

, Cosco

, Brayne

, Lafortune

(2015) Attitudes and preferences towards screening for dementia: A systematic review of the literature. BMC Geriatr 15, 66.

12.

Chambers

, Sivananthan

, Brayne

(2017) Is dementia screening of apparently healthy individuals justified? Adv Prev Med 2017, 9708413–9708413.

13.

Calvin

, Wilkinson

, Starr

, Sudlow

, Hagenaars

, Harris

, Schnier

, Davies

, Fawns-Ritchie

, Gale

, Gallacher

, Deary

(2019) Predicting incident dementia 3-8 years after brief cognitive tests in the UK Biobank prospective study of 500,000 people. Alzheimers Dement 15, 1546–1557.

14.

Turner

, Iliffe

, Downs

, Wilcock

, Bryans

, Levin

, Keady

, O’Carroll

(2004) General practitioners’ knowledge, confidence and attitudes in the diagnosis and management of dementia. Age Ageing 33, 461–467.

15.

Mitchell

, Meader

, Pentzek

(2011) Clinical recognition of dementia and cognitive impairment in primary care: A meta-analysis of physician accuracy. Acta Psychiatr Scand 124, 165–183.

16.

Koch

, Iliffe

, the E-EDp (2010) Rapid appraisal of barriers to the diagnosis and management of patients with dementia in primary care: A systematic review. BMC Fam Pract 11, 52.

17.

Giezendanner

, Monsch

, Kressig

, Mueller

, Streit

, Essig

, Zeller

, Bally

(2019) General practitioners’ attitudes towards early diagnosis of dementia: A cross-sectional survey. BMC Fam Pract 20, 65–65.

18.

Millett

, Naglie

, Upshur

, Jaakkimainen

, Charles

, Tierney

(2018) Computerized cognitive testing in primary care: A qualitative study. Alzheimer Dis Assoc Disord 32, 114–119.

19.

Tierney

, Charles

, Naglie

, Jaakkimainen

, Moineddin

(2017) The effects of computerized cognitive testing of older patients on primary care physicians’ approaches to care: A Canadian study. Alzheimer Dis Assoc Disord 31, 62–68.

20.

Tierney

, Naglie

, Upshur

, Moineddin

, Charles

, Liisa Jaakkimainen

(2014) Feasibility and validity of the self-administered computerized assessment of mild cognitive impairment with older primary care patients. Alzheimer Dis Assoc Disord 28, 311–319.

21.

Bradford

, Kunik

, Schulz

, Williams

, Singh

(2009) Missed and delayed diagnosis of dementia in primary care: Prevalence and contributing factors. Alzheimer Dis Assoc Disord 23, 306–314.

22.

Holsinger

, Boustani

, Abbot

, Williams

(2011) Acceptability of dementia screening in primary care patients. Int J Geriatr Psychiatry 26, 373–379.

23.

Molinuevo

, Rabin

, Amariglio

, Buckley

, Dubois

, Ellis

, Ewers

, Hampel

, Klöppel

, Rami

, Reisberg

, Saykin

, Sikkes

, Smart

, Snitz

, Sperling

, van der Flier

, Wagner

, Jessen

(2017) Implementation of subjective cognitive decline criteria in research studies. Alzheimers Dement 13, 296–311.

24.

Studart

, Nitrini

(2016) Subjective cognitive decline: The first clinical manifestation of Alzheimer’s disease? Dement Neuropsychol 10, 170–177.

25.

Koppara

, Wagner

, Lange

, Ernst

, Wiese

, König

H-H

, Brettschneider

, Riedel-Heller

, Luppa

, Weyerer

, Werle

, Bickel

, Mösch

, Pentzek

, Fuchs

, Wolfsgruber

, Beauducel

, Scherer

, Maier

, Jessen

(2015) Cognitive performance before and after the onset of subjective cognitive decline in old age. Alzheimers Dement (Amst) 1, 194–205.

26.

Mendonça

, Alves

, Bugalho

(2015) From subjective cognitive complaints to dementia: who is at risk? A systematic review. Am J Alzheimers Dis Other Demen 31, 105–114.

27.

Petersen

, Doody

, Kurz

, Mohs

, Morris

, Rabins

, Ritchie

, Rossor

, Thal

, Winblad

(2001) Current concepts in mild cognitive impairment. Arch Neurol 58, 1985–1992.

28.

Manera

, Ben-Sadoun

, Aalbers

, Agopyan

, Askenazy

, Benoit

, Bensamoun

, Bourgeois

, Bredin

, Bremond

, Crispim-Junior

, David

, De Schutter

, Ettore

, Fairchild

, Foulon

, Gazzaley

, Gros

, Hun

, Knoefel

, Olde Rikkert

, Phan Tran

, Politis

, Rigaud

, Sacco

, Serret

, Thümmler

, Welter

, Robert

(2017) Recommendations for the use of serious games in neurodegenerative disorders: 2016 Delphi Panel. Front Psychol 8, 1243.

29.

Zygouris

, Tsolaki

(2015) New technologies and neuropsychological evaluation of older adults: issues and challenges. In Handbook of Research on Innovations in the Diagnosis and Treatment of Dementia, Bamidis

, Tarnanas

, Hadjileontiadis

, Tsolaki

, eds. IGI Global, USA, pp. 1–17.

30.

Coughlan

, Laczó

, Hort

, Minihane

A-M

, Hornberger

(2018) Spatial navigation deficits — overlooked cognitive marker for preclinical Alzheimer disease? Nat Rev Neurol 14, 496–506.

31.

Zygouris

, Ntovas

, Giakoumis

, Votis

, Doumpoulakis

, Segkouli

, Karagiannidis

, Tzovaras

, Tsolaki

(2017) A preliminary study on the feasibility of using a virtual reality cognitive training application for remote detection of mild cognitive impairment. J Alzheimers Dis 56, 619–627.

32.

Messinis

, Tsakona

, Malefaki

, Papathanasopoulos

(2007) Normative data and discriminant validity of Rey’s Verbal Learning Test for the Greek adult population. Arch Clin Neuropsychol 22, 739–752.

33.

Kosmidis

, Vlahou

, Panagiotaki

, Kiosseoglou

(2004) The verbal fluency task in the Greek population: normative data, and clustering and switching strategies. J Int Neuropsychol Soc 10, 164–172.

34.

Duley

, Wilkins

, Hamby

, Hopkins

, Burwell

, Barry

(1993) Explicit scoring criteria for the Rey-Osterrieth and Taylor complex figures. Clin Neuropsychol 7, 29–38.

35.

Efklides

, Yiultsi

, Kangellidou

, Kounti

, Dina

, Tsolaki

(2002) Wechsler Memory Scale, Rivermead Behavioral Memory Test, and Everyday Memory Questionnaire in healthy adults and Alzheimer’s patients. Eur J Psychol Assess 18, 63–77.

36.

Robertson

, Ward

, Ridgeway

, Nimmo-Smith

(1996) The structure of normal human attention: The Test of Everyday Attention. J Int Neuropsychol Soc 2, 525–534.

37.

Zalonis

, Kararizou

, Triantafyllou

, Kapaki

, Papageorgiou

, Sgouropoulos

, Vassilopoulos

(2008) A normative study of the Trail Making Test A and B in Greek adults. Clin Neuropsychol 22, 842–850.

38.

Hutton

(1990) Alzheimer’s disease. In Conn’s Current Therapy, Rakel

, ed. WB Saunders, Philadelphia, pp. 780.

39.

Kounti

, Tsolaki

, Kiosseoglou

(2006) Functional cognitive assessment scale (FUCAS): a new scale to assess executive cognitive function in daily life activities in patients with dementia and mild cognitive impairment. Hum Psychopharmacol 21, 305–311.

40.

Fountoulakis

, Tsolaki

, Iacovides

, Yesavage

, O’Hara

, Kazis

, Ierodiakonou

(1999) The validation of the short form of the Geriatric Depression Scale (GDS) in Greece. Aging Clin Exp Res 11, 367–372.

41.

Grammatikopoulos

, Sinoff

, Alegakis

, Kounalakis

, Antonopoulou

, Lionis

(2010) The Short Anxiety Screening Test in Greek: translation and validation. Ann Gen Psychiatry 9, 1.

42.

Nasreddine

, Phillips

, Bedirian

, Charbonneau

, Whitehead

, Collin

, Cummings

, Chertkow

(2005) The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53, 695–699.

43.

Fountoulakis

, Tsolaki

, Chantzi

, Kazis

(2000) Mini Mental State Examination (MMSE): a validation study in Greece. Am J Alzheimers Dis Other Demen 15, 342–345.

44.

Zygouris

, Giakoumis

, Votis

, Doumpoulakis

, Konstantinos

, Segkouli

, Karagiannidis

, Tzovaras

, Tsolaki

(2015) Can a virtual reality cognitive training application fulfill a dual role? Using the Virtual Super Market cognitive training application as a screening tool for mild cognitive impairment. J Alzheimers Dis 44, 1333–1347.

45.

Eraslan Boz

, Limoncu

, Zygouris

, Tsolaki

, Giakoumis

, Votis

, Tzovaras

, Öztürk

, Yener

(2020) A new tool to assess amnestic mild cognitive impairment in Turkish older adults: virtual supermarket (VSM). Neuropsychol Dev Cogn B Aging Neuropsychol Cogn 27, 639–653.

46.

Pudil

, Novovicová

, Kittler

(1994) Floating search methods in feature selection. Pattern Recognit Lett 15, 1119–1125.

47.

Ferri

, Pudil

, Hatef

, Kittler

(1994) Comparative study of techniques for large-scale feature selection. Mach Intell Pattern Recognit 16, 403–413.

48.

Zhao

, Chellappa

, Nandhakumar

(1998) Empirical performance analysis of linear discriminant classifiers. Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 164–169.

49.

Werner

, Rabinowitz

, Klinger

, Korczyn

, Josman

(2009) Use of the Virtual Action Planning Supermarket for the diagnosis of mild cognitive impairment. Dement Geriatr Cogn Disord 27, 301–309.

50.

Aubin

, Béliveau

M-F

, Klinger

(2018) An exploration of the ecological validity of the Virtual Action Planning–Supermarket (VAP-S) with people with schizophrenia. Neuropsychol Rehabil 28, 689–708.

51.

Nir-Hadad

, Weiss

, Waizman

, Schwartz

, Kizony

(2017) A virtual shopping task for the assessment of executive functions: Validity for people with stroke. Neuropsychol Rehabil 27, 808–833.

52.

Ciesielska

, Sokołowski

, Mazur

, Podhorecka

, Polak-Szabela

, Kędziora-Kornatowska

(2016) Is the Montreal Cognitive Assessment (MoCA) test better suited than the Mini-Mental State Examination (MMSE) in mild cognitive impairment (MCI) detection among people aged over 60? Meta-analysis. Psychiatr Pol 50, 1039–1052.

53.

Pinto

TCC

, Machado

, Bulgacov

, Rodrigues-Júnior

, Costa

MLG

, Ximenes

RCC

, Sougey

(2019) Is the Montreal Cognitive Assessment (MoCA) screening superior to the Mini-Mental State Examination (MMSE) in the detection of mild cognitive impairment (MCI) and Alzheimer’s Disease (AD) in the elderly? Int Psychogeriatr 31, 491–504.

54.

Valenzuela

, Sachdev

(2009) Can cognitive exercise prevent the onset of dementia? Systematic review of randomized clinical trials with longitudinal follow-up. Am J Geriatr Psychiatry 17, 179–187.

55.

Gruters

AAA

, Christie

, Ramakers

IHGB

, Verhey

FRJ

, Kessels

RPC

, de Vugt

(2020) Neuropsychological assessment and diagnostic disclosure at a memory clinic: A qualitative study of the experiences of patients and their family members. Clin Neuropsychologist, doi: 10.1080/13854046.2020.1749936.

56.

Poyser

, Tickle

(2019) Exploring the experience of the disclosure of a dementia diagnosis from a clinician, patient and carer perspective: a systematic review and Meta-ethnographic synthesis. Aging Ment Health 23, 1605–1615.

57.

Visser

LNC

, van Maurik

, Bouwman

, Staekenborg

, Vreeswijk

, Hempenius

, de Beer

, Roks

, Boelaarts

, Kleijer

, van der Flier

, Smets

EMA

(2020) Clinicians’ communication with patients receiving a MCI diagnosis: The ABIDE project. PLoS One 15, e0227282.