Visuoconstructional Abilities of Patients With Subjective Cognitive Decline,Mild Cognitive Impairment and Alzheimer’s Disease

Abstract

Background

Via the Vienna Visuoconstructional Test 3.0 (VVT 3.0) delayed recall we combined the assessment of visuoconstructive abilities and memory and investigated the test’s potential to support diagnostic processes, including staging and the elaboration of a cognitive profile.

Methods

We retrospectively analysed the data of 368 patients of the Department of Neurology, Medical University of Vienna, between 04/2014 and 10/2020 that had performed the VVT 3.0. Our sample involved 70 healthy controls (HC), 29 patients with subjective cognitive decline (SCD), 154 patients with mild cognitive impairment (MCI) and 115 patients with Alzheimer’s disease (AD). We investigated the differences in the VVT 3.0 scores, as well as the VVT’s ability to differentiate between AD and nonAD by calculating receiver-operating-characteristic (ROC) curves, ideal cut-offs and a logistic regression model.

Results

Results stated that the VVT 3.0 delayed recall scores were able to differentiate between all diagnostic groups, respectively, except HC-SCD and SCD-MCI. The ROC analyses determined an AUC of 0.890, 95% CI [0.855; 0.925], P < .001, and the ideal cut-off at 29.5 points that maximised sensitivity at 0.896 and specificity at 0.81. The logistic regression model classified 83.4% of AD patients correctly and delivered a significant Cohen’s Kappa of 0.619 (P < .001).

Conclusion

As the VVT 3.0 revealed satisfactory values of diagnostic accuracy in our sample, it could enrich clinical diagnosing. However, for more clarity about its informative value in other populations, there remains a need for future studies with other samples.

Keywords

visuoconstruction memory delayed recall subjective cognitive decline mild cognitive impairment alzheimer’s disease

Introduction

The International Working Group (IWG)¹ and the US National Institute on Aging-Alzheimer’s Association (NIA-AA)² each describe a concept to classify the stages within the progressive cognitive decline of AD before the final stage of major cognitive decline due to AD. The “asymptomatic at-risk stage of AD” (no symptoms but biomarker evidence for AD pathology) named by the IWG matches with the “preclinical stage of AD” (no impairment in cognition on standard assessments but biomarker evidence) of the NIA-AA’s concept. At the late stage of preclinical AD subtle cognitive decline can be reported subjectively (SCD) or by an informant, is, however, not detectable by typical neurocognitive tests.^1,2 In case of evidenced AD pathology, this stage is called “SCD due to AD”, if there is no proof it is defined as “SCD” only.

As test scores of the earliest Alzheimer’s stages are at the border between normal and diseased, categorical definition by tests leads to wrong classifications. High-performing patients are deemed normal and cause low sensitivity, whereas poor performance in a single time point measurement results in misclassification of truly cognitively healthy subjects as declined.³

When this “subjective” cognitive decline can also be sensed by objective cognitive tests the second stage, “prodromal AD” by the IWG, respectively “MCI due to AD” by the NIA-AA is reached.^1,2 Despite MCI, general cognitive function is maintained, as well as functional activities.⁴

At last, both concepts end with “dementia due to AD” in stage 3, standing for full dependence in instrumental activities of daily living, e.g., making calls, handling money.^1,2,5 In other words, formulated in the DSM-5, major neurocognitive disorder interferes with independence in everyday activities. Moreover, all above-mentioned stages of cognitive deficit may not better be related to another psychiatric, neurological or systemic condition.⁶

Various studies analysed the prevalence of each stage and the annual conversion rates (ACR) to more severe stages (for SCD MCI and dementia, for MCI only dementia).^7-11

In his review Petersen et al. reported several prevalence of MCI, altogether steeply rising with age:” 6.7% for ages 60-64, 8.4% for 65-69, 10.1% for 70-74, 14.8% for 75-79, and 25.2% for 80-84”¹²: 1). They further report a 2-year conversion rate to dementia of 14.9% for MCI patients older than 65 years. This constitutes a three times higher risk to develop dementia due to AD than for HC.¹³

Mitchell et al. found that 17.4% of elderly without objective cognitive impairment report subjective memory difficulties (not yet used standardized definition of “SCD”).¹⁴ Later he conducted a meta-analysis that determined an ACR from SCD to MCI of 6,67% and 2,33% that converted from SCD to dementia, which presents a twofold higher risk to develop dementia of patients with SCD than of controls without SCD.⁹

So although the majority does not develop dementia,¹⁵ this higher risk, which was supported by multiple studies,^7,9,16,17 makes SCD a potential indicator for first manifestations.¹⁸

Cognitive decline in the amnestic domain is considered as typical in MCI due to AD and in AD.^2,6,8,19,20 Moreover, also most studies on cognitive decline scrutinized memory and learning,^15,16 which is ordinarily evaluated by tests that include an immediate and a delayed recall. However, for the differentiation between normal and different stages of cognitive decline due to AD, the combined assessment of various cognitive domains has been more and more proposed.^2,15

According to the American Psychiatric Association cognition includes six distinct domains (complex attention, executive function, learning and memory, language, perceptual motor, or social cognition). Next to memory and learning the association states that dysfunction of executive abilities may co-occur in MCI. Perceptual motor ability and language is usually affected in later stages already belonging to major neurocognitive disorder. However, atypical cases resembling non-amnestic-MCI have been found that present with disabilities in these non-amnestic domains in earlier stages. Visuoconstruction is one ability that belongs to the non-amnestic domain perceptual-motor and one that our study and test involves. It can be assessed by copying, drawing and assembling that all demand hand-eye coordination.⁶

To address this independent cognitive domain Lehrner et al. established the Vienna Visuoconstructional Test (VVT 1.0).²¹ It combines three different copy tasks; firstly, the clock drawing test,²² secondly, the pentagons task of the Mini-Mental State Examination (MMSE)²³ and thirdly, the cube task from the Alzheimer’s Disease Assessment Scale (ADAS-Cog).²⁴ In addition to this, the initial test version was amended by appending a delayed recall. The VVT 3.0 comprises the VVT 3.0 copy and the additional VVT 3.0 delayed recall. The add-on was conducted approximately 30 minutes after the first three tasks and encompasses again the same three patterns. A scoring system including 98 different items for both the immediate and the delayed task leads to the corresponding scores. Besides this, a quotient, the “DI-quotient” (delayed recall score divided by the immediate copy score = DI-quotient), was investigated as an additional VVT 3.0 score.

Since our add-on, the VVT 3.0 delayed recall, obviously is no immediate copy task, the underlying neurologic processes happen in additional, other domains next to visuoconstruction.^25,26 On the one hand there are no patterns presented, which makes it a draw-to-command task. As such it affects higher order executive operations such as planning and active organisation, besides visuospatial abilities, attention, comprehension, and motor execution that are equally addressed as in copy tasks.^27,28 On the other hand the instruction does not name the specific patterns or support recognition by any given cues, which claims the patient to recall them. By that, it requires autonomous commencement and for remembering the patterns approximately 30 minutes later, episodic recent memory is indispensable, more specifically the memory process of recollection or mnemonic strategies.²⁹

The focus of the VVT 3.0 delayed recall therefore rests on not only on visuoconstruction andexecutive functioning, but also memory and learning. To a certain degree the results might also depend on language, attention, visual perception and numeric knowledge,^6,25,27 but this is not what the test should provide information about. Alike the delayed recall score, the DI-quotient represents several cognitive domains since it combines both VVT 3.0 scores.

The purpose of the current study is to make use of a combination of three already-known visuoconstructive tests applied in form of a delayed recall denoted as the VVT 3.0 delayed recall and DI-quotient scores and examine the differences between the different, currently distinguished stages of AD (SCD, MCI, AD) and healthy controls. We evaluated the potential of the VVT 3.0 delayed recall score and the DI-quotient to discriminate between nonAD (HC, SCD, MCI) and AD patients and its concurrent validity. Furthermore, we were able to analyse associations between the VVT 3.0, age, gender, premorbid intelligence quotient (IQ), global cognitive status (Neuropsychological Test Battery Vienna, NTBV) and depression.

Methods

Study Population and Clinical Assessments

This study analysed pseudoanonymized data from a retrospective view. Data collection consisted of patients who were under treatment at the Department of Neurology at the Medical University of Vienna between 04/2014 and 10/2020. This study followed the ethical principles of the Declaration of Helsinki and was again assented by the ethics committee of the Medical University of Vienna. The whole study population received information about legal backgrounds of the participation and signed a declaration of consent.

Our sample consisted of 368 patients in total and comprised 70 healthy participants, 29 patients with SCD, 154 patients with MCI and 115 with AD. Patients with cognitive dysfunction (SCD, MCI, AD) approached directly to the tertiary care setting seeking for help or got referred for examination of their cognitive status. The inclusion in the study proceeded consecutively and incorporated all patients who met eligibility criteria and approved.

Eligibility criteria for diseased patients encompassed possible classification into one of the three groups and not meeting any of the criteria. They received a standard neurological and neuropsychological evaluation and conducted the NTBV.³⁰ The following diagnostic criteria were applied for group-assignment: the research criteria of the Subjective Cognitive Decline Initiative⁷ for SCD, the criteria of Petersen et al.⁴ for MCI, the DSM-V criteria of the American Psychiatric Association⁶ and the criteria of the NINCDS-ADRDA Work Group for AD.³¹ Furthermore, the neuropsychological status included the MMSE²³ before conducting the more comprehensive NTBV.³⁰ The NTBV combines various measuring tools, each requiring primarily one specific cognitive domain Altogether they comprise memory, language, executive functions, psychomotor speed and attention. The concrete tasks are based on separate pre-existing tests listed in Table 3. Additionally, the Beck Depression Inventory II (BDI-II)³² served to identify depressive symptoms, and the Wortschatztest (WST)³³ to evaluate premorbid intelligence.

Cognitively healthy subjects were tested on their suitability in a standardized clinical interview and cognitive screening. Normal cognition was determined with the aid of the Mayo Clinic criteria.³⁴

Exclusion criteria have been maintained from previous studies on the VVT 3.0 and encompassed:

(1). Neurological disorders like cortical stroke or traumatic brain injuries in the past, which were determined by neuroradiologic and clinical examination. (2). Medical conditions possibly interfering with normal cognitive abilities including renal, respiratory, cardiac and hepatic disease. (3). Current major psychiatric disorder according to ICD-10²⁰ other than depressive symptoms. (4). Significant auditory, visual, language or motor deficits. (5). “Less than 8 years of schooling.” (6). Other forms of dementia that are not related to AD. (7). Less than 50 years of age.

Vienna Visuoconstructional Test 3.0 - VVT 3.0

The VVT 3.0 comprised a copy and delayed task. In the immediate copy task participants were asked to copy three objects (analogue clock set at 11:10, two intercepting pentagons and a three-dimensional cube) as accurately as possible. After a delay of approximately 30 minutes the participant was asked to draw the three items from memory (delayed recall task) without giving any cue about the patterns. The retention interval (30 minutes delay) was chosen according to established neuropsychological visual memory tests such as e.g., Rey Complex Figure Test and Recognition Trial, where a time delay of 30 minutes is also used. Both the copy and the delayed recall task were scored broadly³⁵ (maximum 98 points) by study staff that had clinical information about the patients. A transparent foil functioning as a template was used to facilitate and standardize scoring. The VVT 3.0 can be obtained from www.psimistri.com.

Statistical Analyses

Statistical analyses were conducted using the IBM SPSS Statistics for Windows, Version 27.0, and Microsoft Excel for additional calculations. The alpha level was set to 0.05. In cases of pairwise comparisons the Holm-Bonferroni correction was applied (adjusted alpha = aa). In Kruskal-Wallis analyses the IBM SPSS programme revealed only one P-value for each analysis including 6 pairwise comparisons. Only in these and in one other analysis, a correlation analysis with 96 pairwise comparisons, the Bonferroni correction was chosen, as expenditure for ranking all coefficients would have exceeded benefits. For these calculations the adjusted alpha based on the classical Bonferroni correction (aa_c) was quoted. Analyses were performed on the whole study population and all available values. If any value was missing, it was excluded from analyses on this specific variable.

To compare the ranks of the VVT 3.0 scores (delayed recall and DI-quotient) of the diagnostic groups Kruskal-Wallis analyses coupled with a Dunn’s post hoc comparison test were performed. Ensuing, receiver-operating-characteristic (ROC) curves for differentiation between AD and nonAD patients were computed. Ideal cut- were set exploratorily by using the Youden Index. Test values that undercut the threshold were counted as positive results, indicating the presence of AD, scores that exceeded it were counted as the absence of AD (HC, SCD, MCI). Subsequently, sensitivity, specificity, together with positive and negative predictive values (PPV, NPV) and positive and negative likelihood ratios were evaluated for the threshold of the delayed recall score. Diagnosis according to the threshold versus diagnosis previously given by clinicians were depicted in cross tabulations and Cohen’s Kappa was determined.

This finding brought along novelty in so far as most other visuoconstructional assessment tests failed to discriminate between HC and MCI and only showed significant differences in test values of subjects with moderate-to-severe dementia and HC.^36-39

Following this, logistic regression analyses were performed. For this purpose, the VVT 3.0 delayed recall scores were defined as independent variable and the diagnoses AD versus non-AD as binary dependent variable.

Afterwards, again cross tabulations were generated and Cohen’s Kappa was calculated to quantify the agreement between the predicted and the actual diagnoses according to the diagnostic criteria.

In addition to that, Spearman’s correlation analyses were conducted to investigate associations between the VVT 3.0 scores, age, the MMSE, the WST, the BDI-II and the different tested domains by the NTBV. A t-test served to explore correlations between the VVT 3.0 scores and gender. Moreover, we calculated the Spearman’s rho to determine the correlation between all three VVT 3.0 scores (copy, delayed recall, DI-quotient) among each other and to quantify and compare the overlap of the measured constructs.

Results

Sample Characteristics

In sum there were 368 values of delayed test executions: 70 (19.0%) of HC, 29 (7.9%) of patients with SCD, 154 (41.8%) of patients with MCI and 115 (31.3%) of patients with AD. Clinical characteristics for this total subgroup and for each diagnostic group can be found in Table 1.

Table 1.

Sample Characteristics of the VVT 3.0 delayed recall

	HC (n = 70)	SCD (n = 29)	MCI (n = 154)	AD (n = 115)	total sample (n = 368)
Age [median (IQR) in years]	68 (15)	68 (22)	73 (15)	77 (8)	74 (14)
Gender (male/female)	20/48	11/18	91/63	58/57	180/186
MMSE [median (IQR) in points]	28.5 (3)	29 (2)	28 (3)	22 (4)	26 (5)
BDI-II [median (IQR) in points]	3 (8)	9 (13)	8 (10)	8 (9)	8 (10)
WST [median (IQR) in points]	108.5 (18)	114 (15)	104 (17)	99 (20)	107 (21)
Education [median (IQR) in years]	12 (6)	13 (10)	12 (8)	11 (5)	11 (7)
VVT 3.0 immediate recall [median (IQR)]	74.5 (13)	77 (10)	73 (14)	63 (21)	71 (16)
VVT 3.0 delayed recall [median (IQR)]	63.5 (22.75)	64 (33)	45 (40)	0 (16)	40.5 (56)
VVT 3.0 DI-quotient [median (IQR)]	0.86 (0.28)	0.77 (0.44)	0.61 (0.5)	0 (0.29)	0.54 (0.73)

Abbreviations: AD, Alzheimer’s disease; BDI-II, Beck Depression Inventory-II; HC, healthy controls; IQR, interquartile range; MCI, mild cognitive impairment; MMSE, Mini-Mental State Examination; n, absolute number; SCD, subjective cognitive decline; VVT, Vienna Visuoconstructional Test; WST, Wortschatztest.

Group Comparisons

As data was not normally distributed, non-parametric measures were chosen. The medians and IQRs of all three VVT 3.0 scores are listed in Table 1.

Kruskal-Wallis analyses depicted a significant difference between groups in the VVT3.0 delayed recall score with a H(3) = 167.734 (P < .001, aa = .009) and mean ranks of HC 274.08, SCD 253.03, MCI 204.57 and AD 85.82. Dunn’s post hoc comparison test specified this difference as significant between AD and all other diagnostic groups (HC and AD: P < .001, aa = .009, SCD and AD: P < .001, aa .0125, MCI and AD: P < .001, aa = .01) as well as between MCI and healthy controls (P < .001, aa = .0167). In comparisons of HC and SCD, such as SCD and MCI, the p-value, however, did not reach the adjusted alpha levels to be significant (HC and SCD: P = .24, aa = .025, SCD and MCI: P = .368, aa = .05).

Results of Kruskal-Wallis analyses and Dunn’s post hoc analyses on the VVT 3.0 DI-quotient revealed the same significant group differences as the delayed recall did. Exact values for the DI-quotient were: H(3) = 152.085 (P < .001, aa = .009); mean ranks: HC = 273.27, SCD = 238.57, MCI = 203.95, AD = 90.79; Dunn’s post hoc comparison - significant results; HC-AD: P < .001, aa = .009; SCD-AD: P < .001, aa 0.0125; MCI-AD: P < .001, aa = .01; HC-MCI: P < .001, aa = .0167; Dunn’s post hoc comparison - not significant results; HC-SCD: P = .138, aa = .05; SCD-MCI: p = .106, aa = .025.

The ensuing ROC curves for the delayed recall with AD as the positive actual state revealed an area under the curve (AUC) of .890, 95% CI [.855; .925], P < .001. By means of the Youden Index an ideal cut-off of 29.5 points was determined, corresponding with a sensitivity of .896 and a specificity of .81. Figure 1 shows the ROC curve that contains the ideal cut-off at the point of intersection of the two dashed lines. At this point the sum of sensitivity plus specificity is the greatest possible.

Figure 1.

ROC Curve of the VVT 3.0 delayed recall (positive actual state = AD)

The same procedure was applied for the DI-quotient, presenting an AUC of 0.870, 95% CI [.830; .911], P < .001, an ideal cut-off of 0.485, as it entails the maximal sum of sensitivity (.887) and specificity (.763) (see Figure 2).

Figure 2.

ROC Curve of the VVT 3.0 DI-quotient (positive actual state = AD)

Furthermore, the PPV (.682) and the NPV (.945) were calculated for the cut-off of 29.5 points of the VVT 3.0 delayed recall score, as well as positive (4.72, 95% CI [3.63; 6.14]) and negative likelihood ratios (.129, 95% CI [.08; 0,22]). Cross-tabulations for diagnosis according to the VVT 3.0 delayed recall score versus diagnosis given by clinicians can be found in Table 2. Cohen’s Kappa defined the degree of agreement between the diagnosis according to the cut-off of 29.5 points and the diagnosis nonAD versus AD given by clinicians significantly at .650 (P < .001).

Table 2.

Crosstabulation: Diagnosis According to the VVT 3.0 Delayed Recall - Cut-Off <29,5 Versus Diagnosis Made by Clinicians.

			Diagnosis made by clinicians		Total
			nonAD	AD	Total
Diagnosis according to the VVT 3.0 delayed recall - cut-off <29.5	ADneg	n	205	12	217
		% within diagnosis according to the cut-off <29.5	94.5% =NPV	5.5%	100.0%
		% within diagnosis made by clinicians	81.0% =specificity	10.4%	59.0%
		% of Total	55.7%	3.3%	59.0%
	ADpos	n	48	103	151
		% within diagnosis according to the cut-off <29.5	31.8%	68.2% =PPV	100.0%
		% within diagnosis made by clinicians	19.0%	89.6% =sensitivity	41.0%
		% of Total	13.0%	28.0%	41.0%
Total		n	253	115	368
		% within diagnosis according to cut-off <29.5	68.8%	31.3%	100.0%
		% within diagnosis made by clinicians	100.0%	100.0%	100.0%
		% of Total	68.8%	31.3%	100.0%

Abbreviations: AD, Alzheimer’s disease; n, absolute number; VVT, Vienna Visuoconstructional Test.

The binomial logistic regression comprised the VVT 3.0 delayed recall score as independent/predictor variable and the diagnoses HC, SCD, MCI, collectively referred to as nonAD, versus AD as dependent variable. It ascertained significance of P < .001 for a X² = 182.406 and a Nagelkerke R² = .550. Furthermore, the model classified 83.4% of the cases correctly with a sensitivity of .757 and a specificity of .87. The PPV resulted in .725 and the NPV yielded .887.

The B of -0.75 was significant (standard error = .006, P < .001). When inserting this value into the formula of the natural logarithm with the Euler’s number e as a base, it results in .928 with a 95% CI of [.914; .942]. The measure of agreement between the diagnoses predicted by the logistic regression model and the diagnoses previously made by clinicians according to the diagnostic criteria revealed a significant Kappa of .619 (P < .001).

Correlation Analyses

Further on, Spearman’s rank correlation analyses on diseased groups found significant negative associations between the VVT 3.0 scores and age: for the delayed recall -0.389 (P < .001, aa = .016), for the DI-quotient -0.368 (P < .001, aa = .025). In the control group correlations were quantified significantly with a Spearman’s rho of -.339 (p = .004, aa = .0125) for the delayed recall. Correlation between age and the DI-quotient did not reach significance in the HC group (DI-quotient: -.202, p = .094, aa = .016).

Results of the correlations with other test formats such as the MMSE, the WST, the BDI-II, years of education and all subtests of the NTBV are displayed in Table 3.

Table 3.

Spearman’s Rank Correlations of the Vvt 3.0 Scores With Age, the MMSE, the WST, the BDI-II, Years of Education and the Subtests of the NTBV

			VVT 3.0 copy	VVT 3.0 delayed recall	VVT 3.0 DI-quotient
Spearman's rho	Age^a (n = 298)	r	−.259**	−.389**	−.365**
	Age^b (n = 70)	r	−.246	−.339**	−.202
	MMSE (n = 312)	r	.468**	.658**	.617**
	Wortschatztest (n = 213)	r	.127	.176	.162
	BDI-II (n = 210)	r	.104	.064	.014
	Years of education (n = 345)	r	.209**	.300**	.287**
	Domain attention
	AKT time (n = 306)	r	−.454**	−.515**	−.461**
	AKT total/time (n = 305)	r	.456**	.527**	.475**
	Digit-symbol test (WAIS-R) (n = 229)	r	.364**	.452**	.375**
	TMT B (n = 218)	r	−.460**	-.561**	−.494**
	Domain psychomotor speed
	Symbols counting (C.I.) (n = 297)	r	−.486**	−.430**	−.354**
	TMT A (n = 304)	r	−.521**	−.556**	−.500**
	Domain language
	Semantic fluency (SWT) total (n = 227)	r	.326**	.541**	.483**
	Phonematic verbal fluency total (n = 221)	r	.154	.296**	.277**
	Boston Naming Test (mBNT) (n = 305)	r	.311**	.466**	.448**
	Domain memory
	Verbal memory total recall (VSRT) (n = 305)	r	.275**	.497**	.466**
	Verbal memory immediate recall (VSRT) (n = 305)	r	.364**	.674**	.639**
	Verbal memory delayed recall (VSRT) (n = 305)	r	.387**	.681**	.640**
	Verbal memory recognition (VSRT) (n = 305)	r	.279**	.579**	.564**
	Domain executive functions
	Nonverbal fluency (Five Point Test) total (n = 219)	r	.350**	.477**	.416**
	Nonverbal fluency (Five Point Test) perseverations (n = 217)	r	−.244**	−.251**	−.215
	Stroop Test colors time (n = 218)	r	−.325**	−.334**	−.260**
	Stroop Test words time (n = 217)	r	−.363**	−.372**	−.299**
	Stroop Test total/time (n = 217)	r	.363**	.359**	.292**
	Stroop Test interference (III-I) (n = 217)	r	−.267**	−.284**	−.226
	Planning maze test (NAI) time (n = 302)	r	−.506**	−.523**	−.450**
	Planning maze test (NAI) total/time (n = 302)	r	.455**	.520**	.462**
	Interference (TMT B-TMT A) (n = 217)	r	−.422**	−.511**	−.449**
	Interference time (C.I.) (n = 302)	r	−.532**	−.523**	−.447**
	Interference total/time (C.I.) (n = 303)	r	.551**	.532**	.455**

**Correlation is significant at the Bonferroni-Holm corrected level (2-tailed). For the subtests of the NTBV the adjusted alpha level is .0005 (2-tailed).

Abbreviations: AKT, Alters-Konzentrations-Test; BDI-II, Beck Depression Inventory-II; C.I., cerebral Insufficiency test; ID, identification number; mBNT, modified Boston Naming Test; MMSE, Mini-Mental State Examination, N, absolute number; NAI, Nürnberger Altersinventar; NTBV, Neuropsychological Test Battery Vienna; PWT, phonematic verbal fluency; r, Spearman’s rho; SWT, semantic verbal fluency; TMT A, Trail Making Test version A; TMT B, Trail Making Test version B; VSRT, Verbal Selective Reminding Test; VVT, Vienna Visuoconstructional Test; WST, Wortschatztest.

Analysed population includes only diseased patients.

Analysed population includes only HC.

Analyses on the correlation of the different VVT 3.0 scores yielded a significant coefficient of .490 (P < .001, aa = .025) between the copy score and the delayed recall score. Between the copy and the DI-quotient Spearman’s rho quantified the correlation with 0.305 (P < .001, aa = .05), between the delayed recall and the DI-quotient with .961 (P < .001, aa = .0167) (see Table 4).

Table 4.

Spearman’s rank correlations between the three VVT 3.0 scores

			VVT 3.0 copy	VVT 3.0 delayed recall	VVT 3.0 DI-quotient
Spearman’s rho	VVT 3.0 copy	r	1,000	.490^**	.305^**
	VVT 3.0 copy	N	368	368	368
	VVT 3.0 delayed recall	r	.490^**	1,000	.961^**
	VVT 3.0 delayed recall	N	368	368	368
	VVT 3.0 DI-quotient	r	.305^**	.961^**	1,000
	VVT 3.0 DI-quotient	N	368	368	368

** Correlation is significant at the adjusted alpha level of 0.01 (2-tailed).

Abbreviations: N, absolute number, r, Spearman’s rho; VVT, Vienna Visuoconstructional Test.

According to the t-test gender showed no significant associations with any of the VVT 3.0 scores: delayed recall: t = -1.126 (P = .261); DI-quotient: t = -1.234 (P = .218).

Discussion

Our results provide insight into the potential of the VVT 3.0 delayed recall; on the one hand to assess the decline of visuoconstruction combined with memory and executive control in different stages of cognitive decline, on the other hand to determine the diagnostic group membership. Moreover, we studied correlations between various clinical variables and the VVT 3.0 scores.

The distribution of the four diagnostic groups in our study population was dominated by MCI patients and AD patients. This was foreseeable as the Department of Neurology at the University Hospital of Vienna is sought out mainly by patients that report cognitive decline, such as MCI and AD, with medical referral for further assessment or directly by self-initiative. For SCD patients, however, the demarcation between physiological cognitive deterioration as a concomitant of ageing and abnormal cognitive deficits is difficult. Before cognitive decline is not objectively recognisable, the presentation of SCD due to normal ageing or because of AD equal one another.¹⁵ In view of this background it might had contributed to a more balanced distribution and higher sample of SCD patients, if there was more active recruitment of SCD patients. However, Jessen et al. explicitly recommended not to consciously search for affected persons, who do not seek out medical institutions by themselves.¹⁵

When regarding the medians of all three VVT 3.0 scores, we found that the VVT 3.0 delayed recall showed stronger decline than the VVT 3.0 copy. The same applied for the DI-quotient which seemed to follow primarily the distribution of the delayed recall. This circumstance stresses the underlying difference of cognitive domains that are demanded for each subtest. The poorer performance in the delayed recall therefore could be traced back to the most commonly impaired domain in cognitive decline due to AD, memory and learning.² In addition to that, it constitutes a domain that is involved early in the process of cognitive decline due to AD.⁶

Delayed recall task showed more distinct values of the MCI group compared to HC and SCD patients than the copy task. One reason for that could be the that the delayed recall teask taps memory and perhaps other functions, so that test scores not only assess and depend on the visuoconstructive abilities. Indeed, delayed recall was shown to be influenced by the functioning of executive abilities and memory.^26,29

The post hoc comparison analyses provided results that the delayed recall and DI-quotient significantly differed in group comparisons including AD and the comparison between HC and MCI. No significant difference could be found between SCD and MCI, what we had not expected. HC-SCD did not reveal significance either and thus, did fulfil our expectation as SCD by definition does not show declined performance on an objective test.⁷

The demarcation between SCD due to AD and MCI due to AD diagnostically consists only of an objectively measured decline. Practically, however, they are both part of a continuum in the disease progression of AD.¹⁵

Moreover, although HC and SCD per definition both perform above cut-offs of objective tests, there is evidence that SCD patients score slightly lower than HC, but still within the range of normal reference standard.⁴⁰ Forcing the setting of a cut-off would likely lead to misclassifications of both diagnostic groups, producing both low sensitivity and specificity.³ This might theoretically explain, why our results detected significant differences between HC and MCI, but not between SCD and MCI. Practically, there was no indication that the SCD group performed worse than HC. Enlarging sample size might enable detection of that subtle differences.

Reason for missing objective difference in scores between HC and SCD is an immanent part of the definition of SCD per se.¹⁵ Results advocate appropriate diagnosing criteria, as SCD is diagnosed by subjective report only and cannot yet objectively be sensed by a neuropsychological testing method.⁴¹

Comparing our results on the delayed recall with those of Valencia et al.⁴² on the copy task revealed that the post hoc comparison test was significant in the same pairwise group comparisons as the other two VVT 3.0 scores, except for the comparison between HC-MCI. We thus can infer that the delayed recall achieved the aim to allow more gradations in the distinction of different disease stages. This enhancement may again be related to tapping additional cognitive domains, such as memory and executive functioning. While visuoconstructive impairment was not different in HC, SCD or MCI, VVT 3.0 delayed recall differed between MCI and HC as well as MCI and SCD. This supports earlier findings that memory functions are already impaired in MCI.^1,4,6,43 Furthermore, there are studies that provide evidence regarding results of visuoconstructive tests to differentiate between MCI and AD patients.^21,43-45 The inconsistent findings might be based on the different subtypes of MCI.^4,42 As shortly outlined above, “MCI” might present in either the amnestic domain, in one or multiple nonamnestic domains or in both.⁴ If a study sample includes various subtypes or many MCI patients without nonamnestic decline, this could explain missing differentiation by copy tasks. If it comprises a lot of nonamnestic declined patients, it might overestimate copy test scores.

According to the calculated values of sensitivity and specificity for the ideal cut-off of 29.5 points for the delayed recall score, 89.6% of patients with AD are classified as such, and 81% of all nonAD patients are identified correctly. According to the proposed standard of Hosmer and Lemeshow the AUC of the delayed recall and of the DI-quotient can be classified as “excellent”,⁴⁶ as cited in.⁴⁷

The Cohen’s Kappa quantified the strength of interrater reliability between the diagnosis according to the cut-off of 29.5 points and the diagnosis nonAD versus AD according to clinical criteria. This value incorporated incidental consensus⁴⁸ and can be graded as “substantial agreement” following the interpretation of Landis et al.⁴⁹

Despite these satisfactory results, they should be interpreted with caution. As the underlying cut-off was chosen post hoc and by maximizing the sensitivity and specificity through the Youden Index, the test performance might be overestimated.^50,51 The cut-off of 29.5 is the ideal in our sample, but firstly may not provide the same accuracy when applied to another population, secondly may not be the best threshold in another population. Therefore, depending on its final intended use, different cut-offs could and should be evaluated. For instance, high sensitivity and a high PPV might be targeted, while a consequently lower specificity and lower NPV might be tolerated.⁵² This could be obtained by raising the threshold.

Next to ROC analyses and the ideal cut-offs, also the binomial logistic regression model delivered satisfactory results. The Nagelkerke R² indicated that the model explained 55% of the variance of the diagnosis nonAD versus AD by the VVT 3.0 delayed recall score. Altogether the model predicted 83.4% of the group membership correctly, 75.5% of the AD patients and 87% of the nonAD patients. The B of -0.75 (P < .001) predicted that increasing the VVT 3.0 delayed recall score was associated with decreasing odds of suffering from AD. Specifically, increasing the VVT 3.0 delayed recall score by 1 decreased the odds to have AD by a factor of .928, 95% CI [.914; .942]. Cohen’s Kappa revealed “substantial agreement” of 0.619 (P < .001) between the diagnosis met by the logistic regression model and the actual diagnosis observed by clinicians.⁴⁹

To differentiate within predementia stages, various authors recommend the combination of various testing methods.^4,15,22,36 This should include the above-mentioned consideration of non-amnestic domains.^2,7

The VVT 3.0 delayed recall could play a role in the process of diagnosing, specifically in staging and in the elaboration of the cognitive profile. A probable indication could be any type of cognitive impairment, no matter how far progressed. It may greatly serve as an add-on test within the first steps of staging before more costly, more complicated neuropsychological test batteries are applied.⁵² Further steps should always follow, as only the combination of various assessment tools allows obtaining full clarity about the present status, more accurate staging and probable distinction between SCD and MCI.^1,4,15,36

Although the VVT 3.0 delayed recall task requires a time gap of about 30 minutes, the task itself is still easily and quickly administered and scored. Hence participants usually accept it well.

Although the correlation coefficients between age and the VVT 3.0 scores in diseased groups turned out to be moderate, analyses on associations between age and the delayed recall in HC also determined a rho of -.339 (P = .004, aa = .016). The remaining VVT 3.0 scores did not provide significant correlations with age in the control group, which was most likely based on the small sample size. Especially the results in the HC group stressed that the moderate associations between the VVT 3.0 scores and age in diseased groups were not solely due to pathologic impairment. They corroborated findings that delayed recall, visuoconstructional and executive abilities decline as a concomitant of normal ageing.⁵³

If the HC group would not show almost the same size of correlation between age and the delayed recall score as did the diseased group, we could not differentiate if age correlated with the VVT 3.0 scores because of normal or because of pathologic changes. Although it was likely that associations between age and the VVT 3.0 scores in the diseased group were related to both types of changes (cognitive decline due to normal ageing⁵³ and cognitive decline due to AD, whose prevalence and incidence strongly depend on age)⁵⁴, the difference in correlations with age between the HC group and the diseased group was small. Thus, the difference did not show a strongly confounding effect of pathologic changes that increase with age, such as cognitive impairment due to AD.

Even if the effect size was small to moderate for age, or not significant in case of gender, these variables might have confounded our results up to a moderate degree. Calculations were based on the observed scores and not adjusted for any variable, such as age or gender.

As expected, the MMSE revealed large associations with the VVT 3.0 scores. This was not surprising, as the MMSE constitutes a measure for global cognition and itself comprises visuoconstructional assessment by the pentagon task.²³ The nonsignificant results for the WST and the VVT 3.0 scores supported the assumption that the VVT 3.0 does not depend on premorbid intelligence. Further nonsignificant results on the BDI-II reassessed the VVT 3.0 as independent of depression, which corresponds with prior findings.^42,55

The NTBV subtests allowed correlation analyses of different domains with the VVT 3.0. In the domain memory differences between the correlations of the two VVT 3.0 tasks were the strongest. While the delayed recall notably largely correlated with the subtests on memory, the copy task only yielded low to moderate coefficients. This supports our concept of the delayed free recall task addressing episodic memory.²⁹ Other than expected and other than reported before^22,25 the domain executive function did not stand out as a domain that was considerably more required by the draw-to-command task than by the copy task, at least according to the subtests of the NTBV on executive control. The coefficient for the DI-quotient for almost all subtests of the NTBV was located between the coefficients of the delayed recall and of the copy task.

In sum, correlation analyses with the NTBV depicted that the VVT 3.0 delayed recall task did depend on the domains attention, psychomotor speed, language and executive functions up to a moderate extent. The inclusion of these domains had been reported before and was outlined above.^27,28

The difference between the measured domains by the VVT 3.0 copy and the delayed recall could also be depicted by the Spearman’s rho between these two. If they affected completely the same domains, we would have expected a rho of almost 1. The coefficient of 0.490 (P < .001, aa = .025) showed that the domains only correlated moderately and provided evidence that the measured constructs were quite disparate.

The correlation analyses on the VVT 3.0 DI-quotient regarding the delayed recall yielded a three times higher Spearman’s rho than when regarding the copy task. These correlations were also present in our primary and secondary results. In most of our analyses results of the DI-quotient resembled the results of the delayed recall considerably stronger than the copy task. This is most likely based on the mathematical background of calculating the DI-quotient by dividing the delayed recall score by the copy task score.

In general, however, the results on the DI-quotient were in most cases located between the results on the copy task and those on the delayed recall and rarely more extreme. It follows that the quotient does not entail more informative value, than the results on the initial immediate copy and delayed recall scores. Consequently, we suggest dispensing with the additional calculation of the DI-quotient in future studies on the VVT 3.0.

Further Limitations

Further limitations of our design should be discussed. Our results are influenced by the severity of dementia, especially when diagnosis was analysed dichotomously, i.e. nonAD versus AD. For instance, sensitivity would rise and the rate of false positive classifications would simultaneously fall, if more patients with severe dementia were analysed.⁵⁶

Comorbidities or conditions that resemble symptoms of AD could show the same influence on test properties. If they affect subjects of the HC group, they would decrease diagnostic accuracy by increased false positive assignments.⁵⁶ Therefore, the majority of medical conditions that could produce biases in HC were averted by defining comprehensive exclusion criteria. In case of diseased participants, however, the incipient multi-morbidity of AD per se and of predementia stages of AD aggravates diagnosing and obtaining undisturbed results.^6,57-59

Additional confounders such as apathy expressing as a lack of motivation or diminished initiative during the test administration, which are typical of cognitive decline due to AD, could not be controlled for either.¹⁹ Hypothetically, they would lead to lower scores in AD patients and further to overestimation of the VVT 3.0 scores.

Another disturbance might have been produced by impairment in processing speed, difficulties with complex attention or fine motor skills. Their decline, however, must not be pathological, but is known to accompany normal ageing and further can impact various other cognitive abilities.^52,60 Depending on whether these normal age-effects concern the HC or the diseased group, they could cause over- as well as underestimation of our results.

The fact that all our patients either autonomously decided to see a doctor or had social contacts that took action to seek medical advice, might have preselected healthier or socially better integrated subjects for our sample. All these possible biases constitute limitations of our results that should sound a note of caution about generalising our findings too generously.

Summary and Conclusion

To conclude, our newly added delayed recall enhanced the discriminating power and the diagnostic accuracy levels of the VVT 3.0. On the one hand the VVT 3.0 delayed recall scores were able to differentiate between all diagnostic groups, respectively, except HC-SCD and SCD-MCI. On the other hand, they revealed great diagnostic accuracy values in our analyses, including an AUC of .890, 95% CI [.855; .925], P < .001, a sensitivity of .896, a specificity of .81 for the cut-off of 29.5 points and a correct classification by the regression model of 83.4%, which all outperformed the immediate copy task. This improvement is most likely based on the additionally demanded domain memory by the free delayed recall.

Although results on accuracy are highly satisfactory in our study population, they should be interpreted with caution when regarding other population groups and with the background of possible biases. Therefore, there remains a need for further investigations on other study samples in order to assess its potential use in clinical diagnosing, staging and cognitive profiling. The VVT 3.0 might also serve as a convenient tool for monitoring and for prognosis. However, to analyse this use, longitudinal studies with a reasonable follow-up sample and ideally various follow-ups are needed.

Footnotes

Acknowledgments

This manuscript has been accomplished as part of the diploma thesis of Paula Knechtl. We would like to thank Mrs. Mag. Grete Sonnek for proofreading this manuscript.

Declaration of conflicting interests

The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: P. Knechtl has no competing interests to declare. J. Lehrner is CEO of psimistri which owns the website .

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Open access was funded by the Medical University of Vienna.

Ethical Approval

All participants signed an informed consent. The study was approved by the Ethics Committee of the Medical University of Vienna (EC-number: 2227/2020) and was conducted in accordance with the 1964 Helsinki Declaration.

Informed Consent

Written informed consent was obtained from all participants.

Data Availability

The datasets analysed during the study are available from the corresponding author on reasonable request.

ORCID iD

Johann Lehrner

References

Dubois

Feldman

Jacova

, et al. Advancing research diagnostic criteria for alzheimer’s disease: The IWG-2 criteria. The Lancet Neurology. 2014;13(6):614-629. doi:10.1016/s1474-4422(14)70090-0

Albert

Dekosky

Dickson

, et al. The diagnosis of mild cognitive impairment due to alzheimer’s disease: Recommendations from the national institute on aging-alzheimer’s Association workgroups on diagnostic guidelines for alzheimer’s disease. Alzheimer’s & Dementia. 2011;7(3):270-279. doi:10.1016/j.jalz.2011.03.008

Jessen

Wolfsgruber

Wiese

, et al. AD dementia risk in late MCI, in early MCI, and in subjective memory impairment. Alzheimers Dement. 2014;10(1):76-83. doi:10.1016/j.jalz.2012.09.017

Petersen

. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004;256(3):183-194. doi:10.1111/j.1365-2796.2004.01388.x

Dubois

Hampel

Feldman

, et al. Preclinical alzheimer’s disease: Definition, natural history, and diagnostic criteria. Alzheimer's & Dementia. 2016;12(3):292-323.

American Psychiatric Association . Diagnostic and statistical manual of mental disorders: DSM-5: American Psychiatric Association. 2013.

Jessen

Amariglio

Van Boxtel

, et al. A conceptual framework for research on subjective cognitive decline in preclinical alzheimer’s disease. Alzheimers Dement. 2014;10(6):844-852. doi:10.1016/j.jalz.2014.01.001

Lehrner

Gufler

Guttmann

, et al. Annual conversion to alzheimer disease among patients with memory complaints attending an outpatient memory clinic: the influence of amnestic mild cognitive impairment and the predictive value of neuropsychological testing. Wien Klin Wochenschr. 2005;117(18):629-635. doi:10.1007/s00508-005-0428-6

Mitchell

Beaumont

Ferguson

Yadegarfar

Stubbs

. Risk of dementia and mild cognitive impairment in older people with subjective memory complaints: meta-analysis. Acta Psychiatrica Scandinavica. 2014;130(6): 439-451. doi:10.1111/acps.12336

10.

Petersen

Stevens

Ganguli

Tangalos

Cummings

DeKosky

. Practice parameter: Early detection of dementia: Mild cognitive impairment (an evidence-based review). Report of the Quality Standards Subcommittee of the American Academy of Neurology. 2001;56(9):1133-1142. doi:10.1212/wnl.56.9.1133

11.

Jack

Jr Bennett

Blennow

, et al. NIA-AA research framework: Toward a biological definition of alzheimer’s disease. Alzheimers Dement. 2018;14(4), 535-562. doi:10.1016/j.jalz.2018.02.018

12.

Petersen

Lopez

Armstrong

, et al. (2018). Practice guideline update summary: Mild cognitive impairment: Report of the guideline development, dissemination, and implementation subcommittee of the American academy of neurology. Neurology. 2018;90(3):126-135. doi:10.1212/WNL.0000000000004826

13.

Aisen

Petersen

Donohue

Weiner

. Alzheimer’s disease neuroimaging initiative 2 clinical core: Progress and plans. Alzheimers Dement. 2015;11(7):734-739. doi:10.1016/j.jalz.2015.05.005

14.

Mitchell

. The clinical significance of subjective memory complaints in the diagnosis of mild cognitive impairment and dementia: A meta-analysis. Int J Geriatr Psychiatry. 2008;23(11):1191-1202. doi:10.1002/gps.2053

15.

Jessen

Amariglio

Buckley

, et al. The characterisation of subjective cognitive decline. Lancet Neurol. 2020;19(3):271-278. doi:10.1016/s1474-4422(19)30368-0

16.

Jessen

. Prediction of dementia by subjective memory impairment effects of severity and temporal association with cognitive impairment dementia and subjective memory impairment. Archives of General Psychiatry. 2010;67(4):414. doi:10.1001/archgenpsychiatry.2010.30

17.

Van Harten

Mielke

Swenson-Dravis

, et al. Subjective cognitive decline and risk of MCI. Neurology. 2018;91(4):e300-e312. doi:10.1212/wnl.0000000000005863

18.

Koppara

Wagner

Lange

, et al. Cognitive performance before and after the onset of subjective cognitive decline in old age. Alzheimers Dement (Amst). 2015;1(2):194-205. doi:10.1016/j.dadm.2015.02.005

19.

McKhann

Knopman

Chertkow

, et al. The diagnosis of dementia due to alzheimer’s disease: Recommendations from the national institute on aging-alzheimer’s association workgroups on diagnostic guidelines for alzheimer’s disease. Alzheimers Dement. 2011;7(3):263-269. doi:10.1016/j.jalz.2011.03.005

20.

World Health Organization . ICD-10: International statistical classification of diseases and related health problems. (5th ed.): World Health Organization. 2016.

21.

Lehrner

Krakhofer

Lamm

, et al. Visuo-constructional functions in patients with mild cognitive impairment, alzheimer’s disease, and Parkinson’s disease. Neuropsychiatr. 2015;29(3):112-119. doi:10.1007/s40211-015-0141-2

22.

Shulman

. Clock-drawing: Is it the ideal cognitive screening test? Int J Geriatr Psychiatry. 2000;15(6), 548-561.

23.

Folstein

McHugh

. Mini-mental state. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189-198. doi:10.1016/0022-3956(75)90026-6

24.

Rosen

Mohs

Davis

. A new rating scale for alzheimer’s disease. Am J Psychiatry. 1984;141(11):1356-1364. doi:10.1176/ajp.141.11.1356

25.

Royall

Cordes

Polk

. CLOX: An executive clock drawing task. Journal of Neurology, Neurosurgery & Psychiatry. 1998;64(5):588-594. doi:10.1136/jnnp.64.5.588

26.

Shon

Lee

Seo

, et al. Functional neuroanatomical correlates of the executive clock drawing task (CLOX) performance in alzheimer’s disease: A FDG-PET study. Neuroscience. 2013;246:271-280. doi:10.1016/j.neuroscience.2013.05.008

27.

Mendez

Ala

Underwood

. Development of scoring criteria for the clock drawing task in alzheimer’s disease. J Am Geriatr Soc. 1992;40(11):1095-1099. doi:10.1111/j.1532-5415.1992.tb01796.x

28.

Waber

Holmes

. Assessing children’s copy productions of the rey-osterrieth complex figure. J Clin Exp Neuropsychol. 1985;7(3):264-280. doi:10.1080/01688638508401259

29.

Caruso

Perri

Fadda

Caltagirone

Carlesimo

. (2020). Recall and recognition in alzheimer’s disease and frontotemporal dementia. Journal of Alzheimer’s Disease. 2020;77:655-666. doi:10.3233/JAD-200126

30.

Johann

Maly

Gleiß

Auff

. The vienna neuropsychological test battery (VNTB) for detecting dementia—standardization, norms, and validation. Psychologie in Österreich. 2007;4(5):358-365.

31.

McKhann

Drachman

Folstein

Katzman

Price

Stadlan

. Clinical diagnosis of alzheimer’s disease: Report of the NINCDS-ADRDA work group under the auspices of department of health and human services task force on alzheimer’s disease. Neurology. 1984;34(7):939-944. doi:10.1212/wnl.34.7.939

32.

Hautzinger

Keller

Kühner

. Beck Depressions-Inventar (BDI-II). Frankfurt/Main: Harcourt Test Services. 2006.

33.

Schmidt

Metzler

. Wortschatztest. Beltz. 1992.

34.

Greenaway

Smith

Tangalos

Geda

Ivnik

. Mayo older americans normative studies: Factor analysis of an expanded neuropsychological battery. Clin Neuropsychol. 2009;23(1):7-20. doi:10.1080/13854040801891686

35.

Heidinger

Lehrner

. Comparing a visual and verbal semantic memory test on the effects of gender, age and education as assessed in a cognitively healthy sample : A pilot study in the development of the international neuropsychological test profile-a tablet-based cognitive assessment. Neuropsychiatr. 2020;34(3):140-147. doi:10.1007/s40211-020-00355-9

36.

Connor

Seward

Bauer

Golden

Salmon

. Performance of three clock scoring systems across different ranges of dementia severity. Alzheimer Dis Assoc Disord. 2005;19(3):119-127. doi:10.1097/01.wad.0000174948.77113.a6

37.

Lee

Swanwick

Coen

Lawlor

. Use of the clock drawing task in the diagnosis of mild and very mild alzheimer’s disease. Int Psychogeriatr. 1996;8(3):469-476. doi:10.1017/s1041610296002827

38.

Sevigny

Peng

Liu

Lines

. Item analysis of ADAS-Cog: Effect of baseline cognitive impairment in a clinical AD trial. Am J Alzheimers Dis Other Demen. 2010;25(2):119-124. doi:10.1177/1533317509350298

39.

Zec

Landreth

Vicari

, et al. Alzheimer disease assessment scale: A subtest analysis. Alzheimer Dis Assoc Disord. 1992;6(3):164-181. doi:10.1097/00002093-199206030-00004

40.

Jessen

Spottke

Boecker

, et al. Design and first baseline data of the DZNE multicenter observational study on predementia alzheimer’s disease (DELCODE). Alzheimers Res Ther. 2018;10(1):15. doi:10.1186/s13195-017-0314-2

41.

Jessen

. Subjective and objective cognitive decline at the pre-dementia stage of alzheimer’s disease. Eur Arch Psychiatry Clin Neurosci. 2014;264(1):S3-S7. doi:10.1007/s00406-014-0539-z

42.

Valencia

Lehrner

. Assessing visuo-constructive functions in patients with subjective cognitive decline, mild cognitive impairment and alzheimer’s disease with the vienna visuo-constructional test 3.0 (VVT 3.0). neuropsychiatrie. 2021;35(3):147-155. doi:10.1007/s40211-021-00385-x

43.

Nasreddine

Phillips

Bédirian

, et al. The montreal cognitive assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005;53(4):695-699. doi:10.1111/j.1532-5415.2005.53221.x

44.

Numrich

. Visuo-constructional functions in patients with subjective cognitive decline (SCD), mild cognitive impairment (MCI), Alzheimer’s disease (AD) and Parkinson’s disease (PD). (Master’s Thesis). University of Vienna. 2017.

45.

Thomann

Toro

Dos Santos

Essig

Schröder

. Clock drawing performance and brain morphology in mild cognitive impairment and alzheimer’s disease. Brain Cogn. 2008;67(1):88-93. doi:10.1016/j.bandc.2007.11.008

46.

Lemeshow

Sturdivant

Hosmer

. Applied Logistic Regression (3rd ed.): Wiley. 2013.

47.

Mandrekar

. Receiver operating characteristic curve in diagnostic test assessment. Journal of Thoracic Oncology. 2010;5(9):1315-1316.

48.

McHugh

. Interrater reliability: The kappa statistic. Biochem Med (Zagreb). 2012;22(3):276-282.

49.

Landis

Koch

. The measurement of observer agreement for categorical data. Biometrics. 1997;33(1):159-174. doi:10.2307/2529310

50.

Ewald

. Post hoc choice of cut points introduced bias to diagnostic research. J Clin Epidemiol. 2006;59(8):798-801. doi:10.1016/j.jclinepi.2005.11.025

51.

Leeflang

Moons

Reitsma

Zwinderman

. Bias in sensitivity and specificity caused by data-driven selection of optimal cutoff values: Mechanisms, magnitude, and solutions. Clin Chem. 2008;54(4):729-737. doi:10.1373/clinchem.2007.096032

52.

Cohen

Korevaar

Altman

, et al. STARD 2015 guidelines for reporting diagnostic accuracy studies: explanation and elaboration. BMJ Open. 2016;6(11):e012799. doi:10.1136/bmjopen-2016-012799

53.

Harada

Natelson Love

Triebel

. Normal cognitive aging. Clin Geriatr Med. 2013;29(4):737-752. doi:10.1016/j.cger.2013.07.002

54.

Fiest

Roberts

Maxwell

, et al. The prevalence and incidence of dementia due to alzheimer’s Disease: A systematic review and meta-analysis. Journal Canadien des Sciences Neurologiques. 2016;43(S1):S51-S82. doi:10.1017/cjn.2016.36

55.

Krakhofer

. Visuokonstruktive Fähigkeiten bei Patienten mit Alzheimer Krankheit, Mild Cognitive Impairment (MCI) und Parkinson Krankheit. (Master’s Thesis). University of Vienna. 2013.

56.

Rutjes

Reitsma

Vandenbroucke

Glas

Bossuyt

. Case–control and two-gate designs in diagnostic accuracy studies. Clinical Chemistry. 2005;51(8):1335-1341. doi:10.1373/clinchem.2005.048595

57.

Livingston

Huntley

Sommerlad

, et al. Dementia prevention, intervention, and care: 2020 report of the Lancet Commission. Lancet. 2020;396(10248):413-446. doi:10.1016/s0140-6736(20)30367-6

58.

Livingston

Sommerlad

Orgeta

, et al. Dementia prevention, intervention, and care. Lancet. 2017;390(10113):2673-2734. doi:10.1016/s0140-6736(17)31363-6

59.

Stephan

Brayne

Savva

Matthews

. Occurrence of medical co-morbidity in mild cognitive impairment: Implications for generalisation of MCI research. Age Ageing. 2011;40(4):501-507. doi:10.1093/ageing/afr057

60.

Hoogendam

Hofman

Van der Geest

Van der Lugt

Ikram

. Patterns of cognitive function in aging: The rotterdam study. Eur J Epidemiol. 2014;29(2):133-140. doi:10.1007/s10654-014-9885-4