Validation of the Cognitive Telephone Screening Instruments COGTEL and COGTEL+ in Identifying Clinically Diagnosed Neurocognitive Disorder Due to Alzheimer’s Disease in a Naturalistic Clinical Setting

Abstract

Background:

Telephone-based neurocognitive instruments embody valuable tools in identifying cognitive impairment in research settings and lately also in clinical contexts due to the pandemic crisis. The accuracy of the Cognitive Telephone Screening Instrument (COGTEL) in detecting mild- (MiND) and major (MaND) neurocognitive disorder has not been studied yet.

Objective:

Comparison of the utility of COGTEL and COGTEL+, which is enriched with orientation items, with the modified Mini-Mental State Examination (3MS) in detecting MiND and MaND due to Alzheimer’s disease (AD) and assessment of the impact of COGTEL face-to-face-versus telephone administration on individual performance.

Methods:

The study included 197 cognitively intact individuals (CI), being at least 45 years old, 95 and 65 patients with MiND and MaND due to AD, respectively. In 20 individuals COGTEL was administered both in face-to-face and telephone sessions. Statistical analyses included proportional odds logistic regression models, stratified repeated random subsampling used to recursive partitioning to training and validation set (70/30 ratio), and an appropriate F-test.

Results:

All studied instruments were significant predictors of diagnostic outcome, but COGTEL+ and 3MS explained more variance relative to the original COGTEL. Except for the validation regression models including COGTEL in which the average misclassification error slightly exceeded 15%, in all other cases the average misclassification errors (%) were lower than 15%. COGTEL administration modality was not related to systematic over- or underestimation of performance on COGTEL.

Conclusion:

COGTEL+ is a valuable instrument in detecting MiND and MaND and can be administered in face-to-face or telephone sessions.

Keywords

Mild and major neurocognitive disorder modified Mini-Mental State Examination telephone-based neurocognitive testing

INTRODUCTION

Telephone-based neurocognitive assessment embodies a valuable tool in identifying cognitive impairment in variable research settings and lately also in clinical settings [1, 2]. Neurocognitive assessment provides evidence for the presence and severity of cognitive dysfunction, while in its extensive form it yields a profile of the impaired cognitive domains [3, 4]. Such profiles facilitate the detection of the disease that causes the neurocognitive disorder, since diseases causing neurocognitive disorders manifest, albeit not always, with distinct phenotypes of cognitive deficits. For instance, mild and major neurocognitive disorder (MiND and MaND, respectively) due to Alzheimer’s disease (AD), is predominately characterized by deficits in memory and orientation [5, 6]. Using telephone-based neurocognitive tools is a pragmatic strategy to enable early diagnosis of neurocognitive disorders in constellations in which face-to-face assessment is not feasible or appropriate (e.g., long distance between residence and health facilities, mobility difficulties, severe motor or visual impairments) [2, 7]. Particularly in longitudinal studies, neurocognitive assessment over the phone enables frequent, less time-consuming and more cost-effective monitoring of cognitive function. In addition, in the new terrain of the coronavirus disease 2019 (COVID-19) crisis, in which social distancing is an effective measure to decelerate the COVID-19 spread, telephone-based neurocognitive tests form useful and realistic alternatives to in-person neurocognitive assessment in the context of telepsychogeriatric services [8 –10]. Several telephone-based tests have been developed. Nonetheless, most of them are constrained by ceiling and floor effects and have been criticized for being less adept at distinguishing MiND from normal cognition [2].

The Cognitive Telephone Screening Instrument (COGTEL) is a telephone test battery that enables the detailed assessment of performance in six different cognitive domains [11]. It was specifically designed to cover a broad range of cognitive domains with appropriate resolution across the full range of adult cognitive functioning [12]. Mainly based on subtests of the well-known Wechsler memory scale-revised (WMS-R) [13] and Wechsler Adult Intelligence Scale revised (WAIS-R) [14], the COGTEL assesses prospective memory (0 or 1 point), verbal short- and long-term memory (0–8 points each), working memory (0–12 points), verbal fluency (0 to unlimited; as many words as the participant can name within 1 min) and inductive reasoning (0–8 points) [11]. The scores of the six subtests are combined in the form of a weighted total score (7.2×prospective memory + 1.0×verbal short-term memory + 0.9×verbal long-term memory + 0.8×working memory + 0.2×verbal fluency + 1.7×inductive reasoning score). COGTEL can also be administered in face-to-face sessions [15] and administration modality does not significantly affect participant performance [11]. The distribution of total scores was shown to be roughly normally distributed, limiting ceiling effects [11]. COGTEL is a reliable instrument for capturing interindividual differences in cognitive functioning in epidemiological and aging studies [12, 15]. Nonetheless, no studies have reported on the COGTEL accuracy in identifying either MiND or MaND [2].

The primary aim of the study was to assess in the naturalistic setting of old age mental health outpatient units the utility of COGTEL and COGTEL+, being a modified version of COGTEL enriched with six orientation items, in detecting MiND and MaND due to AD and compare it to the Modified Mini-Mental State Examination (3MS) [16, 17]. The secondary study aim was to compare the accuracy of face-to-face COGTEL administration to COGTEL administered via telephone, so that potential effects of administration modality on participant performance on COGTEL are unraveled. We hypothesized 1) that compared with 3MS the telephone instruments would be at least equally efficient in discriminating intact cognition from MiND and MaND and 2) that administration modality would not exert a significant influence on individual performance on COGTEL.

MATERIALS AND METHODS

Participants

The study encompassed older adults who underwent a diagnostic workup between 2017 and 2020 at old age mental health outpatient units in Greece. The study was conducted in accordance with the latest revision of the Declaration of Helsinki and was approved by the study sites’ bioethics and research ethics committees. All participants or authorized representatives gave their written informed consent after a thorough description of the study aims and protocol and prior to study enrolment. Inclusion criteria for the entire sample were 1) (self-) referral for diagnostic evaluation due to cognitive concerns, 2) age≥45 years, 3) diagnosis of MiND or MaND due to AD or absence of a neurocognitive disorder. Exclusion criteria were 1) diagnosis of a neurocognitive disorder caused by a disease other than AD (e.g., frontotemporal lobar degeneration, Parkinson’s disease), 2) mental or neurological disorder or unstable medical condition potentially affecting cognitive function (e.g., major depression, schizophrenia, multiple sclerosis, seizure disorder, head injury, uncontrolled hypothyroidism), 3) hearing or visual difficulties, being potential sources of bias in diagnostic accuracy, 4) insufficient knowledge of the Greek language, and 5) unwillingness to participate in the study.

Clinical diagnoses relied on the findings of a thorough diagnostic workup and were established according to international diagnostic criteria. The diagnostic assessment included a history from the patient and from an informant; neurological and psychiatric examination; laboratory screening and brain imaging (CT or MRI) and the administration of the Mini-Mental State Examination (MMSE) [18] and the Montreal Cognitive Assessment (MoCA) [19], assessing a relatively wide range of cognitive domains (orientation, short-term memory/delayed recall, executive function/visuospatial ability, language abilities, abstraction, animal naming and attention). Of note, MOCA is effective in distinguishing between subjective cognitive complaints and MiND and MaND [19]. The diagnosis of MiND and MaND due to AD was based on the DSM-5 diagnostic criteria [20] and on the diagnostic guidelines of the National Institute on Aging-Alzheimer Association [5, 6]. In individuals without cognitive impairment, neither cognitive deficits nor functional impairment were detected [21, 22]. The clinician who established the diagnoses was blind to the individual performance on COGTEL, COGTEL+ and 3MS.

Neurocognitive instruments

COGTEL- and COGTEL+ validity was compared to 3MS. COGTEL was translated to Greek with advice from the authors of the original COGTEL [11] and afterwards a bilingual clinician (IL) not familiar with the original version of the instrument performed a back translation into English. Comparison of the original English version with the back translation showed that the new version was similar to the original one (see the Supplementary Material). The scoring system was not changed. Like the English original, the Greek COGTEL can be administered in approximately 15 minutes. Due to the principal role of orientation deficits in the AD phenotype [5, 6], performance of individuals on the four temporal- and two spatial orientation MOCA items and COGTEL weighted total score were added together yielding the COGTEL+ score. In twenty randomly chosen participants, COGTEL was also administered in face-to-face sessions. The time between the face-to-face session and COGTEL administration over the telephone did not exceed six days. In twelve cases COGTEL was firstly administered over the phone and afterwards in in-person sessions (positive values of days between the two assessments), while in eight cases face-to-face assessment with COGTEL preceded COGTEL administration over the telephone (negative values of time interval).

3MS was developed to overcome the shortcoming of the MMSE, particularly its narrow range of possible scores and ceiling effects [16, 17]. It has been enriched with four additional items (date and place of birth, word fluency, similarities, and delayed recall of words) and the range of item scores has been increased to produce a maximum of 100 points. These modifications resulted in higher reliability of 3MS compared to MMSE as shown in normal community-dwelling older adults, individuals with MiND and MaND, nursing home residents, geriatric rehabilitation patients and patients with right-hemisphere stroke [16, 17]. MMSE tasks, which are included in the 3MS, were performed only once.

Statistical analyses

Pairwise comparisons and differences across the three study groups, i.e., individuals without cognitive impairment, patients with MiND or MaND, in sex distribution, age, education and test scores were assessed with Pearson Chi-square test, Kruskal Wallis test, Wilcoxon–Mann–Whitney test, as appropriate, since data normality assumption was rejected based on analysis of skewness and kurtosis. Three proportional odds logistic regression models (POLR models) were employed for studying the relationship between diagnostic groups (served as the ordinal dependent variable) and each one of the three different instruments (COGTEL, COGTEL+ and 3MS) taking into account age, sex, and education, which influence cognitive function in older adults [23]. Stratified repeated random subsampling (stratified bootstrap resampling) [24, 25] was used to recursive partitioning to training and validation set (70/30 ratio). The procedure was repeated 20,000 times and the results (parameters estimates over the training data sets and misclassification errors over the training and the validation data sets) were then averaged over the splits. Kernel Fisher discriminant analysis models were also employed to compare the capacity of the three instruments to separate correctly the three study groups [26, 27]. The accuracy of face-to-face COGTEL administration was evaluated and compared to scores of COGTEL administered over the telephone using an appropriate F-test [28], which had been employed in previous studies [29 –31]. The impact of the time interval (in days) between the assessments on individual performance on COGTEL was assessed with a multiple linear regression model with performance on COGTEL administered over the telephone as dependent variable and time interval and performance on COGTEL administered in face-to-face sessions as independent variables.

RESULTS

The study sample encompassed 197 individuals without cognitive impairment, 95 patients with MiND and 65 with MaND due to AD (Table 1). Age (χ² = 98.50, p < 0.001, df = 2), education (χ² =95.19, p < 0.001, df = 2), performance on MOCA (χ² = 264.98, p < 0.001, df = 2), MMSE (χ² = 269.82, p < 0.001, df = 2), COGTEL (χ² = 253.34, p < 0.001, df = 2), COGTEL+ (χ² = 257.68, p < 0.001, df = 2), and 3MS (χ² = 251.90, p < 0.001, df = 2) significantly differed between the groups, so did sex distribution (Pearson χ² = 1.62, p = 0.45, df = 2). Results of pairwise comparison analyses are presented in Table 1.

Table 1

Demographic, neurocognitive, and clinical characteristics of the study sample

	Individuals without cognitive impairment (Group 1, G1)	Mild neuro-cognitive disorder due to AD (Group 2, G2)	Major neuro-cognitive disorder due to AD (Group 3, G3)	Pairwise comparisons
				G1 versus G2	G1 versus G3	G2 versus G3
N	197	95	65
Age, y^*	65.89 (7.71) [45–88]	73.03 (8.11) [52–88]	77.62 (7.78) [56–89]	< 0.001^‡	< 0.001^‡	< 0.001^‡
Education, y^*	12.91 (3.27) [4 –22]	8.97 (4.06) [0–16]	7.46 (4.18) [0–18]	< 0.001^‡	< 0.001^‡	0.007^‡
Sex (female)	146	67	43	0.518^†	0.189^†	0.558^†
MOCA^*	28.48 (1.38) [26 –30]	23.11 (2.71) [16 –29]	14.26 (4.98) [3 –26]	< 0.001^‡	< 0.001^‡	< 0.001^‡
MMSE^*	29.30 (0.76) [28 –30]	25.89 (2.04) [20 –30]	18.69 (4.83) [3 –26]	< 0.001^‡	< 0.001^‡	< 0.001^‡
3MS^*	96.35 (3.44) [81–100]	83.40 (8.32) [66–100]	59.94 (16.11) [11–81]	< 0.001^‡	< 0.001^‡	< 0.001^‡
COGTEL^*	33.30 (8.95) [12–52]	15.97 (5.07) [4.60–36.10]	7.55 (3.86) [0.9–17.6]	< 0.001^‡	< 0.001^‡	< 0.001^‡
COGTEL+^*	39.28 (8.96) [18–58]	21.74 (5.01) [10.60–42.10]	11.54 (4.59) [1.90–22.60]	< 0.001^‡	< 0.001^‡	< 0.001^‡

^*mean (standard deviation) [range]; AD, Alzheimer’s disease; MOCA, Montreal Cognitive Assessment; MMSE, Mini-Mental State Examination; COGTEL, Cognitive Telephone Screening Instrument; COGTEL+, Cognitive Telephone Screening Instrument plus six orientation items of the Montreal Cognitive Assessment; 3MS, Modified Mini-Mental State examination; ^‡ Wilcoxon–Mann–Whitney test; ^†Pearson Chi-square test.

Three POLR models were employed to investigate the relationship between diagnostic status (0: no cognitive impairment, 1: MiND, 2: MaND) and performance on each one of the studied instruments COGTEL, COGTEL+ and 3MS. The three POLR models have been incorporated into the following Google sheet and can be used for estimating the probability of an individual to belong to one of the three diagnostic categories (no cognitive impairment, MiND, MaND) according to her/his performance on tests and demographic characteristics (http://www.des.upatras.gr/amm/economou/NeurocognitiveAssessment.html). In Table 2, the averages of the parameters of the three models along with their 95%bootstrap confidence intervals based on 20,000 stratified bootstrap training sets are presented. Age significantly pertained to diagnostic group in all three models. For example, holding everything else constant, an increase in age by one year increases the expected value of diagnostic status in log odds by 0.0864 according to the findings of the POLR model which included COGTEL as independent variable. Regarding sex, women were classified into less severe diagnostic categories than men with the same characteristics. Education was significantly associated with diagnostic category only in the model which included 3MS, since the 5%bootstrap confidence intervals for the other two models contained zero (Table 2). As expected, higher performance on the studied cognitive instruments pertained to less severe diagnostic category in all models.

Table 2

The averages of the parameters of the three proportional odds logistic regression models along with their 95%bootstrap confidence intervals based on 20,000 stratified bootstrap training sets

		POLR including 3MS as independent variable	POLR including COGTEL as independent variable	POLR including COGTEL+ as independent variable
95%bootstrap confidence intervals	Y > = 1	–25.2961 (–30.4928, –21.1432)	–3.6054 (–6.0619, –1.2968)	–6.2998 (–9.2445, –3.6975)
	Y > = 2	–18.5620 (–22.8152, –14.9827)	0.9355 (–1.239, 3.1899)	1.3497 (–3.7964, 1.0900)
Covariates	Age	0.0788 (0.049, 0.115)	0.0864 (0.0627, 0.1139)	0.0878 (0.0637, 0.1158)
	Sex	–0.3165 (–0.8242, 0.1808)	–0.7540 (–1.2808, –0.2728)	–0.8255 (–1.3972, –0.3180)
	Education	–0.1137 (–0.1779, –0.0548)	–0.0404 (–0.0995, 0.0177)	–0.0249 (–0.088, 0.0370)
	Neurocognitive instrument	–0.3237 (–0.3816, –0.2823)	–0.3834 (–0.448, –0.3371)	–0.4070 (–0.4802, –0.3567)

POLR, proportional odds logistic regression model; 3MS, Modified Mini-Mental State examination; COGTEL, Cognitive Telephone Screening Instrument; COGTEL+, Cognitive Telephone Screening Instrument plus six orientation items of the Montreal Cognitive Assessment.

Differences in misclassification errors between 3MS, COGTEL, and COGTEL+ were unveiled. Misclassification errors (%) over the 20,000 stratified bootstrap training and validation sets along with their 95%bootstrap confidence intervals are presented in Table 3. The POLR models including COGTEL+ and 3MS as dependent variables outperformed the models with COGTEL as dependent variable. Interestingly, except for the validation POLR models including COGTEL in which the average misclassification error slightly exceeded 15%, in all other cases the average misclassification errors (%) were lower than 15%pointing to high classificatory utility of the models. According to the results of the discriminant analysis, the cut off values for detecting MaND and MiND due to AD were 68.5 and 90.3 for 3MS, 9.8 and 21.8 for COGTEL, 15.2 and 27.6 for COGTEL+ (Fig. 1). The misclassification error was 14.6%, 15.7%, and 12.6%for 3MS, COGTEL, and COGTEL+, respectively.

Table 3

The misclassification errors (%) of the three proportional odds logistic regression models along with their 95%bootstrap confidence intervals based on 20,000 stratified bootstrap training and validation sets

	Training sets	Validation sets
POLR including 3MS as independent variable^*	12.68 (10.40,15.20)	13.80 (8.41,19.63)
POLR including COGTEL as independent variable^*	14.62 (12.00,17.20)	15.39 (10.28,21.50)
POLR including COGTEL+ as independent variable^*	12.88 (10.40,15.60)	13.74 (8.41,19.63)

POLR, proportional odds logistic regression mode.

Fig. 1

Weighted density function, scores on the instruments modified mini-mental state examination (3MS), Cognitive Telephone Screening Instrument (COGTEL) and COGTEL plus six orientation items of the Montreal Cognitive Assessment (COGTEL+) and cut-off points for diagnosing minor and major neurocognitive disorder due to Alzheimer’s disease.

The analysis did not unravel a significant effect of administration modality on participant performance on the Greek version of COGTEL. A multiple linear regression model with performance on COGTEL administered over the telephone as dependent variable and time interval (in days) between the telephone- and in-person administration of COGTEL and performance on COGTEL administered in face-to-face sessions as independent variables was employed. The two administration modalities showed a very strong linear relationship (B = 0.964, p < 0.001), while time interval between the assessments was not found to pertain to performance on COGTEL administered in face-to-face sessions (B = –0.072, p = 0.610) according to the multiple regression model which explained 93.81%of variation of performance on COGTEL administered over the telephone (F(2.17) = 128.78, p < 0.001). An F-test was employed to test whether the intercept and the slope of the true relation between the two measurements equaled to zero and unity, respectively, or not, i.e., to test if the observations deviate systematically from the 45-degree line indicating that one administration modality results in performance systematically higher or lower than the other. In Fig. 2, the corresponding scatterplot is presented along with the estimated regression line (solid line) and the 45-degree line (dotted line). The findings indicate that compared to administration over the telephone, face-to-face COGTEL administration yields accurate scores and does not result in a systematic over- or underestimation of performance on COGTEL.

Fig. 2

Scatterplot of the face-to-face COGTEL administration scores versus performance on COGTEL administered over the telephone along with the estimated regression lines (solid line) and the 45-degree line (dotted line).

DISCUSSION

The present study provides first evidence for the utility of COGTEL and COGTEL+, being a version enriched with orientation items, in identifying MiND and MaND due to AD. The novelty of the study design abides 1) in including a relatively large and well characterized sample of patients suffering from neurocognitive disorder due to AD, since no such sample has been assessed with COGTEL so far, 2) in the naturalistic study design based on individuals referred for cognitive concerns to specialized old-age psychiatry outpatient units, as well as 3) in the direct comparison of COGTEL and COGTEL+ with 3MS, which has a well-developed literature in the discrimination of neurocognitive disorder from healthy cognitive aging [17]. Our findings point to significant associations between performance on both COGTEL and COGTEL+ and diagnostic status. Lower scores were shown to pertain to phenotypes characterized by more severe cognitive deficits. Interestingly, COGTEL+ outperformed original COGTEL. The misclassification errors were lower in the case of COGTEL+ compared to COGTEL in both training and validation sets (Table 3) and the discriminant analysis models verified this superiority. The superiority of the modified version of COGTEL to the original version may be attributed to the key role of orientation impairment in the diagnosis of MaND due to AD and in the differential diagnosis between MiND and MaND due to AD [5 , 32]. Even slight temporal or spatial disorientation, including mild difficulty with time relationships or the need for additional assistance in arriving at destinations, substantially interferes with daily life, leads to impairment in activities of daily living and subsequently signals the crossing of the threshold between MiND and MaND in the trajectory of AD [32].

The utility of 3MS was found to be higher than that of COGTEL, as indicated by the misclassification errors of the proportional odds regression- and the discriminant analysis models, but almost equally high with COGTEL+. 3MS is an instrument which captures a broad set of cognitive domains. It assesses temporal and spatial orientation, language, attention, orientation to date and birthplace, verbal short- and long- term memory, naming, verbal fluency, visuoconstruction, abstract reasoning (links between two objects) [17]. Compared to 3MS, COGTEL captures neither visuoconstruction, due to its aim to serve also as telephone-based instrument, nor orientation, language and naming, probably because it is developed as a brief screening tool assessing cognitive functioning across adulthood and not focusing exclusively on age-related (neurodegenerative) neurocognitive disorder, which in its advanced stage is commonly characterized by impaired orientation, language and naming ability [5, 32]. COGTEL covers prospective memory and inductive reasoning, which are not covered in the 3MS, while cognitive domains that are covered in both instruments, are assessed in COGTEL with tasks that enable a more fine-grained differentiation between individuals within the healthy range of functioning (e.g., recall of eight word pairs, of which four are semantically unrelated in COGTEL versus recall of three words in 3MS) [15]. This characteristic of COGTEL does not seem to represent an advantage in the diagnostic workup of individuals with clinically significant cognitive dysfunctions, as indicated by the lack of superiority of COGTEL to 3MS. Nonetheless, it may be of paramount importance for detecting cognitive decline in individuals in the long preclinical phase of AD or other neurodegenerative brain diseases, which is hardly detected by the brief neurocognitive tools that are commonly employed in the diagnostic workup of MiND and MaND [33]. Of note, the addition of the four temporal- and two spatial orientation items to COGTEL led to improvement of the discriminatory power of the instrument. The classification accuracy of COGTEL+ is as high as 3MS in distinguishing between healthy cognitive aging and MiND and MaND.

In line with previous reports, the modality of COGTEL administration was not found to exert a significant influence on individual performance [11]. Despite the potential bias stemming from the possibly unavoidable learning effects in repeated measurement assessments, it is important to shed light on this issue, since flexible combinations of the two forms of administration may be a feasible strategy for large-scale cross-sectional and longitudinal studies. Moreover, only a few researchers have directly compared scores of cognitive screening instruments in a telephone assessment versus a face-to-face assessment [11]. The findings of the employed F-test point out that administration modality neither affects performance on COGTEL nor results in systematic over- or underestimation of cognitive functioning, whilst results of COGTEL administered over the phone and in face-to-face sessions significantly correlate with each other. Thus, it can be reckoned that as the original English one, the Greek version of COGTEL can be interchangeably administered in face-to-face sessions or over the phone, since administration modality does not modulate individual performance on the instrument.

Besides performance on neurocognitive tests, epidemiological characteristics were found to be significantly related to diagnostic status. The POLR models unraveled significant positive associations between age and severity of cognitive impairment, as mirrored in more advanced clinical stages of AD. This observation is in line with previous findings highlighting that the greatest risk factor for AD is advanced age [34]. Interestingly, women were classified into less severe diagnostic categories compared to men with the same demographic and neurocognitive characteristics. This finding seems to be somehow unexpected taking into account that mounting evidence indicates that AD disproportionately affects women in both disease prevalence and rate of symptom progression [35]. It is noteworthy that the clinical diagnosis of MiND and MaND does not exclusively rely on cognitive dysfunction. It is also underpinned by the impact of this dysfunction on performance in activities of daily living [32, 36]. Interestingly, a study which included more than 17,000 community-dwelling older adults unveiled that the prevalence of those with poor complex activities of daily living was higher than 17%in males and only 4.5%in females, showing a significant sex difference [37]. Of note, there is some evidence in support of a protective “functional reserve” influence from continuing being engaged in daily household chores, as women commonly do as age advances, which may preserve functional independence for longer even in the face of cognitive declines [38]. Nonetheless, this finding warrants further investigation before final conclusions can be drawn. Only in the POLR model that included 3MS a significant inverse relation between education and symptom severity was detected. This observation possibly mirrors the fact that in contrast to COGTEL, 3MS scores should be corrected according to the educational attainment of the individual [16, 17]. In the present analyses, no education-adjusted 3MS data were considered because of the lack of such adjustment guidelines for the Greek version of the instrument. Nevertheless, education was considered in all POLR models.

The present study has a number of limitations. First, the evaluation was confined to patients with MiND and MaND due to AD. Hence, we were not in the position to assess the effectiveness of COGTEL and COGTEL+ in detecting neurocognitive disorder caused by other diseases, such as Lewy bodies or cerebrovascular pathologies. Second, the reproducibility and repeatability of the two studied instruments were not evaluated in this study, but as both COGTEL and COGTEL+ assess cognitive functions in a well-standardized and objective manner, the intra- and interrater-related bias is likely to be low [15]. Third, several other screening measures could have been employed within the scope of comparing them with COGTEL and COGTEL+. Nonetheless, no telephone-based neurocognitive instruments have been validated in a Greek-speaking population yet, while 3MS is a tool of high content validity assessing a broad set of cognitive domains and being highly effective in detecting MiND and MaND [16, 17]. Fourth, the present study focuses on total scores and does not shed light on subscale score differences, which facilitate the differential diagnosis of the cause of neurocognitive disorder [39]. Even though the individual variation amongst subtests may be useful and informative, the total score is generally the most clinically reliable point of data for detecting MiND and MaND [39]. Finally, the clinical diagnosis, which was based on a comprehensive diagnostic procedure and on international diagnostic criteria, was used as the ultimate gold standard. Despite the high validity of the diagnostic criteria for AD, the clinical diagnoses are neither always confirmed at autopsy nor always supported by biomarker constellations typical for AD [39 –41]. Thus, possibly erroneous clinical assessments should be also taken into account. The validity of COGTEL and COGTEL+ could therefore be lower than our results suggest.

Overall, the observations of the present study make a strong argument for the clinical utility particularly of the COGTEL+ in detecting not only MaND but also MiND. It should be underscored that we do not opt for substituting comprehensive face-to-face neurocognitive evaluation with COGTEL+ or other telephone-based instruments. Nonetheless, COGTEL+ brevity, content validity, user friendliness, wide range of possible scores and lack of ceiling effects make it a valuable instrument that can be administered in face-to-face or telephone sessions. The development of an online version of the instrument would possibly expand its practicability. Nonetheless, several modifications would be necessary, since input and output modality on following instructions influences cognitive performance and auditory and visual stimuli are not interchangeable [42 –44]. COGTEL+ may prove useful in large scale cross-sectional and longitudinal aging studies as well as in telemedicine diagnostic workup of cognitive impairment in the era of the COVID-19 pandemic crisis and beyond it.

Footnotes

ACKNOWLEDGMENTS

The publication of this article has been financed by the Research Committee of the University of Patras.

Authors’ disclosures available online ().

The supplementary material is available in the electronic version of this article: .

References

Elliott

, Green

, Llewellyn

, Quinn

(2020) Accuracy of telephone-based cognitive screening tests: Systematic review and meta-analysis. Curr Alzheimer Res 17, 460–471.

Carlew

, Fatima

, Livingstone

, Reese

, Lacritz

, Pendergrass

, Bailey

, Presley

, Mokhtari

, Cullum

(2020) Cognitive assessment via telephone: A scoping review of instruments. Arch Clin Neuropsychol 35, 1215–1233.

Salmon

(2019) Neuropsychological assessment of dementia. In The Cambridge Handbook of Clinical Assessment and Diagnosis, Sellbom M, Suhr JA, eds. Cambridge University Press, pp. 416–430.

Casaletto

, Heaton

(2017) Neuropsychological assessment: Past and future. J Int Neuropsychol Soc 23, 778–790.

McKhann

, Knopman

, Chertkow

, Hyman

, Jack

, Kawas

, Klunk

, Koroshetz

, Manly

, Mayeux

, Mohs

, Morris

, Rossor

, Scheltens

, Carrillo

, Thies

, Weintraub

, Phelps

(2011) The diagnosis of dementia due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 7, 263–269.

Albert

, DeKosky

, Dickson

, Dubois

, Feldman

, Fox

, Gamst

, Holtzman

, Jagust

, Petersen

, Snyder

, Carrillo

, Thies

, Phelps

(2011) The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 7, 270–279.

Lacritz

, Carlew

, Livingstone

, Bailey

, Parker

, Diaz

(2020) Patient satisfaction with telephone neuropsychological assessment. Arch Clin Neuropsychol 35, 1240–1248.

Soares

, Silvestre

, Lima

AMO

, de Almondes

(2020) The influence of telemedicine care on the management of Behavioral and Psychological Symptoms in Dementia (BPSD) risk factors induced or exacerbated during the COVID-19 pandemic. Front Psychiatry 11, 577629.

Smith

, Ostinelli

, Macdonald

, Cipriani

(2020) COVID-19 and telepsychiatry: Development of evidence-based guidance for clinicians. JMIR Ment Health 7, e21108.

10.

Numbers

, Brodaty

(2021) The effects of the COVID-19 pandemic on people with dementia. Nat Rev Neurol 17, 69–70.

11.

Kliegel

, Martin

, Jäger

(2007) Development and validation of the Cognitive Telephone Screening Instrument (COGTEL) for the assessment of cognitive function across adulthood. J Psychol 141, 147–170.

12.

Breitling

, Wolf

, Müller

, Raum

, Kliegel

, Brenner

(2010) Large-scale application of a telephone-based test of cognitive functioning in older adults. Dement Geriatr Cogn Disord 30, 309–316.

13.

Elwood

(1991) The Wechsler Memory Scale-Revised: Psychometric characteristics and clinical application. Neuropsychol Rev 2, 179–201.

14.

Franzen

(2002) TheWechsler Adult Intelligence Scale Revised and Wechsler Adult Intelligence Scale-III. In Reliability and Validity in Neuropsychological Assessment, Puente AE, Reynolds CR, Franzen MD, eds. Springer US, Boston, MA, pp. 55–70.

15.

Ihle

, Gouveia

ÈR

, Gouveia

, Kliegel

(2017) The Cognitive Telephone Screening Instrument (COGTEL): A brief, reliable, and valid tool for capturing interindividual differences in cognitive functioning in epidemiological and aging studies. Dement Geriatr Cogn Dis Extra 7, 339–345.

16.

Jones

, Schinka

, Vanderploeg

, Small

, Graves

, Mortimer

(2002) 3MS normative data for the elderly. Arch Clin Neuropsychol 17, 171–177.

17.

van Patten

, Britton

, Tremont

(2019) Comparing the Mini-Mental State Examination and the modified Mini-Mental State Examination in the detection of mild cognitive impairment in older adults. Int Psychogeriatr 31, 693–701.

18.

Fountoulakis

, Tsolaki

, Chantzi

, Kazis

(2000) Mini Mental State Examination (MMSE): A validation study in Greece. Am J Alzheimers Dis Other Demen 15, 342–345.

19.

Poptsi

, Moraitou

, Eleftheriou

, Kounti-Zafeiropoulou

, Papasozomenou

, Agogiatou

, Bakoglidou

, Batsila

, Liapi

, Markou

, Nikolaidou

, Ouzouni

, Soumpourou

, Vasiloglou

, Tsolaki

(2019) Normative data for the Montreal cognitive assessment in Greek older adults with subjective cognitive decline, mild cognitive impairment and dementia. J Geriatr Psychiatry Neurol 32, 265–274.

20.

American Psychiatric Association, DSM-5 Task Force (2013) Diagnostic and statistical manual of mental disorders: DSM-5, American Psychiatric Association, Arlington, VA.

21.

Robb

, Udeh-Momoh

, Wagenpfeil

, Schöpe

, Alexopoulos

, Perneczky

(2017) Biomarkers and functional decline in prodromal Alzheimer’s disease. J Alzheimers Dis 58, 69–78.

22.

Politis

, Alexopoulos

, Vorvolakos

(2020) May neuropsychiatric symptoms be a potential intervention target to delay functional impairment in Alzheimer’s disease? Int Psychogeriatr 32, 689–691.

23.

Bernardelli

, Caruso

, Travaini

, Merzagora

, Gualdi

, Sartori

RDG

, Mari

, Cesari

, Edefonti

(2020) Socio-demographic characteristics and cognitive performance in oldest old subjects asking for driving license renewal. BMC Geriatr 20, 241.

24.

Lokhov

, Kharybin

, Archakov

(2012) Diagnosis of lung cancer based on direct-infusion electrospray mass spectrometry of blood plasma metabolites. Int J Mass Spectrom 309, 200–205.

25.

James

, White

, Kraemer

(2005) Repeated split sample validation to assess logistic regression and recursive partitioning: An application to the prediction of cognitive impairment. Stat Med 24, 3019–3035.

26.

Baudat

, Anouar

(2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12, 2385–2404.

27.

Mika

, Ratsch

, Weston

, Scholkopf

, Mullers

(1999) Fisher discriminant analysis with kernels. In Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468) IEEE, pp. 41–48.

28.

Economou

, Batsidis

, Kounetas

(2021) Evaluation of the OECD’s prediction algorithm for the annual GDP growth rate. Commun Stat Case Stud Data Anal Appl 7, 67–87.

29.

Holden

, Peel

, Thompson

(1985) Survey evidence on expectations formation. In Expectations: Theory and Evidence, Holden K, Peel DA, Thompson JL, eds. Macmillan Education UK, London, pp. 66–93.

30.

Bekri

, Yannopoulos

, Economou

(2019) Methodology for improving reliability of river discharge measurements. J Environ Manage 247, 371–384.

31.

Papadimitropoulos

, Tsikas

, Chassiakos

(2020) Modeling the influence of environmental factors on concrete evaporation rate. J Soft Comput Civil Eng 4, 79–97.

32.

Knopman

, Petersen

(2014) Mild cognitive impairment and mild dementia: A clinical perspective. Mayo Clin Proc 89, 1452–1459.

33.

Jack

, Bennett

, Blennow

, Carrillo

, Dunn

, Haeberlein

, Holtzman

, Jagust

, Jessen

, Karlawish

, Liu

, Molinuevo

, Montine

, Phelps

, Rankin

, Rowe

, Scheltens

, Siemers

, Snyder

, Sperling

(2018) NIA-AA research framework: Toward a biological definition of Alzheimer’s disease. Alzheimers Dement 14, 535–562.

34.

Guerreiro

, Bras

(2015) The age factor in Alzheimer’s disease. Genome Med 7, 106.

35.

Fisher

, Bennett

, Dong

(2018) Sexual dimorphism in predisposition to Alzheimer’s disease. Neurobiol Aging 70, 308–324.

36.

Perneczky

, Pohl

, Sorg

, Hartmann

, Tosic

, Grimmer

, Heitele

, Kurz

(2006) Impairment of activities of daily living requiring memory or complex reasoning as part of the MCI syndrome. Int J Geriatr Psychiatry 21, 158–162.

37.

Tomioka

, Kurumatani

, Hosoi

(2017) Age and gender differences in the association between social participation and instrumental activities of daily living among community-dwelling elderly. BMC Geriatr 17, 99.

38.

Berezuk

, Zakzanis

, Ramirez

, Ruocco

, Edwards

, Callahan

, Black

(2017) Functional Reserve: Experience participating in instrumental activities of daily living is associated with gender and functional independence in mild cognitive impairment. J Alzheimers Dis 58, 425–434.

39.

Alexopoulos

, Ebert

, Richter-Schmidinger

, Schöll

, Natale

, Aguilar

, Gourzis

, Weih

, Perneczky

, Diehl-Schmid

, Kneib

, Förstl

, Kurz

, Danek

, Kornhuber

(2010) Validation of the German revised Addenbrooke’s cognitive examination for detecting mild cognitive impairment, mild dementia in Alzheimer’s disease and frontotemporal lobar degeneration. Dement Geriatr Cogn Disord 29, 448–456.

40.

Alexopoulos

, Roesler

, Werle

, Thierjung

, Lentzari

, Ortner

, Grimmer

, Laskaris

, Politis

, Gourzis

, Kurz

, Perneczky

(2018) Fluid biomarker agreement and interrelation in dementia due to Alzheimer’s disease. J Neural Transm (Vienna) 125, 193–201.

41.

Degenhardt

, Witte

, Case

, Yu

, Henley

, Hochstetler

, D’Souza

, Trzepacz

(2016) Florbetapir F18 PET amyloid neuroimaging and characteristics in patients with mild and moderate Alzheimer dementia. Psychosomatics 57, 208–216.

42.

Yang

, Allen

, Yu

, Chan

RCK

(2015) The influence of input and output modality on following instructions in working memory. Sci Rep 5, 17657.

43.

Cohen

, Horowitz

, Wolfe

(2009) Auditory recognition memory is inferior to visual recognition memory. Proc Natl Acad Sci U S A 106, 6008–6010.

44.

Schmiedchen

, Freigang

, Rübsamen

, Richter

(2013) A Acomparison of visual and auditory representational momentum in spatial tasks. Atten Percept Psychophys 75, 1507–1519.