Cross-Cultural Applicability of the Montreal Cognitive Assessment (MoCA): A Systematic Review

Abstract

The Montreal Cognitive Assessment (MoCA) is widely used to screen for mild cognitive impairment (MCI). While there are many available versions, the cross-cultural validity of the assessment has not been explored sufficiently. We aimed to interrogate the validity of the MoCA in a cross-cultural context: in differentiating MCI from normal controls (NC); and identifying cut-offs and adjustments for age and education where possible. This review sourced a wide range of studies including case-control studies. In addition, we report findings for differentiating dementias from NC and MCI from dementias, however, these were not considered to be an appropriate use of the MoCA. The subject of the review assumes heterogeneity and therefore meta-analyses was not conducted. Quality ratings, forest plots of validated studies (sensitivity and specificity) with covariates (suggested cut-offs, age, education and country), and summary receiver operating characteristic curve are presented. The results showed a wide range in suggested cutoffs for MCI cross-culturally, with variability in levels of sensitivity and specificity ranging from low to high. Poor methodological rigor appears to have affected reported accuracy and validity of the MoCA. The review highlights the necessity for cross-cultural considerations when using the MoCA, and recognizing it as a screen and not a diagnostic tool. Appropriate cutoffs and point adjustments for education are suggested.

Keywords

Cross-cultural dementia mild cognitive impairment Montreal Cognitive Assessment

INTRODUCTION

Mild cognitive impairment (MCI) is conceptualized as a transitional stage between normal aging and dementia and is recognized as a risk factor for AD [1], prompting many researchers to screen for MCI in order to provide early treatment and reduce the risk of progression to dementia [2]. Transitional markers from MCI to dementias are challenging to define and cognitive impairment may result from conditions not related to dementia (e.g., delirium, substance use, sleep disturbances, psychiatric illness, and metabolic conditions). Factors including high premorbid intellectual function and high quality occupational and educational attainment are suggested to provide a ‘cognitive reserve’ [3] conserving existing cognitive abilities even in the presence of biological risk. Due to the lack of stable biomarkers of neurodegeneration [4, 5], diagnosis is reliant on clinical judgment, defined by clinical, cognitive, and functional criteria. Broadly, criteria for MCI relate to concern regarding a change in cognition, impairment in one or morecognitive domains, and preservation of independence in functional abilities [6]. However differences in clinical definitions exist [6, 7] with clinical subtypes such as amnestic and non-amnestic MCI, and MCI due to deficits in single and multiple cognitive domains [8]. Indeed, MCI does not necessarily indicate a progressive degeneration with reversion rates from MCI to normal controls (NC) estimated at approximately 18% [9] and stability rates between 37% to 67% [10]; higher than rates of progression to dementia. Wide variation of incidence and prevalence rates exists for subtypes of MCI [11, 12] related to the differences in diagnostic criteria applied and how they are operationalized (e.g., cognitive assessments used, cut-offs for age, and education) [11].

To facilitate the detection of MCI, many health professionals around the world are using the Montreal Cognitive Assessment (MoCA) as a brief cognitive screen across a variety of clinical settings. The MoCA, developed and validated by Nasreddine et al. [13], is a brief and potentially useful screening tool with high sensitivity and specificity for detecting MCI in persons performing in the normal range on the Mini-Mental State Examination (MMSE). It has been translated into 36 languages and additional alternative forms, yet only a small number have been validated and nearly all versions lack population-based data of individuals aged 80 and older [14]. For instance, the norms and cut-off scores for MCI and dementia of the MoCA are different among five Chinese versions [14] and the cut-off of 26 (+1 for <12 years education) recommended in the original study [13], has been suggested on many forms without validation. Indeed, cultural factors (i.e., the accumulation of shared knowledge, reflected in behaviors, experience, beliefs, values, attitudes [15]), lifestyle factors (i.e., pattern of everyday life given available resources, e.g., employment, diet, activities, living arrangements [16]), and education (which can be considered an aspect of culture where learning and schooling is embedded with the culture [17]) will vary across and within groups. Culture may affect the validity of the cognitive tests used to identify MCI through biases related to the test or administration. For instance, where the construct an item is measuring is ambiguous due to poor translation or lack of cultural equivalent, or where there is unfamiliarity with both testing and stimuli or the related skill does not exist within a cultural repertoire [18].

Because of cultural background and lifestyle differences in Eastern and Western countries, it is necessary to assess the scale in patients from different cultures. The MoCA is a screening instrument that evaluates seven cognitive domains on a single page and scores range from 0 to 30 [13]. The domains are: visuospatial/executive functions, naming, verbal memory registration and learning, attention, abstraction, 5-minute delayed verbal memory, and orientation. Validation of the MoCA in different languages has led to linguistic and cultural modifications. For instance, in the Filipino version of the MoCA, an owl, the image that locals can recognize, is used as the replacement of rhinoceros [19]. The Chinese Beijing Version for example replaced Roman alphabets with Chinese character sequences in Alternating Trail Making and the names were changed to more common Chinese names to reflect local familiarity in the sentence repetition [20]. In the Korean version, it is notable that the phonemic fluency task is replaced with a semantic fluency task [21]. Considering that frequency of phonemes may differ across languages, the scores of phonemic fluency task are not transferable from one country to another [22]. Indeed, the format for some cultures has been drawn into question, for instance Alternating Trail Making and cube copying test may be unfamiliar [23] and sentence repetition where revisions are not considered may not be equivalent becoming overly complex in translation. In contrast, some items in the English version may not be sensitive enough, for instance Chinese speakers tend to perform better on digit span tasks [24].

Furthermore, issues of normal variability in cognitive functioning may lead to false positive (FP) misclassifications (incorrectly indicating that MCI is present). For instance, individuals at the polar ends of a normal distribution (e.g., 7% of healthy people will score at or below 1.5 standard deviations below the norm), will lead to a FP misclassification (i.e., 7% of healthy people would be falsely diagnosed with MCI) based on a cutoff. A normative approach was attempted by Wong et al. [25] using the MOCA (Hong Kong version) and estimated high FP rates among NC highlighting the elevated risk of misclassification when using single cutoffs scores, especially for those older and less educated.

The use of universal cut-offs also leads to a sampling bias (i.e., sample for which the cutoff was established does not match with the individual being assessed) which, if uncontrolled, leads to false conclusions about an individual’s cognitive ability. This may not relate to the cognitive test itself, but how a health care professional may interpret these results and how this may inform clinical decision making.A ‘diagnostic cut-off’ suggests that the cut-off alone can be used to interpret the MoCA rather than considering other factors affecting performance, or being aware that the MoCA is a screen and does not replace more thorough diagnostic assessment. However, where resources and training are limited, there may be a reliance on tools which are cost and time effective. Therefore, education and culturally adjusted ‘cut-offs’ can inform clinical decision making. Furthermore, different domains of cognition have cultural and language loadings. Language and literacy vary within and across cultures and will affect measurement of attention and memory. Performance on tests assessing executive functions are strongly related to education and culture [26, 27] associated to processes of learning, language skills, perceptual skills, and reasoning. As a consequence, individuals are unlikely to perform consistently across the MoCA due to factors other than cognitive decline. Even where the individual may speak English (and tested in English), English proficiency, language, and acculturation (adaptive cultural and psychological adjustment undertaken by individuals moving from one cultural context to another), may affect test performance [28 –31]. In addition, adjustments to assessment, for instance using an interpreter introduces additional methodological issues.

Studies have demonstrated that the culturally and linguistically adapted versions of the MoCA are superior to the MMSE in screening for MCI [20 , 33]. However, due to the differences in Western culture and difficulty in separating the influence of culture from education level, further revision of MoCA adaptations are required to tailor the contents more accurately to local populations across the globe as well as optimizing cut-offs for detecting MCI.

In this article, we address these critical issues by reporting the results of our systematic review. Thus, the aim of this study is to explore the cross-cultural applicability of the measure and suggested cut-offs in more detail. Given that the MoCA is regularly used as a reference standard (i.e., selected as the measure against which other screening tests are evaluated or to inform classification), we aim to highlight considerations specific to language, cultural regions, age, and education when using this measure clinically and as a research tool especially when assessing non-Western English speaking individuals.

The MoCA was originally validated to identify MCI [13]. Due to the variation in classification there is no universally accepted “gold standard” criterion for diagnosis of MCI. As such, this review considers the validations of the MoCA for MCI against recognized diagnostic criteria outlined in the inclusion criteria.

Data differentiating between MCI and dementia, and between dementia and NC are presented, however, these were considered to have limited utility. Cognitive screens alone should not be used in isolation to diagnose dementia. Guidelines [34, 35] indicate the need for comprehensive physical, medical, and cognitive assessment and history taking when it comes to diagnosing forms of dementia. Differentiating MCI and dementia is informed by identification of cognitive decline interfering with performing complex activities. The same applies to differentiating dementia from NC. Nevertheless, dementia diagnosis based on MoCA has recently been reviewed [36], suggesting the quality of the studies was “not good enough” to conclusively recommend its use and that a lower cut-off was likely to be more accurate. While the review reported optimal sensitivity and specificity, due consideration was not given to education or variation in suggested cut-offs. Many studies have emerged since their search in 2012, and from a cross-cultural perspective, given that these studies are validating the MoCA in different populations, these data have been included for reference.

METHODS

The review protocol was registered with PROSPERO (registration no. CRD42016032837). Review followed PRISMA guidelines [37].

Data search

Published and unpublished studies were considered, no language or date restrictions were applied. Search terms were compiled into two concepts 1. (Montreal adj cognitive adj assessment) or MoCA 2. validat$ or norm$ or adapt$ or translat$ or cultur$ or reliab$). Searches were conducted using the following databases: Ovid MEDLINE In-Process & Other Non-Indexed Citations and Ovid MEDLINE, Ovid PsycINFO and Ovid Embase (all years to 09 January 2016). Articles citing the original MoCA paper [13] were screened separately using GoogleScholar.

Inclusion/Exclusion criteria

Criteria for inclusion of studies in the review were (1) dementia or MCI with comparison group, (2) more than 10 cases in each group, and (3) use of MoCA compared with a reference standard. The reference standard was defined as the Diagnostic and Statistical Manual of Mental Disorders (DSM) versions III to 5 [38], International Classification of Diseases –10th edition (ICD-10) [39], the National Institute of Neurological and Communicative Disorders and Stroke and the Alzheimer’s Disease and Related Disorders Association Alzheimer’s criteria (NINCDS-ADRDA) [40], Petersen’s criteria [41], National Institute of Neurological Disorders and Stroke and Association Internationale pour la Recherché et l’Enseignement en Neurosciences (NINDS-AIREN) [42] or expert diagnosis following interview such as the Clinical Dementia Rating (CDR) [43].

Studies were excluded (1) where MoCA score was used as a grouping variable, (2) where studies investigated English speaking, western samples and (3) where there was evidence of co-morbid mental health conditions, e.g., schizophrenia, in the samples.

Selection of studies

Review authors independently reviewed titles and abstracts for inclusion criteria. Any questions regarding eligibility were resolved by seeking additional information from study authors and through discussion with the other author. Figure 1 outlines a PRISMA flow diagram of the systematicreview.

Fig.1

PRISMA flow diagram of the systematic review.

Data extraction

From the selected papers, the following data were extracted: population, recruitment strategy, specification of illness, comparator (diagnostic standard), age, country, education adjustments, cut-off score, sensitivity, and specificity (full details in the Supplementary Material). Study authors were contacted for additional information where necessary. Reasonable attempts were made to translate papers to English (authors sought out translation by native speakers within their affiliated institutions).

Risk of bias across studies

Review authors independently assessed the methodological quality of the included studies using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) instrument [44], designed to evaluate the risk of bias and applicability of diagnostic accuracy studies. Any disagreements were decided by consensus.

The methodological quality in each of the domains (patient selection, execution of the index test and the reference standard, and flow of patients) was often difficult to assess as the required information was not clearly stated in the published reports. The QUADAS-2 scores for each domain and for each sub analysis are included (Supplementary Figures 1–3), a summary is present in Fig. 2.

Fig.2

Risk of bias and applicability concerns graph: review authors’ judgments about each domain presented as percentages across included studies.

For Patient Selection, case-control design studies were included for all analyses, where controls were thought to be a representative sample of the population which produced the cases. However, the case control design leads to spectrum bias (where sensitivity or specificity of the test differs with different patient characteristics or disease features, e.g., severity) resulting in overestimation of test accuracy, which therefore resulted in a high risk of selection bias (differences between baseline characteristics of the groups that are compared) and is recognized as a source of heterogeneity in this review.

In the Index Test domain, over half of the studies were judged as being of unclear risk of bias, mainly due to the absence or unclear reporting on test administration, masking and pre-specification of thresholds. These studies were considered to have low risk of concerns about applicability.

In the Reference Standard domain, all studies used a recognized reference standard that was likely to correctly classify the condition. However, the studies marked as unclear did not report the use of masking clearly and/or how the reference standard was operationalized and applied. Over 85% of the studies were assessed as being of low risk of concern about applicability.

In the Flow and Timing domain, less than a quarter of studies were judged as unclear, mainly due to absence of relevant information about the time intervals between index tests and reference standards and a lack of reporting about whether all participants were included in the final analyses.

Data synthesis

Sensitivity, specificity for optimal cutoffs (as indicated by the study authors) and sample size data were extracted into Review Manager 5.2 [45] for analysis. Where data were stratified to account for education or age, we selected these data. Where exact sample sizes were not specified, the whole group sample was divided for both groups (clinical andcontrol). Analyses were separated into three categories: differentiating MCI from NC, differentiating MCI from dementia, and differentiating dementia from NC. True (TP) and false (FP) positives, and true (TN) and false (FN) negatives were calculated for all studies. Sensitivity and specificity with 95% confidence intervals (CIs) were presented separately for MCI and dementias.

Given the majority of studies were case control studies, accurate estimates of prevalence could not be calculated, limiting further analyses. To overcome the heterogeneity due to cut-offs, the accuracy data were pooled by fitting a summary received operating characteristic curve (SROC).

RESULTS

MCI versus NC

Twenty-six studies in total (N = 19,060) investigated the ability of the MoCA to differentiate MCI from NC. Sensitivity ranged from 55% (at a specificity of 76%) to 96% (at a specificity of 76%). Specificity ranged from 19% (at sensitivity of 96%) to 97% (at sensitivity of 91%). A symmetrical SROC curve was generated with studies given equal weight for analysis (Fig. 3). Studies, covariates and forest plots are presented in Fig. 4.

Fig.3

SROC curve for studies differentiating between MCI and NC.

Fig.4

Forest plot for differentiating between MCI and NC. a, b indicates education or age adjustment within the same study.

While the number of studies and variation with the regions affect pooling of data, we divided up the studies into geographical regions to summarize the descriptive data. It does not suggest homogeneity between regions. Where indicated in the studies, education or age adjustments are reported.

China, Malaysia, and Taiwan

Thirteen studies (N = 16,972) were validated across Chinese and Malay cohorts differentiating MCI from controls. These regions have been combined due to cross over in ethnicities involved in these studies. There were a number of validated adapted versions of the MoCA for this region: Beijing, Cantonese, Changsha, Hong Kong, and Singapore. Suggested cut-offs ranged from 13 to 26 to differentiate MCI from NC. Two studies employing large samples, suggest age [14] and education [23] adjusted cutoffs. Only one study investigated vascular cognitive impairment [46] suggesting a cut-off 26/27. Sensitivity ranged from 55% (at a specificity of 76%) to 96% (at a specificity of 76%). Specificity ranged from 62% (at sensitivity of 75%) to 97% (at sensitivity of 91%).

Lu et al. [23] noted significant effects of education, age, urban or rural residence, and sex on MoCA scores. Zhou et al. [47] raised concerns about applicability of MoCA-Cantonese in their low education sample (sensitivity: 49%). A number of studies retained the original suggestion of 1 point adjustment for <12 years education [14 , 47–51]. An education adjustment for <6 years of 1 point was made by a number of studies [20 , 53], while Tan et al. [14] suggested a 2 point adjustment for ≤6 years of education and Chu et al. [52] 2 points for illiteracy.

Two studies (N = 489) specifically investigated the MoCA-Singapore [54, 55]; both adjusted cut-offs for education at different levels, however, accuracy levels were poor with high FP and FN. Sensitivity ranged from 54% (at a specificity of 51%) to 96% (at a specificity of 31%). Specificity ranged from 19% (at sensitivity of 96%) to 57% (at sensitivity of 59%).

The MoCA-Taiwan was only validated to differentiate between dementia and NC.

South East and Southern Asia (Japan, Korea, Philippines, Sri Lanka, Thailand)

One study (N = 66) differentiating MCI from controls in Japan [56] suggested a cutoff of 25/26, in an educated sample, no education adjustment was used. One study (N = 85) in Thailand [57] suggested a cutoff of 24/25 in a poorly educated sample. One study (N = 152) in Korea [21] suggested a cut-off of 22/23 with a 1 point adjustment for <6 years of education. Julayanont et al. [57] made an adjustment of 1 point for <4 years education and an additional 1 point for meeting the illiteracy criteria. All studies displayed high sensitivity and specificity, but these results were based on small samples.

The MoCA-Philippines and the MoCA-Sri Lanka were only validated to differentiate between dementia and NC.

Europe (Germany, Holland, Poland, Portugal)

One study from each region (N = 639) validated the MoCA differentiating MCI from controls. Where education level was broadly comparable thesuggested cut-offs were 25/26, in the lower education Portuguese sample [32] the suggested cutoff was 22. Sensitivity ranged from 72% (at a specificity of 73%) to 93% (at a specificity of 62%). Specificity ranged from 54% (at sensitivity of 81%) to 77% (at sensitivity of 81%).

While the Polish [58] and German [59] studies retained the original 1 point adjustment, the Dutch study [60] implemented a 2 point adjustment for ≤12 years of education. Freitas et al. [23] did not include an adjustment for their sample.

Additionally, high FP rates were noted in the German, Polish, and Dutch studies with lower end CIs for specificity approaching or falling belowchance.

Middle East (Israel and Turkey)

One study from each region validated the MoCA differentiating MCI from controls. Kaya et al. [33] (N = 196) reported education adjusted cut-offs for the MoCA-Turkey; however there were high FP rates, with specificity around chance level, sensitivity was also low for those less educated.

The MoCA-Hebrew study (N = 154) [61] suggested a cut-off of 26. Education adjustments were not used in these studies.

South America (Brazil, Chile, Columbia)

Two studies (N = 168) validated the MoCA differentiating MCI from controls in Brazil [62, 63]; both reported a cut-off of 25 with adequate sensitivity and specificity. One study [64] (N = 68) validated the MoCA-Chile suggesting a cutoff of 24 in an educated sample. However, the FP rate was high, with wide CIs. In the Columbian sample [65] (N = 216) suggested a cut-off of 23 in an educated sample with adequate sensitivity and specificity.

Memoria et al. [63] retained the original 1 point adjustment, while Gil et al. [65] suggested it should be for <11 years education.

Dementia versus NC

Eighteen studies in total (N = 12,670) investigated the ability of the MoCA to differentiate dementia from NC. Studies, covariates, and forest plots are presented Fig. 5. Dementia types: Alzheimer’sdisease (10), Dementia with Lewy bodies (1), mixed type dementia (4), behavioral variant frontotemporal dementia (1), MCI and dementia –MCI+ (1), vascular dementia (1), and major NCD (1). Cutoffs ranged from 13 to 26. Sensitivity ranged from 75% (at a specificity of 53%) to 100% (at a specificity of 89%). Specificity ranged from 57% (at sensitivity of 75%) to 100% (at sensitivity of 94%).

Fig.5

Forest plot for studies differentiating between dementia and NC. a, b indicates education or age adjustment within the same study.

MCI versus Dementia

Seven studies in total (N = 1,087) investigated the ability of the MoCA to differentiate dementia from NC. Studies, covariates and forest plots are presented in Fig. 6. Cut-offs ranged from 15 to 25. Sensitivity ranged from 81% (at a specificity of 80%) to 100% (at a specificity of 96%). Specificity ranged from 70% (at sensitivity of 89%) to 98% (at sensitivity of 96%).

Fig.6

Forest plot for studies differentiating between MCI and dementia. a, b indicates education or age adjustment within the same study.

Heterogeneity

The studies included in the review are heterogeneous, with different demographic samples, recruitment settings, education levels, and variations in study population (e.g., severity of disease and co-morbidities). The study specific cut-offs adjust for the above factors within their sample populations. However, there are likely other factors such as age, education, spectrum (severity or stage) of the target disorder, operator characteristics (for index test and reference standard) and technical features (including different versions of the test and item adjustment) that have not been accounted for. Diagnostic criteria used will lead to diagnostic variability across studies in identifying MCI and subsequently affect the validity, reliability, sensitivity, and specificity of the MOCA in identifying MCI.

We assessed potential heterogeneity through visual inspection of risk of bias, descriptive data, and forest plots. We specified two factors that may contribute to heterogeneity explored by relating study level co-variates: quality item - selection of patients, and education. These were assessed effect by plotting SROCs by covariate.

We compared high, unclear, and low risk studies for the item, selection of patients, in SROC (summary of pairs of sensitivity & specificity in ROC space). These data suggest that accuracy varies with selection of studies with high/unclear/low (Fig. 7). Studies at high risk of bias reported higher level of accuracy than low risk studies and unclear studies reported lower accuracy.

Fig.7

SROC curve displaying influence of quality item, selection of patients on accuracy estimates.

Due to different reporting of education attainment, heterogeneity was assessed through visual inspection of forest plot (low to high education). It appeared that there was a mild distinction between primary and secondary educated (at the 8-year point). Below 8 years there was no clear pattern of improving sensitivity/specificity as hypothesized (pooled sensitivity/specificity: 74% /70%). For >8 years, sensitivity appears stable while specificity fluctuates (pooled sensitivity/specificity: 86% /70%). As such, there was no clear pattern for the effect of education.

DISCUSSION

The investigation of the cross-cultural applicability of the MoCA in screening for MCI, revealed a wide range of cut-offs both across countries and within region. As a result, the score of an individual could be considered either normal or MCI according to the cut-offs used and it is not possible to recommend adjusted cut-offs. Indeed, apart from China, there have been very few studies validating the measure in each region. Many translated forms of the MoCA have not been validated, and there are no published studies validating its use in large populations (e.g., use in different countries in Africa or the Middle East). A number of studies were not considered to have reached adequate sensitivity with high FP rates [33 , 58]. However, many of the studies indicated adequate to high sensitivity and specificity against reported specific study cut-off scores and education adjustments.

No clear pattern was identified for the effect of education. Years of education may not be equivalent across countries, where quality of education varies. Formal educational levels will differ between countries, where variation will exist in the organization of educational systems, access, learning outcomes and resources (e.g., teachers). Indeed, literacy in later life (as a measure of educational quality) is a stronger predictor of cognitive performance in older individuals than years of education [66]. In addition, some items in the MoCA (clock drawing, phonemic verbal fluency, and verbal abstraction) are likely to be influenced by education [43, 63].

The reviewed studies were primarily case-control studies, with small sample sizes, with some validations only carried out for dementia. As expected, the case-control studies (high risk of bias) appeared to over-estimate sensitivity and specificity, with studies with unclear risk of bias for selection of patients reporting lower accuracy (than low risk studies), possibly indicative of poor methodological quality. Item adjustments may also have effected scores with different forms being used across studies, this is especially pertinent to China where multiple versions exist. In this review, variation in both the diagnostic criteria and operational definitions precluded us from making quantitative interpretations of the influence of both factors. However, given that these variables substantially affect estimation of prevalence rates [11] it is likely that they will exert a strong influence on a comparison between studies.

A comprehensive review of the literature was conducted with attempts made to obtain translations of non-English papers and contact with authors was made where further information or additional data were required. Samples from studies conducted in the context of different specialist settings were included in this review as they were considered to have ecological validity applicable to clinicians in care settings. However, it is recognized that this gives rise to heterogeneity. The diversity of countries that have studied the MoCA both between and within cultures adds to the generalizability of the results. The review does not cover the influence of acculturation and use of interpreters which are likely to affect administration and reliability of scores. In a study of acculturation effects in North America, high FP rates and lower cut-offs were reported (even where English language was proficient) [67]. When using MoCA applicable to their country of origin, it is necessary to consider acculturation (positive and negative effects), regional variation, e.g., urban/rural, regional language variation, and quality of education.

Given the aging immigrant population across Western countries and the anticipated increase in dementia prevalence worldwide in coming decades, earlier and more accurate detection of MCI in these populations will be required. Although the MoCA is an effective brief cognitive assessment tool that has shown strong psychometric properties in several countries and shows particular advantage in culturally and linguistically diverse populations, our results highlight the need for caution when using MoCA as a screen for MCI without appropriately accounting for the effects of education, language, and cultural diversity. Hu et al. [20] found several cultural differences in practice in a Chinese version of MoCA. Subjects with lower levels of education did not understand how to perform the ‘Alternating Trail Making’ test. This has also been reported in a study in subjects from Hong Kong [68]. In the ‘Naming tests’, 58.2% of subjects were unable to name the ‘rhinoceros’ and ‘camel’ since the animal is not within the realm of their general understanding or vocabulary. In the word memory test, the original words are not easy to memorize for Chinese people especially the words ‘velvet’ and ‘church’, which fall outside Chinese cultural background. In the ‘Abstract thinking’ test, the ‘watch’ and ‘ruler’ were only correctly remembered by 41.8% of subjects, and even among those of a higher level of education they were only correctly identified 50% of the time [20].

Due to the differences in cultures, further revision is required to tailor the contents more accurately to local populations and to highlight the difficulty in separating the influence of culture from education level. In immigrant populations, additional caution should be given to interpretation of scores when using forms for the country of origin or English language forms due to acculturation effects.

It is also necessary to redefine cut-off rates based on age and education-specific normative values so that individual test items can be modified appropriately, enabling better content validity (i.e., type and quality of the test items to assess mild impairments in cognitive functioning). There is need for future research to provide consistent data reporting conventions to facilitate direct comparison across countries. Future research, may also make use of education classification systems (e.g., International Standard Classification of Education) [65] to facilitate more accurate cross-cultural comparisons of the effects of education. Validations need to be carried out on different versions, especially those available on the MoCA website as these can appear to be endorsed by the creators. Further research could also explore the effect of acculturation on MoCA performance as this is especially pertinent given the aging migratory populations. Identification of salient domains which show cultural variability or have greater/less reliability could also inform development of MoCA forms for unvalidated populations.

In conclusion, the findings of this review suggest that it would be more appropriate to use cutoffs and point adjustments for education as suggested by validation authors (not those of the original paper). The ‘low’ quality of validation papers and likelihood that accuracy is overestimated highlighted the need for clinicians and researchers to use the MoCA appropriately, that is, as a brief, cognitive screening tool to identify individuals who may need an additional comprehensive examination. Furthermore, it is important to stress that test scores should be interpreted in light of other clinical data, such as clinical history, collateral information, and behavioral observations [25]. It should be noted that although the MoCA has been adapted across languages and cultures, it takes little account of acculturation factors. Utilizing cognitive screens can be implemented to obtain objective data, however, this review indicates the limitations and suitability of using versions of the MOCA within the context of individual and cultural factors.

In addition, the reported range of cut-offs in validated papers suggests that there is questionable utility in using unvalidated measures and direct translations.

DISCLOSURE STATEMENT

Authors’ disclosures available online (http://j-alz.com/manuscript-disclosures/16-1042r3).

Footnotes

Appendix

References

Levey

, Lah

, Goldstein

, Steenland

, Bliwise

(2006) Mild cognitive impairment: An opportunity to identify patients at high risk for progression to Alzheimer’s disease. Clin Ther 28, 991–1001.

Ritchie

, Artero

, Touchon

(2001) Classification criteria for mild cognitive impairment: A population-based validation study. Neurology 56, 37–42.

Stern

(2012) Cognitive reserve in ageing and Alzheimer’s disease. Lancet Neurol 11, 1006–1012.

Jansen

, Ossenkoppele

, Knol

, Tijms

, Scheltens

, Verhey

FRJ

, Visser

, Aalten

, Aarsland

, Alcolea

, Alexander

, Almdahl

, Arnold

, Baldeiras

, Barthel

, van Berckel

BNM

, Bibeau

, Blennow

, Brooks

, van Buchem

, Camus

, Cavedo

, Chen

, Chetelat

, Cohen

, Drzezga

, Engelborghs

, Fagan

, Fladby

, Fleisher

, van der Flier

, Ford

, Förster

, Fortea

, Foskett

, Frederiksen

, Freund-Levi

, Frisoni

, Froelich

, Gabryelewicz

, Gill

, Gkatzima

, Gómez-Tortosa

, Gordon

, Grimmer

, Hampel

, Hausner

, Hellwig

, Herukka

S-K

, Hildebrandt

, Ishihara

, Ivanoiu

, Jagust

, Johannsen

, Kandimalla

, Kapaki

, Klimkowicz-Mrowiec

, Klunk

, Köhler

, Koglin

, Kornhuber

, Kramberger

, Van Laere

, Landau

, Lee

, de Leon

, Lisetti

, Lleó

, Madsen

, Maier

, Marcusson

, Mattsson

, de Mendonça

, Meulenbroek

, Meyer

, Mintun

, Mok

, Molinuevo

, Møllergård

, Morris

, Mroczko

, Van der Mussele

, Na

, Newberg

, Nordberg

, Nordlund

, Novak

, Paraskevas

, Parnetti

, Perera

, Peters

, Popp

, Prabhakar

, Rabinovici

, Ramakers

IHGB

, Rami

, Resende de Oliveira

, Rinne

, Rodrigue

, Rodríguez-Rodríguez

, Roe

, Rot

, Rowe

, Rüther

, Sabri

, Sanchez-Juan

, Santana

, Sarazin

, Schröder

, Schütte

, Seo

, Soetewey

, Soininen

, Spiru

, Struyfs

, Teunissen

, Tsolaki

, Vandenberghe

, Verbeek

, Villemagne

, Vos

SJB

, van Waalwijk van Doorn

LJC

, Waldemar

, Wallin

ÅK

, Wiltfang

, Wolk

, Zboch

, Zetterberg

(2015) Prevalence of Cerebral Amyloid Pathology in Persons Without Dementia. JAMA 313, 1924.

Saleem

, Herrmann

, Swardfager

, Eisen

, Lanctôt

(2015) Inflammatory markers in mild cognitive impairment: A meta-analysis. J Alzheimers Dis 47, 669–679.

Albert

, DeKosky

, Dickson

, Dubois

, Feldman

, Fox

, Gamst

, Holtzman

, Jagust

, Petersen

, Snyder

, Carrillo

, Thies

, Phelps

(2011) The diagnosis of mild cognitive impairment due to Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 7, 270–279.

Petersen

(2011) Mild cognitive impairment. N Engl J Med 364, 2227–2234.

Jak

, Bondi

, Delano-Wood

, Wierenga

, Corey-Bloom

, Salmon

, Delis

(2009) Quantification of five neuropsychological approaches to defining mild cognitive impairment. Am J Geriatr Psychiatry 17, 368–375.

Canevelli

, Grande

, Lacorte

, Quarchioni

, Cesari

, Mariani

, Bruno

, Vanacore

(2016) Spontaneous reversion of mild cognitive impairment to normal cognition: A systematic review of literature and meta-analysis. J Am Med Dir Assoc 17, 943–948.

10.

Pandya

, Clem

, Silva

, Woon

(2016) Does mild cognitive impairment always lead to dementia? A review. J Neurol Sci 369, 57–62.

11.

Ward

, Arrighi

, Michels

, Cedarbaum

(2012) Mild cognitive impairment: Disparity of incidence and prevalence estimates. Alzheimers Dement 8, 14–21.

12.

Roberts

, Knopman

(2013) Classification and epidemiology of MCI. Clin Geriatr Med 29, 753–772.

13.

Nasreddine

, Phillips

, Bedirian

, Charbonneau

, Whitehead

, Collin

, Cummings

, Chertkow

(2005) The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53, 695–699.

14.

Tan

, Li

, Gao

, Wang

, Zhao

, Yu

, Du

, Zhang

, Cui

, Wang

, Li

, Yang

, Yu

, Xia

, Zhou

(2015) Optimal cutoff scores for dementia and mild cognitive impairment of the Montreal Cognitive Assessment among elderly and oldest-old Chinese population. J Alzheimers Dis 43, 1403–1412.

15.

Porter

, Samovar

(1994) An introduction to intercultural communication. In Intercultural communication: A reader, Samovar

, Porter

, eds. Wadsworth, Belmont, CA, pp. 4–26.

16.

Zapf

(1987) Individualisierung und Sicherheit: Untersuchungen zur Lebensqualität in der Bundesrepublik Deutschland, Beck.

17.

Ardila

, Ostrosky-Solis

, Mendoza

(2000) Learning to read is much more than learning to read: A neuropsychologically based reading program. J Int Neuropsychol Soc 6, 789–801.

18.

van de Vijver

, Tanzer

(2004) Bias and equivalence in cross-cultural assessment: An overview. Eur Rev Appl Psychol 54, 119–135.

19.

Dominguez

, Orquiza

MGS

, Soriano

, Magpantay

, Esteban

, Corrales

, Ampil

(2013) Adaptation of the Montreal Cognitive Assessment for elderly Filipino patients. East Asian Arch Psychiatry 23, 80–85.

20.

, Zhou

, Hu

, Huang

, Wei

, Qi

, Huang

, Xu

(2013) Cross-cultural difference and validation of the Chinese version of Montreal Cognitive Assessment in older adults residing in Eastern China: Preliminary findings. Arch Gerontol Geriatr 56, 38–43.

21.

Lee

J-YJ

, Lee

, Cho

, Na

, Jeon

, Kim

, Lee

, Youn

, Kwon

, Lee

J-YJ

, Cho

(2008) Brief screening for mild cognitive impairment in elderly outpatient clinic: Validation of the Korean version of the Montreal Cognitive Assessment. J Geriatr Psychiatry Neurol 21, 104–110.

22.

Tombaugh

, Kozak

, Rees

(1999) Normative data stratified by age and education for two measures of verbal fluency: FAS and animal naming. Arch Clin Neuropsychol 14, 167–77.

23.

, Li

, Zhou

, Wang

, Zuo

, Jia

X-F

, Song

, Jia

(2011) Montreal Cognitive Assessment in detecting cognitive impairment in Chinese elderly individuals: A population-based study. J Geriatr Psychiatry Neurol 24, 184–190.

24.

Hedden

, Park

, Nisbett

, Ji

L-J

, Jing

, Jiao

(2002) Cultural variation in verbal versus spatial neuropsychological function across the life span. Neuropsychology 16, 65–73.

25.

Wong

, Law

, Liu

, Wang

, Lo

, Lau

, Wong

, Mok

(2015) Montreal Cognitive Assessment: One cutoff never fits all. Stroke 46, 3547–3550.

26.

Friedman

, Miyake

, Corley

, Young

, DeFries

, Hewitt

(2006) Not all executive functions are related to intelligence. Psychol Sci 17, 172–179.

27.

Lewis

, Koyasu

, Oh

, Ogawa

, Short

, Huang

(2009) Culture, executive function, and social understanding. New Dir Child Adolesc Dev 2009, 69–85.

28.

Razani

, Burciaga

, Madore

, Wong

(2007) Effects of acculturation on tests of attention and information processing in an ethnically diverse group. Arch Clin Neuropsychol 22, 333–341.

29.

Razani

, Murcia

, Tabares

, Wong

(2006) The effects of culture on WASI test performance in ethnically diverse individuals. Clin Neuropsychol 21, 773–741.

30.

Boone

, Victor

, Wen

, Razani

, Pont

(2007) The association between neuropsychological scores and ethnicity, language, and acculturation variables in a large patient population. Arch Clin Neuropsychol 22, 355–365.

31.

Fernández

, Marcopulos

(2008) A comparison of normative data for the Trail Making Test from several countries: Equivalence of norms and considerations for interpretation. Scand J Psychol 49, 239–246.

32.

Freitas

, Simões

, Alves

, Santana

(2013) Montreal Cognitive Assessment. Alzheimer Dis Assoc Disord 27, 37–43.

33.

Kaya

, Aki

, Can

, Derle

, Kibaro lu

, Barak

(2014) Validation of Montreal Cognitive Assessment and discriminant power of Montreal Cognitive Assessment subtests in patients with mild cognitive impairment and Alzheimer dementia in Turkish population. J Geriatr Psychiatry Neurol 27, 103–109.

34.

Jack

, Albert

, Knopman

, McKhann

, Sperling

, Carrillo

, Thies

, Phelps

(2011) Introduction to the recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 7, 257–262.

35.

Pathway-Dementia

NICE

(2006) Dementia: Supporting people with dementia and their carers in health and social care NICE guidelines [CG42] Published date: November 2006. Last updated: April 2014.

36.

Davis

DHJ

, Creavin

, Yip

JLY

, Noel-Storr

, Brayne

, Cullum

(2015) Montreal Cognitive Assessment for the diagnosis of Alzheimer’s disease and other dementias. Cochrane Database Syst Rev, CD010775.

37.

Moher

, Liberati

, Tetzlaff

, Altman

, PRISMA

Group

(2009) Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Med 6, e1000097.

38.

American Psychiatric Association (2000) Diagnostic and Statistical Manual of Mental Disorders, Washington, DC.

39.

World Health Organization (1993) The ICD-10 classification of mental and behavioural disorders: Diagnostic criteria for research.

40.

Hyman

, Phelps

, Beach

, Bigio

, Cairns

, Carrillo

, Dickson

, Duyckaerts

, Frosch

, Masliah

, Mirra

, Nelson

, Schneider

, Thal

, Thies

, Trojanowski

, Vinters

H V.

, Montine

(2012) National Institute on Aging–Alzheimer’s Association guidelines for the neuropathologic assessment of Alzheimer’s disease. Alzheimers Dement 8, 1–13.

41.

Petersen

(2004) Mild cognitive impairment as a diagnostic entity. J Intern Med 256, 183–94.

42.

Roman

, Tatemichi

, Erkinjuntti

, Cummings

, Masdeu

, Garcia

, Amaducci

, Orgogozo

J-M

, Brun

, Hofman

, Moody

, O’Brien

, Yamaguchi

, Grafman

, Drayer

, Bennett

, Fisher

, Ogata

, Kokmen

, Bermejo

, Wolf

, Gorelick

, Bick

, Pajeau

, Bell

, DeCarli

, Culebras

, Korczyn

, Bogousslavsky

, Hartmann

, Scheinberg

(1993) Vascular dementia: Diagnostic criteria for research studies: Report of the NINDS-AIREN International Workshop. Neurology 43, 250–250.

43.

Morris

(1993) The Clinical Dementia Rating (CDR): Current version and scoring rules. Neurology 43, 2412–2412.

44.

Whiting

, Rutjes

AWS

, Westwood

, Mallett

, Deeks

, Reitsma

, Leeflang

MMG

, Sterne

JAC

, Bossuyt

PMM

, QUADAS-2 Group (2011) QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155, 529–36.

45.

(2012) Review Manager (RevMan).

46.

Q-Y

, Jin

, Ding

B-R

, Yang

, Lei

Z-H

, Bai

, Zhang

Y-D

, Tang

X-Q

(2013) Reliability, validity, and optimal cutoff score of the Montreal Cognitive Assessment (Changsha version) in ischemic cerebrovascular disease patients of Hunan province, China. Dement Geriatr Cogn Dis Extra 3, 25–36.

47.

Zhou

, Zhu

, Zhang

, Wang

, Li

, Lv

, Ng

, Yu

, Wang

(2014) The influence of education on Chinese version of Montreal Cognitive Assessment in detecting amnesic mild cognitive impairment among older people in a Beijing rural community. ScientificWorldJournal 2014, 689456.

48.

Dong

, Lee

, Hilal

, Saini

, Wong

, Chen

CL-H

, Venketasubramanian

, Ikram

(2013) Comparison of the Montreal Cognitive Assessment and the Mini-Mental State Examination in detecting multi-domain mild cognitive impairment in a Chinese sub-sample drawn from a population-based study. Int Psychogeriatrics 25, 1831–1838.

49.

(2008) The validity of the Montreal cognitive assessment (Cantonese version) as a screening tool for mild cognitive impairment in Hong Kong Chinese (Thesis). University of Hong Kong, Pokfulam, Hong Kong SAR. https://dx-doi-org.web.bisu.edu.cn/10.5353/th_b4100566

50.

, Li

, Huang

(2012) The Beijing version of the Montreal Cognitive Assessment as a brief screening tool for mild cognitive impairment: A community-based study. BMC Psychiatry 12, 156.

51.

Zhao

, Liu

, Shen

, Zhao

(2011) Reasonable neuropsychological battery to identify mild cognitive impairment. Med Hypotheses 76, 50–53.

52.

Chu

, Ng

KHY

, Law

ACK

, Lee

, Kwan

(2015) Validity of the cantonese Chinese montreal cognitive assessment in Southern Chinese. Geriatr Gerontol Int 15, 96–103.

53.

Yeung

, Wong

, Chan

, Leung

JLM

, Yung

(2014) A validation study of the Hong Kong version of Montreal Cognitive Assessment (HK-MoCA) in Chinese older adults in Hong Kong. Hong Kong Med J 20, 504–510.

54.

, Chew

, Narasimhalu

, Kandiah

(2013) Effectiveness of Montreal Cognitive Assessment for the diagnosis of mild cognitive impairment and mild Alzheimer’s disease in Singapore. Singapore Med J 54, 616–619.

55.

, Feng

, Lim

, Chong

, Lee

, Yap

, Tsoi

, Liew

, Gao

, Collinson

, Kandiah

, Yap

(2015) Montreal Cognitive Assessment for screening mild cognitive impairment: Variations in test performance and scores by education in Singapore. Dement Geriatr Cogn Disord 39, 176–185.

56.

Fujiwara

, Suzuki

, Yasunaga

, Sugiyama

, Ijuin

, Sakuma

, Inagaki

, Iwasa

, Ura

, Yatomi

, Ishii

, Tokumaru

, Homma

, Nasreddine

, Shinkai

(2010) Brief screening tool for mild cognitive impairment in older Japanese: Validation of the Japanese version of the Montreal Cognitive Assessment. Geriatr Gerontol Int 10, 225–232.

57.

Julayanont

, Tangwongchai

, Hemrungrojn

, Tunvirachaisakul

, Phanthumchinda

, Hongsawat

, Suwichanarakul

, Thanasirorat

, Nasreddine

(2015) The Montreal Cognitive Assessment-Basic: A screening tool for mild cognitive impairment in illiterate and low-educated elderly adults. J Am Geriatr Soc 63, 2550–2554.

58.

Magierska

, Magierski

, Fendler

, Kloszewska

, Sobow

(2012) Clinical application of the Polish adaptation of the Montreal Cognitive Assessment (MoCA) test in screening for cognitive impairment. Neurol Neurochir Pol 46, 130–139.

59.

Costa

, Fimm

, Friesen

, Soundjock

, Rottschy

, Gross

, Eitner

, Reich

, Schulz

, Nasreddine

, Reetz

(2012) Alternate-form reliability of the Montreal cognitive assessment screening test in a clinical setting. Dement Geriatr Cogn Disord 33, 379–384.

60.

Thissen

AJAM

, van Bergen

, de Jonghe

JFM

, Kessels

RPC

, Dautzenberg

PLJ

(2010) [Applicability and validity of the Dutch version of the Montreal Cognitive Assessment (moCA-d) in diagnosing MCI]. Tijdschr Gerontol Geriatr 41, 231–240.

61.

Lifshitz

, Dwolatzky

, Press

(2012) Validation of the Hebrew version of the MoCA test as a screening instrument for the early detection of mild cognitive impairment in elderly individuals. J Geriatr Psychiatry Neurol 25, 155–161.

62.

Martinelli

, Cecato

, Bartholomeu

, Montiel

(2014) Comparison of the diagnostic accuracy of neuropsychological tests in differentiating Alzheimer’s disease from mild cognitive impairment: Can the Montreal Cognitive Assessment be better than the Cambridge cognitive examination? Dement Geriatr Cogn Dis Extra 4, 113–121.

63.

Memoria

, Yassuda

, Nakano

, Forlenza

(2013) Brief screening for mild cognitive impairment: Validation of the Brazilian version of the Montreal cognitive assessment. Int J Geriatr Psychiatry 28, 34–40.

64.

Munoz-Neira

, Henriquez Chaparro

, Delgado

, Brown

, Slachevsky

(2014) Test Your Memory-Spanish version (TYM-S): A validation study of a self-administered cognitive screening test. Int J Geriatr Psychiatry 29, 730–740.

65.

Gil

, Ruiz de Sánchez

, Gil

, Romero

, Pretelt Burgos

(2015) Validation of the Montreal Cognitive Assessment (MoCA) in Spanish as a screening tool for mild cognitive impairment and mild dementia in patients over 65 years old in Bogotá, Colombia. Int J Geriatr Psychiatry 30, 655–662.

66.

Sisco

, Gross

, Shih

, Sachs

, Glymour

, Bangen

, Benitez

, Skinner

, Schneider

, Manly

(2015) The role of early-life educational quality and literacy in explaining racial disparities in cognition in late life. J Gerontol B Psychol Sci Soc Sci 70, 557–567.

67.

Goldstein

, Ashley

A V.

, Miller

, Alexeeva

, Zanders

, King

(2014) Validity of the Montreal Cognitive Assessment as a screen for mild cognitive impairment and dementia in African Americans. J Geriatr Psychiatry Neurol 27, 199–203.

68.

Wong

, Xiong

, Kwan

PWL

, Chan

AYY

, Lam

WWM

, Wang

, Chu

WCW

, Nyenhuis

, Nasreddine

, Wong

LKS

, Mok

VCT

(2009) The validity, reliability and clinical utility of the Hong Kong Montreal Cognitive Assessment (HK-MoCA) in patients with cerebral small vessel disease. Dement Geriatr Cogn Disord 28, 81–87.