Evaluation of the Accuracy of Cognitive Screening Tests in Detecting Dementia Associated with Alzheimer’s Disease: A Hierarchical Bayesian Latent Class Meta-Analysis

Abstract

Background:

Montreal Cognitive Assessment (MoCA) and Mini-Mental State Examination (MMSE) are neuropsychological tests commonly used by physicians for screening cognitive dysfunction of Alzheimer’s disease (AD). Due to different imperfect reference standards, the performance of MoCA and MMSE do not reach consensus. It is necessary to evaluate the consistence and differentiation of MoCA and MMSE in the absence of a gold standard for AD.

Objective:

We aimed to assess the accuracy of MoCA and MMSE in screening AD without a gold standard reference test.

Methods:

Studies were identified from PubMed, Web of Science, CNKI, Chinese Wanfang Database, China Science and Technology Journal Database, and Cochrane Library. Our search was limited to studies published in English and Chinese before August 2021. A hierarchical Bayesian latent class model was performed in meta-analysis when the gold standard was absent.

Results:

A total of 67 studies comprising 5,554 individuals evaluated for MoCA and 76,862 for MMSE were included in this meta-analysis. The pooled sensitivity was 0.934 (95% CI 0.905 to 0.954) for MoCA and 0.883 (95% CI 0.859 to 0.903) for MMSE, while the pooled specificity was 0.899 (95% CI 0.859 to 0.928) for MoCA and 0.903 (95% CI 0.879 to 0.923) for MMSE. MoCA was useful to rule out dementia associated with AD with lower negative likelihood ratio (LR-) (0.074, 95% CI 0.051 to 0.108). MoCA showed better performance with higher diagnostic odds ratio (DOR) (124.903, 95% CI 67.459 to 231.260).

Conclusion:

MoCA had better performance than MMSE in screening dementia associated with AD from patients with mild cognitive impairment or healthy controls.

Keywords

Alzheimer’s disease meta-analysis Mini-Mental State Examination Montreal Cognitive Assessment sensitivity specificity

INTRODUCTION

Alzheimer’s disease (AD), the most common cause of dementia, is a progressive neurodegenerative condition characterized by cognitive impairment, memory loss, behavioral abnormality, and social disorders [1, 2]. At present, over 50 million people around the world suffer from dementia, and this number is predicted to rise to 152 million by 2050 in low- and middle-income countries with an aging population [1]. AD affects both individuals and their families with huge social, economic, and health burdens; therefore, a great concern has been raised in the global public health field [1, 2]. Moreover, AD is an irreversible and incurable condition [1, 3]. Thus, early-stage screening and the accurate identification of AD are of significant importance in clinical practice.

Screening, assessing, and diagnosing AD is a stepwise process in primary care, and screening cognitive impairment in clinical practice settings can increase recognition of AD dementia [3, 4]. Cognitive screening tests are a relatively inexpensive neuropsychological measurement that has been recommended as an important component of diagnostic tests in detecting dementia associated with AD [5]. Cognitive screening tests such as the Montreal Cognitive Assessment (MoCA) [6] and Mini-Mental State Examination (MMSE) [7] are commonly used by physicians for screening of cognitive impairment in AD patients [8 –10]. Early detection with a positive screening of MMSE and MoCA is the first step in facilitating a full diagnosis and early appropriate clinical interventions [4, 10]. At present, dozens of validation studies showed MoCA was superior to MMSE in detecting cognitive impairment of mild AD by the international reference standards [11 –13].

In fact, full diagnostic reference standards for AD are mainly based on clinical diagnosis and biomarkers, which are recommended by the National Institute of Neurological and Communicative Diseases and Stroke/Alzheimer’s Disease and Related Disorders Association (NINCDS-ADRDA) [5], National Institute on Aging-Alzheimer’s Association (NIA-AA) [14], International Working Group (IWG) [15], and Diagnostic and Statistical Manual of Mental Disorders (DSM) [16]. Currently, the NIA-AA research framework defines AD in vivo by biomarkers [14], which is more accurate than clinical observations and usually regarded as the “gold standard” by some researchers. However, none of the biomarkers is as sensitive as the direct examination of tissue at autopsy [14]. Researchers reported that the overall accuracy of clinical diagnosis, biomarkers, neuroimaging, and genetic testing in diagnosing AD are never 100% accurate when confirmed by autopsy validation [17 –21]. Khan and Alkon [17] found that the sensitivity and specificity of clinical diagnosis were 78% and 20% and biomarkers were 97% and 100%, respectively. Additionally, the sensitivity of NINCDS-ADRDA clinical criteria ranged from 76% to 93%, and the specificity ranged from 55% to 91%, compared with neuropathological verification [19]. Until recently, the best standard to define AD is by longitudinal follow-up with biomarkers in living people and by postmortem examination based on clinical symptoms [14 , 22]. Nevertheless, current biomarkers used in AD research are either expensive or invasive and are difficult to obtain in the preliminary preclinical screening [14]. A brain autopsy is also an unavailable option for the living people. Thus, validation studies of MoCA and MMSE in detecting dementia associated with AD were always based on imperfect reference standards. There is evidence that ignoring the imperfect reference standard could result in substantial bias in pooled estimations to evaluate the accuracy of tests in meta-analysis [23, 24]. This type of bias is called imperfect gold standard bias in the field of statistics [23]. Clinical research without biomarkers remains a significant challenge [14]. An accurate screening test is an essential prelude to early discovery and intervention. Therefore, it is necessary to evaluate the accuracy of MoCA and MMSE in the absence of a gold standard to assess the imperfect gold standard bias in meta-analysis. In addition, characteristics of subjects (i.e., sex, age, education) in published studies could affect different positivity thresholds for cognitive screening tests [25, 26]. Various positivity thresholds then become another source of inconsistency in diagnostic performance [27]. The hierarchical summary receiver operating characteristic (HSROC) model is a recommended method to assess the variation when there are different positivity cutoff values in the included diagnostic accuracy studies [27]. Dendukuri et al. developed a hierarchical Bayesian latent-class approach based on the HSROC model to control the imperfect gold standard bias with a hope to produce unbiased pooled outcomes in the absence of a gold standard [24].

So far, several reviews and meta-analyses have studied the performance of the MoCA and MMSE in screening mild cognitive impairment (MCI) or all-cause dementia [28 –32]. Pinto et al. showed the pooled area under the curve (AUCs) varied from 0.87 to 0.99 for MoCA, and 0.67 to 0.99 for MMSE in screening AD in the elderly [28]. Two meta-analyses used the HSROC model to evaluate the accuracy of MMSE at various cutoff points for all-cause dementia and subtypes (Alzheimer’s disease dementia, Lewy body dementia, vascular dementia, and frontotemporal dementia) with international reference standards [30, 31]. Currently, outcomes of cognitive screening tests in published meta-analyses vary tremendously due to different reference standards. To our knowledge, there is no review and meta-analysis in the literature on the accuracy of MoCA and MMSE in detecting cognitive impairment of AD in the absence of a gold standard.

In this study, we performed a meta-analysis with the hierarchical Bayesian latent-class model for all available eligible studies to evaluate the performance of MoCA and MMSE for AD screening. We aimed to compare the accuracy of MoCA and MMSE in screening for AD from MCI or normal cognitive function in the absence of a gold standard. It is significant to verify the conclusions of the published studies using statistical methods to pool sensitivity and specificity in a meta-analysis without a gold standard reference.

METHODS

Data collection and search strategy

We conducted this meta-analysis following the Preferred Reporting Items for Systematic Reviews and Meta-analyses extension for Diagnostic Test Accuracy Studies (PRISMA-DTA) guidelines [33] (see Supplementary Table 1). A comprehensive search strategy combining the key words and Medical Subject Heading (MeSH) terms was utilized to identify relevant studies in PubMed, Web of Science, China National Knowledge Infrastructure, Chinese Wanfang Database, China Science and Technology Journal Database, and Cochrane Library. Our search was limited to studies published in English and Chinese before July 15, 2021. Key words used were “Alzheimer’s disease”, “Montreal Cognitive Assessment”, “Mini-Mental State Examination”, “AD”, “MOCA”, “MMSE”, “Sensitivity and Specificity”. Reference lists of previous reviews and meta-analyses were identified and reviewed to reveal other potentially relevant studies. The strategy used for searching PubMed database can be found in Supplementary Table 2.

Inclusion and exclusion criteria

We selected studies based on the following criteria: 1) patients with AD and MCI, and healthy controls (HC); 2) validation study using the MoCA and MMSE as cognitive screening tests (both with a total score of 30); 3) constructed validity of the above cognitive screening tests compared to the reference standard (recommendations from NIA-AA, NINCDS-ADRDA and IWG, the DSM criteria and other approved criteria); 4) reported the results in English or Chinese with sufficient information to reconstruct a 2×2 contingency table. The exclusion criteria were: 1) the study included patients with cognitive impairment caused by other diseases (post-stroke, Parkinson’s disease, cerebral infarction, and vascular dementia), other systemic diseases (e.g., thyroid dysfunction, severe anemia, syphilis, HIV, or abnormalities in vitamin B12), major psychiatric disorders (e.g., psychotic disorder, bipolar disorder, substance abuse), psychotropic drugs (antidepressant, antipsychotics, hypnotics, etc.) and severe or unstable medical condition; 2) the publication provided insufficient information to construct a contingency table. The detailed reasons for exclusion are listed in the flow diagrams (Fig. 1). Two authors (F.J.L. and Z.J.) independently screened selected eligible studies for inclusion.

Fig. 1

PRISMA Flow chart of study selection. MoCA (a) and MMSE (b).

Data extraction

Data from each selected study were extracted independently by two authors (F.J.L. and Z.J.). Discrepancies in opinion were resolved by discussion with two additional reviewers (X.N.W. and H.P.Z). If there were different validation groups in the same study, we included all of them. If multiple reference standards were provided in the article, we extracted all of them for our analysis. If more than one cutoff were reported in a study, we selected the optimal value with the best balance of sensitivity and specificity.

Sensitivity, specificity, numbers of true positive (TP), false negative (FN), false positive (FP), and true negative (TN) values were collected from the eligible study. Descriptive information for patient characteristics (sample size, age, gender, and education status) were extracted for analysis. We also listed the study information including title, first author, year of publication, country of study, cognitive type, reference standard, and cutoff values. When needed, we contacted the corresponding author online to ask for missing information. An article was dropped from our analysis if there was no response from the author.

Quality assessment

Two authors (F.J.L. and Z.J.) independently assessed the risk of bias and applicability of the diagnostic accuracy studies using the Quality Assessment of Diagnostic Accuracy Studies 2 (QUADAS-2) checklist [34]. We focused on four domains: patient selection, index test, reference standard, and flow and timing. Risk of bias and concern of applicability for each domain were categorized as “low”, “high”, or “unclear”. Any disagreement was resolved by discussion with two additional reviewers (X.N.W. and H.P.Z).

Statistical analysis

With available information extracted from the eligible studies, we converted the results of two cognitive screening tests into dichotomous (AD versus non-AD). The TP, FP, FN, and TN values in contingency tables were directly obtained or calculated indirectly according to the sensitivity and specificity. We used the HSROC model of Rutter and Gatsonis [27] to evaluate variation in diagnostic accuracy and cutoff values. The pooled sensitivity, specificity, diagnostic odds ratio (DOR), positive and negative likelihood ratio (LR+, LR-) and their associated 95% confidence intervals (CIs) were jointly estimated by a hierarchical Bayesian latent class model [24] in the absence of a gold standard. HSROC curves were drawn to show the diagnostic effects under different cut-off values.

LRs, including LR+ and LR-, were generated by sensitivity and specificity. It is a ratio of the probability that a test result is correct to the probability that the test is incorrect [35, 36]. We used LR statistics to summarize the properties of cognitive screening tests with higher LR+ meaning “rule in disease” and lower LR- meaning “rule out disease” in clinical research [35, 36]. Furthermore, the post-test probability can be calculated by LR and pre-test probability using Bayes theorem [35, 36]. We assessed that the pre-test probability of AD was 35% according to the reported characteristics and prevalence of all included studies [37, 38]. The results were displayed in Fagan plot.

Heterogeneity was evaluated using Cochran’s Q test or inconsistency index (I²), with p < 0.05 or I² > 50% indicating significant heterogeneity [39]. Meta-regression analyses were performed to explore potential sources of heterogeneity. Deeks’ analysis was used to assess publication bias, with p < 0.05 suggesting that publication bias might exist [40].

All results of meta-analysis were performed by R (version 3.6.3), Stata (version 14.0) and the Cochrane Collaboration Review Manager (version 5.4) statistical software.

RESULTS

Study identification and characteristics

A total of 67 studies with at least one of the two cognitive screening tests (MoCA and MMSE) conducted across more than 20 countries were included in this meta-analysis, with 29 datasets from 26 studies for MoCA (Fig. 1a) and 67 datasets from 56 studies for MMSE (Fig. 1b). Of these, 16 were from China, 9 from Brazil, 6 from Korea, 6 from the United States, and 5 from Spain. The sample size ranged from 31 to 17,555 with a total of 76,862 individuals for MMSE and 5,554 for MoCA included in our final analysis. The detailed characteristics of the eligible studies were summarized in Table 1.

Table 1

Characteristics of included studies

Author	Year	Country	N	Age, y±SD	Population	Female (%)	Education, y±SD	Reference standard	Index test	TP	FP	FN	TN
A) Montreal Cognitive Assessment (MoCA)
Memoria [41]	2013	BR	82	74.79±5.67	AD	69.51	21.23±6.74	NINCDS-ARDRA, Petersen’s criteria	MoCA≤22	22	0	2	58
Lee [42]	2008	KR	196	69.81±6.77	AD	64.80	8.03±3.60	DSM-IV, CDR0.5 or 1, Petersen’s criteria	MoCA≤19/20	41	5	3	147
Nasreddine [6]	2005	CA	183	74.81±8.23	Mild AD/HC	59.56	11.65±3.99	DSM-IV, NINCDS-ARDRA	MoCA≤26	93	12	0	78
Ng [43]	2013	SG	109	68.02±9.66	Mild AD/MCI	47.71	8.75±4.81	NINCDS-ARDRA, Petersen’s criteria	MoCA≤25	54	15	6	34
Matias-Guiu [44]	2017	ES	160	78.27±6.52	Mild AD/HC	65.00	7.49±4.70	NIA-AA, GDS = 4	MoCA≤14	65	11	27	57
Chang [45]	2012	TW, CN	149	67.33±9.19	Very-mild AD/HC	44.97	9.57±4.50	DSM-IV, NINCDS-ARDRA, CDR = 1	MoCA≤22/23	43	12	9	85
Magierska [46]	2012	PL	115	73.96±6.17	AD/(MCI + HC)	73.91	12.34±4.59	DSM-IV, ICD, CDR, Petersen’s criteria	MoCA≤19	31	14	5	65
Nogueira [47]	2018	PT	410	69.13±9.28	AD/HC	60.98	7.23±4.32	NINCDS-ADRDA	MoCA < 17	192	8	8	202
Freitas [48]	2012	PT	270	72.30±7.86	AD/HC	59.40	6.34±4.25	DSM-IV-TR, NINCDS-ADRDA	MoCA < 17	79	4	11	176
Roalf [8]	2013	US	461	74.32±8.77	AD/HC	63.56	14.11±4.02	CERAD-NB, DSRS	MoCA≤23	302	6	19	134
Hu [49]	2013	CN	218	67.60±5.02	AD/HC	55.96	9.17±2.78	DSM-IV-TR, NINCDS-ADRDA	MoCA≤25/26	66	6	6	140
Tsai [50]	2012	TW, CN	136	79.07±6.35	AD/HC	36.03	8.88±4.83	NINCDS-ADRDA	MoCA≤21/22	96	2	2	36
Chu [51]	2015	HK, CN	179	74.45±6.71	AD/HC	70.39	6.11±4.94	NINCDS-ADRDA	MoCA≤19/20	60	9	4	106
Mellor [26]	2016	CN	760	72.03±8.21	AD/HC	53.55	8.76±4.58	DSM-IV	MoCA≤19.5	50	92	0	618
Wang [9]	2013	TW, CN	129	76.94±5.59	AD/HC	50.39	7.34±4.69	DSM-IV, NINCDS-ARDRA	MoCA≤21/22	64	11	3	51
Wang [52]	2011	CN	44	61.10±2.89	AD/HC	U	U	DSM-IV	MoCA≤26	19	4	3	18
Fujiwara [53]	2010	JP	66	76.90±4.75	AD/HC	67.90	12.21±2.64	DSM-IV, NINCDS-ARDRA	MoCA≤25/26	30	4	0	32
Costa [54]	2012	GER	130	65.95±9.07	AD/HC	42.31	11.63±2.77	Petersen’s criteria, NINCDS-ARDRA	MoCA≤22	29	3	1	97
Karunaratne [55]	2011	LK	98	64.13±12.37	AD/HC	34.69	10.35±3.11	DSM-IV	MoCA≤24	48	10	1	39
Li [56]	2018	CN	157	79.93±6.35	AD/HC	28.03	12.09±4.24	NINCDS-ADRDA	MoCA≤23.5	100	6	2	50
Nogueira [47]	2018	PT	440	69.40±8.98	AD/MCI	60.45	6.80±4.28	NINCDS-ADRDA	MoCA < 14	170	36	30	204
Wang [57]	2010	CN	118	73.00±10.00	AD/MCI	20.70	U	DSM-IV, NINCDS-ARDRA, Petersen’s criteria	MoCA≤18.5	42	21	3	52
Dai [58]	2010	CN	31	75.55±8.35	AD/MCI	50.00	U	DSM-IV, NINCDS-ARDRA, Petersen’s criteria	MoCA≤20.5	14	1	0	16
Montiel [59]	2013	BR	97	77.31±7.04	AD/MCI	61.86	6.46±1.94	DSM-IV, NINCDS-ARDRA, Petersen’s criteria	MoCA≤21	45	11	7	34
Roalf [8]	2013	US	447	74.73±8.33	AD/MCI	58.39	13.76±4.22	CERAD-NB, DSRS	MoCA≤19	247	25	74	101
Cecato [60]	2017	BR	97	77.34±7.00	AD/MCI	62.89	U	DSM-IV, NIA-AAA	MoCA≤20.5	46	11	7	33
Montiel [59]	2013	BR	91	75.31±7.56	AD/HC	68.13	6.43±1.93	DSM-IV, NINCDS-ARDRA	MoCA≤23	51	0	1	39
Rashedi [12]	2021	IR	80	65.47±8.52	AD/HC	62.50	10.76±5.59	DSM-V, CDR	MoCA < 20	56	2	3	19
Liu [61]	2021	CN	100	69.25±9.54	AD/HC	44.00	7.50±1.50	DSM-V	MoCA < 20	42	19	8	31
B) Mini-Mental State Examination (MMSE)
Monsch [62]	1995	CH	120	70.66±8.67	AD/HC	50.83	11.83±2.58	DSM-III, NINCDS-ARDRA	MMSE≤26	52	0	18	50
Bartos [63]	2016	CZ	760	70.45±8.61	Early AD/HC	64.87	13.86±3.02	NIA-AA	MMSE≤27	95	137	15	514
Galasko [64]	1990	US	98	70.38±7.20	Mild AD/HC	55.41	14.35±3.03	NINCDS-ADRDA	MMSE≤23	24	1	0	73
Matias-Guiu [44]	2017	ES	160	78.27±6.52	Mild AD/HC	65.00	7.49±4.70	NIA-AA, GDS = 5	MMSE≤24	72	12	20	56
Nasreddine [6]	2005	CA	183	74.81±8.23	Mild AD/HC	59.56	11.65±3.99	DSM-IV, NINCDS-ARDRA	MMSE≤26	73	0	20	90
Bartos [63]	2016	CZ	725	70.03±8.47	Mild AD /HC	65.37	13.90±3.02	NIA-AA	MMSE≤26	62	72	14	579
Galasko [64]	1990	US	124	70.38±7.20	Moderate AD/HC	55.41	14.35±3.03	NINCDS-ADRDA	MMSE≤24	40	1	10	73
Chang [45]	2012	TW, CN	149	67.33±9.19	Very-mild AD/HC	44.97	9.57±4.50	DSM-IV, NINCDS-ARDRA, CDR = 1	MMSE≤26/27	49	16	3	81
Magierska [46]	2012	PL	115	73.96±6.17	AD/(MCI + HC)	73.91	12.34±4.59	DSM-IV, ICD, Petersen’s criteria, AD:CDR = 1; MCI:CDR = 0.5; HC:CDR = 0	MMSE≤26	29	18	7	61
Alexopoulos [65]	2006	GER	104	69.09±9.50	AD/HC	47.12	10.03±2.56	NINCDS-ADRDA	MMSE≤27/28	48	18	2	36
Camozzato [66]	2011	BR	133	72.68±6..90	AD/HC	59.40	5.14±3.23	NINCDS-ADRDA	MMSE≤22	60	10	6	57
Wang [9]	2013	TW, CN	129	76.94±5.59	AD/HC	50.39	7.34±4.69	DSM-IV, NINCDS-ARDRA	MMSE≤24/25	59	9	8	53
Forno [67]	2006	IT	109	74.19±8.40	AD/HC	63.30	7.67±4.08	Neurological and neuropsychological evaluation	MMSE≤25	40	4	5	60
Mellor [26]	2016	CN	750	72.03±8.21	AD/HC	53.54	8.76±4.58	DSM-IV	MMSE≤23.5	49	49	0	652
Roalf [8]	2013	US	461	74.32±8.77	AD/HC	63.56	14.11±4.02	CERAD-NB, DSM-IV	MMSE≤28	308	4	13	136
Kalbe [68]	2004	U	266	68.40±10.87	AD/HC	83.03	10.66±1.89	DSM-IV, NINCDS-ARDRA	MMSE≤21	111	20	10	125
Iatraki [69]	2014	GR	206	69.22±11.74	AD/HC	48.51	8.33±4.37	NINCDS-ADRDA	MMSE≤23/24	25	17	10	154
Aprahamian [70]	2011	BR	106	79.40±6.84	AD/HC	71.70	U	NINCDS-ADRDA	MMSE≤17.5	64	11	2	29
Heo [71]	2012	KR	89	71.94±9.59	AD/HC	65.17	8.55±4.94	NINCDS-ADRDA	MMSE≤23/24	41	8	9	31
Hu [49]	2013	CN	218	67.60±5.02	AD/HC	U	9.17±2.78	DSM-IV-TR, NINCDS-ADRDA	MMSE≤24/25	66	0	6	146
Zhang [72]	2020	CN	134	70.08±6.07	AD/HC	62.69	11.34±3.20	NINCDS-ADRDA	MMSE≤24	27	2	0	105
Nogueira [47]	2018	PT	410	69.13±9.28	AD/HC	60.98	7.23±4.32	NINCDS-ADRDA	MMSE < 27	178	6	22	204
Kim [73]	2005	KR	164	73.14±7.24	AD/HC	32.93	5.28±5.39	DSM-IV, NINCDS-ARDRA	MMSE≤19/20	66	8	16	74
Konagaya [74]	2007	JP	135	74.63±7.07	AD/HC	74.81	11.25±2.53	DSM-IV, NINCDS-ADRDA	MMSE≤26	45	4	4	82
Kuslansky [75]	2004	US	371	79.00±5.40	AD/HC	60.92	12.76±3.31	NINCDS-ADRDA	MMSE≤26	42	97	6	226
Ladeira [76]	2009	BR	164	71.80±7.71	AD/HC	73.00	11.08±6.26	Consensus diagnoses	illiterate: MMSE < 20, 1–4 y education: MMSE < 25, 4–8 y education: MMSE < 26, 9 y education and above: MMSE < 28	79	24	2	59
Chu [51]	2015	HK, CN	179	74.45±6.71	AD/HC	70.39	6.11±4.94	NINCDS-ADRDA	MMSE≤24/25	60	2	4	113
Li [56]	2018	CN	157	79.93±6.35	AD/HC	28.03	12.09±4.24	NINCDS-ADRDA	MMSE≤24	83	6	19	50
Salami [77]	2020	IR	174	72.32±0.68	AD/HC	50.57	U	NINCDS-ADRDA	MMSE≤18.5	54	5	6	109
Chandler [78]	2005	US	190	74.30±7.30	AD/HC	63.16	14.45±2.74	NINCDS-ADRDA	MMSE≤26.5	88	5	7	90
Tierney [79]	2003	CA	124	71.65±7.50	AD/HC	54.84	14.03±3.20	DSM-III-R, NINCDS-ARDRA	MMSE≤23	20	21	9	74
Baek [80]	2016	KR	188	71.35±8.64	AD/HC	65.43	10.28±5.22	NINCDS-ADRDA, CDR, CDR-SOB	MMSE≤23	89	12	8	79
De Araujo [81]	2018	BR	135	75.96±7.99	AD/HC	27.41	8.49±6.55	DSM-IV, NINCDS-ARDRA	MMSE < 24	51	14	19	51
Ansari [82]	2010	IR	113	70.17±14.93	AD/HC	55.75	U	DSM-IV	MMSE≤23	98	0	2	13
Kourtesis [83]	2020	GR	78	73.67±6.28	AD/HC	50.00	11.93±3.24	NINCDS-ADRDA	MMSE≤22	31	1	9	37
Karunaratne [55]	2011	LK	98	64.13±12.37	AD/HC	34.69	10.35±3.11	DSM-IV	MMSE≤26	39	0	10	49
Freitas [48]	2013	PT	270	72.30±7.856	AD/HC		6.34±4.25	DSM-IV-TR, NINCDS-ADRDA	MMSE < 26	77	13	14	167
Choi [84]	2016	KR	4236	71.30±9.03	AD/HC	70.07	7.41±5.49	DSM-IV, NINCDS-ARDRA	mild to moderate AD: MMSE≤26/27; moderate to severe AD: MMSE≤20/21	2771	388	90	987
Tsai [50]	2012	CN	136	79.07±6.35	AD/HC	36.03	8.88±4.83	NINCDS-ADRDA	MMSE≤25/26	93	1	5	37
Wojtynska [85]	2016	PL	139	73.31±9.93	AD/HC	61.87	12.59±3.14	NINCDS-ADRDA	MMSE≤23	106	8	3	22
Wang [52]	2011	CN	44	61.10±2.89	AD/HC	U	U	DSM-IV	MMSE≤26	13	3	9	19
Wu [86]	2002	CN	16095	63.39±7.60	AD/HC	59.60	U	DSM-IV	< 1 y education: MMSE < 18, 1 6 y education: MMSE < 21,> 7 y education: MMSE < 25	271	569	12	15243
Xu [87]	2012	CN	297	71.43±9.80	AD/HC	51.20	U	DSM-IV	Illiterate: MMSE≤17, Primary school: MMSE≤20, Secondary school or above: MMSE≤24	37	19	6	235
Zhu [88]	2010	CN	82	73.2±12.17	AD/HC	65.85	U	DSM-IV, NINCDS-ADRDA	U	33	6	8	35
Cacho [89]	2010	ES	132	72.95±5.12	AD/HC	62.88	U	DSM-IV	MMSE≤23/24	57	3	9	63
Schrijnemaekers [90]	2006	UK	47	76.82±8.12	AD/MCI	53.19	17.66±3.61	NINCDS-ADRDA, CAMDEX	MMSE≤25.5	24	3	4	16
Pozueta [91]	2011	ES	105	74.36±6.90	AD/MCI	61.90	U	DSM-IV, NINCDS-ARDRA, ICD10	MMSE≤26	32	11	18	44
Roalf [8]	2013	US	447	74.73±8.33	AD/MCI	58.39	13.76±4.22	CERAD-NB, DSRS	MMSE≤25	247	21	74	105
Ahn [92]	2010	KR	136	U	AD/MCI	U	U	NINCDS-ARDRA, Petersen’s criteria	MMSE≤25.5	78	5	15	38
Zhang [72]	2020	CN	134	70.16±6.27	AD/MCI	63.43	11.10±3.25	NINCDS-ADRDA	MMSE≤24	27	17	0	90
Nogueira [47]	2018	PT	440	69.40±8.98	AD/MCI	60.45	6.80±4.28	NINCDS-ADRDA, Petersen’s criteria	MMSE < 26	168	46	32	194
Montiel [59]	2013	BR	97	77.31±7.04	AD/MCI	61.86	6.46±1.94	DSM-IV, NINCDS-ARDRA, Petersen’s criteria	MMSE≤26	36	11	16	34
Chapman [93]	2016	US	12697	74.63±9.15	AD/MCI	51.95	14.85±3.28	NINCDS-ADRDA, IWG criteria	MMSE≤24	4041	494	2773	5389
Chandler [78]	2005	US	155	73.78±7.48	AD/MCI	57.42	14.06±2.86	NINCDS-ADRDA, Petersen’s criteria	MMSE≤25.5	84	10	11	50
Cebi [94]	2020	TR	105	72.04±7.14	AD/MCI	58.10	7.40±3.54	NINCDS-ADRDA, Petersen’s criteria	MMSE≤24	58	7	5	35
Baek [80]	2016	KR	323	72.35±7.94	AD/MCI	60.99	10.90±4.98	NINCDS-ADRDA, CDR, CDR-SOB	MMSE≤23	81	29	16	197
Dai [58]	2010	CN	31	75.55±8.35	AD/MCI	50.00	U	DSM-IV, NINCDS-ARDRA, Petersen’s criteria	MMSE≤25.5	12	1	2	16
Monllau [95]	2007	ES	451	65.74±9.97	AD/(MCI + HC)	58.09	8.25±4.82	DSM-IV, NINCDS-ARDRA	MMSE≤24	95	26	16	314
Cacho [89]	2010	ES	153	73.07±5.11	AD/(MCI + HC)	63.40	U	DSM-IV, Petersen’s criteria	MMSE≤23/24	57	14	9	73
Spering [96]	2012	US	6813	74.00±10.10	AD/(MCI + HC)	48.90	17.50±1.60	NINCDS-ARDRA	MMSE≤27	2470	485	478	3380
Spering [96]	2012	US	6843	74.00±10.10	AD/(MCI + HC)	48.90	17.50±1.60	NINCDS-ARDRA	MMSE≤27	2934	826	154	2929
Miranda [97]	2018	BR	84	73.00±7.40	AD/(MCI+HC)	67.80	11.00±5.60	NIA-AA, CDR, Petersen’s criteria	MMSE≤26	21	12	2	49
Montiel [59]	2013	BR	91	75.31±7.56	AD/HC	68.13	6.43±1.93	DSM-IV, NINCDS-ARDRA	MMSE≤29	48	7	4	32
Chapman [93]	2016	US	17555	73.22±9.50	AD/HC	61.16	15.27±3.10	NINCDS-ADRDA	MMSE≤24	4565	161	2249	10580
Pezzotti [98]	2008	IT	179	U	AD	73.50	U	NINCDS-ADRDA	MMSE≤21	87	12	8	72
Trapp [99]	2021	GER	655	76.19±7.96	AD/HC	80.81	12.03±2.00	ICD10	MMSE≤26	410	16	109	120
Placido [100]	2021	BR	66	78.05±6.28	AD/MCI	U	12.39±1.91	DSM-4, CDR, Petersen’s Criteria	MMSE≤25	39	4	4	19

CN, China; HK, Hong Kong; TW, Taiwan; SG, Singapore; PT, Portugal; KR, Korea; ES, Spain; TN, Tunisia; CH, Switzerland; CZ, Czech; US, America; BR, Brazil; CA, Canada; GR, Greece; IN, India; IR, Iran; IT, Italy; JP, Japan; LK, Sri Lanka; PL, Poland; TR, Turkey; UK, United Kingdom; GER, Germany; TP, true positive; FN, false negative; FP, false positive; TN, true negative; AD, Alzheimer’s disease; MCI, mild cognitive impairment; HC, health counterparts; U, According to the articles, the content of this section is unclear.

Quality of included studies

Results for risk of bias are shown according to the QUADAS-2 (Fig. 2 for MoCA, and Fig. 3 for MMSE). The overall methodological quality of included studies was considered moderate (Fig. 4).

Fig. 2

Risk assessment of included studies of MoCA using QUADAS-2.

Fig. 3

Risk assessment of included studies of MMSE using QUADAS-2.

Fig. 4

Summary risks of bias and applicability concerns. The overall methodological quality of included studies was moderate.

Results of meta-analysis

Summary estimates from the meta-analysis are presented in Table 2. The pooled sensitivity was 0.934 (95% CI 0.905 to 0.954) for MoCA and 0.883 (95% CI 0.859 to 0.903) for MMSE, while the pooled specificity was 0.899 (95% CI 0.859 to 0.928) for MoCA and 0.903 (95% CI 0.879 to 0.923) for MMSE. MoCA showed higher sensitivity (93.4%) than MMSE, while the two tests presented a fairly high specificity (around 90%).

Table 2

Results of meta-analysis and Bayes analysis

Parameters	MoCA	MMSE
Sensitivity (95% CI)	0.934 (0.905,0.954)	0.883 (0.859,0.903)
Specificity (95% CI)	0.899 (0.859,0.928)	0.903 (0.879,0.923)
DOR (95% CI)	124.903 (67.459,231.260)	70.430 (50.001,97.260)
LR+ (95% CI)	9.220 (6.541, 12.996)	9.135 (7.293,11.441)
LR- (95% CI)	0.074 (0.051,0.108)	0.130 (0.107,0.157)

As for LR, the LR+ was 9.220 (95% CI 6.541 to 12.996) for MoCA and 9.135 (95% CI 7.293 to 11.441) for MMSE, while the LR- was 0.074 (95% CI 0.051 to 0.108) for MoCA and 0.130 (95% CI 0.107 to 0.157) for MMSE. The overall DOR was 124.903 (95% CI 67.459 to 231.260) for MoCA, and 70.430 (95% CI 50.001 to 97.260) for MMSE. MoCA had better performance than MMSE with a much higher DOR value.

The HSROC curve presented summary points, 95% confidence region and 95% prediction region (Fig. 5), which intuitively illustrated the performance of the two cognitive screening tests in detecting dementia associated with AD. The forest plots revealed the sensitivity and specificity of MoCA and MMSE (Fig. 6).

Fig. 5

HSROC curves for diagnosing AD. MoCA (a); MMSE (b). Pooled sensitivities and specificities in detecting AD are given by red solid box. HSROC curves, the 95% confidence region (pink dotted line) as well as the 95% prediction region (blue dotted line) are also given.

Fig. 6

Forest plots for sensitivity and specificity. MoCA (a), MMSE (b).

Results of Bayes analysis

Clinical utilization of the two tests for detecting cognitive impairment of AD was evaluated using the LRs. Post-test probability was calculated based on the Bayes’ theorem. We assessed pre-test probability of 35%, which was reported in the statistical analysis section, and the corresponding post-test probability has been shown using Fagan’s nomograms (Fig. 7). The Fagan plots suggest that MoCA (Fig. 7a) and MMSE (Fig. 7b) are equally informative with an 83% post-test probability of AD following a “positive” test result. However, MoCA was more useful to rule-out the screening with a 4% post-test probability of AD following a “negative” MoCA result, as compared to the MMSE (7%).

Fig. 7

Fagan plot analysis to evaluate the diagnostic accuracy of cognitive screening tests in detecting Alzheimer’s disease. An axis on the left represents the prior log-odds (pre-test probability), an axis in the middle represents the log likelihood ratio (LR) and an axis on the right represents the posterior log-odds (post-test probability).

Results of heterogeneity exploration and publication bias

The Q-test demonstrated heterogeneity (p < 0.01) and the Higgins I² statistics (I² > 50%) demonstrated considerable heterogeneity among studies in terms of sensitivity and specificity of the two cognitive screening tests (Fig. 6), with 88.29% sensitivity and 87.38% specificity for MoCA (Fig. 6a), and 98.57% sensitivity and 98.44% specificity for MMSE (Fig. 6b).

Meta-regression analyses were performed to discuss the sources of heterogeneity. Seven dimensions including publication year (before 2010 versus after 2010), number of patients (more than or equal to 200 patients versus less than 200 patients), country or region (studies from Asia versus other continents), population type (whether MCI patients were included or not), average age of population (over 70 years old versus less than or equal to 70 years old), average education level of population (more than 12 years versus less than or equal to 12 years), and reference standard (only one reference standard versus multiple reference standards) were analyzed in our meta-regression. Meta-regression results show the seven dimensions mainly lead to the heterogeneity and are independently associated with sensitivity and specificity of MoCA (p < 0.05, see Supplementary Table 3) and MMSE (p < 0.05, see Supplementary Table 4). Results of meta-regression analyses are presented by figures as well (see Supplementary Figure 1).

Results of Deeks’ funnel plot and Deeks’ asymmetry test (Fig. 8) suggest that the likelihood of publication bias in the included studies is low for MoCA (p = 0.90, Fig. 8a) and MMSE (p = 0.25, Fig. 8b).

Fig. 8

Deeks’ funnel plot. (a) Deeks’ funnel plot for MoCA. A p value of 0.90 indicates absence of publication bias. (b) Deeks’ funnel plot for MMSE. A p value of 0.25 indicates absence of publication bias. ESS, effective sample size.

DISCUSSION

In this study, we conducted a meta-analysis to evaluate the accuracy of MoCA and MMSE in distinguishing AD from MCI or HC in the absence of a gold standard. Our meta-analysis revealed MoCA had better performance with a pooled sensitivity of 93.4%, a pooled specificity of 89.9%, LR+ of 9.220, LR- of 0.074, and DOR of 124.903. MMSE showed slightly inferior performance with a pooled sensitivity of 88.3%, a pooled specificity of 90.3%, LR+ of 9.135, LR- of 0.130, and DOR of 70.430, respectively. Results of our study are consistent with findings of a previous systematic review that MoCA performs better and is more sensitive than MMSE in discriminating AD from MCI or HC [28]. Similarly, Creavin et al. [30] used a HSROC model and reported the sensitivity and specificity of MMSE in detecting cognitive dysfunction of all-cause dementia were 0.85 (95% CI 0.74 to 0.92) and 0.90 (95% CI 0.82 to 0.95), respectively. However, several systematic reviews, including Creavin’s study, had insufficient data of AD dementia subtype when evaluating the accuracy of the MMSE, and did not assess the accuracy of MoCA [30, 31]. Our meta-analysis investigated the pooled accuracy of MMSE and MoCA for AD screening simultaneously and we addressed the limitations of previous studies. MoCA proved to be more sensitive and superior to MMSE in tracking AD in most primary studies [6 , 46–48]. Nevertheless, one study suggested that the performance of MMSE was equivalent to that of MoCA [51]. The discrepancy might be attributed to the different cutoff value for positivity and the population classification criteria caused by different age, education, and stages of cognitive impairment. Moreover, Mellor and Tsiakiri reported the effects of social and demographic factors including sex, age, and education on the accuracy of MMSE and MoCA to detect dementia associated with AD, which could result in different positivity thresholds for cognitive screening tests [25, 26].

This study demonstrated that MoCA and MMSE were equally informative to “rule-in” dementia associated with AD following a positive result, and MoCA was more useful to “rule-out” dementia associated with AD following a “negative” result. To date, lack of a gold standard is acknowledged as a vitally important problem in assessing the accuracy of disease screening tests. Different reference standards were used in the process of diagnosis due to the absence of a gold standard, which might result in heterogeneity among studies [23, 24]. Our study is the first study to perform a meta-analysis assessing the accuracy of MoCA and MMSE tests for screening of AD simultaneously, and to overcome limitations of traditional approach by using the most advanced and rigorous hierarchical Bayesian latent-class model. This statistical method takes into account not only within- and between-study variability, but also different cut-off points across studies to justify the imperfect reference standard bias [24].

In a hierarchical Bayesian latent-class model, the true disease status (AD versus non-AD) can be treated as a dichotomous latent variable, and a hierarchical structure with two levels (a within-study level for a specific study and a common between-study level for all studies) account for variability in the distribution of screening results in each study and variation in using difference positivity thresholds across studies. An assumption that the screening results of MoCA and MMSE tests are conditionally independent given the true disease status is set in this hierarchical latent-class model. We used the Bayesian method to estimate the parameters of interest for this model. Since it is difficult to find analytical solution for the marginal posterior distributions of specified parameters, we used the Monte Carlo Markov Chain (MCMC) with Gibbs sampling techniques to obtain the posterior estimators for parameters of interest. The advantage of this statistical method is it allowed for variation in cutoff values, different reference standards can be modeled, and imperfect gold standard bias can be investigated. However, conditional independence between two cognitive screening tests is a strong assumption in this analysis, thus it is hard to check the validity [23]. In reality, the cognitive screening tests results may be correlated due to common items in scales. In this study, we did not consider the conditional dependence among test results. Further studies need to assess the performance of tests without a gold standard when the results of cognitive screening tests have conditional dependence.

De Roeck et al. [101] reported that MoCA is a promising screening tool, but the specificity to detect the cognitive impairment of early AD is low. Ashford and Schmitt stated that MMSE provided a poor estimate of patient severity early in the AD time-course and late in the course due to ceiling and floor effects [102]. In this study, we did not conduct subgroup analysis of the severity of AD due to the limitations of data information. Hence, we could not verify the findings of the prior studies in which a screening test was suitable for the different stages of AD. Test items had better psychometric characteristics and were important to improve the measurement of severity and progression with respect to the time-course of AD [102]. Factor-structure items of the cognitive impairment were vital to identify the severity of AD based on a “time-index” model and item response theory [102, 103]. However, we focused on the overall accuracy of cognitive screening tests in this study. A further meta-analysis is needed to define the contribution of each item of cognitive screening tests in estimating the severity on the disability continuum of AD.

Moderate risk of bias occurred for index tests in methodological quality based on the QUADAS-2 criteria, as the sensitivity and specificity values we extracted corresponded to the optimal cut-off points in most studies. The heterogeneity among studies was large, likely because the collected results were from different versions of cognitive screening tests, and some of these tests were translated into many different languages, such as the Persian [12, 82], Georgian [11], Chinese [49], Sinhala [55], Korean [42], Brazilian [13], Iranian [77], and Greek [69, 83]. We also included a telephone version MMSE, which was strongly correlated with the in-person MMSE [66]. Computerized assessments have also been created to assess cognition or memory in recent years [104]. Moving from the MoCA to a computerized test and evaluating its screening accuracy are important issues in the detection of dementia associated with AD [61, 105]. Further studies are needed to address this new approach. Moreover, participants with different age or education levels in the included studies might also contribute to some levels of clinical heterogeneity. Furthermore, results of meta-regression suggested that publication year, number of patients, country or region, population type, average age of population, average education of population, and reference standard were also sources of heterogeneity. These seven dimensions were independently significantly associated with the accuracy of the two cognitive screening tests.

To make our findings more representative of participants recruited from around the world, we searched publications in English and Chinese from multiple source databases. We also comprehensively searched the original literatures and reference citations of reviews from each database so that the possibility of publication bias was relatively small, which is one strengths of this meta-analysis. Another strength is a large number of primary studies were included. Nonetheless, there were some limitations in this meta-analysis. First, heterogeneity between included studies was inevitable. We used meta-regression to discuss the sources of heterogeneity from a limited seven dimensions due to the little information from the primary studies. Second, this study did not consider the effect of potential covariates including age, sex, education, Apolipoprotein E (APOE) genotypes, and race on the accuracy of MoCA and MMSE in screening AD in the primary studies. Subgroup analysis on age and education of individuals showed certain influence of these two factors on sensitivity and specificity of MoCA and MMSE in screening AD. Third, this study did not conduct subgroup sensitivity analysis for the severity of AD, and the test items of MoCA and MMSE for the severity on the disability continuum of AD were not assessed due to insufficient data in primary studies. Our review only assessed the performance of the two cognitive screening tests in screening AD from MCI or HC but did not evaluate their classification ability in screening MCI from HC, or AD patients of different periods and dementia associated with other diseases. However, subgroup analysis was conducted to evaluate the accuracy of MoCA and MMSE in screening AD from different populations. Fourth, reliable biomarkers are essential to increase diagnostic accuracy, allowing for earlier diagnosis and better participant selection [22]. Therefore, it is possible that subjects with very mild impairment were misclassified as normal using MMSE and MoCA. However, we did not combine biomarker information to recheck these subjects due to insufficient data in the primary studies. This limitation might have affected the estimations of pooled sensitivity and specificity in screening AD.

Conclusions

In conclusion, the present study found that MoCA had a higher screening capacity in discriminating AD from MCI or HC. Moreover, MoCA was more useful to “rule-out” AD. Further studies with higher quality and lower heterogeneity are needed to confirm these findings.

Footnotes

ACKNOWLEDGMENTS

This study was funded by National Natural Science Foundation of China (Grant No. 81903408), and Young Core Personal Project & Beijing Outstanding Talent Training Project (2018000020124G136).

The funder/ sponsor had no role in the design and conduct of the study, collection, management, analysis, and interpretation of the data; preparation, review, or approval of the manuscript; and decision to submit the manuscript for publication.

We thank Professor Henry Xiang, MD, MPH, PhD, MBA at the Ohio State University College of Medicine and Mrs. Megan Armstrong at the Abigail Research at Nationwide Children’s Hospital, Ohio, USA for editing the final version of the manuscript.

Authors’ disclosures available online ().

The supplementary material is available in the electronic version of this article: .

References

(2021) 2021 Alzheimer’s disease facts and figures. Alzheimers Dement 17, 327–406.

Lane

, Hardy

, Schott

(2018) Alzheimer’s disease. Eur J Neurol 25, 59–70.

Borson

, Frank

, Bayley

, Boustani

, Dean

, Lin

, McCarten

, Morris

, Salmon

, Schmitt

, Stefanacci

, Mendiondo

, Peschin

, Hall

, Fillit

, Ashford

(2013) Improving dementia care: The role of screening and detection of cognitive impairment. Alzheimers Dement 9, 151–159.

Galvin

, Sadowsky

(2012) Practical guidelines for the recognition and diagnosis of dementia. J Am Board Fam Med 25, 367–382.

McKhann

, Drachman

, Folstein

, Katzman

, Price

, Stadlan

(1984) Clinical diagnosis of Alzheimer’s disease: Report of the NINCDS-ADRDA Work Group^* under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease. Neurology 34, 939–944.

Nasreddine

, Phillips

, Bédirian

, Charbonneau

, Whitehead

, Collin

, Cummings

, Chertkow

(2005) The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment. J Am Geriatr Soc 53, 695–699.

Folstein

, Folstein

, McHugh

(1975) “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 12, 189–198.

Roalf

, Moberg

, Xie

, Wolk

, Moelter

, Arnold

(2013) Comparative accuracies of two common screening instruments for classification of Alzheimer’s disease, mild cognitive impairment, and healthy aging. Alzheimers Dement 9, 529–537.

Wang

CSM

, Pai

, Chen

, Hou

, Chien

, Huang

(2013) Montreal Cognitive Assessment and Mini-Mental State Examination performance in patients with mild-to-moderate dementia with Lewy bodies, Alzheimer’s disease, and normal participants in Taiwan. Int Psychogeriatr 25, 1839–1848.

10.

Tsai

, Chen

, Chu

, Yang

, Chung

, Liao

, Chou

(2016) Comparing the sensitivity, specificity, and predictive values of the Montreal Cognitive Assessment and Mini-Mental State Examination when screening people for mild cognitive impairment and dementia in Chinese population. Arch Psychiatr Nurs 30, 486–491.

11.

Janelidze

, Mikeladze

, Bochorishvili

, Dzagnidze

, Kapianidze

, Mikava

, Khatiashvili

, Kakhiani

, Mirvelashvili

, Shiukashvili

, Nadareishvili

(2017) Validity of the Georgian Montreal Cognitive Assessment for the screening of mild cognitive impairment and dementia. Am J Alzheimers Dis Other Demen 32, 36–40.

12.

Rashedi

, Foroughan

, Chehrehnegar

(2021) Psychometric properties of the Persian Montreal Cognitive Assessment in mild cognitive impairment and Alzheimer disease. Dement Geriatr Cogn Dis Extra 11, 51–57.

13.

Pinto

TCC

, Machado

, Costa

MLG

, Santos

MSP

, Bulgacov

, Rolim

APP

, Silva

, Rodrigues-Júnior

, Sougey

, Ximenes

RCC

(2019) Accuracy and psychometric properties of the Brazilian version of the Montreal Cognitive Assessment as a brief screening tool for mild cognitive impairment and Alzheimer’s disease in the initial stages in the elderly. Dement Geriatr Cogn Disord 47, 366–374.

14.

Clifford

R. JJ

, Bennett

, Blennow

, Carrillo

, Dunn

, Haeberlein

, Holtzman

, Jagust

, Jessen

, Karlawish

, Liu

, Molinuevo

, Montine

, Phelps

, Rankin

, Rowe

, Scheltens

, Siemers

, Snyder

, Sperling

, Contributors (2018) NIA-AA Research Framework: Toward a biological definition of Alzheimer’s disease. Alzheimers Dement 14, 535–562.

15.

Dubois

, Feldman

, Jacova

, Hampel

, Molinuevo

, Blennow

, DeKosky

, Gauthier

, Selkoe

, Bateman

, Cappa

, Crutch

, Engelborghs

, Frisoni

, Fox

, Galasko

, Habert

, Jicha

, Nordberg

, Pasquier

, Rabinovici

, Robert

, Rowe

, Salloway

, Sarazin

, Epelbaum

, de Souza

, Vellas

, Visser

, Schneider

, Stern

, Scheltens

, Cummings

(2014) Advancing research diagnostic criteria for Alzheimer’s disease: The IWG-2 criteria. Lancet Neurol 13, 614–629.

16.

Ganguli

, Blacker

, Blazer

, Grant

, Jeste

, Paulsen

, Petersen

, Sachdev

(2011) Classification of neurocognitive disorders in DSM-5: A work in progress. Am J Geriatr Psychiatry 19, 205–210.

17.

Khan

, Alkon

(2010) Early diagnostic accuracy and pathophysiologic relevance of an autopsy-confirmed Alzheimer’s disease peripheral biomarker. Neurobiol Aging 31, 889–900.

18.

Durand-Martel

, Tremblay

, Brodeur

, Paquet

(2010) Autopsy as gold standard in FDG-PET studies in dementia. Can J Neurol Sci 37, 336–342.

19.

Gaugler

, Kane

, Johnston

, Sarsour

(2013) Sensitivity and specificity of diagnostic accuracy in Alzheimer’s disease: A synthesis of existing evidence. Am J Alzheimers Dis Other Demen 28, 337–347.

20.

Scheltens

, Rockwood

(2011) How golden is the gold standard of neuropathology in dementia? Alzheimers Dement 7, 486–489.

21.

Beach

, Monsell

, Phillips

, Kukull

(2012) Accuracy of the clinical diagnosis of Alzheimer disease at National Institute on Aging Alzheimer Disease Centers, 2005-2010. J Neuropathol Exp Neurol 71, 266–273.

22.

Obrocki

, Khatun

, Ness

, Senkevich

, Hanrieder

, Capraro

, Mattsson

, Andreasson

, Portelius

, Ashton

, Blennow

, Schöll

, Paterson

, Schott

, Zetterberg

(2020) Perspectives in fluid biomarkers in neurodegeneration from the 2019 biomarkers in neurodegenerative diseases course-a joint PhD student course at University College London and University of Gothenburg. Alzheimers Res Ther 12, 20.

23.

Zhou

, Obuchowski

, McClish

(2011) Statistical Methods in Diagnostic Medicine, JohnWiley & Sons, Inc.

24.

Dendukuri

, Schiller

, Joseph

, Pai

(2012) Bayesian meta-analysis of the accuracy of a test for tuberculous pleuritis in the absence of a gold standard reference. Biometrics 68, 1285–1293.

25.

Tsiakiri

, Vadikolias

, Tripsianis

, Vlotinou

, Serdari

, Terzoudi

, Heliopoulos

(2021) Influence of social and demographic factors on the Montreal Cognitive Assessment (MoCA) test in rural population of North-Eastern Greece. Geriatrics (Basel) 6, 43.

26.

Mellor

, Lewis

, McCabe

, Byrne

, Wang

, Zhu

, Cheng

, Yang

, Dong

, Xiao

(2016) Determining appropriate screening tools and cut-points for cognitive impairment in an elderly Chinese sample. Psychol Assess 28, 1345–1353.

27.

Rutter

, Gatsonis

(2001) A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med 20, 2865–2884.

28.

Pinto

, Machado

, Bulgacov

, Rodrigues-Júnior

, Costa

MLG

, Ximenes

RCC

, Sougey

(2018) Is the Montreal Cognitive Assessment (MoCA) screening superior to the Mini-Mental State Examination (MMSE) in the detection of mild cognitive impairment (MCI) and Alzheimer’s disease (AD) in the elderly? Int Psychogeriatr 31, 491–504.

29.

Ciesielska

, Sokolowski

, Mazur

, Podhorecka

, Polak-Szabela

, Kędziora-Kornatowska

(2016) Is the Montreal Cognitive Assessment (MoCA) test better suited than the Mini-Mental State Examination (MMSE) in mild cognitive impairment (MCI) detection among people aged over 60? Meta-analysis. Psychiatr Pol 50, 1039–1052.

30.

Creavin

, Wisniewski

, Noel-Storr

, Trevelyan

, Hampton

, Rayment

, Thom

, Nash

KJE

, Elhamoui

, Milligan

, Patel

, Tsivos

, Wing

, Phillips

, Kellman

, Shackleton

, Singleton

, Neale

, Watton

, Cullum

(2016) Mini-Mental State Examination (MMSE) for the detection of dementia in clinically unevaluated people aged 65 and over in community and primary care populations. Cochrane Database Syst Rev 2016, CD011145.

31.

Arevalo-Rodriguez

, Smailagic

, Roqué

, Figuls

, Ciapponi

, Sanchez-Perez

, Giannakou

, Pedraza

, Bonfill Cosp

, Cullum

(2015) Mini-Mental State Examination (MMSE) for the detection of Alzheimer’s disease and other dementias in people with mild cognitive impairment (MCI). Cochrane Database Syst Rev, CD010783.

32.

Tsoi

KKF

, Chan

JYC

, Hirai

, Wong

SYS

, Kwok

TCY

(2015) Cognitive tests to detect dementia: A systematic review and meta-analysis. JAMA Intern Med 175, 1450–1458.

33.

McInnes

MDF

, Moher

, Thombs

, McGrath

, Bossuyt

, Clifford

, Cohen

, Deeks

, Gatsonis

, Hooft

, Hunt

, Hyde

, Korevaar

, Leeflang

MMG

, Macaskill

, Reitsma

, Rodin

, Rutjes

AWS

, Salameh

, Stevens

, Takwoingi

, Tonelli

, Weeks

, Whiting

, Willis

(2018) Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: The PRISMA-DTA Statement. JAMA 319, 388–396.

34.

Whiting

, Rutjes

, Westwood

, Mallett

, Deeks

, Reitsma

, Leeflang

, Sterne

, Bossuyt

(2011) QUADAS-2: A revised tool for the quality assessment of diagnostic accuracy studies. Ann Intern Med 155, 529–536.

35.

Deeks

, Altman

(2004) Diagnostic tests 4: Likelihood ratios. BMJ 329, 168–169.

36.

Akobeng

(2007) Understanding diagnostic tests 2: Likelihood ratios, pre- and post-test probabilities and their use in clinical practice. Acta Paediatr 96, 487–491.

37.

Gaser

, Franke

, Klöppel

, Koutsouleris

, Sauer

(2013) BrainAGE in mild cognitive impaired patients: Predicting the conversion to Alzheimer’s disease. PLoS One 8, e67346.

38.

Panegyres

, Rogers

, McCarthy

, Campbell

, Wu

(2009) Fluorodeoxyglucose-positron emission tomography in the differential diagnosis of early-onset dementia: A prospective, community-based study. BMC Neurol 9, 41.

39.

Higgins

, Thompson

, Deeks

, Altman

(2003) Measuring inconsistency in meta-analyses. BMJ 327, 557–560.

40.

Deeks

, Macaskill

, Irwig

(2005) The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol 58, 882–893.

41.

Memória

, Yassuda

, Nakano

, Forlenza

(2013) Brief screening for mild cognitive impairment: Validation of the Brazilian version of the Montreal cognitive assessment. Int J Geriatr Psychiatry 28, 34–40.

42.

Lee

, Lee

, Cho

, Na

, Jeon

, Kim

, Lee

, Youn

, Kwon

, Lee

, Cho

(2008) Brief screening for mild cognitive impairment in elderly outpatient clinic: Validation of the Korean version of the Montreal Cognitive Assessment. J Geriatr Psychiatry Neurol 21, 104–110.

43.

, Chew

, Narasimhalu

, Kandiah

(2013) Effectiveness of Montreal Cognitive Assessment for the diagnosis of mild cognitive impairment and mild Alzheimer’s disease in Singapore. Singapore Med J 54, 616–619.

44.

Matías-Guiu

, Valles-Salgado

, Rognoni

, Hamre-Gil

, Moreno-Ramos

, Matías-Guiu

(2017) Comparative diagnostic accuracy of the ACE-III, MIS, MMSE, MoCA, and RUDAS for screening of Alzheimer disease. Dement Geriatr Cogn Disord 43, 237–246.

45.

Chang

Y-T

, Chang

C-C

, Lin

H-S

, Huang

C-W

, Chang

W-N

, Lui

C-C

, Lee

C-C

, Lin

Y-T

, Chen

C-H

, Chen

N-C

(2012) Montreal cognitive assessment in assessing clinical severity and white matter hyperintensity in Alzheimer’s disease with normal control comparison. Acta Neurol Taiwan 21, 64–73.

46.

Magierska

, Magierski

, Fendler

, Kłoszewska

, Sobów

(2012) Clinical application of the Polish adaptation of the Montreal Cognitive Assessment (MoCA) test in screening for cognitive impairment. Neurol Neurochir Pol 46, 130–139.

47.

Joana

, Sandra

, Diana

, Jorge

, Isabel

(2018) Validation study of the Alzheimer’s disease assessment scale-cognitive subscale (ADAS-Cog) for the Portuguese patients with mild cognitive impairment and Alzheimer’s disease. Clin Neuropsychol 32, 46–59.

48.

Freitas

, Simões

, Alves

, Santana

(2013) Montreal Cognitive Assessment: Validation study for mild cognitive impairment and Alzheimer disease. Alzheimer Dis Assoc Disord 27, 37–43.

49.

, Zhou

, Hu

, Huang

, Wei

, Qi

, Huang

, Xu

(2013) Cross-cultural difference and validation of the Chinese version of Montreal Cognitive Assessment in older adults residing in Eastern China: Preliminary findings. Arch Gerontol Geriatr 56, 38–43.

50.

Tsai

, Lee

, Wang

, Shia

, Nasreddine

, Fuh

(2012) Psychometrics of the Montreal Cognitive Assessment (MoCA) and its subscales: Validation of the Taiwanese version of the MoCA and an item response theory analysis. Int Psychogeriatr 24, 651–658.

51.

Chu

, Ng

, Law

, Lee

, Kwan

(2015) Validity of the Cantonese Chinese Montreal Cognitive Assessment in Southern Chinese. Geriatr Gerontol Int 15, 96–103.

52.

Wang

, Zhu

, Huang

, Cong

, Sheng

(2011) Comparison and application of the Montreal Cognitive Assessment and the Mini-Mental State Examination in screening Alzheimer’s disease. Heilongjiang Med Pharm 34, 40.

53.

Fujiwara

, Suzuki

, Yasunaga

, Sugiyama

, Ijuin

, Sakuma

, Inagaki

, Iwasa

, Ura

, Yatomi

, Ishii

, Tokumaru

, Homma

, Nasreddine

, Shinkai

(2010) Brief screening tool for mild cognitive impairment in older Japanese: Validation of the Japanese version of the Montreal Cognitive Assessment. Geriatr Gerontol Int 10, 225–232.

54.

Costa

, Fimm

, Friesen

, Soundjock

, Rottschy

, Gross

, Eitner

, Reich

, Schulz

, Nasreddine

, Reetz

(2012) Alternate-form reliability of the Montreal Cognitive Assessment screening test in a clinical setting. Dement Geriatr Cogn Disord 33, 379–384.

55.

Karunaratne

, Hanwella

, de Silva

(2011) Validation of the Sinhala version of the Montreal Cognitive Assessment in screening for dementia. Ceylon Med J 56, 147–153.

56.

, Zhang

, Zhu

, He

, Zhang

, Tian

(2018) Construct validity and reliability of the Test Your Memory Chinese version in older neurology outpatient attendees. Int J Ment Health Syst 12, 1–11.

57.

Wang

, Liu

, Gao

, Xie

, Zhou

, Cheng

, Zhang

(2010) Exploration of the Cut-off point of the Chinese version of the Montreal cognitive assessment among retired soldiers in Beijing. Chinese J Health Care Med 12, 271–273.

58.

Dai

(2010) Application of neuropsychological tests in diagnosing mild cognitive impairment [M], Nankai University.

59.

Montiel

, Cecato

, Bartholomeu

, Martinelli

(2013) Evaluation of Montreal cognitive assessment for the differential diagnosis of mild cognitive impairment and Alzheimer’s disease in elderly patients with more than 5 years of schooling: Data from a Brazilian sample. Adv Aging Res 2, 121–129.

60.

Cecato

, Martinelli

, Izbicki

, Yassuda

, Aprahamian

(2016) A subtest analysis of The Montreal Cognitive Assessment (MoCA): Which subtests can best discriminate between healthy controls, mild cognitive impairment and Alzheimer’s disease? Int Psychogeriatr 28, 825–832.

61.

Liu

, Chen

, Zhou

, Shang

, Xu

, Zhang

, He

, Zhao

, Du

, Wang

, Zhang

, Bergeron

, Ding

, Ashford

, Zhong

(2021) Validity of the MemTrax Memory Test compared to the Montreal Cognitive Assessment in the detection of mild cognitive impairment and dementia due to Alzheimer’s disease in a Chinese cohort. J Alzheimers Dis 80, 1257–1267.

62.

Monsch

, Foldi

, Ermini-Fünfschilling

, Berres

, Taylor

, Seifritz

, Stähelin

, Spiegel

(1995) Improving the diagnostic accuracy of the Mini-Mental State Examination. Acta Neurol Scand 92, 145–150.

63.

Bartos

, Raisova

(2016) The Mini-Mental State Examination: Czech norms and cutoffs for mild dementia and mild cognitive impairment due to Alzheimer’s disease. Dement Geriatr Cogn Disord 42, 50–57.

64.

Galasko

, Klauber

, Hofstetter

, Salmon

, Lasker

, Thal

(1990) The Mini-Mental State Examination in the early diagnosis of Alzheimer’s disease. Arch Neurol 47, 49–52.

65.

Alexopoulos

, Greim

, Nadler

, Martens

, Krecklow

, Domes

, Herpertz

, Kurz

(2006) Validation of the Addenbrooke’s Cognitive Examination for detecting early Alzheimer’s disease and mild vascular dementia in a German population. Dement Geriatr Cogn Disord 22, 385–391.

66.

Camozzato

, Kochhann

, Godinho

, Costa

, Chaves

(2011) Validation of a telephone screening test for Alzheimer’s disease. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn 18, 180–194.

67.

Gloria

, Paola

, Federica

, Florinda

, Enzo

, Jason

, Maria

, Patrizio

(2006) Use of an Italian version of the telephone interview for cognitive status in Alzheimer’s disease. Int J Geriatr Psychiatry 21, 126–133.

68.

Kalbe

, Kessler

, Calabrese

, Smith

, Passmore

, Brand

, Bullock

(2004) DemTect: A new, sensitive cognitive screening test to support the diagnosis of mild cognitive impairment and early dementia. Int J Geriatr Psychiatry 19, 136–143.

69.

Iatraki

, Simos

, Lionis

, Zaganas

, Symvoulakis

, Papastefanakis

, Panagiotakis

, Pantelidakis

, Papadopoulos

, Tziraki

(2014) Cultural adaptation, standardization and clinical validity of the Test Your Memory dementia screening instrument in Greek. Dement Geriatr Cogn Disord 37, 163–180.

70.

Ivan

, Eduardo

, Juliana

, Sanches

(2011) Screening for Alzheimer’s disease among illiterate elderly: Accuracy analysis for multiple instruments. J Alzheimers Dis 26, 221–229.

71.

Heo

, Lee

, Park

, Ahn

, Kim

(2012) Validation of the Korean Addenbrooke’s Cognitive Examination for diagnosing Alzheimer’s dementia and mild cognitive impairment in the Korean elderly. Appl Neuropsychol Adult 19, 127–131.

72.

Zhang

, Wang

, Deng

, Fei

, Jin

, Pan

, Cai

, Albano

, Zhong

(2019) Five-minute cognitive test as a new quick screening of cognitive impairment in the elderly. Aging Dis 10, 1258–1269.

73.

Kim

, Lee

, Jhoo

, Youn

, Suh

, Jun

, Seo

, Woo

(2005) Diagnostic accuracy of Mini-Mental Status Examination and revised Hasegawa Dementia Scale for Alzheimer’s disease. Dement Geriatr Cogn Disord 19, 324–330.

74.

Yoko

, Yukihiko

, Hideyuki

, Akinori

, Tomoyuki

, Toshiki

(2007) Validation of the Telephone Interview for Cognitive Status (TICS) in Japanese. Int J Geriatr Psychiatry 22, 695–700.

75.

Kuslansky

, Katz

, Verghese

, Hall

, Lapuerta

, LaRuffa

, Lipton

(2004) Detecting dementia with the Hopkins Verbal Learning Test and the Mini-Mental State Examination. Arch Clin Neuropsychol 19, 89–104.

76.

Ladeira

, Diniz

, Nunes

, Forlenza

(2009) Combining cognitive screening tests for the evaluation of mild cognitive impairment in the elderly. Clinics (Sao Paulo) 64, 967–973.

77.

Salami

, Alinaghipour

, Daneshvar

, Hamidi

, Agahi

, Soheili

, Akbari

, Taba

SME

(2020) Adapted MMSE and TYM cognitive tests: How much powerful in screening for Alzheimer’s disease in Iranian people. Aging Mental Health 24, 1010–1017.

78.

Chandler

, Lacritz

, Hynan

, Barnard

, Allen

, Deschner

, Weiner

, Cullum

(2005) A total score for the CERAD neuropsychological battery. Neurology 65, 102–106.

79.

Tierney

, Herrmann

, Geslani

, Szalai

(2003) Contribution of informant and patient ratings to the accuracy of the mini-mental state examination in predicting probable Alzheimer’s disease. J Am Geriatr Soc 51, 813–818.

80.

Baek

, Kim

, Park

, Kim

(2016) The validity and reliability of the Mini-Mental State Examination-2 for detecting mild cognitive impairment and Alzheimer’s disease in a Korean population. PLoS One 11, e0163792.

81.

Araujo NB

, Nielsen

, Engedal

, Barca

, Coutinho

, Laks

(2018) Diagnosing dementia in lower educated older persons: Validation of a Brazilian Portuguese version of the Rowland Universal Dementia Assessment Scale (RUDAS). Braz J Psychiatry 40, 264–269.

82.

Ansari

, Naghdi

, Hasson

, Valizadeh

, Jalaie

(2010) Validation of a Mini-Mental State Examination (MMSE) for the Persian population: A pilot study. Appl Neuropsychol 17, 190–195.

83.

Panagiotis

, Eleni

, Christina

, Foteini

, Sharon

(2020) A comparison of the Greek ACE-III, M-ACE, ACE-R, MMSE, and ECAS in the assessment and identification of Alzheimer’s disease. J Int Neuropsychol Soc 26, 825–834.

84.

Choi

, Park

(2016) Three screening methods for cognitive dysfunction using the Mini-Mental State Examination and Korean Dementia Screening Questionnaire. Geriatr Gerontol Int 16, 252–258.

85.

Wojtyńska

, Szcześniak

(2016) DemTect®–effective to assess MCI and dementia–validation study of the Polish language version. Aging Ment Health 20, 510–516.

86.

, Zhou

, Como

(2002) Application of a screening scale of Alzheimer’s disease in rural area of China. Chinese Mental Health J 16, 242–245.

87.

(2012) Research on the application of the Chinese version of GOCOG in cognitive screening for the elderly in community [M], Huazhong University of Science and Technology.

88.

Zhu

, Zhu

, Qiao

(2010) Diagnosis value of mini-mental stale examination combined with the delay memory and verbal fluency in Alzheimer’s disease. J North China Coal Med Univ 12, 156–157.

89.

Cacho

, Benito-Leon

, Garcia-Garcia

, Fernandez-Calvo

, Vicente-Villardon

, Mitchell

(2010) Does the combination of the MMSE and clock drawing test (mini-clock) improve the detection of mild Alzheimer’s disease and mild cognitive impairment? J Alzheimers Dis 22, 889–896.

90.

Schrijnemaekers

, De Jager

, Hogervorst

, Budge

(2006) Cases with mild cognitive impairment and Alzheimer’s disease fail to benefit from repeated exposure to episodic memory tests as compared with controls. J Clin Exp Neuropsychol 28, 438–455.

91.

Pozueta

, Rodríguez-Rodríguez

, Vazquez-Higuera

, Mateo

, Sánchez-Juan

, González-Perez

, Berciano

, Combarros

(2011) Detection of early Alzheimer’s disease in MCI patients by the combination of MMSE and an episodic memory test. BMC Neurol 11, 78.

92.

Ahn

, Chin

, Park

, Lee

, Suh

, Seo

, Na

(2010) Seoul Neuropsychological Screening Battery-dementia version (SNSB-D): A useful tool for assessing and monitoring cognitive impairments in dementia patients. J Korean Med Sci 25, 1071–1076.

93.

Chapman

, Bing-Canar

, Alosco

, Steinberg

, Martin

, Chaisson

, Kowall

, Tripodis

, Stern

(2016) Mini Mental State Examination and Logical Memory scores for entry into Alzheimer’s disease trials. Alzheimers Res Ther 8, 9.

94.

Cebi

, Babacan

, Tanor

, Gurvit

(2020) Discrimination ability of the Short Test of Mental Status (STMS) compared to the Mini Mental State Examination (MMSE) in the spectrum of normal cognition, mild cognitive impairment, and probable Alzheimer’s disease dementia: The Turkish standardization study. J Clin Exp Neuropsychol 42, 450–458.

95.

Monllau

, Pena-Casanova

, Blesa

, Aguilar

, Bohm

, Sol

, Hernandez

(2007) [Diagnostic value and functional correlations of the ADAS-Cog scale in Alzheimer’s disease: Data on NORMACODEM project]. Neurologia 22, 493–501.

96.

Spering

, Hobson

, Lucas

, Menon

, Hall

, O’Bryant

(2012) Diagnostic accuracy of the MMSE in detecting probable and possible Alzheimer’s disease in ethnically diverse highly educated individuals: An analysis of the NACC database. J Gerontol A Biol Sci Med Sci 67, 890–896.

97.

Miranda

DDC

, Brucki

SMD

, Yassuda

(2018) The Mini-Addenbrooke’s Cognitive Examination (M-ACE) as a brief cognitive screening instrument in Mild Cognitive Impairment and mild Alzheimer’s disease. Dement Neuropsychol 12, 368–373.

98.

Pezzotti

, Scalmana

, Mastromattei

, Di Lallo

(2008) The accuracy of the MMSE in detecting cognitive impairment when administered by general practitioners: A prospective observational study. BMC Fam Pract 9, 29.

99.

Trapp

, Röder

, Heid

, Billman

, Daiber

, Hajak

(2021) Sensitivity and specificity of the Bamberg Dementia Screening Test’s (BDST) full and short versions: Brief screening instruments for geriatric patients that are suitable for infectious environments. BMC Med 19, 65.

100.

Plácido

, Ferreira

, Araújo

, Silva

, Ferreira

, Guimarães

, de Carvalho

, Laks

, Deslandes

(2021) Beyond the Mini-Mental State Examination: The use of physical and spatial navigation tests to help to screen for mild cognitive impairment and Alzheimer’s disease. J Alzheimers Dis 81, 1243–1252.

101.

De Roeck

, De Deyn

, Dierckx

, Engelborghs

(2019) Brief cognitive screening instruments for early detection of Alzheimer’s disease: A systematic review. Alzheimers Res Ther 11, 21.

102.

Ashford

, Schmitt

(2001) Modeling the time-course of Alzheimer dementia. Curr Psychiatry Rep 3, 20–28.

103.

Ashford

, Kolm

, Colliver

, Bekian

, Hsu

(1989) Alzheimer patient evaluation and the mini-mental state: Item characteristic curve analysis. J Gerontol 44, 139–146.

104.

van der Hoek

, Nieuwenhuizen

, Keijer

, Ashford

(2019) The MemTrax Test compared to the Montreal Cognitive Assessment estimation of mild cognitive impairment. J Alzheimers Dis 67, 1045–1054.

105.

Rose

, Gilbertson

, Cottrell

, Tampi

(2021) Cognitive screening for adult psychiatric outpatients: Comparison of the Cognivue(®) to the Montreal Cognitive Assessment. World J Psychiatry 11, 265–270.