Performance on the mini-mental state exam and the Montreal cognitive assessment in a sample of old age psychiatric patients

Abstract

Objective:

This study assesses to what extent the Mini-Mental State Exam and the Montreal Cognitive Assessment scores may predict the presence of dementia in a sample of typical old age psychiatric patients who may or may not have temporally or permanently reduced cognitive abilities.

Methods:

A total of 141 inpatients completed the Mini-Mental State Exam and the Montreal Cognitive Assessment at arrival. All patients were subsequently diagnosed during their stay at the age-psychiatric unit. Receiver operating characteristics and analysis of variance were used to compare the results of the two tests for different patient groups.

Results:

The Montreal Cognitive Assessment is slightly more sensitive and specific than the Mini-Mental State Exam for dementia prediction. Age, sex, and education only account for approximately 2% of the variance in both tests. Patients with more than one diagnosis across the diagnostic groups included in this study (dementia, psychoses, affective disorder, and depression) performed significantly poorer on both tests than patients with a single diagnosis.

Conclusions:

Both tests are efficient in detecting cognitive impairment, but neither test can effectively exclude other reasons for low test results in our sample of elderly psychiatric patients. The sensitivity for ruling out dementia is 27 points for the Mini-Mental State Exam and 23 points for the Montreal Cognitive Assessment in the current patient sample.

Keywords

Mental health/old age psychiatry geriatrics/gerontology neurology dementia depression screening

Introduction

Screening tests are useful in detecting suspected cognitive decline. Since there are several cognitive functional areas, screening tests need to cover all cognitive areas and be sensitive enough to indicate cognitive decline even though only a single cognitive area has been affected. The Mini-Mental State Examination (MMSE)¹ and the Montreal Cognitive Assessment (MoCA)² are two commonly used screening tests for cognitive deficits. One or both tests are typically included in the screening battery for incoming patients in psychiatric clinics for the elderly, used to direct further cognitive assessment. Low scores on MoCA and MMSE do not necessarily indicate cognitive decline as individuals with chronically lower cognitive abilities, for example, intellectual developmental disorder, may naturally have a low score.

The population referred to an inpatient psychiatric clinic for the elderly are generally older than 65 years of age and include a diverse patient population and several diagnostic groups. Depression and anxiety are prevalent and have a high rate of comorbidity,³ while other common patient groups include schizophrenia and dementia. Also, patients with dementia often have comorbid diagnoses, such as depression, which may or may not be secondary to dementia.⁴ Disease comorbidity in addition to increased age may influence MMSE and MoCA scores to varying degrees, and since the cognitive decline is a natural part of aging, the effects of age become more influential on test scores over time. Each of these factors, or a combination of them, may lead to permanent or temporary cognitive decline. In this complicated environment, screening for dementia disease becomes challenging. Education level, gender, and ethnicity are other moderating factors in cognitive tests.^5,6

To screen incoming patients, the ability to identify the presence of dementia (sensitivity) is more important than the ability to identify the absence of dementia (specificity), but for a cost-efficient assessment, a trade-off where both sensitivity and specificity are near their peaks is preferred as a cut-off to identify possible dementia patients from the patient pool. The cut-offs used vary greatly between patient studies and are specific to the patient groups being studied^7–9 and the language version used or ethnicity of the patients.^6,10,11 There is no consensus on the appropriate cut-off score in an elderly psychiatric environment, although both MoCA and MMSE are well established. Several studies on elderly patients have pointed out that the established cut-off scores for MMSE and MoCA may be too rigorous. Both the MMSE and the MoCA tests have a score range from 0 to 30 points.^2,9,12,13 The MMSE cut-off score most commonly used for determining cognitive deficits in patients aged 65 years and above is 24 points.¹³ Cut-offs that are as low as 20 points have been suggested for low-educated elderly.¹² For the MoCA test, the cut-off points for the elderly vary greatly, from a 23–25-point range depending on ethnicity in an American study⁶ to 20–21 points in a Brazilian study.¹¹ These results include one point added for patients with less than 12 years of education since education level greatly affects performance. The established cut-off point for ruling out a cognitive decline in elderly patients from normal controls is 26 points.²

A meta-analysis of 34 English language dementia studies¹⁴ using cut-off scores ranging from 27/30 to 22/30 concluded that the MMSE was only “modestly effective at ruling-out dementia in specialist settings.” In a recent meta-study, Pinto et al.¹¹ concluded that 80% of the included 34 articles found that the MoCA test was superior to the MMSE test in discriminating between individuals with mild cognitive impairment and no cognitive impairment, while they concluded that both tests were accurate in the detection of Alzheimer’s disease (AD). However, these studies also find that the MMSE has acceptable sensitivity to identify dementia. The MMSE and MoCA tests overlap but weigh the cognitive domains differently. The MMSE assesses verbal abilities, which are often affected in advanced AD and other forms of dementia, such as Dementia with Lewy Bodies (DLB)¹⁵ and Parkinson’s disease (PD).¹⁶ However, reduced verbal abilities are seldom the first sign of cognitive decline, so a focus on language-related abilities is not suitable for the detection of early symptoms of dementia or mild cognitive impairments (MCI).

The different focus of the tests can be seen in a large multicentre study.¹⁷ Dementia patients associated with executive dysfunction scored higher on MMSE relative to MoCA than dementia patients where executive function is less affected. The MoCA test includes additional trail making and clock-drawing tests, is less focused on verbal ability than the MMSE, and is better able to distinguish between early and severe signs of cognitive decline,¹⁸ due to the additional focus on visuospatial construction, attention and calculation, executive function, and delayed memory. Subsets of these cognitive functions are often affected early in the course of Alzheimer’s disease and several other forms of dementia. The MoCA test is more demanding to perform, which prevents some severely impaired patients from completing it. Failure to perform the MoCA is considered a result, indicating the need for further cognitive assessment. Since the MMSE and the MoCA are short tests with different profiles and difficulty levels, both tests are often used in parallel for quality assurance in many institutions and memory clinics. Low results on both tests increase the probability of a cognitive deficit. In addition, a low result on the MoCA and a normal result on the MMSE may indicate MCI, while a very low result on the MMSE may indicate a severe cognitive deficit.

Clinics in several countries have migrated from using MMSE to MoCA and created conversion formulas and tables to understand the MoCA results considering their experience with MMSE.¹⁰ However, as reviewed here, these conversions may not be appropriate in a diverse patient population since the tests weigh the cognitive domains differently. Also, due to the diversity of the patient population, where any patients may be permanently or temporarily cognitive declined, disentangling dementia in a screening session becomes more challenging.

The purpose of the study is two-fold. (1) MMSE has been in use over decades and is well established, so replacing it comes with a cost since most personnel know how to administer and score it. We have been using MoCA in parallel, and we wanted to establish whether to replace MMSE with MoCA or if we gain sensitivity by screening with both tests, so all patients scoring below a threshold would undergo further diagnostic assessment. (2) Several studies are comparing the MMSE with MoCA, but none we have found includes the typical patient distribution typically found in an age-psychiatric inpatient clinic. Several conditions may affect cognition in these patients, and we wanted to know if sensitivity and specificity for identifying dementia patients are different in this patient distribution than for those populations used to calculate cut-off scores in earlier research, and if the MMSE and MoCA identify cognitive decrement in other than dementia patients within this sample. We hypothesized that the MoCA test is more sensitive than the MMSE, and that cut-offs for distinguishing dementia patients from other age-psychiatric patients are lower than the recommended cut-offs used for distinguishing dementia patients from normally functioning elderly.

Material and methods

The study compares two commonly used screening tests for detecting cognitive impairment and was classified as a prospective quality control study.

Tests

The Norwegian translations of MMSE-NR¹⁹ and MoCA²⁰ were used.

Ethics

The research was conducted according to the World Medical Association Declaration of Helsinki. The research was reviewed and waived approval by the Regional Ethics Board of south-eastern Norway and approved by the data protection office at Oslo University Hospital. The participants gave written consent to their participation in the study.

Participants

All new patients arriving for assessment at the outpatient and inpatient clinics of the department of old age psychiatry at Oslo University Hospital over a 2-year period were tested with the MMSE and the MoCA. MoCA/MMSE sessions included are the first tests administered to these patients when they first arrived at the hospital. At that point only the reason for referral was known about their cognitive state. They received a subsequent diagnosis that was unknown at the time of the initial screening with MMSE and MoCA and received appropriate treatment and care. This study evaluates the initial results of the MMSE and MoCA tests only, and patients are categorized according to the primary diagnosis they subsequently received.

All patients were further assessed according to International Classification of Diseases, 10th Revision (ICD-10) criteria,²¹ using interdisciplinary methods (e.g. neuropsychological assessment, neurological evaluations, psychiatric consultations, imaging diagnostics). Interdisciplinary methods were used when establishing a diagnosis, including clinical observations, interviewing, self-reporting, interview with care givers, neuropsychological evaluation (processing speed and attention, language, memory, visuoperceptual ability, abstraction and problem solving, personality testing/behavior monitoring), geriatric examination, and imaging (magnetic resonance (MR), computed tomographic (CT)). Only the subsequent diagnostic results and the initial MMSE and MoCA results are extracted for this article.

A total of 141 patients who were legally able to consent were included. The patient characteristics are summarized in Table 1. Seventeen patients with temporary diagnoses including non-psychotic organic conditions within the F6 range of ICD-10²¹ and without a heterogeneous pattern of addiction within the F10-F19 range of ICD-10 were excluded from the analysis. Some patients were tested with the MMSE or the MoCA alone or not at all due to difficulties in performance, refusal to perform, or untrained admitting personnel. Eighty-two patients performed the MoCA, 110 patients performed the MMSE, and a total of 62 patients performed both tests. To predict the presence of dementia, the patient groups were clustered according to their main ICD-10²¹ diagnosis: Group 1 (Dementia: F01-F03), Group 2 (Psychoses: F20, F22, F25, Schizophrenia and other psychoses including two F5 delirium patients without dementia, with delusions, and six F31 patients with psychotic symptoms), Group 3 (Affective: F41, F43, including anxiety, behavioral disorder, personality disorder), and Group 4 (Depression: F32, F33, including nine F31 patients without psychotic symptoms). About 35% of patients received diagnoses falling in two or more of these groups. Second diagnosis for dementia patients were depression (27%), schizophrenia/other psychoses (27%), affective disorders (7%), and other/unspecified (17%). The second diagnosis for psychoses patients was depression (12%), affective disorders (8%), and other/unspecified (12%). Second diagnosis for affective disorders were depression (22%), psychoses (8%), and other/unspecified (11%). The second diagnosis for depression patients was psychoses (15%), affective disorders (8%), and other/unspecified (2%). All patients with a dementia diagnosis were placed in Group 1, and all other patients were grouped based on their main diagnosis. Since the diverse pattern of comorbidity would require a much greater patient sample, the secondary diagnosis was not included in the analysis.

Table 1.

Demographics.

	Men (n)	Women (n)	Age (years)	Education (years)
Dementia disease (Group 1)	10	18	78 (59–89)	9.1
Psychoses (Group 2)	12	24	75 (62–90)	11.2
Affective disorder (Group 3)	7	19	77 (69–86)	10.3
Depression (Group 4)	16	35	77 (64–92)	11.1
Total / average	45	96	77 (59–92)	10.6

Analysis

The raw data were used when including age, sex, and education as covariates. When analysis did not include these covariates, one point was added according to the MoCA manual for low education. The effect on test scores of education level, age, multiple diagnoses, and patient-group was analyzed with between-subject analysis of variance (ANOVA) tests using IBM SPSSTM ver24. The main effects of age, education, and sex were analyzed first and later included as co-variables in the fixed effects analysis for group and number of diagnoses. The sensitivity and specificity of the tests for dementia patients as a criterion group were analyzed with receiver operating curve analysis (ROC)²² with all other patients as a comparison group. This condition corresponds to the detangling of dementia among a representative sample in a psychiatric clinic. There was an uneven distribution of education, with large clusters at 7 years (elementary school) and 10 years (high school equivalent). For this reason, patients were clustered in three equally sized groups (less than 9 years, 9–10 years, more than 10 years) to visualize differences in education. Patients were also divided into 3 equally sized age groups (less than 74 years, 74–80 years, more than 80 years).

Results

The main effects of education, age, and sex were tested individually to determine whether they should be included as covariates in further analysis. The MMSE (F(2, 91) = 7.12, p = 0.001) and MoCA results (F(2, 73) = 12.3, p < 0.001) were affected by education level, and we found that the group with highest education level scored highest, as shown in Figure 1. The demographic data (Table 1) also show that dementia patients have more than 1 year less education than the average patients, consistent with the observation that education is an important factor for preventing or delaying dementia.²³ Both the MMSE (F(2, 91) = 3.75, p < 0.05) and the MoCA results (F(2, 73) = 7.9, p = 0.001) were also affected by age group, with the oldest age range scoring the lowest on both tests, as shown in Figure 1. The effect of sex was not significant for the MoCA (F(2, 73) = 3.77, p = 0.06), but it was significant for the MMSE (F(2, 73) = 4,55, p < 0.05) and sex is therefore also included as a covariate in addition to education level and age in the analysis.

Figure 1.

Test scores for education level groups, age groups, sex, and diagnostics groups.

Diagnostic group (1, 2, 3, 4) and the number of diagnoses (one, more than one) were fixed factors in the successive ANOVA analysis, controlling for education, age, and sex, and the effect of main diagnostic group was significant for both tests (MMSE: F(3, 109) = 6.80, p < 0.001; MoCA: F(3, 81) = 9.30, p < 0.001). Table 2 shows that the dementia group scored lower than the other diagnostic groups, and a separate analysis with the dementia patients excluded showed no significant group effects. Patients with more than one diagnosis overlapping other diagnostic groups (35% of patients) performed poorer on both tests (MMSE: F(1, 109) = 4.94, p < 0.005; MoCA: F(1, 81) = 6.41, p < 0.001), but the interaction between diagnostic group and number of diagnoses was not significant for either test (MMSE: F(3, 109) = 2.58, p > 0.05; MoCA: F(3, 81) = 1.13, p > 0.05).

Table 2.

Group results.

	MoCA score	MMSE score	MMSE – MoCA (Δ)
Dementia disease (Group 1)	13.9	20.4	6.5
Psychoses (Group 2)	19.7	24.0	4
Affective disorder (Group 3)	20.4	24.0	3.6
Depression (Group 4)	21.6	25.6	4

MoCA: Montreal Cognitive Assessment; MMSE: Mini-Mental State Exam.

The difference between MMSE scores and MoCA scores is larger for dementia patients than for other patient groups. The average point scores are approximately 4 points higher on the MMSE than on the MoCA scale for all patient groups, except in the dementia group, for which the score difference is 6.5 points. The ROC characteristics for the dementia patients are higher for the MoCA (A = 0.86, p < 0.001) than for the MMSE (A = 0.75, p < 0.001) (Figure 2).

Figure 2.

ROC characteristics for the MMSE and MoCA tests.

The ROC analysis shows that the sensitivity for ruling out dementia is 27 points for the MMSE and 23 points for the MoCA in the current patient sample, and the specificity for determining dementia is 18 for the MMSE and 11 for the MoCA, confirming that both tests are unable to determine a specific disease or disorder (88% of dementia patients scored lower than 27 on the MMSE while 87% scored lower than 20 on the MoCA; 100% scored lower than 23 on the MoCA).

Discussion

The dementia group scored significantly lower than other patient groups on both MMSE and the MoCA at the group level, but the results show that both tests are only moderately sensitive in predicting the presence of dementia. Both tests are poor in excluding other reasons for low test results due to low ROC specificity, particularly in the current sample of elderly psychiatric patients. This finding is consistent with the literature.^1,2 The ROC analysis shows that the sensitivity for ruling out dementia is 27 points for the MMSE and 23 points for the MoCA in the current patient sample when only the main diagnosis is considered. Note that these values are only valid for this specific patient population, but it suggests that an MMSE cut-off score of 24¹³ is too low to screen dementia patients when screening incoming patients in a general psychiatric clinic. The suggested cut-off for MoCA of 26 points for detecting cognitive decline is sufficiently sensitive for this specific patient population.

Since the detection of dementia among other conditions is more important than the inclusion of false positives, it could be argued that both tests serve their purpose in detecting dementia for the present patient population. Also, it may be argued that future screening tests should opt for higher sensitivity and perhaps sacrifice some specificity to identify all individuals who would benefit from further evaluation.

A study using Taiwanese test versions suggest that a formula combining MMSE results with Word Recall test results and a Visuospatial Index is a better predictive model for detecting MCI in Parkinson’s patients than the MoCA.¹⁰ So, if these or other tests are routinely screened in a clinic, more work should be done to develop predictive models based on a combination of the screening tests already performed. Most literature reviewed here suggests that MoCA is more sensitive than MMSE for the detection of cognitive decline in the elderly population. There was no significant difference in MMSE or MoCA scores between the non-demented patient groups in the present sample, although patients referred to a psychiatric clinic scored lower than expected from the general age-matched population. We did not see significant differences in performance between patients assigned to the psychoses, affective, or depression groups on either the MMSE or the MoCA, underlining the fact that normal or just below normal MMSE and MoCA scores are expected for these groups. These patients should not fail the MMSE or MoCA, but low cooperation in the testing situation may be a contributing factor to the poor results found in all groups.²⁴ Thirty-five percent of patients in the present sample had multiple diagnoses across the four diagnostic groups at the time of data collection. These patients performed poorer than average patients on both the MMSE and the MoCA, with an average 5-point difference on both scales, indicating that patients with multiple disease conditions generally perform indistinguishably from patients with dementia in the current sample of elderly psychiatric patients.

Five patients diagnosed with a form of psychoses, depression, or paranoia at the time of testing followed by a later dementia diagnosis performed well on the MMSE. Four of these patients scored above 25/30 and one scored 5/30 points. None of those patients were tested with the MoCA, and the MMSE results could not identify cognitive deficits in four out of five potential MCI patients. The current data do not include information regarding why the MoCA test was not performed, but inability to perform the MoCA test is an important result that suggest that further assessment should be performed. Another individual example of better than expected results was found in one patient with Alzheimer’s and 16 years of education, who scored 4/5 on the clock-test, 26 on the MMSE, and 20 on the MoCA. This exemplifies that a highly educated person may score higher than average despite having a form of dementia. There is also one example in the dataset in which a patient with Parkinson’s who was later diagnosed with dementia scored 20 on the MMSE and 22 on the MoCA, indicating that the MMSE may be more specific for certain patient groups or individuals. Although there could be other explanations for these example cases, including variability in testability, these examples illustrate the limitation of both tests both in the detection and elimination of dementia.

Limitations

The study has several limitations. The patient population in this study is heterogeneous, and the composition may vary between clinics, cultures, and languages. Also, the medication use at entry to the clinic was not controlled for, and medication may have temporarily affected the cognitive state of individual patients. Also, patients were at different psychological state at entry, which may have affected their testability. The sample size included all available subjects but was relatively small, and thus, sample size calculation was not performed.

Conclusion

The short-term cost of replacing the MMSE with MoCA or another alternative must be weighed against the benefit. MMSE combined with other measures may be equally or more sensitive. The study shows that the cut-off scores should be at least 27 for MMSE when screening patients for further assessment, and at least 23 for the MoCA test for sensitivity and specificity required in this patient sample. However, if established cut-off values are used, they should not be replaced with these values, since sensitivity is more important than specificity when screening incoming patients for possible cognitive deficits.

Supplemental Material

MMSE-NR_and_MoCA_Norwegian_version_-_material_1 – Supplemental material for Performance on the mini-mental state exam and the Montreal cognitive assessment in a sample of old age psychiatric patients

Supplemental material, MMSE-NR_and_MoCA_Norwegian_version_-_material_1 for Performance on the mini-mental state exam and the Montreal cognitive assessment in a sample of old age psychiatric patients by Maria Stylianou Korsnes in SAGE Open Medicine

Supplemental Material

Supplementary_material_-_MMSE_-_MoCA_-_English_versions – Supplemental material for Performance on the mini-mental state exam and the Montreal cognitive assessment in a sample of old age psychiatric patients

Supplemental material, Supplementary_material_-_MMSE_-_MoCA_-_English_versions for Performance on the mini-mental state exam and the Montreal cognitive assessment in a sample of old age psychiatric patients by Maria Stylianou Korsnes in SAGE Open Medicine

Footnotes

Author contributions

M.S. Korsnes designed the study. M.S. Korsnes supervised the data collection and wrote the article. M.S. Korsnes was responsible for the statistical design of the study and carried out the statistical analysis.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Ethical approval

The study was reviewed by the regional ethics committee (REK) of South-Eastern Norway (2013/1845). They considered the study to be a quality improvement study outside their mandate. A notification form “for internal quality register, quality study, health research and other research outside REK’s mandate, as well as whole registers as a basis for several studies and quality purposes” was then reviewed and approved by the data protection office at Oslo University Hospital.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was funded by Oslo University Hospital.

Informed consent

Written informed consent was obtained from all subjects. All subjects included had decisional capacity at the time of study and we did not include patients without capability of consent.

ORCID iD

Maria Stylianou Korsnes

Supplemental material

Supplemental material for this article is available online.

References

Folstein

McHugh

PR.

Mini-mental state: a practical method for grading the cognitive state of patients for the clinician. J Psychiat Res 1975; 12: 189–219.

Nasreddine

Phillips

Bédirian

, et al. The Montreal cognitive assessment (MoCA): a brief screening tool for mild cognitive impairment. J Am Geriatr Soc 2005; 53(4): 695–699.

Gorman

JM.

Comorbid depression and anxiety spectrum disorders. Depress Anxiet 1996; 4: 160–168.

Korsnes

Ulstein

ID.

Cognitive effects of late-life depression: review of neuropsychological findings. J Behav Brain Sci 2014; 4: 141–157.

Mungas

Reed

Farias

, et al. Age and education effects on relationships of cognitive test scores with brain structure in demographically diverse older persons. Psychol Aging 2009; 24(1): 116–128.

Milani

Marsiske

Cottler

, et al. Optimal cutoffs for the Montreal cognitive assessment vary by race and ethnicity. Alzheim Dement 2018; 10: 773–781.

Damian

Jacobson

Hentz

, et al. The Montreal cognitive assessment and the mini-mental state examination as screening instruments for cognitive impairment: item analyses and threshold scores. Dement Geriatr Cogn Disord 2011; 31(2): 126–131.

Malek-Ahmadi

Power

Belden

, et al. Age- and education-adjusted normative data for the montreal cognitive assessment (MoCA) in older adults age 70-99. Neuropsychol Dev Cogn B Aging Neuropsychol Cogn 2015; 22(6): 755–756.

Luis

Keegan

Mullan

Cross-validation of the Montreal cognitive assessment in community-dwelling older adults residing in the Southeastern US. Int J Geriatr Psychiatry 2009; 24(2): 197–201.

10.

Lee

, et al. Evaluating mild cognitive dysfunction in patients with Parkinson’s disease in clinical practice in Taiwan. Nature Sci Report 2020; 10: 1014.

11.

Pinto

TCC

Machado

Bulgacov

, et al. Is the Montreal cognitive assessment (MoCA) screening superior to the mini-mental state examination (MMSE) in the detection of mild cognitive impairment (MCI) and Alzheimer’s disease (AD) in the elderly. Int Psychogeriatr 2019; 31(4): 491–504.

12.

MacKenzie

Copp

Shaw

, et al. Brief cognitive screening of the elderly: a comparison of the mini-mental state examination (MMSE), abbreviated mental test (AMT) and mental status questionnaire (MSQ). Psychol Med 1996; 26: 427–430.

13.

Gallo

Wittink

MN.

Cognitive assessment. In: Gallo

Bogner

Fulmer

, et al. (eds) Handbook of geriatric assessment. 4th ed. Sudbury, MA: Jones and Bartlett, 2006, pp.105–151.

14.

Gross

McMillan

Chandrasekaran

, et al. Sentence processing in Lewy body spectrum disorder: the role of working memory. Brain Cogn 2012; 78(2): 85–93.

15.

Mitchell

AJ.

A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res 2009; 43(4): 411–431.

16.

Bastiaanse

Leenders

KL.

Language and Parkinson’s disease. Cortex 2009; 45(8): 912–914.

17.

Bergeron

Flynn

Verret

, et al. Multicenter validation of an MMSE-MoCA conversion table. J Am Geriatr Soc 2017; 65(5): 1067–1072.

18.

Dong

Lee

Basri

, et al. Montreal cognitive assessment is superior to the mini-mental state examination in detecting patients at higher risk of dementia. Int Psychogeriatr 2012; 24(11): 1749–1755.

19.

Strobel

Engedal

MMSE-NR. Norsk revidert mini mental status evaluering. Revidert og utvidet manual. Oslo: Nasjonalt Kompetansesenter for Aldring og Helse, 2008.

20.

van Walsem

Tyvoll

. The Montreal cognitive assessment (MoCA) Norwegian version. 2010, http://www.mocatest.org/

21.

World Health Organization. The ICD-10 classification of mental and behavioural disorders: diagnostic criteria for research. Geneva: World Health Organization, 1993.

22.

Hanley

McNeil

BJ.

The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143(1): 29–36.

23.

Larsson

Traylor

Malik

, et al. Modifiable pathways in Alzheimer’s disease: Mendelian randomization analysis. Brit Med J 2017; 259: j5375.

24.

Fink

Carpenter

Jr Sacks

, et al. Testability and group phenomenon in schizophrenia. Compr Psychiatry 1977; 18(1): 23–29.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

1.22 MB

0.48 MB