Abstract
Background:
Patients with amnestic mild cognitive impairment (aMCI) have an increased risk of dementia. However, conversion rate varies. Therefore, predicting the dementia conversion in these patients is important.
Objective:
We aimed to develop a nomogram to predict dementia conversion in aMCI subjects using neuropsychological profiles.
Methods:
A total of 338 aMCI patients from two hospital-based cohorts were used in analysis. All patients were classified into 1) verbal, visual, or both, 2) early or late, and 3) single or multiple-domain aMCI according to the modality, severity of memory dysfunction, and multiplicity of involved cognitive domains, respectively. Patients were followed up, and conversion to dementia within 3 years was defined as the primary outcome. Our patients were divided into a training data set and a validation data set. The associations of potential covariates with outcome were tested, and nomogram was constructed by logistic regression model. We also developed another model with APOE data, which included 242 patients.
Results:
In logistic regression models, both modalities compared with visual only (OR 4.44, 95% CI 1.83–10.75, p = 0.001), late compared to early (OR 2.59, 95% CI 1.17–5.72, p = 0.019), and multiple compared to single domain (OR 3.51, 95% CI 1.62–7.60, p = 0.002) aMCI were significantly associated with dementia conversion within 3 years. A nomogram incorporating these clinical variables was constructed on the training data set and validated on the validation data set. Both nomograms with and without APOE data showed good prediction performance (c-statistics ≥ 0.75).
Conclusions:
This study showed that several neuropsychological profiles of aMCI are significantly associated with imminent dementia conversion, and a nomogram incorporating these clinical subtypes is simple and useful to help to predict disease progression.
INTRODUCTION
Mild cognitive impairment (MCI), especially the amnestic form, is regarded as a transitional state between normal aging and Alzheimer’s disease (AD) [1, 2]. However, not all amnestic MCI (aMCI) patients progress to AD dementia (ADD); some remain stable or even revert to normal cognition [3 –5], suggestive of a heterogeneous entity. Thus, finding good predictors for AD progression in aMCI patients has attracted considerable attention because of the clinical implications of such a model.
Previous studies investigated several neuropsychological characteristics which are predictable of the progression of aMCI to ADD. Poor performance in episodic memory tests, cognitive impairment in other domains such as frontal/executive function, and depression are all known to predict conversion to AD in aMCI subjects [6 –9]. In fact, even among aMCI subjects, those with more severe memory impairment or multi-domain cognitive deficits have higher AD conversion rates than those with milder memory impairment or only memory involvement [2 , 11].Also, the study from our group found that aMCI patients with verbal memory dysfunction had a higher risk of dementia conversion than patients with visual only memory dysfunction [12]. However, while these studies have shown differences at the group level, they did not predict the risk of dementia at the individual level. Therefore, it is crucial to develop risk prediction models that can distinguish individuals who would convert to ADD from those who would not.
The risk prediction model has been widely used in the field of medicine for prediction of cancer risk or cardiovascular disease prognosis. The proper use of risk prediction models, such as the nomogram, is important for patient counseling, management, risk stratification, and sometimes for the determination of early enrollment in clinical trials. Among many risk prediction models, the nomogram method isclinically useful because it is graph-based and simple, which makes it easy to communicate with patients.
In this study, we therefore aimed to construct a nomogram to predict the risk of dementia in aMCI patients using clinical features and neuropsychological results. We hypothesized that various clinical and neuropsychological features of aMCI would have independent effects on the risk of dementia conversion, and that a combination of those features could make it feasible to accurately and intuitively predict dementia conversion at the individual level.
METHODS
Subjects
A total of 338 aMCI patients, who had been followed at least three years after baseline neuropsychological tests, were included from two separate registry studies: 107 aMCI patients from the Memory Disorder Clinic in Samsung Medical Center (Enrolled from June 2007 to December 2011) and 231 aMCI patients from the Clinical Research Center for Dementia of South Korea (CREDOS) study, a nationwide multicenter cohort study of cognitive disorders involving 31 memory disorder clinics at universities and general hospitals in South Korea (Enrolled from November 2005 to December 2011) [13]. These two cohorts used a common standardized diagnostic assessment, including diagnostic criteria for normal cognition, MCI, and AD. All aMCI subjects met the following criteria based on Petersen’s criteria for MCI: 1) subjective memory complaint by patients or caregivers; 2) normal general cognition as defined by scores on the Korean version of the Mini-Mental State Examination (MMSE) within 1.0 SD of the norms for age- and education-matched normal subjects; 3) normal activities of daily living (ADL), as judged not only clinically but also using the ADL scale described below; 4) objective memorydysfunction evidenced by low scores (≥1.0 SD below the norms) on either verbal or visual memory tests; and 5) not demented.
All subjects underwent a detailed clinical interview using a semi-structured questionnaire, neurological examination, and a comprehensive neuropsychological battery, as described in previous studies [14]. ADL were assessed using a questionnaire form, the Seoul Instrumental ADL (S-IADL). This questionnaire has scoring system on a four-point Likert scale ranging from 0 to 3, with higher scores indicating lower function, and has been standardized with a cut-off point of 7 for impaired ADLs [15]. Apolipoprotein E (APOE) genotype test was performed in 242 (71.6%) out of 338 subjects.
We excluded patients who had secondary causes of cognitive deficits confirmed with laboratory tests including vitamin B12/folate determination, syphilis serology, and thyroid functioning tests, and those with structural lesions on conventional brain MRI such as territorial infarction, intracranial hemorrhage, brain tumor, hydrocephalus, traumatic brain injury, or severe white matter hyperintensities, according to the modified Fazekas ischemic scale. Participants with clinical presentations reminiscent of other types of dementia such as dementia with Lewy bodies, progressive supranuclear palsy, or frontotemporal lobar degeneration were also excluded.
The Institutional Review Board of the Samsung Medical Center approved this study, and informed consent was obtained from all participants andcaregivers.
Neuropsychological assessments
All participants underwent the Seoul Neuropsychological Screening Battery (SNSB), a standardized neuropsychological battery described in detail elsewhere [12, 16]. Four major cognitive domains were evaluated: memory, language, visuospatial, and frontal/executive function. Scores on delayed recall items of Rey-Osterrieth Complex Figure TesT (RCFT), and Seoul Verbal Learning Test (SVLT) were used to determine whether subjects had visual and verbal memory dysfunction, respectively. Scores on the Korean version of the Boston Naming Test (K-BNT) and RCFT copy were used to evaluate language and visuospatial dysfunction, respectively. Frontal/executive function was evaluated from three groups of tests: motor executive function (contrasting program, Go/no-go, fist-edge-palm, alternating hand movement, alternative square and triangle, and Luria loop); Controlled Oral Word Association Test (COWAT); and Stroop tests. Frontal/executive dysfunction was operationally defined as impairment in at least two of these three groups of tests. Scores in each specific cognitive test were considered to be abnormal when they were lower than the –1.0 SD (16th percentiles) of the age and education matched norms.
Classification of aMCI subjects
We classified aMCI subjects into several subgroups by the modality, severity of involved memory dysfunction, and multiplicity of involved cognitive domains, based on previous results from our groups [12 , 16–18]. First, we classified aMCI subjects into three subgroups according to the modality of involved memory dysfunction: subjects with only visual memory dysfunction (Visual-aMCI), those with only verbal memory dysfunction (Verbal-aMCI), and those with both visual and verbal memory dysfunction (Both-aMCI) [12, 17]. Therefore, Verbal-, Visual-, and Both-aMCI patients have scores lower than –1.0 SD of norms in the delayed recall tests of SVLT, RCFT, and both SVLT and RCFT, respectively. Second, we classified aMCI subjects according to the severity of memory dysfunction. If aMCI subjects had scores on delayed recall tests between 1.0 and 1.5 SD below norms, they were considered to have mild memory dysfunction and classified as early stage aMCI (E-aMCI); if subjects had scores greater than 1.5 SD below norms in either SVLT or RCFT delayed recall tests, they were considered to have severe memory dysfunction and classified as late stage aMCI (L-aMCI) [18, 19]. Finally, aMCI subjects were classified according to the multiplicity of involved cognitive domains [16]. Patients with isolated memory dysfunction were assigned as single-domain aMCI (Single-aMCI), and subjects having memory impairment plus other cognitive deficits, such as language and visuospatial dysfunction, were assigned as multiple-domain aMCI (Multiple-aMCI).
Follow-up
All patients underwent a standardized neuropsychological battery at least twice. A diagnosis of dementia was made based on the criteria of the fourth edition of the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV) and required evidence of cognitive deficits (confirmed by neuropsychological testing) and social and/or occupational dysfunction (confirmed by impairments of ADL). For a diagnosis of probable AD, the criteria of the National Institute of Neurological and Communicative Disorders and Stroke (NINCDS) and the Alzheimer’s Disease and Related Disorders Association (ADRDA) were used [20]. A consensus panel and an experienced neurologist reviewed the interview records and neuropsychological results of each aMCI patient and confirmed the AD diagnosis in the CREDOS and SMC cohorts, respectively.
To make the model, we specified the primary outcome as 3-year dementia conversion and included variables such as age, sex, years of education, BMI, three neuropsychological features (modality, severity, and multiplicity), and APOE4 as potential predictors.
Analyses
Statistical analyses
Continuous variables which did not meet the assumption of linearity were converted into categorical variables by grouping values into ≥ 2 categories. The association of potential predictors with outcome was tested by univariable and multivariable logistic regression models. A stepwise method was used for variable selection; a variable had to be significant at the 0.25 level before it could be entered into the model, while a variable in the model had to be significant at the 0.15 level for it to remain. Multicollinearity was checked using variance inflation factor (VIF). Multiple comparisons were corrected by Bonferroni’s method. Statistical analysis was executed using SAS version 9.4 (SAS Institute, Cary, NC) and R 3.3.2 (Vienna, Austria; http://www.R-project.org/). Nomograms were formulated based on the results of multivariable analysis by R 3.2.2 (http://www.r-project.org) with rms packages, which was also used for nomogram validation.
Training and validation data sets for model construction
To make the models, we split the data into two groups, one for training and the other for validation. The training data set consisted of all patients from the SMC data set and 50% of patients randomly selected from CREDOS data. The validation data set was composed of the remaining 50% of CREDOS patients. Among predictors, we selected age and sex because of their biological importance in AD development, and further selected BMI, education level, and neuropsychological patterns such as modality, severity, and multiplicity because these variables affected conversion of aMCI to dementia in our previous studies [12 , 21]. Also, we made two versions of the predictive model, with and without APOE genotype data, because only 71.6% of patients had APOE genotype data. In the model without APOE genotype data (Model 1), the training and validation data sets were composed of 222 and 116 patients, respectively. In the model with APOE genotype data (Model 2), the training and validation data sets included 167 and 75 patients, respectively (Table 1).
Demographic and clinical characteristics of the subjects
Values are presented as mean ± SD or number of cases (percentage). BMI, body mass index (kg/m2); aMCI, amnestic mild cognitive impairment; ADD, Alzheimer’s disease dementia.
Development of risk scores and nomogram
The multivariable logistic regression model was used for creating the nomogram. We assigned a point value to each variable using the beta coefficients from the regression model. Based on the estimated regression coefficients, we ranked the estimated effects disregarding statistical significance as well as direction (absolute beta value). The most important thing was to determine which predictor had the biggest impact in the model, then sequentially to assign other predictors based on their proportions relative to the most impactful predictor. The summed total point value from the point axis for each predictor was allocated to an overall risk score, using the linear predictor method (by the package of rms in R).
Validation of risk prediction models and nomograms
To validate the predictive accuracy of our prediction models, nomogram performance was quantified with respect to discrimination and calibration. Discrimination was quantified with a concordance index (C-index), which is the area under the receiver operating characteristic (ROC) curve. Calibration was graphically assessed with the relationship between the actual observed probabilities and predicted probabilities (calibration curve). Internal validation of performance was estimated with a bootstrapping method (1,000 replications) and a 10-fold crossvalidation method.
RESULTS
Patient demographics and clinical characteristics
Of the 338 subjects with aMCI, 222 (65.7%) were female. The mean age at the baseline evaluation was 71.6 (SD = 7.3) years (range 43–88). Mean follow-up duration was 2.92 (SD = 1.99) years. A total of 208 (61.5%) subjects were diagnosed as converters during 3 years of follow-up. Patients whose test results remained stable (n = 87), reverted to normal (within 1.0 SD of normal in all cognitive domains, n = 27) or converted to non-memory type (non-aMCI, n = 16) during the 3 years of follow-up were included as nonconverters (n = 130). The clinical characteristics of patients in the training and validation sets are shown in Table 1, and the comparisons of converters and nonconverters in two models are shown in Supplementary Table 1.
Developments of risk prediction models
Logistic regression models were used to test the association of the outcome with potential predictors. Univariable logistic regression results are shown in Supplementary Table 1. A multivariable logistic regression model with stepwise selection showed that age (OR 1.10, 95% CI 1.05–1.15) and neuropsychological features were significantly predictive of conversion to dementia. In particular, Both-aMCI (OR 4.30, 95% CI 1.95–9.47), L-aMCI (OR 2.15, 95% CI 1.06–4.36), and Multiple-aMCI (OR 3.60, 95% CI 1.78-7.29) had a significantly higher dementia risk compared to Visual-aMCI, E-aMCI, and Single-aMCI, respectively (Table 2, Model 1). When the same analysis was performed in the patients with APOE genotype data, the presence of APOE4 was predictive of conversion to dementia (OR 4.71, 95% CI 2.12–10.49), but L-aMCI compared to E-aMCI was no longer predictive of conversion to dementia (Table 2, Model 2).
Multivariable analysis of clinical and neuropsychological factors associated with conversion to dementia
*The total points made from the sum of each point indicate the overall risk score.
The constructed nomograms are shown in Fig. 1. A specific point was matched to each variable based on the beta coefficients from these regression analyses. The total points made from the sum of each point indicate the overall risk score. This can be applied to predict the risk of dementia conversion as shown in Table 3.
Nomogram.
Risk of dementia conversion for low, intermediate and high-risk individuals according to the dementia risk score
Validation analysis
Supplementary Table 2 shows the validation results in terms of discrimination capabilities. The prediction performance of Model 1 was 0.77 in the original data and 0.75/0.79 in internal/external validation. These values were even higher in the model with APOE data, as C-statistics were 0.80 in original data, and above 0.75/0.82 in internal/external validation. The nonparametric calibration curves showed that the bias corrected calibration plot was close to the idealpredicted values, which means the nomograms were well calibrated (Supplementary Figure 1).
Additionally, we obtained the positive predictive and negative predictive values using the data in this study (Supplementary Table 3). If total points are higher than 140, the positive predictive value for dementia conversion in 3 y is 0.85 and 0.89 in Model 1 and Model 2, respectively. Also, if total points are 73 or lower, negative predictive value for dementia conversion in 3 years is about 0.89 and 0.87 in Model 1 and Model 2, respectively.
DISCUSSION
In this study, we developed and validated a nomogram to predict the risk of conversion to dementia after 3 years in aMCI subjects based on clinical and neuropsychological features in a nationwide multicenter aMCI cohort study. Our results suggested that this nomogram is effective and visually insightful in predicting dementia conversion in aMCI patients.
The risk prediction models in this study showed fair to good discrimination (C-statistics of 0.78 in Model 1, 0.80 in Model 2) and calibration capabilities. Previous studies have incorporated biomarker information such as Aβ in cerebrospinal fluid, amyloid deposition on PET, and atrophy measured on brain MRI into prediction models [22 –25], showing accuracies ranging from 60 to 90% [22 –26]. However, biomarker studies are relatively expensive, not easy to conduct, and sometimes unavailable depending on the clinical situation. In addition, data from advanced biomarker studies sometimes require complicated data processing procedures that clinical sites may not be able to perform. Therefore, our model based on neuropsychological tests is useful, because we can easily get neuropsychological test results in a clinical setting and intuitively predict dementia conversion using the nomogram. Also, this risk score makes it possible to predict dementia conversion quantitatively by showing the probability. We expect that this will be helpful for patient-doctorcommunication regarding a prognosis.
We found that age, but not sex or educational levels, was associated with the risk of conversion to dementia in this analysis. Old age and low education are well-known risk factors for dementia in cognitively normal individuals, while ADD patients with early age of onset and with higher education show more rapid progression than ADD patients with late age of onset and with lower education [27 –29]. To our knowledge, the effects of age and education level on conversion of aMCI to dementia have not yet been fully investigated. In this study, age was the most powerful risk factor even after controlling for neuropsychological features. For example, an 85-year-old aMCI patient with single, early stage, visual amnestic features had a higher risk score for conversion to dementia than a 45-year-old aMCI patient with multiple, late stage, combined visual and verbal amnestic features. However, education level was not predictive of conversion to dementia in aMCI patients. In epidemiologic studies, the prevalence of AD was higher in females than in males. Previous studies also showed that females are at higher risk of developing AD than males[30, 31]. However, we did not find that female sex was predictive of conversion to AD.
In this study, the presence of APOE4 was also predictive of conversion to dementia after controlling for age, education, and neuropsychological features. APOE4 is one of the most important risk factors for development of dementia in cognitively normal individuals, while it is unclear whether APOE4 affects progression of ADD [32]. Some studies showed that ADD APOE4 carriers showed faster decline in cortical thickness than ADD APOE4 non-carriers, while others suggested that APOE4 did not affect progression of ADD. Consistent with a previous study showing that APOE4 is associated with increased risk for progression to ADD in MCI patients [33], our findings also suggested that APOE4 is one of the most important risk factors for conversion of aMCI to dementia.
We found that neuropsychological features of aMCI were independently predictive of conversion to dementia, consistent with previous studies. However, while previous studies showed that each neuropsychological feature affected conversion to dementia, they did not control for each other. In this study, when we converted these features into the risk score, the risk of dementia progression was in the descending order of Both-aMCI (modality), multiple-aMCI (multiplicity), and late stage-aMCI (severity). Interestingly, when APOE genotype data was included in the analysis, the effect of severity on dementia risk disappeared. This suggests that in aMCI patients, the contribution of severity of memory impairment might be attenuated in the presence of APOE4.
In this study, we developed and validated risk prediction models of conversion to dementia based on a nationwide multicenter cohort study for aMCI. There are several limitations in this study. First, despite the strength in being a multicenter study in Korea, the results of this study may not be generalizable to multinational patients. External validation in another cohort, such as the Alzheimer’s Disease Neuroimaging Initiative, is required to demonstrate its applicability in all aMCI patients. Because most studies have been performed in Western countries, there is an unmet need for developing risk prediction models in Asian aMCI patients. Second, Model 2, which included APOE genotype data (71.6%), had a relatively small number of patients. Finally, in this cohort, clinical diagnosis of aMCI and AD did not include amyloid PET imaging. While most patients who converted to dementia would be likely to have AD, we were not able to confirm AD pathology in vivo. Therefore, a future longitudinal study that extends this nomogram approach to subjects with amyloid PET data might be helpful to confirm the results of this study. Nonetheless, our study is strengthened by the clinical usefulness and applicability of the risk score system, which was developed simply based on neuropsychological test results. Furthermore, in current circumstances under which clinical studies of AD disease modifying treatments trials are active, clinic-based risk scores help to prioritize the MCI patients who are at highest risk of disease progression.
Footnotes
ACKNOWLEDGMENTS
This research was supported by a fund (2016-ER6203-00) by Research of Korea Centers for Disease Control and Prevention; The National Research Foundation of Korea (NRF) grant funded by the Korean government (MSIP; Nos. NRF-2015R1C1A2A01053281, NRF-2017R1A2B2005081); The Brain Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT & Future Planning (2016M3C7A1913844).
