Rethinking Dementia Risk Prediction: A Critical Evaluation of a Multimodal Machine Learning Predictive Model

The introduction of artificial intelligence (AI)-driven Clinical Decision Support Systems (CDSS) and, in particular, of machine learning, is expected to greatly influence disease prediction in the field of dementia, with early identification of pre-clinical dementia status [1]. Ding et al. added original knowledge, exploring a “Multimodal Machine Learning for 10-year Dementia Risk Prediction: the Framingham Heart Study” [2], aimed at integrating traditional clinical assessments with advanced technology, including big data and AI to identify individuals at higher risk to develop dementia in the next decade.

Ding et al. performed a multimodal approach, capitalizing neuropsychological testing (NP), MRI measures, and a set of clinical risk factors for 10-year dementia prediction, showing that the employment of machine learning models provides an efficient strategy utilizing the non-linear information of predictors with potential clinical implications for preventative or prioritizing interventions to mitigate and/or reduce the burden of the disease.

The study is built upon a high-quality database and a robust methodology, offering a newly comprehensive pipeline for predictive systems in clinical settings, demonstrating that, unlike classical statistical methods that prioritize hypotheses and inferences, machine learning prioritizes predictions, excelling with extensive datasets, and effectively handling numerous patients and features. Moreover, machine learning can analyze vast, unconstrained data, making its full adaptability invaluable across different data analysis fields, including mental sciences [3].

Previous studies have used one or two types of datasets to predict high-risk groups [4–6]. Notably, the authors showed that the model incorporating all three modalities (NP, MRI, clinical risk factors) performed best, surpassing single-modality models. In particular, MRI measures were the most influential, addressing the value of integrating neuroimaging into predictive assessment models for dementia.

However, all the models showed a lower performance with increasing baseline age, especially for people aged 75 and more, suggesting a diminished accuracy in the prediction of incident dementia in older individuals.

As the population ages, the concept of frailty becomes increasingly relevant and may be considered a harbinger between aging and the development of dementia in later life. Frailty is defined as a state of an individual’s increased vulnerability to developing dependency and/or mortality or higher rates of adverse clinical outcomes when exposed to a physiological or psychological stressor [7]. Accumulating evidence has shown that different degrees of frailty may provide a buffer against brain neuropathological burden so that a reduced brain burden is needed before signs of dementia become evident when a state of frailty is present [8]. Similarly, frailty may shape the onset and clinical course of dementia of Alzheimer’s type (AD) [9], modulating the accumulation of brain AD-related hallmarks.

Based on this background, the dynamic nature of frailty may be associated with people’s earlier and late-life experiences, forming an individual’s resilience towards age or insult and tipping the balance between brain resilience and cognitive decline in later life. As a result, the epigenetic influence of lifestyle choices as a person ages may help protect against dementia by maintaining brain connectivity, plasticity, and networking even in very old ages.

The authors included the Fried phenotype [10] for baseline frailty stratification, which did not exert any impact on the prediction accuracy compared to the other clinical variables. The static construct of Fried phenotype may be a limitation for the dynamic nature of frailty, which allows multiple assessments of individuals’ clinical trajectories to capture changes in degrees of frailty over time. The accumulation of clinical deficits to detect frailty based on Rockwood construct [11] seems to fit better and pursue this scope, depicting the highly individualized accumulation over a life exposure. As a result, the evaluation of the relationship between frailty and brain resilience is particularly critical since an individual’s lifestyle habits (such as earlier/past social and cognitive engagements) may cumulate or have influenced current levels of activities and overall engagement. In line with that, recent evidence has underscored that cognitive reserve (CR), defined as the cumulation of one’s life experiences through intellectual, cognitive, and leisure activities, and the social environment substantially interact with frailty and that targeting CR and frailty may implement early prevention of mortality [12].

Based on this bidirectional interaction between frailty and dementia in older individuals, the reliability of a baseline single-point assessment static model to predict the dynamic nature of dementia in late life may have some major pitfalls. Trajectories of the patients are highly predictive of the development, transition, and future course of the disease, facilitating effective and tempestive care; AI can actually provide suggestions for challenging future directions. However, there are some substantial challenges related to the combination and modeling of data from trajectories and baseline variables (multimodal, sociodemographic, or genomic data). Patients’ trajectories, in fact, are highly dimensional, which reflects difficulties for AI solutions. A remedy could be achieved by reducing such dimensionality to lower levels to exclude overfitting. Moreover, newly developed computational strategies for missing data, and higher accuracy for standardization in data collection will be needed to comply with the dynamic nature of trajectories. Moreover, shorter time intervals to detect health objective measurements will be necessary, requiring further harmonization of data. Either way, these AI solutions shall be validated by randomized trials to confirm their effectiveness by assessing the interpretability and explainability of the models in clinical practice [13].

Furthermore, although limitations have been clearly exposed, MRI measures may be biased by multiple confounding factors such as the huge heterogeneity in brain aging. The accurate prediction of brain aging differences along trajectories of cognitive changes, on the basis of multiple health factors, is still a challenge. Indeed, brain aging is both a neuroimaging-based biomarker of aging that may be weighted [14] and a heterogenous clinical construct that can be barely identified with specific patterns of cognitive aging. Mediating the synergistic role of brain aging with individualized lifestyle habits and health indices is critical [15], since such a huge heterogeneity in cognitive aging patterns increases in late life, adding degrees of uncertainties [16].

Moreover, this study did not consider subtypes of dementia aside from AD such as mixed dementia [17] [18], which may be considered the prevalent types of dementia in very old individuals, adding misclassification biases to both baseline diagnostic accuracy and the predictive ability of the model.

Eventually, these findings originally address the need to collect holistic cognitive data that facilitate the application of multimodal machine learning in the field of dementia.

The use of machine learning holds the promise to anticipate the diagnosis of AD, in the era of newer disease-modifying therapies.

The collection of high-quality clinical data is mandatory and the use of appropriate AI-based techniques that match patient’s clinical phenotypes is crucial for the extensive reliability of medical AI. Both data quantity and quality are of key relevance, as biased or low-quality data can lead to skewed results [5]. As such, black-box machine learning techniques may add difficulty in understanding the relationship between clinical variables. Furthermore, when the model has to include a wide set of clinical variables, such as in the case of the multidimensional assessment of older adults, that reflects their clinical complexity and the person-centered approach beyond the single disease approach, the feasibility and interpretability of AI models may ultimately diminish.

In conclusion, medical researchers should continue embracing machine learning, adapting this evolving landscape to the prevention and treatment of chronic diseases, including dementia. Welcoming these changes and harnessing AI potential can pave the way to more insightful and data-driven research in the field of mental science. However, the real-world application is still in its infancy and further studies will be needed, based on reliable data, and aware of the importance of combining patients’ multidimensionality with the explainability and interpretability of predictions.