ChatGPT as a Diagnostic Aid in Alzheimer’s Disease: An Exploratory Study

INTRODUCTION

The diagnosis of Alzheimer’s disease (AD) involves the comprehensive assessment of cognitive symptoms and performance metrics, functional abilities, and neurological signs by specialized physicians. Cognitive evaluations typically entail neuropsychological tests covering general cognitive function, memory, language, and executive function. Functional and neurological assessments are typically conducted through clinical examinations and require receiving information from third-parties, typically family members from patients. The diagnosis of the underlying neurobiological entity of AD in living individuals requires neuroimaging and biomarkers, which may provide evidence on the accumulation of amyloid-beta and tau and neurodegeneration [1]. In the diagnostic process, the presence of medical conditions such as cardiovascular disease, obesity, and diabetes is also considered due to their association with increased risk of AD and multifactorial dementia [2]. The diagnosis of AD also includes categorizing patients into different stages, most commonly as belonging to the mild, moderate, or advanced stages, with many patients seeking consultation during the mild stage. Furthermore, the diagnostic process also considers preclinical and early clinical or prodromal stages. Patients may present with mild cognitive impairment (MCI), which involves objectively verifiable cognitive changes while, crucially, are still being able to perform daily tasks independently [3]. MCI can be a precursor to AD, in which case it may also be called prodromal AD, although not all individuals with MCI ultimately develop AD or other dementia [3]. To summarize, the diagnosis of AD involves a battery of assessments, including history intake, neuropsychological and neurological tests, neuroimaging, and biomarkers. This comprehensive approach allows for the classification of patients into specific stages of the disease, including its preclinical phases.

Given the complexity of AD diagnosis, we explored the potential of artificial intelligence, particularly Chat-Generative Pre-Trained Transformer (ChatGPT), to aid in the complex process of diagnosis. In the ever-evolving landscape of artificial intelligence, ChatGPT stands as a remarkable achievement in the ongoing quest to enhance human-machine communication. ChatGPT operates on the principles of deep learning, employing an extensive neural network model with the capacity to comprehend and produce text with a nuanced understanding of context, tone, and intent [4]. ChatGPT is increasingly finding applications in medicine. A growing body of research emphasizes the utility of ChatGPT in various medical contexts, such as patient education, interaction, and the dissemination of healthcare information [5–10]. ChatGPT has even demonstrated some levels of accuracy in diagnosing medical conditions [11–16]. This underscores the potential for ChatGPT to play a valuable role in Medicine, including aiding in the complex process of AD diagnosis.

While the potential of ChatGPT in medical diagnosis has been explored in various medical conditions, little research has been conducted on its application to AD. A study has attempted topredict AD from spontaneous speech using ChatGPT [17]. This study has shown that text embedding, as generated by ChatGPT, can be used to not only detect patients with AD from healthy controls but also to infer the patients’ cognitive scores. This study was however solely based on speech data. Using neuropsychological data, a study [18] has investigated the ability of ChatGPT to assist in AD screening. This study has assessed the ability of ChatGPT to provide general interpretations of neuropsychological test scores for a patient with mild AD. Results demonstrated that ChatGPT could accurately interpret the scores obtained on neuropsychological tests. However, ChatGPT has showed several limitations as it did not utilize standardized scores and did not specify the affected cognitive domains. It is important to note that the preliminary study by El Haj et al. [18] was concerned with the interpretation of neuropsychological data and did not directly address the diagnosis of AD. The present study expands upon the study by El Haj et al. [18]. The present study goes beyond the interpretation of neuropsychological data and aims to assess whether ChatGPT can effectively integrate neuropsychological, medical, neurological, neuroimaging, and biomarker information to contribute to the diagnostic process of AD. Importantly, the present study incorporates multiple case scenarios to evaluate the ability of ChatGPT to classify patients into specific stages of AD, including its preclinical phases. This broader assessment seeks to address the complex and multifaceted nature of AD diagnosis.

In summary, while research has shown the potential of ChatGPT in the field of medical diagnosis for various conditions [11–16], there is a dearth of knowledge about its application in the context of AD and neurological conditions in general. While a prior study by El Haj et al. [18] explored the ability of ChatGPT to interpret neuropsychological data in a single patient with AD, it did not directly address its broader potential in assisting with the diagnosis of AD. The present study thus assesses whether ChatGPT can effectively synthesize information from various medical domains to contribute to the intricate diagnostic process of AD. Furthermore, the study assesses the ability of ChatGPT to classify patients into distinct stages of AD, including its preclinical stages. To achieve this, we provided ChatGPT with a comprehensive dataset comprising patient histories, complaints, physical examinations, as well as biomarker, neuroimaging, and neuropsychological data. The patients presented varying degrees of AD severity, with one patient at the MCI stage. To validate the accuracy of ChatGPT in AD diagnosis, we compared its assessments with those made by neurologists and geriatricians who were blinded to the aims of the study. Additionally, we enlisted other blinded physicians to determine whether the diagnoses were formulated by their human colleagues or by artificial intelligence, akin to a Turing Test. Our expectations were that ChatGPT would demonstrate a high level of accuracy in AD diagnosis, potentially providing valuable insights into the diagnostic process.

METHODS

DISCUSSION

Within the constantly evolving landscape of artificial intelligence, we investigated whether AD diagnosis can be aided or supplemented using ChatGPT. We provided ChatGPT with descriptions of four cases: one representing the mild stage of AD, one representing the moderate stage of AD, one representing the advanced stage of AD, and representing MCI. ChatGPT accurately diagnosed the four cases. Similar diagnoses were made by two blinded professionals. Interestingly, we invited other blinded professionals to decide whether these diagnoses were made by colleagues or artificial intelligence, and they could not distinguish between the two, thereby, ChatGPT passing the Turing Test. These findings suggest that ChatGPT can be a useful tool for symptom assessment and may aid with the clinical diagnosis of AD.

AD is a complex disorder as multiple mechanisms (e.g., cognitive, neurobiological, and even environmental) are involved in its pathogenesis and progression, and it is stills unclear how each mechanism contributes to the disease and specific features of its clinical presentation. AD diagnosis is a challenging clinical task as it requires considering a wide range of parameters. Moreover, there is not a single test that can definitively diagnose AD. The diagnosis is typically based on a combination of medical history, clinical evaluation, and various biomarker assessments. The AD diagnosis is also challenging because clinicians need to consider that symptoms may overlap with those of other medical conditions, especially at the mild stage of AD where memory problems and other cognitive manifestations can overlap with normal age-related changes and other dementing disorders. Also, AD progresses somewhat differently in each individual, making it challenging to establish a standard diagnosis for each stage of the disease. Our study, however, provides initial evidence for the potential value of ChatGPT in the diagnosis of AD. As evident by ChatGPT’s responses, its diagnostic process is built upon the successful analysis of a variety of cognitive, clinical, neurological and biomarker information, in addition to the patients’ histories and complaints. Thus, ChatGPT can perform certain basic diagnostic tasks for AD. ChatGPT can even situate patients within a precise stage of AD. Clinicians have the option to utilize ChatGPT in a supplementary capacity. For instance, ChatGPT can be furnished with comprehensive case descriptions to help identify and delineate primary symptoms. Subsequently, clinicians can assess and verify this analysis to expedite the diagnostic procedure. Furthermore, the diagnostic approach of ChatGPT can serve as a valuable model for trainees, offering them a basis for symptom recognition and, in turn, aiding in the development of their capacity to evaluate and address any irregularities.

While ChatGPT may prove to be a valuable tool in aiding the diagnosis of AD, it is imperative to emphasize that, in its current developmental stage, it falls far short of replacing clinicians. Physicians are trained to deliver comprehensive diagnoses by considering the myriad mechanisms underlying the onset and progression of AD. This was, somehow, evidenced in our study as the blinded physicians automatically diagnosed Case 1 as mild stage AD, unlike ChatGPT who only did so after prompting. However, ChatGPT automatically situated the other cases into their respective stage of AD, possibly because it built upon our previous prompt. We note with interest and surprise that ChatGPT demonstrated the ability to learn from its previous interaction with a user. Another concern with the use of ChatGPT in diagnosis is that the clinical interview, an integral component of the diagnostic process, should be administered by physicians. Hence, it is vital to view ChatGPT as a complementary tool to clinical judgment, and it should always be employed by physicians who possess the training and expertise to detect and interpret AD symptoms.

Our study can be situated within the growing body of research on the application of ChatGPT in Healthcare [5–10, 19–24] and, specifically, in the area of medical diagnosis [11–16]. This research shows how ChatGPT may demonstrate high levels of accuracy in diagnosing medical conditions. Our study also extends a previous study showing that ChatGPT could accurately interpret scores of neuropsychological tests [18]. However, the present study is original in that it provides evidence of the ability of ChatGPT to diagnose AD and, critically, its ability to classify patients into specific stages of AD, including its preclinical phases.

One potential limitation of our study may be the use of generated cases. These cases were what one may consider typical “textbook” examples. The consideration of including data related to real cases could further strengthen the generalizability of our findings, although ethical aspects must be carefully considered. Adherence to ethical and privacy considerations is paramount when integrating ChatGPT into the diagnostic process. For example, the matter of data ownership is a subject of debate, as ChatGPT may use data to enhance its learning process. This is why we chose to deliberately create the four cases rather than use actual patient data. Privacy and professional confidentiality pose core ethical challenges when employing Large Language Models in the medical domain. The deployment of personal assistants powered by these models raises significant concerns regarding patient privacy and practitioner confidentiality. The sensitivity of data transmitted to these “personal assistants” requires scrutiny by regulatory bodies to mitigate potential intrusions in the medical domain. Another ethical challenge lies in the potential misuse of language models. The unregulated application of language models poses a prominent risk in the medical domain. Therefore, it is crucial to implement control measures aligned with legal frameworks to prevent any deleterious misuse of these potent linguistic tools.

In conclusion, while the use of ChatGPT in AD diagnosis is promising, it should be seen as an adjunct to clinical judgment rather than a replacement. The findings of this study open new avenues for the application of artificial intelligence in medical diagnosis, ultimately benefiting patients and contributing to the advancement of the field. It is a step toward harnessing the potential of Artificial Intelligence to improve diagnosis as well as the lives of individuals affected by AD and similar conditions.

AUTHOR CONTRIBTUIONS

Mohamad El Haj (Conceptualization; Investigation; Methodology; Writing – original draft); Claire Boutoleau Bretonnière (Resources; Supervision; Validation); Karim Gallouj (Methodology; Supervision; Validation); Nathalie Wagmann (Resources; Supervision; Validation); Pascal Antoine (Project administration; Resources); Dimitrios Kapogiannis (Supervision; Validation; Writing – review & editing); Guillaume Chapelet (Investigation; Methodology; Supervision; Validation; Writing – review & editing).