Abstract
Background:
Magnetic resonance imaging (MRI) provides objective information about brain structural atrophy in patients with Alzheimer’s disease (AD). This multi-structural atrophic information, when integrated as a single differential index, has the potential to further elevate the accuracy of AD identification from normal control (NC) compared to the conventional structure volumetric index.
Objective:
We herein investigated the performance of such an MRI-derived AD index, AD-Resemblance Atrophy Index (AD-RAI), as a neuroimaging biomarker in clinical scenario.
Method:
Fifty AD patients (19 with the Amyloid, Tau, Neurodegeneration (ATN) results assessed in cerebrospinal fluid) and 50 age- and gender-matched NC (19 with ATN results assessed using positron emission tomography) were recruited in this study. MRI-based imaging biomarkers, i.e., AD-RAI, were quantified using AccuBrain®. The accuracy, sensitivity, specificity, and area under the ROC curve (AUC) of these MRI-based imaging biomarkers were evaluated with the diagnosis result according to clinical criteria for all subjects and ATN biological markers for the subgroup.
Results:
In the whole groups of AD and NC subjects, the accuracy of AD-RAI was 91%, sensitivity and specificity were 88% and 96%, respectively, and the AUC was 92%. In the subgroup of 19 AD and 19 NC with ATN results, AD-RAI results matched completely with ATN classification. AD-RAI outperforms the volume of any single brain structure measured.
Conclusion:
The finding supports the hypothesis that MRI-derived composite AD-RAI is a more accurate imaging biomarker than individual brain structure volumetry in the identification of AD from NC in the clinical scenario.
Keywords
INTRODUCTION
Alzheimer’s disease (AD) is an occult and progressive neurodegenerative disease and is the most common type of dementia [1, 2]. The majority of AD patients seeking medical consultation are in the moderate-to-severe stage, and it is usually too late for the treatment to be effective [3]. Early and accurate identification of patients with AD is the key to achieve satisfactory treatment efficacy [4]. The main reasons for the delayed medical treatment include: 1) it is commonly believed that memory loss is a normal physiological phenomenon of aging and 2) the clinical symptoms of some AD patients are atypical, and the methods to confirm AD (such as cerebrospinal fluid examination and positron emission tomography, CSF and PET) are complicated and expensive [5 –7]. Therefore, it is in high demand to have a non-invasive AD identification tool that is highly consistent with the clinical diagnosis or neurobiological tests, easy to implement, and objective.
The neuropathological changes of AD are continuous and irreversible processes, manifested by amyloid deposition, tau protein phosphorylation, and neuron death [8, 9]. Macroscopically, the main manifestation is the atrophy of specific brain structures, which can be visualized in magnetic resonance imaging (MRI), and thus is a powerful tool for detection and differentiation of neurodegenerative diseases [10, 11]. Hippocampal atrophy is a neuroimaging biomarker of AD, and the degree of atrophy is related to the risk of dementia development [12, 13]. AccuBrain® (BrainNow Medical Technology Ltd, China) is a National Medical Products Administration and Conformite Europeenne marked cloud-based brain MRI analysis software for clinical use that automatically segments and quantifies brain structures in around 20 minutes. Based on the brain structure volumetry results, the AD Resemblance Atrophy Index (AD-RAI) was derived and has the potential to assess the cognitive decline in mild cognitive impaired patients and cognitively unimpaired subjects [14].
Although the predictive value of AD-RAI for cognitive decline has been demonstrated[14], the validation of its diagnosis performance with clinical and biological result is still lacking. The main objective of this study is to investigate the consistency of MRI-derived atrophy composite index (i.e., AD-RAI) in the identification of AD from NC. The performance of AD-RAI will be compared with conventionally used volumetry of the AD-related brain structure and lobe atrophy features.
METHODS
Subjects
Fifty patients with 3D T1W MRI clinically diagnosed as “sporadic AD” were acquired retrospectively from the clinical database in Sun Yat-sen University and the Third Affiliated Hospital of Shenzhen University from 2015 to 2019. The use of these data was approved by the Medical Research Ethics Committee of Sun Yat-sen Memorial Hospital of Sun Yat-sen University and the Medical Research Ethics Committee of The Third Affiliated Hospital of Shenzhen University. Fifty cognitively normal subjects were selected, according to their gender and age to match with the AD group, from the data acquired in the CU-RISK study (N = 31) [15] and SEEDS study (N = 19) [16] conducted in The Chinese University of Hong Kong, and the ethical approval was granted by The Chinese University of Hong Kong – New Territories East Cluster Clinical Research Ethics Committee. Aside from MRI data, the subjects in SEEDS study underwent 11C-PIB (Pittsburgh Compound B) PET, 18F-T807 (AV-1451) PET, and FDG PET to evaluate the amount of amyloid β, tau deposition, and neurodegeneration, respectively.
The inclusion criteria for AD patients were: 1) patients clinically diagnosed as sporadic AD according to the 2011 NIA-AA guidelines [17]; 2) aged between 45–90 years, right-handed, cognitive impairment manifested mainly by memory decline, Mini-Mental State Examination (MMSE)<26; 3) with good quality 3D T1W brain MRI scanned; 4) with no other known brain diseases. Exclusion criteria were: 1) patients with non-memory-related cognitive impairment as the first symptom; 2) patients with obvious mental behavior abnormalities; 3) patients who could not cooperate with cognitive assessments; 4) patients who had medical history of brain surgery; 5) patients undergoing radiotherapy in the brain; 6) the patients with carbon monoxide poisoning, alcoholism, and metabolic brain injury.
The inclusion criteria for selecting cognitively unimpaired Chinese people from CU-RISK and SEEDS into this study were 1) age between 45–90 years; 2) age- and sex-matched with the subjects in the AD group; 3) MMSE ≥27.
ATN biomarkers assessment
According to the update of the diagnosis standard of AD by NIA-AA in 2018 [18], the diagnostic criteria for the Amyloid, Tau, Neurodegeneration (ATN) system composed of three biomarkers of amyloid-β (A), pathological tau (T), and neurodegeneration (N). The diagnostic criteria are: 1) A + and T + indicating an AD pathology, 2) only A + indicating AD continuum, 3) negative A indicating the exclusion of AD pathology. Among the subjects, 19 normal healthy people and 19 AD patients underwent ATN examination of CSF or PET. CSF was detected by enzyme-linked immunosorbent assay. The test result of A + is Aβ1–42 < 550 pg/mL or Aβ1–42/Aβ1–40 ≤0.1, and the test result of T + is phosphorylated tau-181 protein >61 pg/mL and the test result of N + is total tau protein >452 pg/mL. In the normal control subgroup (N = 19), 11C-PIB and 18F-T807 PET-CT was conducted in the Department of Nuclear Medicine & PET of Hong Kong Sanatorium & Hospital, Hong Kong, to quantify amyloid and tau, respectively. The scanning was performed 35 min after the 11C-PIB intravenous injection, and the 18F-T807 PET-CT was conducted within one week’s time. The 18F-T807 PET imaging was performed 85 min after the intravenous injection. PET images were analyzed by detecting the level of protein tracer in brain, where A + is defined as the increase uptake of 11C-PIB was visually observed in AD-related brain regions, and global retention ≥1.42; T + is defined as when 18F-T807 uptake was visually observed in regions with Tau deposition known for AD, and SUVR ≥1.14.
MR image acquisition and image processing
The brain MRI of AD patients were collected using Philips Achieva (Philips Medical Systems, Best, The Netherlands) 3.0T MRI scanners in The Sun Yat-sen Memorial Hospital and Siemens Aera (Siemens Medical System, Iselin, NJ) 1.5T MRI scanner in The Third Affiliated Hospital of Shenzhen University. The MRI data of NC were acquired using Philips Achieva TX (Philips Medical Systems, Best, The Netherlands) 3.0T MRI scanners in The Chinese University of Hong Kong. Detailed imaging parameters are listed in Supplementary Table 1.
All T1W images were analyzed using the AccuBrain® v1.2 system to automatically quantified 68 structural metrics. The brain structure segmentation and quantification of AccuBrain® are based on a multi-atlas non-rigid registration scheme. Then the predefined brain parcellation labels were transformed to the individual brain image and fused to produce the final segmentation results. The segmentation accuracy of hippocampus in AccuBrain® has been validated in the European Alzheimer’s Disease Consortium-Alzheimer’s Disease Neuroimaging Initiative Harmonized Protocol standard data set [19]. The clinical applicability of AccuBrain® has been demonstrated in diseases as dementia [14, 20], post-stroke [21], small vessel disease [22], temporal lobe epilepsy [23], and Cushing disease [24].
Quantitative brain volumetry
AccuBrain® automatically segmented the brain regions to obtain the absolute volume (AV), relative volume (RV), and percentile of the brain regions. AV is the actual volume of the brain region (in ml). RV is the ratio of the absolute volume of the brain structure to the individual’s own intracranial volume (ICV). The brain lobe atrophy ratio (AR) is the ratio of the CSF volume to the brain parenchyma in the lobe. Percentile is the percentile of RV or AR among healthy subjects of the same age group. The quantitative medial temporal lobe atrophy (QMTA) is the ratio of the inferior lateral ventricle to the ipsilateral hippocampus volume. Lower percentile of the brain structure suggests higher level of brain atrophy, whereas lower percentile of ventricle and lobe atrophy implies the opposite.
AV reflects the size of the real brain structure, but it is easily affected by age and skull development. RV is an index for evaluating brain structure atrophy after correcting the total intracranial volume and the percentile is an index of brain structural atrophy after age adjustment, which can reflect the individualized degree of brain atrophy.
AD-RAI is a composite index (range 0-1) derived from the support vector machine (SVM) implemented in AccuBrain® assessing the similarity of the volumetric atrophy pattern of a given individual to typical AD patients. The larger the AD-RAI is, the greater the similarity to the brain atrophy pattern in AD patients. AD-RAI > 0.50 suggests the individual brain structure is more similar to AD instead of to healthy subjects [14].
Statistical analysis
We evaluated the performance of AD-RAI in AD identification via the area under the receiver operating characteristic (ROC) curve (AUC), and used DeLong’s method to statistically compare the AUC of the ROC curves. When comparing single or different indicators, the classifier model used is the binary classifier. The AccuBrain® quantitative brain structure indexes were compared between AD and NC groups using t-test. All differences were considered statistically significant at p < 0.05. Statistical analysis was conducted using IBM SPSS Statistics 25.0.0 and MedCalc.
RESULTS
Demographic information of subjects
The demographic information of the included subjects is shown in Table 1. There was no significant statistical difference in age, gender, and education between AD and NC group. Significant differences were seen in the history of memory decline, MMSE, and AD-RAI (p < 0.001).
Characteristics of subjects
N, number; MMSE, Mini-Mental State Examination.
Classification accuracy of AD-RAI as an independent biomarker
According to the clinical diagnostic criteria
The accuracy of AD-RAI calculated from AccuBrain® was assessed by comparing with the clinical diagnostic criteria (NIA-AA 2011). The accuracy of AD-RAI to diagnose AD was 91%, the sensitivity was 88%, and the specificity was 96%. The AUC of AD-RAI was 92%. The results are shown in Fig. 2A and 2C. However, there were 6 clinically diagnosed AD patients with AD-RAI less than 0.50 and 2 NC subjects with AD-RAI greater than 0.50. The patient information and brain volume percentiles are shown in Supplementary Table 2.

The characteristic structures of atrophy in AD brain structure. Group-wise box plot and sample images (2D segmentation, 3D surface, and the raw image) of AD (N=50) and NC (N=50) groups.

The ROC curves of AD-RAI and hippocampus, inferior lateral ventricle, bilateral temporal lobe atrophy ratio and QMTA in the differentiation of AD from NC. ROC curves of (A) AD-RAI and RV of the individual structures on whole group; (B) AD-RAI and RV of the individual structures on the subgroup; (C) AD-RAI and percentile of the individual structures on the whole group; (D) AD-RAI and percentile of the subgroup.
According to the ATN biological classification criteria
The accuracy of AD-RAI of classifying biologically confirmed 19 AD and 19 NC was 100%, the sensitivity was 100%, the specificity was 100%, and the AUC was 100%. The results are shown in Fig. 2B and 2D.
Brain volumetry in AD and NC
Characteristic brain structures in AD and NC
Among the volumetric measures described in Quantitative brain volumetry, we found that the bilateral hippocampus, bilateral amygdala, thalamus-proper, putamen, lateral temporal lobe, cingulate lobe, insular lobe, and QMTA were significantly atrophic in the AD group (p < 0.001), and the lateral ventricle, third ventricle, and inferior lateral ventricle were significantly enlarged (p < 0.001) (Table 2).
The volumetric results (AV, RV, and percentile) of brain structures in AD and NC
AV, absolute volume, RV, relative volume; L, left; R, right; *NC (AV) versus AD (RV), p < 0.05; †; NC (AV) versus AD (RV), p < 0.05; ‡NC (percentile) versus AD (percentile), p < 0.05.
Accuracy of AD identification using AD-RAI and single brain structural volumetry
Hippocampus, inferior lateral ventricle, bilateral temporal lobe, and medial temporal lobe are important diagnostic markers of AD. RV and percentile are metrics for evaluating brain atrophy. The results of brain structure segmentation, RV, and percentile are shown in Fig. 1. Here we compared the RV and percentile of each cognitive related brain structure (hippocampus, inferior lateral ventricle, lateral temporal lobe, and medial temporal lobe) with a composite index, AD-RAI in the identifying AD. ROC curves were shown in Fig. 2. Using the clinical diagnostic criteria, AD-RAI (AUC = 92%) was better than single structural biomarkers (AUChipp = 84%, AUCILV = 88%, AUCTemporal _ L = 81%, AUCTemporal _ R = 75%, AUCQMTA = 92%) when RV was used (Fig. 2A). These AUCs were only slightly different when percentile was used (Fig. 2C). Using the ATN biological diagnostic standard, AD-RAI (AUC = 100%) also outperformed the single structure volumetry (AUChipp= 82%, AUCILV = 89%, AUCTemporal _ L = 87%, AUCTemporal _ R = 87%, AUCQMTA= 94%) when RV was used (Fig. 2B). These AUCs were only slightly different when percentile was used (Fig. 2D).
DISCUSSION
In this study, we evaluated the classification performance of AD-RAI, a composite computer-generated AD atrophy index calculated based on structural MRI, with the diagnosis achieved using clinical criteria and biological criteria. Based on the clinical criteria, the AUCs of AD-RAI in classification of AD and NC was as high as 92%, which outperformed single AD-associated brain structure. The sensitivity and specificity of AD-RAI were 88% and 96%, respectively, when 0.5 was used as the cutoff value, which supported the applicability of AccuBrain® in the clinical scenario. According to the subgroup analysis based on 19 AD and 19 NC with ATN results, the classification results of AccuBrain® were also shown to be consistent with the biological diagnostic criteria.
The inter-scanner reproducibility of AD-RAI on the same cohort of 11 subjects (age 20–70 years) scanned using 1.5T Siemens and 3.0T Phillips MRI scanners was demonstrated in the Supplementary Material [25]. The mean of the absolute difference between the AD-RAI acquired using two scanners in this subject group was 0.0697.
In this study, the brain volumetry derived from AccuBrain® confirmed the brain regional atrophy pattern of AD reported frequently in the literature, e.g., volume reduction of brain structures like hippocampus, amygdala, notable atrophy of temporal lobe, cingulate lobe, insular lobe, and medial temporal lobe (measured as QMTA in this study), as well as the enlargement of (inferior) lateral ventricle. In clinical practice, the determination of brain structure atrophy is based on the visual rating of neuroradiologists or neurologists [10, 26], and therefore the precise degree of the atrophy is hard to assess and may have considerable inter-rater variation [27]. In addition, the atrophy assessment is usually performed for each individual structure separately, and the logic to compose a final judgment from multiple structures is usually ambiguous. The PET-CT or CSF examination of biological markers such as Aβ and tau is based on the level of biological markers of AD brain structure to identify and diagnose, with high specificity and sensitivity [6, 8]. However, PET-CT protein tracer is difficult to prepare and preserve, and the examination of biological markers is expensive, which hampers its use in clinical practice and screening [28]. AD-RAI is a brain atrophy degenerative similarity index quantified based on brain structure MRI volumetry, and has been proven to be able to predict cognitive decline. Being a clinically certified automatic quantification software, AccuBrain® has been applied in clinical diagnosis. This study further verifies the clinical applicability of AD-RAI in diagnosing AD, given the fact that MRI is widely used in clinical practice and AD-RAI yields high identification accuracy in AD [11, 19].
The research of applying machine learning in assisting AD diagnosis is a hot topic for decades [29, 30]. However, existing studies are mainly focused on developing machine learning models based on research tools which are usually time-consuming and with limited clinical validation [31]. The machine learning methods usually present as ‘black box’ and the lack of clinical interpretability hampers their clinical applicability [32]. In addition, different from the existing classifiers, which only gives a ‘yes/no’ result, AD-RAI is an automatically generated index with a continuous value to assess the ‘likelihood’ of the similarity of an individual brain to typical AD brains.
It has been reported that there are 4 subtypes in AD with distinct brain atrophy patterns [33 –35]. In this study, there are 8 subjects whose AD-RAI results are inconsistent with the diagnosis. In Supplementary Table 2, the percentiles of the brain volumes of these 8 cases 6 AD patients and 2 NC subjects are shown. Among them, there are 1 case of atypical AD with no atrophy of AD-related structures, 2 cases belonging to the hippocampal-sparing AD, and 3 cases being non-atrophic AD. The 2 NC with AD-RAI>0.5 are present with severe brain atrophy, which may actually show a higher chance to have cognitive decline in the future, and their longitudinal followup is still in progress. AD-RAI is an index derived from volumetry of multiple brain structures using machine learning algorithm based on clinically diagnosed AD patients and NC, where there was much higher proportion of typical AD than atypical ones. That explains why its result will deviate from the diagnosis when the brain atrophy is not prominent [36, 37]. AD-RAI is the index for identifying AD purely based on MRI evidence, and thus it should be considered as an imaging biomarker. The final clinical diagnosis should be made with clinical association and biological markers when achievable [28, 38]. Absolutely accurate AD diagnosis is a difficult problem—given the real ‘ground truth’, i.e., pathological evidences are not feasible in clinical practice, and the accuracy of biological markers themselves is around 90%–95%, the accuracy of this MRI-based composite imaging biomarker is already clinically applicable. Last but not least, this study only focused on classification of AD and NC, and the ability of the MRI volumetric measures to differentiate different types of dementia and AD subtypes deserves further investigation.
This study confirmed that the machine learning derived index AD-RAI is a valuable imaging biomarker in assisting AD diagnosis. There are some limitations in this study. First of all, the sample size with CSF examination results is relatively small, which limits the generalization ability of the statistical findings. Secondly, all subject involved in the current study are all East Asian in ethnicity, though difference between AD, MCI, and NC can also be derived in [14] from the ADNI database. Confirmation of the current finding deserves cross validation studies performed on multicenter and large-scale datasets in the future.
In conclusion, AD-RAI as a composite AD-like brain atrophy imaging biomarker identifies AD patients accurately compared with the clinical diagnosis criteria and the biological research framework. Though brain atrophy presented in MRI is well-studied and simple, and composite indexes like AD-RAI are clinically practical implementations of neuroimaging biomarkers, and should be used with clinical and biological association.
Footnotes
ACKNOWLEDGMENTS
This study was supported by the grant provided by funding from the National Nature Science Foundation of China (grant numbers 81870836), China Postdoctoral Science Foundation (2019TQ0384, 2019M660225), and the Natural Science Foundation of Guangdong Province (grant numbers 2017A030313459, 2020A1515010210).
