Automatic Classification on Multi-Modal MRI Data for Diagnosis of the Postural Instability and Gait Difficulty Subtype of Parkinson’s Disease

Abstract

Background: Patients with the postural instability and gait difficulty subtype (PIGD) of Parkinson’s disease (PD) are a refractory challenge in clinical practice. Despite previous attempts that have been made at studying subtype-specific brain alterations across PD population, conclusive neuroimaging biomarkers on patients with the PIGD subtype are still lacking. Machine learning-based classifications are a promising tool for differential diagnosis that effectively integrate complex and multivariate data.

Objective: Our present study aimed to introduce the machine learning-based automatic classification for the first time to distinguish patients with the PIGD subtype from those with the non-PIGD subtype of PD at the individual level.

Methods: Fifty-two PD patients and forty-five normal controls (NCs) were recruited and underwent multi-modal MRI scans including a set of resting-state functional, 3D T1-weighted and diffusion tensor imaging sequences. By comparing the PD patients with the NCs, features that were not conducive to the subtype-specific classification were ruled out from massive brain features. We applied a support vector machine classifier with the recursive feature elimination method to multi-modal MRI data for selecting features with the best discriminating power, and evaluated the proposed classifier with the leave-one-out cross-validation.

Results: Using this classifier, we obtained satisfactory diagnostic rates (accuracy = 92.31%, specificity = 96.97%, sensitivity = 84.21% and AUC_max = 0.9585). The diagnostic agreement evaluated by the Kappa test showed an almost perfect agreement with the existing clinical categorization (Kappa value = 0.83).

Conclusions: With these favorable results, our findings suggested the machine learning-based classification as an alternative technique to classifying clinical subtypes in PD.

Keywords

Parkinson’s disease functional neuroimaging diagnosis machine learning support vector machines

INTRODUCTION

Parkinson’s disease (PD) is a common neurodegenerative disease with varying degrees of cardinal manifestations, including resting tremor, rigidity and bradykinesia [1]. Based on distinct major symptoms, the heterogeneous entity of PD patients can be grouped into the postural instability and gait difficulty– dominant (PIGD), the tremor-dominant (TD) and the mixed subtypes [2, 3]. Patients with the PIGD subtype have a relatively malignant course [3], including shorter life expectancy, faster progression, worse prognosis, and higher risk of complications [4]. Routine dopamine replacement therapies [5] and surgical procedures [6] work with less success in treating predominant PIGD symptoms in the long term [6]. The PIGD subtype is thus a refractory group of PD population that needs further discrimination from other subtypes of PD. In clinical practice, the identification of subtypes is based on the clinical scales, which may incorporate subjective judgement. For this reason, an efficient and unbiased way to distinguishing patients with the PIGD subtype is essential for making appropriate treatment plans.

In the last few years, there has been increased interest in subtype-specific brain changes of PD and in particular, of PD patients with the subtype of PIGD [7 –12]. It has been generally recognized that multiple neural mechanisms participate in the presence of distinct phenotypes of PD [2 , 14]. Brain alterations in structure [12 , 16] and function [12 , 18] have been frequently reported in comparisons between the PIGD and the non-PIGD subtypes; however, the consensus of the specific brain biomarkers for the PIGD subtype is still far from being reached. Hence, a workable integration of multi-modal data is required to facilitate differential diagnosis of the PIGD subtype.

Along with the advances in machine learning algorithms, the machine learning-classifiers make it possible to effectively analyze complex and multivariate functional magnetic resonance imaging (fMRI) data [19]. Machine learning-based pattern recognition techniques such as support vector machine (SVM) [20, 21] have been widely applied to patient classification across various neurological and psychiatric disorders, with excellent results have been obtained [22 –26]. The SVM algorithm allows studying brain characterizations at the individual level and is sensitive to subtle effects in the brain, exhibiting its high translational potential in clinical practice [24].

In the present study, we aimed to first introduce an SVM-based classification to distinguish patients with the PIGD subtype of PD from those with the non-PIGD subtypes, and then predicted classification outcomes would be compared to the existing clinical diagnosis categorization results. This proposed classification was done by preparing features from structural and functional MRI datasets of PD patients, constructing the classifier with the SVM algorithm, and assessing its performance using indices of sensitivity, specificity and accuracy, receiver operating characteristic (ROC) analysis, and the individual-by-individual agreements test as compared to the acknowledged categorization. As we would show, our newly proposed classification appeared to give satisfactory diagnostic results of expectation.

MATERIAL AND METHODS

Subjects

A total of 97 subjects (M/F 55/42, mean age 57.99 ± 9.50y) participated in the present study, of which 52 were PD patients (M/F 31/21, mean age 58.86 ± 9.02y) and 45 were normal controls (NC, M/F 24/21, mean age 56.98 ± 10.02y). All the patients were diagnosed according to the UK Parkinson’s Disease Society Brain Bank criteria for PD in the Department of Neurology at the Second Affiliated Hospital of Zhejiang University School of Medicine. The NCs were recruited from the communities in Hangzhou who were voluntary to participate in the study. Subjects with history of self-reported or clinically observed neurological/psychiatric disorders were excluded from the study. Using the clinical categorization proposed by Jankovic et al. [3], the recruited PD patients were grouped into the PIGD subtype (19 patients, M/F 11/8, mean age 58.68 ± 8.38y), TD subtype (25 patients, M/F 16/9, mean age 56.96 ± 8.85y) and mix subtype (8 patients, M/F = 4/4, mean age 65.25 ± 9.19y). Here the TD and mix subtypes were merged into one single group termed the non-PIGD group due to the main concern of the present study focusing on patients with the PIGD subtype of PD. Sixteen in 52 of the recruited patients were newly diagnosed and untreated. In the rest of the 36 treated patients, one used to receive irregular Traditional Chinese Medicine treatment. At least a 12-hour withdrawal from medication was required for each treated patient before the examination. Clinical examinations of the Unified Parkinson’s Disease Rating Scale (UPDRS), the Hoehn and Yahr scale (H&Y), duration of disease, and the Mini-Mental State Examination (MMSE) were investigated for all the recruited patients. The duration of disease was defined as the time since the self-conscious parkinsonian signs were present until the time when he/she participated in the study. Clinical examinations ensured the subjects were able to complete independent examination without extra aids. All the subjects provided written informed consent to give permission to participate in the study. The entire protocol has been reviewed and approved by the Ethics Committee of the second affiliated hospital of Zhejiang university school of medicine.

MRI acquisition

Multi-modal MR data were acquired on a 3.0T GE Signa EXCITE scanner with an 8-channel phase-array head coil for all the subjects at rest. The functional images were acquired with an echo-planar imaging (EPI) MRI sequence (TR/TE = 2000/30 ms; matrix = 64×64; FOV = 24×24 cm²; flip angle =80°; 23 slices; and thickness = 5mm). The DTI scans were acquired in the axial plane with EPI sequence. Thirty-one non-collinear diffusion sensitization gradients (b = 1000 s/mm²) and another non-weighted diffusion image (b₀ = 0 s/mm²) were performed once with the following parameters: TR/TE = 2000/30 ms; FOV = 24 × 24 cm²; acquisition matrix = 128×128, 38 slices, thickness = 3 mm and flip angle = 90°. To improve anatomical structures, high resolution T1-weighted MR images were scanned using an axial fast spoiled gradientrecalled-echo (3D-FSGPR) sequence with following parameters: TR/TE = 5100/1.2 ms; matrix =256×256; FOV = 24×24 cm²; thickness = 1.2 mm; space = 0 mm; and 124 slices. During scanning, all the subjects were at rest with their eyes closed to remain immobile and physiologically stable. Foam pads and earplugs were used to reduce head motion artifacts.

Data preprocessing

Resting-state fMRI (rs-fMRI) data

The rs-fMRI data were analyzed using the Statistical Parametric Mapping toolbox (SPM8, Wellcome Trust Centre for Neuroimaging, London, UK) and REST toolbox (version 1.8) [27] in MATLAB R2009b (The MathWorks Inc, Natick, MA, USA). The first ten points of the data were discarded for magnetization equilibrium within the initial MR signals. The preprocessing workflow was performed including slice timing correction, realignment, normalization to the MNI template (McGill University, Montreal QC, Canada), spatial smoothing with an 8 mm Gaussian full width at half maximum (FWHM) to reduce space noise, removal of linearly signal-drifting trend, and a temporal band-pass filter (0.01–0.08 Hz) for all voxel series that eliminated physiological signal noise. The time series of the white matter (WM), cerebrospinal fluid (CSF), and 6 rigid body motion parameters were regressed out as nuisance variables. Two radiologists thoroughly visually checked all the data in case severe artifacts and distortion existed. Some subtle motor distortions can be adjusted, whereas the subjects with severe head motion in any direction greater than 2.5 mm or a rotational degree more than 2.5 degree during scanning were excluded from this study. For rs-fMRI data, we measured the indices of regional homogeneity (ReHo) [28] and global amplitude of low-frequency fluctuation (ALFF) [29] for each subject. In recent years, ALFF has been widely used to delineate the extent of spontaneous neuronal activities in disease states [29 –32]. ReHo is another approach for reflecting regional spontaneous activities [28]. Regional changes of ALFF and ReHo in PD have been extensively found in subcortical and cortical regions [33 –35]. For details of these measurements, please refer to our previous work [23].

T1-weighted data

The T1-weighted images were processed using SPM8 in the MATLAB R2009b. The voxel based morphology (VBM) analysis is a commonly used method to study brain structural changes [12 , 37], which was performed with a refined registration method of DARTEL-VBM [38] to examine volumetric differences in both whole-brain and regional grey matter (GM) between the three groups. In patients with PD, GM loss was observed in areas involving the basal ganglia nuclei [12 , 39–41], calcarine cortex [42, 43] and visual cortex [44, 45] and in particular, in the PIGD subtype compared to the TD subtype [12]. All the T1-weighted images were first manually reoriented along the anterior-posterior commissure before starting analysis and were screened by eyes for any apparent artifact. Then the reoriented images were segmented into GM, WM and CSF probability maps in the individual native space. The DARTEL method was performed by modeling the shape of individual brain to generate a series of increasingly accurate group-level GM, WM and CSF templates [38]. Lastly, with the final study-specific templates, all the GM, WM and CSF images in the native space were normalized into the MNI standard space and then preserved by the “modulation” option respectively, where the modulated images allow us to achieve absolute volumes of grey matter tissue while unmodulated images represent the relative concentration of these structures [46]. The normalized GM images were smoothed (FWHM = 8 mm) and the voxel size was resliced to 1.5 mm × 1.5 mm × 1.5 mm. To screen for signals outside the brain, a whole brain mask was used.

Diffusion tensor imaging (DTI) data

For all the subjects, the DTI data were analyzed using the FSL package and the BET toolbox (version 4.1.0, FMRIB Analysis Group, Oxford, UK). The head motions and eddy currents were first corrected with an affine registration approach. To eliminate effects outside the brain, intracranial signals were extracted to generate a binary brain mask. Using the tensor construction at a voxel level, the indices of fractional anisotropy (FA), mean diffusivity (MD), radial diffusivity (RD), and axial diffusivity (AD) were calculated in the native space for each subject. Clear evidence has shown that WM loss in PD is a frequent occurrence (reviewed in [47]). In particular, WM patterns differ in distinct PD subtypes [15 , 49]. The WM group-wise statistics were performed in a voxel-by-voxel fashion in SPM. Maps of FA, MD, RD and AD were then normalized to the MNI space within a white matter mask. The normalized images were resampled into a voxel size of 2 × 2 × 2 mm³ followed by smooth with an 8 mm FWHM. No subjects were excluded in this study.

Feature extraction and selection

After data prepressing, for each subject, we have obtained GM, WM and CSF maps from T1-weighted data, ALFF, ReHo maps from rs-fMRI data and FA, MD, RD, AD maps from DTI data. To get rid of the features that were excessively redundant for the patients with PD, those multi-modal measurements of both the NC and the merged PD patients (merged by groups of the PIGD and the non-PIGD) were first compared, respectively. Then, the features that are not conducive to the classification of PD subtypes were removed, reducing computational load for the following feature selection. Unpaired two-sample t tests at a voxel-wise group level were performed with a statistical threshold set at P < 0.001 and cluster size >10 voxels with no correction limited. The regionsshowing significant functional and/or structural alterations in the merged PD group were set as regions of interest (ROI) for the subsequent subtype-specific feature extraction. Multi-modal features were extracted for each ROI using the Marsbar toolbox (version 0.43, http://marsbar.sourceforge.net/).

The SVM is a supervised, multivariate classification method consisting of training and testing phases [21, 50], where the training phase is to develop an algorithm for discriminating the two groups while the testing phase is to evaluate the classification performance on a blind prediction of unseen data [19, 24]. The classifier algorithm was developed using MATLAB platform (version 8.3.0.532) and the LIBSVM toolbox (http://www.csie.ntu.edu.tw/cjlin/libsvm/). We applied an SVM-Recursive Feature Elimination (SVM-RFE) algorithm embedded in 100 times of a 5-fold cross validation to construct SVM classification models. The nonlinear radial base function (RBF) kernel function was used in transforming features derived from training dataset to a higher space [51]. To identify the effective features with the highest discriminating power, we employed the SVM-RFE algorithm that is an feature selection strategy proposed by Guyon et al. [52, 53]. The SVM-RFE algorithm uses the weighted coefficients in SVM classification models to rank features and iteratively eliminates non-informative features with the smallest ranking criteria [52], which has been used in many clinical applications [19 , 54–59]. The SVM-RFE algorithm was implemented in this study to select features with the highest discriminating power as inputs to the SVM algorithm. In each iteration, a new SVM classification model was trained and a new weight coefficient map was generated. The ranking criterion we used was the value of each weight coefficient vector ∥w_i∥. Features with low weight coefficient vector ∥w_i∥ were iteratively removed from the feature dataset, whilst features with high weight coefficient vector ∥w_i∥ were retained [52 , 60]. The output of SVM-RFE algorithm is a ranked feature vector from the highest weight coefficient to the smallest weight coefficient. We repeated a 5-fold cross validation for 100 times to determine the feature size for each patient. The final feature size was determined when the classification model reached the highest averaged classification accuracy on the testing dataset after 100 times of 5-fold cross validation. The overall procedure of feature extraction and selection was displayed in Fig. 1. Our implementation of the SVM-RFE was delineated in a pseudo-code provided in the Supplementary materials (S-Figure 1).

Classifier performance assessment

After feature selection, the dimensions of feature vectors for each PD patient were reduced from 319 to 20. The preserved features for each patient were aligned into a single vector, which were then used to construct the SVM classifier for the PIGD and non-PIGD groups of PD. The main procedure included two steps: grid search for parameter C and G, and the leave-one-out cross-validation (LOOCV). The grid search method was employed to select the optimal parameter C and the kernel function parameter G. We used the LOOCV method to estimate the performance of the classifier. Given a training set of data samples, the classifier removes one data sample in each trial and trains the classifier on the rest of the data samples, with the removed sample used for model testing [61] (Fig. 2). The output of the proposed SVM classifier was a set of predicted values, according to which we identified the type of the subtypes to which subtype the test samples belong. Using these output values of the classifier, we drew a scatter plot for the two PD subtypes.

The performance of the SVM model was assessed using measures of accuracy, sensitivity, specificity and maximum area under the curve in the ROC analysis, which were defined as follows: $Accuracy = \frac{TP + TN}{TP + FN + TN + FP}$ (1) $Sensitivity = \frac{TP}{TP + FN}$ (2) $Specificity = \frac{TN}{TN + FP}$ (3) where true positive (TP) represented the number of patients with the PIGD group predicted correctly by the SVM classifier, false negative (FN) was the number of patients with the PIGD group predicated wrongly as those with the non-PIGD group, true negative (TN) was the number of non-PIGDs predicted correctly by the SVM classifier, and false positive (FP) was the number of patients with the non-PIGD group predicted incorrectly as those with the PIGD group. Then, we calculated the area under the ROC curve, which quantified the overall ability of the classifier to discriminate patients from the two subtypes. To assess the agreement of the predicted classification outcomes and the acknowledged clinical categorization results, we further performed the Kappa test on the both labels to compute individual-by-individual diagnostic agreements with the Kappa value. The Kappa value is a robust statistic commonly used to assess classifier performance in machine learning, which is able to detect agreement exceeds chance level [62].

RESULTS

Demographic characteristics and clinical assessments were displayed in Table 1. All the PD patients and NCs age- and sex-mached. For the two PD subtypes, there were no statistic differences observed in age, sex composition, disease duration, the H&Y stage, MMSE scale and UPDRS. The comparisons of multi-modal brain patterns between the NC group and the merged PD group showed significant alterations in regions covering frontal, parietal, occipital, temporal cortices and cerebellum in patients with PD. More details please refer to Supplementary material (Fig. 3 and S-Table 1).

In the performance analysis of classification, the proposed classifier discriminated patients with the PIGD subtype with a diagnostic accuracy, sensitivity and specificity of 92.31%, 84.21% and 96.97%, respectively. The positive predictive value (PPV) was up to 94.12% and the negative predictive value (NPV) was 91.43%. The ROC analysis showed the maximum area under the ROC curve (AUC_max) reached 0.9585 (Fig. 4). The individual-by-individual agreement test showed the Kappa value was 0.83, suggesting an almost perfect diagnostic agreement of the proposed classifier with the existing clinical categorization. The scatterplot of the predicted values of the classifier exhibited the power of effectively discriminating the PIGD and non-PIGD subtypes (Fig. 5).

DISCUSSION

The present study for the first time introduced a SVM-based classifier for the differential diagnosis of patients with the PIGD subtype of PD using multi-modal MRI data. In general, our proposed classifier exhibited satisfactory classification power, with the diagnostic accuracy, sensitivity, specificity, and the maximum AUC up to 92.31%, 84.21%, 96.97%, and 0.9585, respectively. Moreover, the Kappa value of 0.83 suggested an excellent diagnostic agreement between the proposed classifier performance and the current clinical categorization.

Patients with the PIGD and non-PIGD subtypes of PD have been linked to different associated clinical manifestations [3, 63], pathologic patterns [14, 64], genetics [65], and degrees of risk in developing complications [66 –68]. These findings have been revealed and confirmed by means of neuroimaging techniques, where functional and structural differences between the two subtypes have been often observed by studies across various modalities, including positron emission tomography (PET) [69], single photon emission computed tomography (SPECT) [17, 70] and fMRI [12 , 72]. However, conclusive and definitive brain biomarkers specific for the PIGD subtype are still lacking. Thus, based on prior studies [57 , 74] investigating single modality data, we integrated functional and structural MR data (including T1-weighted and DTI data) in one single study to analyze patients with specific PD subtypes. Multi-modal features of the ALFF, ReHo, brain volumes, and DTI properties of WM have been extensively used to detail alterations in brain functions and structures [12 , 76], giving substantial support for better understanding clinical disorders.

Multi-modal MR data give massive features for constructing the classifier though, where features excessively more than examples can lead to overfitting [19]. One way to overcome the adverse effect of overfitting is to select only effective features, which is essential for developing a classifier with a high predictive accuracy [77]. The SVM-RFE algorithm uses weighted coefficients as the selection criterion, which facilitates the full use of the training dataset and avoids overfitting [52, 77]. Recently, advances in the SVM-based classifiers have allowed us to classify patients with PD and in particular, with the phenotypically specific PD patients. The SVM-based imaging recognition classifiers can help distinguish PD patients from healthy controls with SPECT images [78] and the combination of T1-weighted and functional MR images [73], separate PD patients and suspected PD patients with susceptibility-weighted images (SWI) [79] and DTI [80] scans and, of note, diagnose patients with predominant tremor from those having essential tremor with rest tremor with structural MR images [25]. In these previous attempts, the SVM-based classifications have exhibited excellent performance.

Previous successful applications have given us substantial support for the use of SVM-based classifier in differentiating patients with the PIGD subtype of PD. To our best knowledge, there have been no studies reporting the exact diagnostic rates for phenotypic patients with PD. In the performance analysis, our classifier showed a high diagnostic accuracy, sensitivity, specificity, and maximum AUC value, exhibiting a promising diagnostic power in discrimination of the patients with the PIGD subtype of PD. However, it was shown that some PD patietns were wrongly grouped by our classifier, which could be due to the following two reasons. First, the diagnostic rates of PD itself need to be improved. The misdiagnosis and missed diagnosis probably have occurred at the first unpaired two-sample t test procedure. Patients with predominant parkinsonian symptoms but not PD were wrongly recruited in the present study. In a prospective clinicopathological study, an autopsy confirmed diagnosis of PD was only 76% [81]. Even though using the UK Parkinson’s disease brain bank criteria for clinical diagnosis, there are about 10% of patients that have been diagnosed as PD in life still subject to an alternative diagnosis during post-mortem inspections [82]. Besides, these disabling clinical phenomena, such as freezing of gait, balance disturbance, postural instability can separately exist in PD [83 –85], which are also often observed in many other parkinsonisms [86]. Thus, current diagnostic criteria of PD in clinical practice are not an exact science, which may constrain the effectivity and validity of the pattern recognition in the consequent procedures for specific PD patients. Second, the diagnosis of the PIGD subtype of PD is dynamic, with subtype transition from the non-PIGD to PIGD subtype in patients with PD have been observed [66]. Thus, overlapping brain patterns existing between patients of the two subtypes may by nature interfere MRI pattern discrimination in the machine learning procedure. It should be stressed again that the main purpose of our present work is to develop a fast, automatic classification comparable to the existing clinical categorization, and thus to substitute for the complex and subjective large-scale computing. In light of the aforementioned, the Kappa test exactly fitted our objective to estimate the diagnostic agreement between the proposed classifier performance and the existing clinical categorization. Since the Kappa value ranges from –1 to 1, where the value calculated for classifiers approaching “1” represents that the classifier performance is more realistic rather than by random chance and the statistic of 0.81–1.00 indicates an almost perfect agreement [87, 88]. The Kappa value of 0.83 in the diagnostic agreement for the predicted classification results again demonstrated that this classifier was comparable to the existing clinical categorization.

In conclusion, we for the first time introduced the machine learning-based classification to distinguish patients with the PIGD subtype of PD from those with the non-PIGD subtype at the individual level with multi-modal MRI scans. This classifier showed a promising diagnostic accuracy of 92.31% (AUC_max = 0.9585) and an almost perfect agreement for differential diagnosis with clinical categorization (Kappa statistic = 0.83). With these satisfactory results, we successfully proposed an automatic, accurate and specified classification for distinguishing patients with the PIGD subtype of PD; moreover, we demonstrated the availability and validity of this SVM-based classification in the differential diagnosis of specific PD populations.

CONFLICTS OF INTEREST

None.

Footnotes

ACKNOWLEDGMENTS

This study was supported by the National Science & Technology Supporting Program, China (Grant No. 2012BAI10B04) and the Natural Science Foundation of Zhejiang Province, China (Grant No. LY12H09006). We would like to express our gratitude to all the participants in this study. Furthermore, we also thank Dr. Dan Long for his initial attempts in this project and Dr. Hsu-lei Lee for her critical and careful amendments.

Appendix

References

Lees

, Hardy

, & Revesz

(2009) Parkinson’s disease. Lancet, 373, 2055–2066.

Zetusky

, Jankovic

, & Pirozzolo

(1985) The heterogeneity of Parkinson’s disease: Clinical and prognostic implications. Neurology, 35, 522–526.

Jankovic

, McDermott

, Carter

, Gauthier

, Goetz

, Golbe

, Huber

, Koller

, Olanow

, Shoulson

, et al. (1990) Variable expression of Parkinson’s disease: A base-line analysis of the DATATOP cohort. The Parkinson Study Group. Neurology, 40, 1529–1534.

Auyeung

, Tsoi

, Mok

, Cheung

, Lee

, Li

, & Yeung

(2012) Ten year survival and outcomes in a prospective cohort of new onset Chinese Parkinson’s disease patients. J Neurol Neurosurg Psychiatry, 83, 607–611.

Hughes

, Daniel

, Ben-Shlomo

, & Lees

(2002) The accuracy of diagnosis of parkinsonian syndromes in a specialist movement disorder service. Brain, 125, 861–870.

St George

, Nutt

, Burchiel

, & Horak

(2010) A meta-regression of the long-term effects of deep brain stimulation on balance and gait in PD. Neurology, 75, 1292–1299.

Burn

, Rowan

, Allan

, Molloy

, O’Brien

, & McKeith

(2006) Motor subtype and cognitive decline in Parkinson’s disease, Parkinson’s disease with dementia, and dementia with Lewy bodies. J Neurol Neurosurg Psychiatry, 77, 585–589.

Zaidel

, Arkadir

, Israel

, & Bergman

(2009) Akineto-rigid vs. tremor syndromes in Parkinsonism. Curr Opin Neurol, 22, 387–393.

Melzer

, Watts

, MacAskill

, Pitcher

, Livingston

, Keenan

, Dalrymple-Alford

, & Anderson

(2012) Grey matter atrophy in cognitively impaired Parkinson’s disease. J Neurol Neurosurg Psychiatry, 83, 188–194.

10.

Ferris

, Marella

, Smerkers

, Barchet

, Gershman

, Matsuno-Yagi

, & Yagi

(2013) A phenotypic model recapitulating the neuropathology of Parkinson’s disease. Brain Behav, 3, 351–366.

11.

Bunzeck

, Singh-Curry

, Eckart

, Weiskopf

, Perry

, Bain

, Duzel

, & Husain

(2013) Motor phenotype and magnetic resonance measures of basal ganglia iron levels in Parkinson’s disease. Parkinsonism Relat Disord, 19, 1136–1142.

12.

Rosenberg-Katz

, Herman

, Jacob

, Giladi

, Hendler

, & Hausdorff

(2013) Gray matter atrophy distinguishes between Parkinson disease motor subtypes. Neurology, 80, 1476–1484.

13.

Thenganatt

, & Jankovic

(2014) Parkinson disease subtypes. JAMA Neurol, 71, 499–504.

14.

Selikhova

, Williams

, Kempster

, Holton

, Revesz

, & Lees

(2009) A clinico-pathological study of subtypes in Parkinson’s disease. Brain, 132, 2947–2957.

15.

Bohnen

, Muller

, Zarzhevsky

, Koeppe

, Bogan

, Kilbourn

, Frey

, & Albin

(2011) Leucoaraiosis, nigrostriatal denervation and motor symptoms in Parkinson’s disease. Brain, 134, 2358–2365.

16.

Chan

, Ng

, Rumpel

, Fook-Chong

, Li

, & Tan

(2014) Transcallosal diffusion tensor abnormalities in predominant gait disorder parkinsonism. Parkinsonism Relat Disord, 20, 53–59.

17.

Mito

, Yoshida

, Yabe

, Makino

, Tashiro

, Kikuchi

, & Sasaki

(2006) Brain SPECT analysis by 3D-SSP and phenotype of Parkinson’s disease. J Neurol Sci, 241, 67–72.

18.

Tessitore

, Amboni

, Esposito

, Russo

, Picillo

, Marcuccio

, Pellecchia

, Vitale

, Cirillo

, Tedeschi

, & Barone

(2012) Resting-state brain connectivity in patients with Parkinson’s disease and freezing of gait. Parkinsonism Relat Disord, 18, 781–787.

19.

Pereira

, Mitchell

, & Botvinick

(2009) Machine learning classifiers and fMRI: A tutorial overview. Neuroimage, 45, S199–209.

20.

Boser

, Guyon

, & Vapnik

(1992) A training algorithm for optimal margin classifiers. COLT ’92 Proceedings of the fifth annual workshop on Computational learning theory, pp. 144–152.

21.

Vapnik

, & Vapnik

(1998) Wiley. Statistical learning theory–New York.

22.

Kloppel

, Stonnington

, Chu

, Draganski

, Scahill

, Rohrer

, Fox

, Jack

Jr , Ashburner

, & Frackowiak

(2008) Automatic classification of MR scans in Alzheimer’s disease. Brain, 131, 681–689.

23.

Long

, Wang

, Xuan

, Gu

, Xu

, Kong

, & Zhang

(2012) Automatic classification of early Parkinson’s disease with multi-modal MR imaging. PLoS One, 7, e47714.

24.

Orru

, Pettersson-Yeo

, Marquand

, Sartori

, & Mechelli

(2012) Using Support Vector Machine to identify imaging biomarkers of neurological and psychiatric disease: A critical review. Neurosci Biobehav Rev, 36, 1140–1152.

25.

Cherubini

, Nistico

, Novellino

, Salsone

, Nigro

, Donzuso

, & Quattrone

(2014) Magnetic resonance support vector machine discriminates essential tremor with rest tremor from tremor-dominant Parkinson disease. Mov Disord, 29, 1216–1219.

26.

Mwangi

, Ebmeier

, Matthews

, & Steele

(2012) Multi-centre diagnostic classification of individual structural neuroimaging scans from patients with major depressive disorder. Brain, 135, 1508–1521.

27.

Song

, Dong

, Long

, Li

, Zuo

, Zhu

, He

, Yan

, & Zang

(2011) REST: A toolkit for resting-state functional magnetic resonance imaging data processing. PLoS One, 6, e25031.

28.

Zang

, Jiang

, Lu

, He

, & Tian

(2004) Regional homogeneity approach to fMRI data analysis. Neuroimage, 22, 394–400.

29.

Zang

, He

, Zhu

, Cao

, Sui

, Liang

, Tian

, Jiang

, & Wang

(2007) Altered baseline brain activity in children with ADHD revealed by resting-state functional MRI. Brain Dev, 29, 83–91 .

30.

Chou

, Chi

, & Chow

(2004) Sources of income and depression in elderly Hong Kong Chinese: Mediating and moderating effects of social support and financial strain. Aging Ment Health, 8, 212–221.

31.

Han

, Wang

, Zhao

, Min

, Lu

, Li

, He

, & Jia

(2011) Frequency-dependent changes in the amplitude of low-frequency fluctuations in amnestic mild cognitive impairment: A resting-state fMRI study. Neuroimage, 55, 287–295.

32.

Mennes

, Zuo

, Kelly

, Di Martino

, Zang

, Biswal

, Castellanos

, & Milham

(2011) Linking inter-individual differences in neural activation and behavior to intrinsic brain dynamics. Neuroimage, 54, 2950–2959.

33.

Zhang

, Wei

, Hu

, Zhang

, Zhou

, Li

, Wang

, Feng

, Yin

, Xie

, & Wang

(2013) Specific frequency band of amplitude low-frequency fl uctuation predicts Parkinson’s disease. Behav Brain Res, 252, 18–23.

34.

Kwak

, Peltier

, Bohnen

, Muller

, Dayalu

, & Seidler

(2012) L-DOPA changes spontaneous low-frequency BOLD signal oscillations in Parkinson’s disease: A resting state fMRI study. Front Syst Neurosci, 6, 52.

35.

, Long

, Zang

, Wang

, Hallett

, Li

, & Chan

(2009) Regional homogeneity changes in patients with Parkinson’s disease. Hum Brain Mapp, 30, 1502–1510.

36.

Biswal

, Lal

, Rath

, & Mohanti

(1995) Hemostatic radiotherapy in carcinoma of the uterine cervix. Int J Gynaecol Obstet, 50, 281–285.

37.

Price

, Paviour

, Scahill

, Stevens

, Rossor

, Lees

, & Fox

(2004) Voxel-based morphometry detects patterns of atrophy that help differentiate progressive supranuclear palsy and Parkinson’s disease. Neuroimage, 23, 663–669.

38.

Ashburner

(2007) A fast diffeomorphic image registration algorithm. Neuroimage, 38, 95–113.

39.

O’Neill

, Schuff

, Marks

Jr , Feiwell

, Aminoff

, & Weiner

(2002) Quantitative 1H magnetic resonance spectroscopy and MRI of Parkinson’s disease. Mov Disord, 17, 917–927.

40.

Krabbe

, Karlsborg

, Hansen

, Werdelin

, Mehlsen

, Larsson

, & Paulson

(2005) Increased intracranial volume in Parkinson’s disease. J Neurol Sci, 239, 45–52.

41.

Pitcher

, Melzer

, Macaskill

, Graham

, Livingston

, Keenan

, Watts

, Dalrymple-Alford

, & Anderson

(2012) Reduced striatal volumes inParkinson’s disease: A magnetic resonance imaging study. Transl Neurodegener, 1, 17.

42.

Ibarretxe-Bilbao

, Junque

, Marti

, & Tolosa

(2011) Cerebral basis of visual hallucinations in Parkinson’s disease: Structural and functional MRI studies. J Neurol Sci, 310, 79–81.

43.

Velu

, Mullen

, Noh

, Valdivia

, Poizner

, Baram

, & de Sa

(2014) Effect of visual feedback on the occipital-parietal-motor network in Parkinson’s disease with freezing of gait. Front Neurol, 4, 209.

44.

Nishio

, Hirayama

, Takeda

, Hosokai

, Ishioka

, Suzuki

, Itoyama

, Takahashi

, & Mori

(2010) Corticolimbic gray matter loss in Parkinson’s disease without dementia. Eur J Neurol, 17, 1090–1097.

45.

Meppelink

, de Jong

, Teune

, & van Laar

(2011) Regional cortical grey matter loss in Parkinson’s disease without dementia is independent from visual hallucinations. Mov Disord, 26, 142–147.

46.

Mechelli

APCJ

, Friston

, & Ashburner

(2005) Voxel-based morphometry of the human brain: Methods and alications. Curr Med Imaging Rev, 1, 105–113.

47.

Bohnen

, & Albin

(2011) White matter lesions in Parkinson disease. Nat Rev Neurol, 7, 229–236.

48.

Lee

, Kim

, Lee

, An

, Kim

, & Jung

(2009) The severity of leukoaraiosis correlates with the clinical phenotype of Parkinson’s disease. Arch Gerontol Geriatr, 49, 255–259.

49.

, Huang

, Xuan

, Xu

, Li

, Sun

, Yu

, Wang

, Luo

, & Zhang

(2014) Greater loss of white matter integrity in postural instability and gait difficulty subtype of Parkinson’s disease. Can J Neurol Sci, 41, 763–768.

50.

Vapnik

(2013) The nature of statistical learning theorySpringer Science & Business Media.

51.

Chang

Y-W

, Hsieh

C-J

, Chang

K-W

, Ringgaard

, & Lin

C-J

(2010) Training and testing low-degree polynomial data mappings via linear SVM. J Mach Learn Res, 11, 1471–1490.

52.

Guyon

, Weston

, Barnhill

, & Vapnik

(2002) Gene selection for cancer classification using support vector machines. Mach Learn, 46, 389–422.

53.

Rakotomamonjy

(2003) Variable selection using svm based criteria. J Mach Learn Res, 3, 1357–1370.

54.

Burges

(1998) A tutorial on support vector machines for pattern recognition. Data Min Knowl Discov, 2, 121–167.

55.

Zhou

, & Tuck

(2007) MSVM-RFE: Extensions of SVM-RFE for multiclass gene selection on DNA microarray data. Bioinformatics, 23, 1106–1114.

56.

Yoon

, & Kim

(2009) Mutual information-based SVM-RFE for diagnostic classification of digitized mammograms. Pattern Recognit Lett, 30, 1489–1495.

57.

Wee

, Yap

, Shen

, & Alzheimer’s Disease Neuroimaging Initiative (2013) Prediction of Alzheimer’s disease and mild cognitive impairment using cortical morphological patterns. Hum Brain Mapp, 34, 3411–3425.

58.

Wee

, Yap

, Li

, Denny

, Browndyke

, Potter

, Welsh-Bohmer

, Wang

, & Shen

(2011) Enriched white matter connectivity networks for accurate identification of MCI patients. Neuroimage, 54, 1812–1822.

59.

Calderoni

, Retico

, Biagi

, Tancredi

, Muratori

, & Tosetti

(2012) Female children with autism spectrum disorder: An insight from mass-univariate and pattern classification analyses. Neuroimage, 59, 1013–1022 .

60.

De Martino

, Valente

, Staeren

, Ashburner

, Goebel

, & Formisano

(2008) Combining multivariate voxel selection and support vector machines for mapping and classification of fMRI spatial patterns. Neuroimage, 43, 44–58.

61.

Larrañaga

, Calvo

, Santana

, Bielza

, Galdiano

, Inza

, Lozano

, Armañanzas

, Santafé

, & Pérez

(2006) Machine learning in bioinformatics. Brief Bioinform, 7, 86–112.

62.

Amthauer

(2008) Applying Machine Learning Methods to Suggest Network Involvement and Functionality of Genes in Saccharomyces cerevisiaeProQuest.

63.

Wickremaratchi

, Knipe

, Sastry

, Morgan

, Jones

, Salmon

, Weiser

, Moran

, Davies

, Ebenezer

, Raha

, Robertson

, Butler

, Ben-Shlomo

, & Morris

(2011) The motor phenotype of Parkinson’s disease in relation to age at onset. Mov Disord, 26, 457–463.

64.

Paulus

, & Jellinger

(1991) The neuropathologic basis of different clinical subgroups of Parkinson’s disease. J Neuropathol Exp Neurol, 50, 743–755.

65.

Alcalay

, Mejia-Santana

, Tang

, Rosado

, Verbitsky

, Kisselev

, Ross

, Louis

, Comella

, Colcher

, Jennings

, Nance

, Bressman

, Scott

, Tanner

, Mickel

, Andrews

, Waters

, Fahn

, Cote

, Frucht

, Ford

, Rezak

, Novak

, Friedman

, Pfeiffer

, Marsh

, Hiner

, Siderowf

, Caccappolo

, Ottman

, Clark

, & Marder

(2009) Motor phenotype of LRRK2 G2019S carriers in early-onset Parkinson disease. Arch Neurol, 66, 1517–1522.

66.

Alves

, Larsen

, Emre

, Wentzel-Larsen

, & Aarsland

(2006) Changes in motor subtype and risk for incident dementia in Parkinson’s disease. Mov Disord, 21, 1123–1130.

67.

Negre-Pages

, Grandjean

, Lapeyre-Mestre

, Montastruc

, Fourrier

, Lepine

, Rascol

, & DoPaMi

PSG

(2010) Anxious and depressive symptoms in Parkinson’s disease: The French cross-sectionnal DoPaMiP study. Mov Disord, 25, 157–166.

68.

Burn

, Landau

, Hindle

, Samuel

, Wilson

, Hurt

, Brown

, & Group P-PS (2012) Parkinson’s disease motor subtypes and mood. Mov Disord, 27, 379–386.

69.

Muller

, Frey

, Petrou

, Kotagal

, Koeppe

, Albin

, & Bohnen

(2013) beta-Amyloid and postural instability and gait difficulty in Parkinson’s disease at risk for dementia. Mov Disord, 28, 296–301.

70.

Eggers

, Kahraman

, Fink

, Schmidt

, & Timmermann

(2011) Akinetic-rigid and tremor-dominant Parkinson’s disease patients show different patterns of FP-CIT single photon emission computed tomography. Mov Disord, 26, 416–423.

71.

Herman

, Rosenberg-Katz

, Jacob

, Auriel

, Gurevich

, Giladi

, & Hausdorff

(2013) White matter hyperintensities in Parkinson’s disease: Do they explain the disparity between the postural instability gait difficulty and tremor dominant subtypes?. PLoS One, 8, e55193.

72.

Zhang

, Wei

, Hu

, Xie

, Zhang

, Wu

, & Wang

(2015) Akinetic-rigid and tremor-dominant Parkinson’s disease patients show different patterns of intrinsic brain activity. Parkinsonism Relat Disord, 21, 23–30.

73.

Long

, Wang

, Xuan

, Gu

, Xu

, Kong

, & Zhang

(2012) Automatic classification of early Parkinson’s disease with multi-modal MR imaging. PloS One, 7, e47714–47719.

74.

Wen

, Wu

, Liu

, Li

, & Yao

(2013) Abnormal baseline brain activity in non-depressed Parkinson’s disease and depressed Parkinson’s disease: A resting-state functional magnetic resonance imaging study. Plos One, 8, e63691.

75.

Qiu

, Han

, Lv

, Jiang

, Tian

, Zhuo

, Su

, Lin

, & Zhang

(2011) Regional homogeneity changes in heroin-dependent individuals: Resting-state functional MR imaging study. Radiology, 261, 551–559.

76.

Salsone

, Cerasa

, Arabia

, Morelli

, Gambardella

, Mumoli

, Nistico

, Vescio

, & Quattrone

(2014) Reduced thalamic volume in Parkinson disease with REM sleep behavior disorder: Volumetric study. Parkinsonism Relat Disord, 20, 1004–1008.

77.

Guyon

, A (2003) An introduction to variable and feature selection. J Mach Learn Res, 3, 1157–1182.

78.

Prashanth

, Roy

, Mandal

, & Ghosh

(2014) Automatic classification and prediction models for early Parkinson’s disease diagnosis from SPECT imaging. Expert Syst Appl, 41, 3333–3342.

79.

Haller

, Badoud

, Nguyen

, Barnaure

, Montandon

, Lovblad

, & Burkhard

(2013) Differentiation between Parkinson disease and other forms of Parkinsonism using support vector machine analysis of susceptibility-weighted imaging (SWI): Initial results. Eur Radiol, 23, 12–19.

80.

Haller

, Badoud

, Nguyen

, Garibotto

, Lovblad

, & Burkhard

(2012) Individual detection of patients with Parkinson disease using support vector machine analysis of diffusion tensor imaging data: Initial results. Am J Neuroradiol, 33, 2123–2128.

81.

Rajput

, Rozdilsky

, & Rajput

(1991) Accuracy of clinical diagnosis in parkinsonism–a prospective study. Can J Neurol Sci, 18, 275–278.

82.

Hughes

, Daniel

, & Lees

(2001) Improved accuracy of clinical diagnosis of Lewy body Parkinson’s disease. Neurology, 57, 1497–1499.

83.

Nutt

, Bloem

, Giladi

, Hallett

, Horak

, & Nieuwboer

(2011) Freezing of gait: Moving forward on a mysterious clinical phenomenon. Lancet Neurol, 10, 734–744.

84.

Jonasson

, Ullen

, Iwarsson

, Lexell

, & Nilsson

(2015) Concerns about falling in Parkinson’s Disease: Associations with disabilities and personal and environmental factors. J Parkinsons Dis, 5, 341–349.

85.

Ozinga

, Machado

, Miller Koop

, Rosenfeldt

, & Alberts

(2015) Objective assessment of postural stability in Parkinson’s disease using mobile technology. Mov Disord, 30, 1214–1221.

86.

Bohlhalter

, & Kaegi

(2011) Parkinsonism: Heterogeneity of a common neurological syndrome. Swiss Med Wkly, 141, w13293.

87.

Brennan

, & Silman

(1992) Statistical methods for assessing observer variability in clinical measures. BMJ, 304, 1491–1494.

88.

McHugh

(2012) Interrater reliability: The kappa statistic. Biochem Med (Zagreb), 22, 276–282.