A Study of Assisted Screening for Alzheimer’s Disease Based on Handwriting and Gait Analysis

Abstract

Background:

Alzheimer’s disease (AD) is a progressive neurodegenerative disease that is not easily detected in the early stage. Handwriting and walking have been shown to be potential indicators of cognitive decline and are often affected by AD.

Objective:

This study proposes an assisted screening framework for AD based on multimodal analysis of handwriting and gait and explores whether using a combination of multiple modalities can improve the accuracy of single modality classification.

Methods:

We recruited 90 participants (38 AD patients and 52 healthy controls). The handwriting data was collected under four handwriting tasks using dot-matrix digital pens, and the gait data was collected using an electronic trail. The two kinds of features were fused as inputs for several different machine learning models (Logistic Regression, SVM, XGBoost, Adaboost, LightGBM), and the model performance was compared.

Results:

The accuracy of each model ranged from 71.95% to 96.17%. Among them, the model constructed by LightGBM had the best performance, with an accuracy of 96.17%, sensitivity of 95.32%, specificity of 96.78%, PPV of 95.94%, NPV of 96.74%, and AUC of 0.991. However, the highest accuracy of a single modality was 93.53%, which was achieved by XGBoost in gait features.

Conclusions:

The research results show that the combination of handwriting features and gait features can achieve better classification results than a single modality. In addition, the assisted screening model proposed in this study can achieve effective classification of AD, which has development and application prospects.

Keywords

Alzheimer’s disease feature fusion gait handwriting machine learning

INTRODUCTION

With the acceleration of population aging around the world, the incidence rate of neurodegenerative disease (ND) in the elderly is increasing year by year. 1 Among them, AD is the most common ND, and its incidence increases exponentially with age. 2 Currently, approximately 50 million people worldwide suffer from AD. It is expected that the number of patients will double by 2050. 3 Due to the aging of society, the increase in AD patients is quite significant and will become an expensive public health burden in the coming years. 4 The characteristic of AD is the gradual loss of memory and other cognitive functions, which lasts for a long time. 5 Its early stage is not easily detected, and when patients show obvious symptoms, the disease usually progresses to the middle and late stages. Mild cognitive impairment (MCI) is a transitional stage between normal elderly individuals and AD, with approximately 80% of MCI patients transitioning to AD. 6 Although AD patients cannot be cured, identifying the early stage can lead to timely intervention measures, greatly delaying the progression of the disease and controlling symptoms. The initial and most common symptom of AD is short-term memory loss, where patients may forget the latest events and repeat the same problem. Afterward, the problem-solving ability and execution ability are compromised. Patients may also experience language disorders, impairment of visual-spatial skills, 7 and motor disorders such as slowed gait and reduced flexibility.

Currently, the diagnosis of AD requires a comprehensive clinical assessment, involving various cognitive assessment scales, imaging tests, fluid examinations, genetic testing, and eye-tracking technology. 8 However, existing diagnostic methods have certain limitations. Firstly, the scoring system of the scales is based on qualitative descriptions of symptoms, corresponding to some respective scores assigned by physicians, introducing a degree of subjectivity. The most widely used cognitive assessment scales currently include the Mini-Mental State Examination (MMSE) 9 and the Montreal Cognitive Assessment (MoCA). 10 Issues such as multicultural communication and subjective judgment can lead to inaccurate screening when using these scales. Secondly, assessing patients in a stressful medical environment may not accurately reflect their real-life behavioral abilities. Thirdly, medical imaging technologies are not only expensive but also pose some risks to patients’ bodies, making them difficult to access in regions with limited medical resources. In particular, in positron emission tomography (PET) scans, only a few hospitals have the capability for Aβ-PET and Tau-PET examinations, making this technology quite limited. Therefore, there is an urgent need to identify methods for early screening of AD to develop more effective intervention or treatment plans, thereby slowing disease progression and reducing the burden on families and society.

With the development of computer technology and artificial intelligence, utilizing computer technology to screen AD has become a promising approach, assisting in identifying early-stage AD patients. 11 Modality refers to a form in which things happen or exist. In the field of artificial intelligence, multimodality refers to the fusion of data from multiple sources to achieve information integration and complementarity between different modalities. In the realm of computer-assisted AD screening, each biomarker represents a modality. A single modality can reflect different impairments in patients. By merging data from different modalities and inputting them into machine learning or deep learning models, a comprehensive assessment of patients can be provided, enhancing the accuracy of classification. In the screening process for AD, handwriting and gait are two commonly used biomarkers.

Handwriting is a complex process involving cognitive domains such as fine motor skills and kinesiology and is often impaired by AD. 12 It is considered a crucial source of information for assessing neurological disorders. There is evidence that an individual’s health can be distinguished based on their performance in several simple handwriting tasks. 13 Numerous studies have explored the connection between AD, MCI patients, and handwriting characteristics. The results of these studies have shown that it is possible to effectively screen normal, AD, and MCI patients. Common abnormalities observed in AD patients include slow writing speed, reduced writing pressure, increased writing thinking time, more pen strokes, and writing errors.12,14,15 , 12,14,15 Ghaderyan et al. 16 recruited 13 subjects with mild cognitive impairment, 15 with AD, and 15 healthy participants. Their handwriting features under the four writing tasks were input into the sparse non-negative least-square classifier, which achieved 95.3% accuracy in distinguishing AD patients from normal people. Nicole et al. 17 recruited 66 AD patients and 64 healthy participants to finish handwriting tasks. They used Random Forest and the Decision Tree to distinguish two groups of subjects. The highest accuracy achieved 72.30%. Giuseppe et al. 18 proposed a multi-classifier approach to distinguish AD patients from normal people. This multi-classifier framework consisted of 6 basic machine learning models, such as Random Forest, SVM, Logistic regression, and so on. The outputs of the classifiers were eventually combined by a majority vote. The experimental results proved that classification using a multi-classifier works better than a single classifier.

Gait refers to the posture and movements of the body when a person walks, involving coordinated movements of the legs and body. Normal gait is a complex process that requires the coordination of multiple body parts to maintain balance and propel the body forward. Gait disturbances affect many elderly individuals, and the relationship with cognitive function is closely intertwined. Both share the same physiopathological basis and have recently been emphasized as potential indicators of cognitive decline. 19 Gait changes not only serve as markers of ND but also as one of the assessment components for dementia. 20 Gait disturbances associated with AD are mainly increased variability due to executive function, attention impairments, and memory problems. Studies by Grande 21 and Castrillo et al. 22 suggest that AD patients may experience gait disturbances in the early stage. Therefore, assessing relevant gait indicators can aid in detecting the early stage of AD and cognitive impairment. The most common gait abnormalities in AD patients include slowed walking speed, shortened stride length, and increased gait variability.23 –25 In recent years, many have begun to investigate the use of machine learning techniques to screen for AD through gait. Ghoraani et al. 26 proposed a diagnostic algorithm to detect MCI and AD health based only on gait and machine learning. They collected gait data from the subjects under single and dual-tasking and entered them into support vector machine (SVM), which achieved the highest accuracy of 86%. Chen et al. 27 proposed a method of using machine learning models to predict different types of MCI patients. The recruited subjects performed walking, timing, and jumping tests. The collected gait parameters were used in model based on SVM and principal component analysis (PCA), which achieved the highest accuracy of 91.67%. Mahmoud et al. 28 proposed the first application of Kinect V.2 camera and SVM to detect AD patients from normal people through the Timed Up and Go test. The model classified the two groups with an accuracy of 97.75%.

Thus, many studies have been conducted to prove that two types of behavioral data, handwriting, and gait, can achieve better classification performance with machine learning models. They can reflect different and complementary features of cognitive impairments associated with AD. 29 Combining features of these two modalities for analysis can provide higher accuracy in screening AD patients.

This paper proposes an assisted screening framework for AD based on multimodal analysis of handwriting and gait. The framework employs a feature fusion method and achieves effective classification of AD with an accuracy of 96.17%. The contributions of this paper are as follows:

An assisted screening framework for AD based on multimodal analysis of handwriting and gait is proposed to achieve rapid, noninvasive screening for the early stage of AD.

Study showed that the combination of these two kinds of behavioral data was better than the two alone for screening AD.

This study can be applied to healthcare resource-poor areas or grassroots elderly institutions such as communities to achieve collective screening of AD and management in the elderly population.

MATERIALS AND METHODS

The framework of the early AD screening proposed in this paper is shown in Fig. 1, which consists of five stages: data acquisition, feature extraction, feature selection and fusion, classification and evaluation, and statistical analysis. The following subsections describe the details of these five stages.

Fig. 1

The framework of the early AD screening.

Data acquisition

Participants

This study recruited participants from Huzhou Rehabilitation Hospital. The recruited participants were right-handed. Their native language was Chinese, and they were excluded from psychiatric disorders, history of substance abuse, and disorders of the hands, waist, and legs that would affect writing and walking. All participants met possible AD criteria 30 and were in the early stage of AD. All participants were examined on the MMSE. 9 After screening, the study included a total of 90 participants, consisting of 38 probable AD patients and 52 healthy controls (HC). This study was conducted under the Medical Ethics Committee of Huzhou University (20230702) and followed the ethical code stated by the Helsinki Declaration. Before the experiment, all participants signed informed consent forms, voluntarily agreeing to participate in this study. Table 1 provides the basic demographic information for the AD and HC groups.

Table 1

Basic demographic information for the AD and HC groups

Characteristics	AD (N = 38)	HC (N = 52)	p
Gender (n, %)			0.572
Male	11 (28.9)	18 (34.6)
Female	27 (71.1)	34 (65.4)
Age (avg±std)	84.18±7.165	85.77±4.977	0.454
Years of education (avg±std)	8.84±3.413	10.10±2.592	0.051
MMSE (avg±std)	19.68±3.488	27.11±1.833	<0.001

AD, Alzheimer’s disease; HC, healthy control; MMSE, Mini-Mental State Examination.

Equipment

For the collection of handwriting data, in previous studies, most researchers used digital tablets or tablet computers when screening AD patients.31,32, 31,32 To fit the usual writing habits of the elderly and to control other variables, the Tstudy dot-matrix digital pens and dot-matrix paper were chosen as the writing tools in this study. The dot-matrix can provide the coordinate parameter information for the dot-matrix digital pen to record the trajectory, which can be printed out on ordinary paper. When the pen tip is pressed, the pressure sensor and camera inside the pen are triggered. The dot coordinates, writing sequence, pressure data, movement speed, and other information are transferred to the inbuilt processor and finally output via Bluetooth. In this study, the subjects wrote with a dot-matrix digital pen and A4 paper covered with dot-matrix, which fit the subjects’ daily writing habits.

Regarding the formulation of handwriting tasks, Kjerstin et al. 33 proposed as early as 1996 that when screening cognitive impairment in the elderly, geometric replication and writing ability are commonly assessed. Among these tasks, those most sensitive to subtle cognitive changes include copying cubes and pentagons, which can be complemented by tasks such as copying. Based on this, considering the characteristics of writing tasks from previous research14,34,35 , 14,34,35 and demographic features of the Chinese elderly, this study ultimately defined four writing tasks, including two graphic tasks and two text tasks: 1) Copying two crossed pentagons; 2) Copying a cube; 3) Picture naming. Subjects were shown three pictures: a flower, a bed, and a lamp. Subjects were asked to describe them with a single word and write it down; 4) Copying a sentence. The sentence is “”, which means “Happy every second, joy every day.” The specific writing tasks are shown in Table 2. We merged four writing tasks into a complete handwriting dataset.

Table 2

Four writing tasks

For the collection of gait data, this study employed the GAITRite walkway. GAITRite is a portable, single-layer pressure-sensitive walkway, measuring 6 meters in length and 0.6 meters in width. It is embedded with pressure-sensitive sensors that can measure temporal and spatial parameters and easily identify gait abnormalities. Subjects wore shoes and walked 2-3 rounds on the walkway at their usual walking speed and manner. The relevant data were uploaded to the GAITRite system on the computer. To minimize the impact of subjects’ nervousness on the results during the experiment, the GAITRite system automatically merged data from several rounds for each subject before exporting the final results.

Experimental procedure

The process of collecting data in this study was divided into two main stages. Firstly, the collection of subjects’ handwriting data. Subjects were invited to enter a quiet room and sit at a table where a dot-matrix digital pen and A4 paper with tasks and dots printed on it were placed. Before the start of the trial, the researcher explained the writing process and requirements to the subjects. After ensuring that the subject fully understood the writing requirements, the subject began to write at a normal speed.

The second stage was the collection of subjects’ gait data. After a short break from writing, subjects entered another corridor which was paved with GAITRite walkway. To prevent the effects of acceleration and deceleration, markers were placed 1 meter away from the beginning and 1 meter away from the end of the walkway to serve as the start and end of the gait measurement. 24 The researcher guided the subject to stand at the beginning marker. Before the start of the trial, the researcher explained the walking procedure and requirements to the subject. After the researcher said “begin", the subject started walking from the beginning marker at the same speed and posture as their usual walking pace, proceeding along the walkway until they reached the endpoint marker. To minimize the impact of subjects’ nervousness on the results during the experiment, this study requires subjects to walk continuously for 2-3 rounds.

Feature extraction

Based on the characteristics of AD patients and previous research, this study included a total of 30 handwriting features and 32 gait features.

Due to the characteristics of velocity and pressure in writing of AD patients, the extracted handwriting features include 1 item in the pressure category (average pressure), 2 items in the time category (time in air, time on paper), and 7 items in the writing form category (number of strokes, task score, single stroke length, x-axis/y-axis stroke length, tilt, repeated dots). Additionally, there are 10 items in the velocity category (average velocity, maximum/minimum velocity, maximum/minimum velocity on x-axis/y-axis direction, average velocity on x-axis/y-axis direction, number of times velocity slows down), and 10 items in the acceleration category (average acceleration, maximum/minimum acceleration, maximum/minimum acceleration on x-axis/y-axis direction, average acceleration on x-axis/y-axis direction, number of times acceleration slows down).

Due to the occurrence of some qualitative writing errors in AD patients during writing, and according to previous studies,14,35, 14,35 the task score is scored according to the following rules. Task 1 has a maximum score of 3 points, with 1 point each for meeting the following criteria: drawing a pentagon with 5 clear angles and sides; intersecting two pentagons to form a quadrilateral; each side’s length is roughly equal. Task 2 has a maximum score of 4 points, with 1 point each for meeting the following criteria: drawing in three dimensions; correct orientation of the figure; correct drawing of the figure’s lines; and basic parallelism of opposite sides. The remaining two text tasks are scored based on the correctness of each word, with 1 point awarded for each correctly written word. To standardize the full scores for each task, this study maps the scores for each task to percentages, setting the full scores at 10 points. The 30 handwriting features and their meanings are shown in Table 3.

Table 3

30 handwriting features

No.	Feature	Unit	Meaning
Pressure category
1	Average pressure	level (0–1024)	The average pressure when writing
Time category
2	Time in air	ms	Time for pen to move in the air
3	Time on paper	ms	Time for pen to move on the paper
Form category
4	Number of strokes	n	Number of all strokes to complete a task
5	Task score	score	Score for completing a task under the appropriate scoring rule
6	Single stroke length	mm	The length of a particular stroke
7, 8	x-axis/y-axis stroke length	mm	The length of a particular stroke on x-axis/y-axis
9	Tilt	degree	The angle of inclination at which the writing direction intersects the horizontal line
10	Repeated dots	n	Number of repeated dots in a stroke
Velocity category
11	Average velocity	mm/s	The average velocity when writing
12, 13	Maximum/minimum velocity	mm/s	The maximum/minimum velocity when writing
14, 15, 16, 17	Maximum/minimum velocity on x-axis/y-axis direction	mm/s	The maximum/minimum velocity on x-axis/y-axis direction when writing
18, 19	Average velocity on x-axis/y-axis direction	mm/s	The average velocity on x-axis/y-axis direction when writing
20	Number of times velocity slows down	times	Number of times writing velocity slows down
Acceleration category
21	Average acceleration	m/s²	The average acceleration when writing
22, 23	Maximum/minimum acceleration	m/s²	The maximum/minimum acceleration when writing
24, 25, 26, 27	Maximum/minimum acceleration on x-axis/y-axis direction	m/s²	The maximum/minimum acceleration on x-axis/y-axis direction when writing
28, 29	Average acceleration on x-axis/y-axis direction	m/s²	The average acceleration on x-axis/y-axis direction when writing
30	Number of times acceleration slows down	times	Number of times writing acceleration slows down

Due to the characteristics of walking velocity and stride in AD patients, the extracted gait features include 3 items in the velocity category (velocity, stride velocity, stride velocity std dev), 20 items in the time category (step time differential, cycle time differential, step time, cycle time, swing time, stance time, single supp. time, double supp. time, swing % of cycle, stance % of cycle, single supp % cycle, double supp % cycle, heel-off-on time, step time std dev, stride time std dev, swing time std dev, stance time std dev, single supp time std dev, double supp time std dev, heel-off-on std dev), 7 items in the length category (step length differential, step length, stride length, HH base support, step length std dev, stride length std dev, supp base on std dev). Additionally, it also contains 2 other features (cadence, toe in / out). The 32 gait features and their meanings are shown in Table 4.

Table 4

32 gait features

No.	Feature	Unit	Meaning
Velocity category
1	Velocity	cm/s	The walking velocity
2	Stride velocity	cm/s	The ratio of distance to time spent for each stride length
3	Stride velocity std dev	cm/s	The standard deviation of stride velocity
Time category
4	Step time	s	Time taken to walk a step
5	Cycle time	s	Time spent on a complete walking cycle
6, 7	Step time/cycle time differential	s	The difference in stride time /cycle time between right and left foot
8	Swing time	s	Time from when a foot leaves the ground to when it hits the bottom again
9	Stance time	s	Time from when a foot starts to land to when it leaves the ground
10, 11	Single/double supp. time	s	Time spent supporting the body on the ground with one/both feet during a complete walking cycle
12, 13	Swing/stance % of cycle		The proportion of swing/support time in a complete walking cycle
14, 15	Single/double supp % cycle		The proportion of time that one/both feet support the body on the ground in a complete walking cycle
16	Heel-off-on time	s	The heel on time for the next foot minus the heel off time for the current foot
17, 18, 19, 20	Step/stride/swing/stance time std dev	s	The standard deviation of step, stride, swing, and support time
21, 22	Single/double supp time std dev	s	The standard deviation of single/double foot support time
23	Heel-off-on std dev	s	The standard deviation of the heel on time for the next foot minus the heel off time for the current foot
Length category
24	Step length differential	cm	The difference in step length
25, 26	Step/stride length	cm	Single step length/stride length
27	HH base support	cm	Heel to heel (HH) base of support is the horizontal distance from the heel point of one footfall to the line of progression of the opposite foot
28, 29	Step/stride length std dev	cm	The standard deviation of step/stride length
30	Supp base on std dev	cm	The standard deviation of the horizontal distance between the feet and the center point
Others
31	Cadence	steps/min	Steps taken per minute
32	Toe in / out	degree	The angle between the mild-line of the sole and the line of advancement, positive when the mild-line is outside the advancement line, negative when it is inside the advancement line

Feature selection and fusion

To preprocess the extracted data, missing values were imputed using the mean, and then feature selection was performed on the handwriting dataset and gait dataset. To reduce the dimensionality of the feature space, mitigate the curse of dimensionality associated with high-dimensional feature vectors, and identify the most useful features for screening AD patients, this study employed the Sequential Forward Selection (SFS) algorithm. The basic idea of the SFS algorithm is to start with the initial feature set, add one feature at a time, evaluate the accuracy of each classification model, and continue this process until further inclusion of features does not lead to accuracy improvement. After obtaining the optimal feature subsets for both handwriting and gait features separately, the handwriting and gait features were concatenated to create the fused feature vector.

Classification model and evaluation

The purpose of classification is to distinguish AD patients from normal subjects and obtain an AD assisted screening model. Therefore, this study utilized PyCharm Community Edition 2022.2×64 for programming. Classification models, including Logistic Regression (LR), Support Vector Machine (SVM), eXtreme Gradient Boosting (XGB), Adaptive Boosting (Adaboost), and LightGBM (LGB), were established. We chose to use the default values for the hyperparameters or to adjust them manually and empirically. The hyperparameters of each classification model are shown in Table 5. The performance of these classifiers was assessed using 10-fold cross-validation iterated 20 times. In each iteration, the dataset would be shuffled anew. We calculated the average of 20 iterations as the final result. Evaluation of the classification models was based on metrics such as accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), Receiver Operating Characteristic (ROC) curve, and Area Under the Curve (AUC) value. Additionally, to assess the stability of these models, the variance of these parameters was also computed.

Table 5

The hyperparameters of each classification model

Classification model	Hyperparameters
LR	solver: liblinear; penalty: 12; max_iter: 100
SVM	kernel: rbf; gamma: scale; probability: True
Adaboost	base_estimator: DecisionTreeClassifier (max_depth = 8); n_estimators: 100
XGBoost	max_depth: 6; learning_rate: 0.3; n_estimators: 100
LightGBM	max_depth: –1; learning_rate: 0.1; n_estimators: 100

Furthermore, to verify whether the fused features had better classification performance, this paper used the same method to construct classification models for handwriting features and gait features respectively. Additionally, the accuracies of handwriting and gait, iterated 20 times, were subjected to Wilcoxon test when compared with the fused features, ensuring that the differences were statically valid.

Statistical analysis

To explore the different characteristics of AD patients in writing and walking compared to normal elderly people, this study employed independent sample Student’s t-test and Mann-Whitney U test using SPSS 26.0. The choice between the two tests was based on whether the samples satisfied the normality test and ANOVA. The difference was considered statistically significant at p < 0.05.

RESULTS

Statistical analysis

In the basic demographic information for the AD and HC groups shown in Table 1, there was a significant difference between the two groups in terms of MMSE scores (p < 0.001) as shown by the Mann-Whitney U test. There was no significant difference in gender (p = 0.572), age (p = 0.454), and education (p = 0.051).

Using the Mann-Whitney U test for handwriting data showed that there was a significant difference between the AD and HC groups in 28 features such as repeated dots, time in air, and average velocity (p < 0.05). The use of independent sample Student’s t-test and Mann-Whitney U-test for gait data showed that there was a significant difference between the AD and HC groups in 4 features, namely cycle time differential, step length std dev, stride length std dev, and supp base on std dev (p < 0.05). The handwriting and gait features with significant differences are shown in Table 6.

Table 6

The handwriting and gait features with significant differences

Features	Median (IQR)		z value	p
	AD	HC
Handwriting features
Repeated dots	6 (25)	3 (13)	–14.330	<0.001
Time in air	8780.00 (17792)	5654.00 (8928)	–13.627	<0.001
Time in paper	364.00 (392)	326.00 (404)	–5.097	<0.001
Single stroke length	7.361 (6.772)	7.688 (7.590)	–2.975	0.003
x-axis stroke length	3.959 (4.662)	4.126 (5.180)	–3.033	0.002
y-axis stroke length	4.514 (5.587)	4.773 (6.253)	–2.044	0.041
Average velocity	17.726 (16.181)	20.369 (17.282)	–7.495	<0.001
Maximum velocity	59.866 (47.783)	66.276 (51.875)	–6.526	<0.001
Minimum velocity	1.547 (2.139)	1.850 (2.815)	–5.331	<0.001
Average velocity on x-axis direction	10.604 (13.111)	12.438 (14.435)	–6.807	<0.001
Average velocity on y-axis direction	12.545 (15.111)	14.187 (16.762)	–5.336	<0.001
Maximum velocity on x-axis direction	35.458 (38.542)	40.700 (43.097)	–5.963	<0.001
Minimum velocity on x-axis direction	0 (1.542)	0 (1.542)	–3.740	<0.001
Maximum velocity on y-axis direction	41.625 (41.933)	44.708 (46.250)	–4.748	<0.001
Minimum velocity on y-axis direction	0 (1.542)	0 (1.542)	–4.462	<0.001
Number of times velocity slows down	6 (8)	5 (8)	–3.738	<0.001
Average acceleration	0.466 (0.456)	0.542 (0.512)	–7.196	<0.001
Maximum acceleration	1.7525 (1.618)	2.027 (1.809)	–6.621	<0.001
Minimum acceleration	0.022 (0.090)	0.029 (0.123)	–4.866	<0.001
Average acceleration on x-axis direction	0.330 (0.394)	0.397 (0.458)	–7.106	<0.001
Average acceleration on y-axis direction	0.415 (0.461)	0.472 (0.548)	–6.000	<0.001
Maximum acceleration on x-axis direction	1.098 (1.269)	1.279 (1.413)	–6.051	<0.001
Minimum acceleration on x-axis direction	0 (0.064)	0.013 (0.064)	–5.734	<0.001
Maximum acceleration on y-axis direction	1.321 (1.380)	1.477 (1.554)	–5.284	<0.001
Minimum acceleration on y-axis direction	0.009 (0.064)	0.015 (0.090)	–4.634	<0.001
Number of times acceleration slows down	5 (8)	4 (8)	–4.512	<0.001
Number of strokes	35 (26)	30 (24)	–11.843	<0.001
Task score	10 (2.5)	10 (0)	–30.584	<0.001
Gait features
Cycle Time Differential	0.004 (0.002)	0.003 (0.002)	–2.145	0.032
Step Len Std Dev	2.309 (0.865)	2.033 (0.823)	–2.548	0.011
Stride Len Std Dev	3.415 (1.221)	3.028 (1.160)	–2.952	0.003
Supp Base On Std Dev	2.252 (0.869)	1.968 (0.641)	–2.528	0.011

AD, Alzheimer’s disease; HC, healthy control; IQR, interquartile range.

The results of statistical analysis indicated that there was no significant difference in gender, age, and education between AD and HC groups. However, there was a significant difference in MMSE scores (p < 0.001). The U-test results for handwriting data showed significant differences in most handwriting features between the two groups, especially in terms of writing time, writing velocity, and writing form. It can be concluded that AD patients tend to have more strokes, and an increased number of repeated dots for a particular stroke, possibly due to uncertainty about the correctness of their writing, leading to repetition and pauses in the middle of writing. Regarding time, the AD group exhibited longer time in air and time on paper, indicating increased thinking time during writing. In terms of velocity and acceleration, the AD group had lower average velocity, horizontal velocity, and vertical velocity compared to the HC group. Additionally, the AD group tended to make more writing errors. The U-test results for gait data indicated that only four features show significant differences. The AD group has a greater cycle time differential, step length standard deviation, stride length standard deviation, and support base on standard deviation compared to the HC group. It can be concluded that AD patients exhibit irregular walking rhythm, greater variability in step length during walking, and instability in the support base while walking.

Classification performance

To determine which features are most effective for screening AD patients, before training classification models, this study used the SFS algorithm to screen the optimal feature subsets for handwriting and gait. For LR, the optimal handwriting feature subset had 8 items (average pressure, time in air, time on paper, minimum velocity, average velocity on y-axis direction, minimum velocity on x-axis direction, minimum acceleration on x-axis direction, task score); The optimal gait feature subset had 5 items (cycle time differential, HH base support, swing % of cycle, stride velocity std dev, heel-off-on std dev). For SVM, the optimal handwriting feature subset had 5 items (repeated dots, time in air, average velocity, number of strokes, task score); The optimal gait feature subset had 3 items (velocity, step time, HH base support). For XGBoost, the optimal handwriting feature subset had 4 items (repeated dots, time in air, number of strokes, task score); The optimal gait feature subset had 3 items (velocity, step time differential, HH base support). For Adaboost, the optimal handwriting feature subset had 4 items (repeated dots, time in air, number of strokes, task score); The optimal gait feature subset had 4 items (velocity, step time differential, stride velocity, HH base support). For LGB, the optimal handwriting feature subset had 5 items (repeated dots, time in air, minimum velocity on x-axis direction, number of strokes, task score); The optimal gait feature subset had 3 items (velocity, step length differential, HH base support).

To explore whether combining handwriting and gait modalities has better screening effects, in this study, handwriting features, gait features, and fusion features were used to classify the two groups separately by 10-fold cross-validation iterated 20 times. The parameters used to evaluate the classification performance of models were accuracy (Acc.), sensitivity (Se.), specificity (Sp.), PPV, NPV, ROC curve, and AUC value. The results showed that among the classification accuracies obtained on the fusion features, LightGBM had the highest accuracy of 96.17%, sensitivity of 95.32%, specificity of 96.78%, PPV of 95.94%, NPV of 96.74%, and AUC of 0.991 as shown in Table 7. The ROC curve is shown in Fig. 2. It proves that our proposed scheme can effectively classify the AD and HC groups and has application value in assisting the screening of AD.

Fig. 2

ROC curves were calculated to differentiate AD patients and HC under fused features iterated 20 times. The black dashed line represents the performance of a random classifier.

Table 7

Classification performance of each classifier on fusion features. Values were obtained by 20 iterations of 10-fold cross validation

	LR	SVM	XGB	Adaboost	LGB
Acc.	71.95%	74.70%	94.01%	95.07%	96.17%
(Variance)	(0.0086%)	(0.0104%)	(0.0082%)	(0.0092%)	(0.0035%)
Se.	56.77%	61.40%	92.05%	92.56%	95.32%
(Variance)	(0.0205%)	(0.0172%)	(0.0231%)	(0.0367%)	(0.0143%)
Sp.	83.14%	84.53%	95.46%	96.95%	96.78%
(Variance)	(0.0071%)	(0.0198%)	(0.0119%)	(0.0135%)	(0.0036%)
PPV	72.36%	75.41%	94.01%	95.93%	95.94%
(Variance)	(0.0246%)	(0.0271%)	(0.0155%)	(0.0214%)	(0.0049%)
NPV	72.60%	75.13%	94.45%	94.83%	96.74%
(Variance)	(0.0063%)	(0.0062%)	(0.0097%)	(0.0166%)	(0.0064%)

Acc., Accuracy; Se., Sensitivity; Sp., Specificity; PPV, positive predictive value; NPV, negative predictive value; Variance: the variance obtained from running classifiers 20 times.

In addition, as shown in Fig. 3, the highest accuracies of classification using handwriting features and gait features alone were 91.94% and 93.53% respectively. The classification accuracies of all five classifiers on the fusion features were higher than those of the separate modalities. The result of Wilcoxon tests also marked in Fig. 3. The experimental results showed significant differences among LR, Adaboost, and LGB models, whether in the handwriting modality or the gait modality, compared to the combined modality. In SVM, there were significant differences between the gait and combined modalities. In XGB, there were significant differences between the handwriting and combined modalities. It is indicated that the combination of the two modalities of handwriting and gait was better than the use of the two alone in the classification of AD. It also proved the superiority of our proposed scheme.

Fig. 3

Comparison of classification accuracy of each classifier on different modalities. ^*p < 0.05; ^**p < 0.001.

DISCUSSION

This study combined handwriting and gait features to distinguish AD patients from normal elderly people, and compared the performance of different classifiers. Our first conclusion was that better classification results can be obtained by combining data from both modalities of the subjects, which can effectively classify the AD group from the HC group, and had good application value. The second conclusion was that combining data from both modalities can obtain better classification results than a single modality. In addition, the differences in handwriting and gait that we observed in the AD and HC groups were equally meaningful for screening.

It has been demonstrated that handwriting involves several cognitive domains, including language, visuospatial skills, memory, and attention, most of which are affected by AD 12 The experimental results of this study found that out of 30 handwriting features, 28 of them were significantly different. This study finds that people with AD wrote with an increase in the total number of strokes as well as in the number of repeated dots for each stroke, and with shorter stroke lengths. These findings are the same as previous studies.14,35, 14,35 In terms of time, this study can observe that longer time in air and time on paper for each stroke in the AD group. It can respond to the fact that AD patients spend more time thinking before and while writing than the HC group, and to some extent reflect the impaired cognitive ability of AD patients. This finding is also similar to previous studies.31,36, 31,36 In terms of writing velocity, this study can observe that the average velocity, maximum and minimum velocity, and the number of times the velocity slowed down were smaller in the AD group than in the HC group, both horizontally and vertically, as well as the same in terms of writing acceleration. However, there was no significant difference between the AD and HC groups in terms of writing pressure. But we can also observe that the average pressure of the AD group (523.90 level) was slightly greater than that of the HC group (521.51 level). This finding is consistent with the findings of Zhao et al.37,38, 37,38 However, it has also been shown that writing pressure decreases rather than increases in AD patients. 39 Overall, the main characteristics of AD patients in writing are increased thinking time, slower writing velocity, as well as writing errors and increased number of strokes. The present study obtained an accuracy of 91.94% on handwriting features, which was of high value as an aid to screening. In addition, to evaluate the effectiveness of our proposed method, we also conducted experiments using the DARWIN dataset. 40 The DARWIN dataset includes data from 174 participants. Each participant completed 25 writing tasks. After experiments, we achieved the highest accuracy of 83.99% on task 23, while the highest accuracy in the original paper was 82.00%, which also proved that our method is applicable on public datasets. The specific experimental results are shown in Table 8.

Table 8

Comparative experiments results with DARWIN dataset

Task	Our proposed method	Original paper
1	66.76%	65.86%
2	75.26%	67.14%
3	71.27%	68.00%
4	69.54%	71.86%
5	73.59%	73.71%
6	74.87%	74.86%
7	75.29%	78.00%
8	75.29%	71.71%
9	77.61%	77.43%
10	70.07%	69.29%
11	67.19%	64.86%
12	70.20%	67.14%
13	70.62%	68.00%
14	78.30%	71.86%
15	74.08%	73.71%
16	75.29%	74.86%
17	79.44%	78.00%
18	71.21%	71.71%
19	79.67%	77.43%
20	72.39%	71.43%
21	75.78%	72.86%
22	74.22%	75.00%
23	83.99%	82.00%
24	73.99%	72.57%
25	76.31%	73.71%

In addition, compared with neuropsychological tests, which are highly specialized in content and require a high level of screening personnel, the dot-matrix digital pens used in the present study to collect handwriting data are inexpensive and easy to use. So that the present technology can be applied to grass-roots geriatric institutions with scarce healthcare resources to achieve group screening and management of the elderly.

Gait involves the cognitive domains of complex attention, executive function, and perceptual-motor function. The experimental results of this study found significant differences in only 4 out of 32 gait features. This study can find that the AD group had a greater cycle time differential in walking than the HC group. Cycle time differential refers to the difference in time spent on two consecutive walking cycles of the subjects. Under normal conditions, the gait cycles of the left and right feet are close to each other, and if there is gait abnormality or asymmetry, it may lead to different gait cycles of the left and right feet. Therefore, it can conclude that AD patients show asymmetry of the right and left feet and gait rhythm disorder when walking, which may be due to cognitive impairment and imbalance of bilateral muscle strength. This finding is consistent with the previous study. 41 There were also significant differences in the two features of step length standard deviation and stride length standard deviation. It can observe that the AD group’s step length and stride length standard deviation were greater than the HC group. These two features refer to the standard deviation of the step length/stride length of the subjects during walking and are used to measure the degree of change in the step length/stride length of the subjects. Therefore, it can conclude that the AD patients show gait instability during walking. The large variation in stride length also reflects the asymmetry of stride length between the right and left feet of AD patients, which may be due to factors such as lesions in certain parts of the brain and neuronal damage.42,43, 42,43 In addition, there was a significant difference in the standard deviation of the base of support. We can find that the support base standard deviation in the AD group was larger than that in the HC group. The support base refers to the horizontal distance between the two feet and the center point, which is also known as the step width. Therefore, it can conclude that AD patients have greater variability in step width and experience problems such as gait instability and decreased movement coordination when walking. This conclusion is also consistent with previous study. 44

Many studies have reported that gait in AD patients is characterized by slowed gait velocity, shortened stride length, etc.23 –25 However, in this study, the two features were not significant. But the overall mean velocity and mean stride length of the AD group were smaller than those of the HC group. Overall, this study found that cycle time differential, step length/stride length standard deviation, and support base standard deviation increased in AD patients during normal walking. Also, walking velocity and stride length were smaller in AD patients than in normal elderly.

Handwriting and gait cover different cognitive domains. Better classification performance can be obtained by analyzing these two modalities together. This study can be applied to the community, nursing homes, and other grassroots geriatric organizations to achieve group screening and management of the elderly. However, this study also has some limitations. First, this study only included two modalities: handwriting and gait. In the future, other information about the subjects can be collected, such as data that is commonly available on smart wristbands or watches. In addition, clinical data, eye movements, EEG, and other information can also be included in the study.45,46, 45,46 Second, this study divided the AD and HC groups controlling only for variables such as age, education, habitual hand, and diseases that can affect writing and walking. Future studies could control for variables such as writing habits and exercise habits of the elderly to improve the accuracy of the sample. In addition, many studies introduce dual tasks47 –49 (e.g., counting down numbers while walking normally) when studying gait changes in AD patients. This method can better reflect the cognitive impairment of the patients, and then better classification results can be harvested. This practice can be adopted in the future, which will help the study of gait in AD patients.

In addition, in this study, the feature subsets with significant differences in Table 5 were input into each classification model, but the classification effect obtained was not as good as the optimal feature subset derived from the SFS algorithm. After analyzing, there may be several reasons. First, the statistical test may only consider the significance of individual features without considering the correlation or interaction between features, resulting in missing some important features. Second, the feature selection algorithm starts with the contribution of features to the model, which can find the optimal feature subset in a more comprehensive search space, not only relying on the significance of features.

Conclusion

This study proposed an assisted screening framework for AD based on multimodal analysis of handwriting and gait, 38 AD patients and 52 healthy controls were included. Handwriting data of the subjects were collected using dot-matrix digital pens and dot-matrix papers. Gait data were collected using the GAITRite walkway. Differences in handwriting and gait features between the AD and HC groups were analyzed using statistical methods. In addition, five classification models (LR, SVM, XGBoost, Adaboost, LightGBM) were trained on handwriting features, gait features, and fusion features using 10-fold cross-validation iterated 20 times. The experimental results showed that the fusion features obtained the highest test accuracy of 96.17% on LightGBM, with an AUC value of 0.991. It demonstrated that our proposed assisted screening framework can effectively classify AD patients and HC. This study also found that the classification effect of combining two modalities was higher than that of a single modality, proving the classification superiority of multimodal feature fusion. This study can be applied in communities, nursing homes, and other primary geriatric institutions to achieve group-based screening and management of the elderly and to provide ideas for solving the multimodal problem of geriatric diseases.

AUTHOR CONTRIBUTIONS

Hengnian Qi (Conceptualization; Project administration; Writing – review & editing); Xiaorong Zhu (Data curation; Methodology; Writing – original draft); Yinxia Ren (Data curation; Methodology); Xiaoya Zhang (Data curation); Qizhe Tang (Resources; Software); Chu Zhang (Writing – review & editing); Qing Lang (Funding acquisition; Project administration); Lina Wang (Conceptualization; Data curation; Resources).

Footnotes

ACKNOWLEDGMENTS

We are very grateful to the Huzhou Rehabilitation Hospital for its help in the data collection.

FUNDING

This work was supported by the Chaomi S & T Company Cooperation Project (No. HK16003), the National Natural Science Foundation of China (No. 72174061), the Scientific Research Fund of Zhejiang Provincial Education Department (No. Y202250185), China Scholarship Council foundation (No. 202308330251).

CONFLICT OF INTEREST

The authors have no conflict of interest to report.

DATA AVAILABILITY

The data supporting the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.

References

Yin

, Lin

and Feng

. Prospects for the application of new medical imaging technologies in neurodegenerative diseases of the elderly. Geriatr Health Care 2023; 29: 177–181.

Castellani

, Rolston

and Smith

. Alzheimer disease. Dis Mon 2010; 56: 484–546.

Alzheimer’s disease facts and figures

Alzheimers Dement 2023; 19: 1598–1695.

Reitz

, Brayne

and Mayeux

. Epidemiology of Alzheimer disease. Nat Rev Neurol 2011; 7: 137–152.

Huang

and Mucke

. Alzheimer mechanisms and therapeutic strategies. Cell 2012; 148: 1204–1222.

Alzheimer’s Association. 2015 Alzheimer’s disease facts and figures. Alzheimers Dement 2015; 11: 332–384.

Silva

MVF

, Loures

CMG

, Alves

LCV

, et al. Alzheimer’s disease: risk factors and potentially protective measures. J Biomed Sci 2019; 26: 33.

Ren

, Yin

, Wang

, et al. China Alzheimer’s disease report 2021. J Diagn Concepts Pract 2021; 20: 317–337.

Folstein

, Folstein

and McHugh

. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12: 189–198.

10.

Chen

, Xu

, Chu

, et al. Validation of the chinese version of montreal cognitive assessment basic for screening mild cognitive impairment. J Am Geriatr Soc 2016; 64: e285–e290.

11.

Chu

and Xu

. A review of studies on early classification of Alzheimer’s disease based on computer-aided diagnostic techniques. Comput Eng Sci 2022; 44: 879–893.

12.

Fernandes

, Montalvo

, Caligiuri

, et al. Handwriting changes in Alzheimer’s disease: a systematic review. J Alzheimers Dis 2023; 96: 1–11.

13.

Pereira

, Pereira

, Rosa

, et al. Handwritten dynamics assessment through convolutional neural networks: An application to Parkinson’s disease identification. Artif Intell Med 2018; 87: 67–77.

14.

, Zhang

, Wei

, et al. A study of auxiliary screening for Alzheimer’s disease based on handwriting characteristics. Front Aging Neurosci 2023; 15: 1117250.

15.

Delazer

, Zamarian

and Djamshidian

. Handwriting in Alzheimer’s disease. J Alzheimers Dis 2021; 82: 727–735.

16.

Ghaderyan

, Abbasi

and Saber

. A new algorithm for kinematic analysis of handwriting data; towards a reliable handwriting-based tool for early detection of Alzheimer’s disease. Expert Syst Appl 2018; 114: 428–440.

17.

Cilia

, De Stefano

, Fontanella

, et al. Handwriting analysis to support alzheimer’s disease diagnosis: a preliminary study. In: Computer Analysis of Images and Patterns: 18th International Conference, CAIP Salerno, Italy, Proceedings, Part II, Springer, 2019, pp. 143–151.

18.

De Gregorio

, Desiato

, Marcelli

, et al. A multi classifier approach for supporting Alzheimer’s diagnosis based on handwriting analysis. In: Pattern Recognition ICPR International Workshops and Challenges: Virtual Event, Proceedings, Part I, Springer, 2021, pp. 559–574.

19.

Dyer

, Lawlor

and Kennelly

. Gait speed, cognition and falls in people living with mild-to-moderate Alzheimer disease: data from NILVAD. BMC Geriatr 2020; 20: 117.

20.

Zhang

. The relationship between gait and cognitive function and its clinical significance. Prac Geriatr 2021; 35: 1182–1186.

21.

Grande

, Triolo

, Nuara

, et al. Measuring gait speed to better identify prodromal dementia. Exp Gerontol 2019; 124: 110625.

22.

Castrillo

, Olmos

, Rodríguez

, et al. Gait disorder in a cohort of patients with mild and moderate Alzheimer’s disease. Am J Alzheimers Dis Other Demen 2016; 31: 257–262.

23.

Axer

, Axer

, Sauer

, et al. Falls and gait disorders in geriatric neurology. Clin Neurol Neurosurg 2010; 112: 265–274.

24.

Pieruccini-Faria

, Black

, Masellis

, et al. Gait variability across neurodegenerative and cognitive disorders: Results from the Canadian Consortium of Neurodegeneration in Aging (CCNA) and the Gait and Brain Study. Alzheimers Dement 2021; 17: 1317–1328.

25.

Beauchet

, Launay

, Sekhon

, et al. Association of hippocampal volume with gait variability in pre-dementia and dementia stages of Alzheimer disease: Results from a cross-sectional study. Exp Gerontol 2019; 115: 55–61.

26.

Ghoraani

, Boettcher

, Hssayeni

, et al. Detection of mild cognitive impairment and Alzheimer’s disease using dual-task gait assessments and machine learning. Biomed Signal Process Control 2021; 64: 102249.

27.

Chen

, Lien

, Wu

, et al. Gait-based machine learning for classifying patients with different types of mild cognitive impairment. J Med Syst 2020; 44: 107.

28.

Seifallahi

, Mehraban

, Galvin

, et al. Alzheimer’s disease detection using comprehensive analysis of timed up and go test via Kinect V.2 camera and machine learning. IEEE Trans Neural Syst Rehabil Eng 2022; 30: 1589–1600.

29.

Yamada

, Shinkawa

, Kobayashi

, et al. Combining multimodal behavioral data of gait, speech, and drawing for classification of Alzheimer’s disease and mild cognitive impairment. J Alzheimers Dis 2021; 84: 315–327.

30.

Albert

, DeKosky

, Dickson

, et al. The diagnosis of mild cognitive impairment due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement 2011; 7: 270–279.

31.

Chai

, Wu

, Li

, et al. Classification of mild cognitive impairment based on handwriting dynamics and qEEG. Comput Biol Med 2023; 152: 106418.

32.

Müller

, Herde

, Preische

, et al. Diagnostic value of digital clock drawing test in comparison with CERAD neuropsychological battery total score for discrimination of patients in the early course of Alzheimer’s disease from healthy individuals. Sci Rep 2019; 9: 3543.

33.

Ericsson

, Forssell

, Holmén

, et al. Copying and handwriting ability in the screening of cognitive dysfunction in old age. Arch Gerontol Geriatr 1996; 22: 103–121.

34.

and Chang

. Characterization of the fine motor problems in patients with cognitive dysfunction - A computerized handwriting analysis. Hum Mov Sci 2019; 65: S01679457(17)308412.

35.

Wei

, Zhang

, et al. A study of handwriting characteristics of elderly people with mild cognitive impairment and its applied value. Chin Gen Prac 2023; 26: 1224–1233.

36.

Afonso

, Álvarez

, Martínez

, et al. Writing difficulties in Alzheimer’s disease and mild cognitive impairment. Read Writ 2019; 32: 217–233.

37.

Zhào

, Zhang

, Xia

, et al. Digital handwriting analysis of characters in Chinese patients with mild cognitive impairment. J Vis Exp 2021; (169): 10.3791/61841.

38.

Croisile

. Agraphia in Alzheimer’s disease. Dement Geriatr Cogn Disord 1999; 10: 226–230.

39.

Wang

, Abazid

, Houmani

, et al. Online signature analysis for characterizing early stage Alzheimer’s disease: a feasibility study. Entropy 2019; 21: 956–956.

40.

Cilia

, De Gregorio

, De Stefano

, et al. Diagnosing Alzheimer’s disease from on-line handwriting: A novel dataset and performance benchmarking. Eng Appl Artif Intell 2022; 111: 104822.

41.

De Cock

, Fransen

, Perkisas

, et al. Comprehensive quantitative spatiotemporal gait analysis identifies gait characteristics for early dementia subtyping in community dwelling older adults. Front Neurol 2019; 10: 313.

42.

Rosano

, Aizenstein

, Studenski

, et al. A regions-of-interest volumetric analysis of mobility limitations in community-dwelling older adults. J Gerontol A Biol Sci Med Sci 2007; 62: 1048–1055.

43.

Rosano

, Brach

, Studenski

, et al. Gait variability is associated with subclinical brain vascular abnormalities in high-functioning older adults. Neuroepidemiology 2007; 29: 193–200.

44.

Webster

, Merory

and Wittwer

. Gait variability in community dwelling adults with Alzheimer disease. Alzheimer Dis Assoc Disord 2006; 20: 37–40.

45.

Beltrán

, García-Vázquez

, Benois-Pineau

, et al. Computational techniques for eye movements analysis towards supporting early diagnosis of Alzheimer’s disease: a review. Comput Math Methods Med 2018; 2018: 2676409.

46.

Rodrigues

, Bispo

, Garrett

, et al. Lacsogram: a new EEG tool to diagnose Alzheimer’s disease. IEEE J Biomed Health Inform 2021; 25: 3384–3395.

47.

Lowe

, MacAulay

, Szeles

, et al. Dual-task gait assessment in a clinical sample: implications for improved detection of mild cognitive impairment. J Gerontol B Psychol Sci Soc Sci 2020; 75: 1372–1381.

48.

. Single-task or dual-task? Gait assessment as a potential diagnostic Tool for Alzheimer’s dementia. J Alzheimers Dis 2021; 84: 1183–1192.

49.

Higuma

, Sanjo

, Mitoma

, et al. Whole-day gait monitoring in patients with Alzheimer’s disease: a relationship between attention and gait cycle. J Alzheimers Dis Rep 2017; 1: 1–8.