The impact of in-air features on the diagnosis of developmental dysgraphia

Abstract

Handwriting problems, also known as dysgraphia, are defined as a disorder or difficulty in producing written language associated with writing mechanics. The occurrence of handwriting problems among elementary students varies from 10 to 34%. With negative impacts on educational performance, handwriting problems cause low self-confidence and disappointment in the students. In this research, a pen-tablet was employed to sample children’s handwriting, which revealed online features of handwriting such as kinematic and temporal features as well as wrist and hand angles and pen pressure on the surface. This digitizer could also extract the online handwriting features when the pen was not in contact with the surface. Such features are called in-air features. The purpose of this study was to propose a method for diagnosing dysgraphia along with an evaluation of the impact of in-air features on the diagnosis of this disorder. A rich dataset (OHF-1) of online handwriting features of dysgraphic and non-dysgraphic students was prepared. After the extraction of a huge set of features and choosing a feature selection method, three machine learning methods, i.e. SVM, Random Forest and AdaBoost were compared and with the SVM method, an accuracy of 85.7% in diagnosing dysgraphia was achieved, when both in-air and on-surface features were included. However, while using purely in-air data or merely on-surface features, accuracies of 80.9% and 71.4% were achieved, respectively. Our findings showed that in-air features had a significant amount of information related to the diagnosis of dysgraphia. Consequently, they might serve as a significant part of the dysgraphia diagnosis.

Keywords

Handwriting Analysis Identification of Dysgraphia In-Air Analysis Machine Learning Online Handwriting Features

1 Introduction

Handwriting problems in children have been documented with various developmental disabilities. Moreover, abnormal patterns of handwriting have been reported in a wide variety of mental and neurological disorders e.g., depression, OCD, Parkinson’s disease, schizophrenia, as well as developmental coordination disorder. Knowing the factors associated with poor handwriting performance is beneficial to guiding correction and reconstruction strategies. Scientists trust that the components related to handwriting performance are motor planning, eye-hand coordination, visual-motor coordination, and hand functioning [1].

Children with impaired handwriting, write illegible contents, which are often graded inappropriately by the teachers. Handwriting problems commonly cause the students to suffer from poor reading skills and wasting too much time doing their homework. Handwriting problems reveal when reading becomes difficult for children or when they write too slowly. Consequently, the quality and quantity of handwriting can be interpreted in terms of readability and speed, respectively [1].

The occurrence of handwriting problems among elementary students varies from 10 to 34%. Such handwriting problems may have intense consequences for an individual regarding self-image and success in school [2]. The importance of handwriting in school performance is well documented. Poor writing skills have a direct influence on educational performance giving rise to low self-confidence and low self-regard, leading to plenty of frustration and disappointment in the child, the parents, and the teacher. Handwriting problems are closely related to low academic achievement and low self-confidence. Compared to their peers, poor handwriting skills in these children can increase the time required for writing tasks such as doing homework. Sometimes, students cannot read their messy handwriting, which leads to inadequacy in note-taking, chaos in their homework organization, and rejection by their peers in teamwork, which is quite common in classroom learning. In the classroom, the children’s need for concentration and effort while writing increases, which may cause a reduction in the attention to spelling, grammar, and lesson content that, in turn, may further jeopardize their academic achievement [3].

Researchers have realized that handwriting problems may have severe consequences for the student’s overall success, well-being, attitudes, and emotional behavior [4, 5]. These findings illustrate the significance of identifying handwriting problems as soon as possible, as preventive and corrective measures [6].

Since the traditional diagnosis of dysgraphia is time-consuming, many children with the disorder are not currently diagnosed, and therefore, no treatment is provided. With the high speed of an automated test, more students may be diagnosed and treated. Students with lower degrees of disorder, whose condition is not empirically diagnosable, can be identified using the automated test. Moreover, with the automatic diagnosis system, one can have better control over the treatment period and compare the treatment results with those from the previous times. In this study, it was attempted to diagnose the dysgraphia disorder automatically, which owing to the high speed of the automated test and lack of necessity for a specialist, helps to screen a broader range of students.

Traditional handwriting studies were limited to observable result-oriented tests. Nonetheless, computers and digitizer-based technology have led to the development of an innovative method for handwriting evaluation. This means that a digital tablet can provide insight into the process of writing during the actual function of writing [1].

As well as being used in the diagnosis of the dysgraphic disorder, online handwriting features, have been used to identify other diseases and disorders such as Parkinson’s and attention deficit hyperactivity disorder. Drotar et al. [7] quantitatively evaluated Parkinson’s disease using online in-air features. Rosenblum et al. [8] utilized online spatial, temporal, and pressure features to diagnose Parkinson’s disease. Each of the studies [9 –17] has used several features extracted from online pen data to examine the correlation between Parkinson’s disease and handwriting and its diagnosis.

In [2], using temporal, characteristics of dysgraphia were identified by employing handwriting classification methods and utilizing spatial and pressure features (total time for doing the writing task, total time for doing the task in the fifth decile, total time for doing the task in the first decile, time fraction elapsed on paper, average letter spacing, average pressure, and pressure standard deviation) and the method of linear support vector machines. J. Mekyska et al. [18] used a set of features such as velocity, acceleration, vibration, direction, duration, time of the pen movement in the air, and the angle of the pen, with the random forest method of linear differential analysis to diagnose dysgraphia. In comparing children with and without dysgraphia, using the analysis of dynamic features of handwriting, Hen-Herbst et al. [19] concluded that children with dysgraphia needed more time to complete their tasks and draw larger lengths and widths than the control group. Besides, in a study that dealt with the behavioral effects such as those of handwriting on performance, individuals were divided into negative, positive, and neutral groups by inducing mental load; then, they were compared based on analyzing their handwriting, and it was demonstrated that negative groups produced narrower lengths and widths of pen strokes than the neutral group. In contrast, the positive group needed shorter durations than the neutral group [20]. Morello et al. [21], analyzed the correlations between motor parameters in normal and dysgraphic children using motor features reported that they spent more time on doing the assigned tasks. The results showed that online handwriting features can be effective in diagnosing dysgraphia [22 –26].

The purpose of this study was to diagnose dysgraphia and underline the importance of the in-air features of the pen in the diagnosis of this disorder. To do this, 101 students were asked to copy a couple of verses on a pen-tablet. After collecting the raw pen data for each subject, a vast extraction of kinematic, temporal, and spatial features was conducted. The vector data was converted into scalar values with various statistical functions. After the feature selection step by the SVM method, the samples were categorized into three groups depending on the utilization of in-air data, on-surface data, or both types of data. Ultimately, the effective features in each group were displayed, and a number of them were examined.

A new dataset (OHF-1) of online handwriting features of dysgraphic and non-dysgraphic students, with the participation of 101 students and 15 teachers was introduced as well. This dataset included pen-tablet data, handwritten images, and personal information in the form of 505 files for 101 students, which are described in detail in section 2.2.

The contributions of the present work are as follows:

Demonstrating the importance of in-air features in the diagnosis of the dysgraphic disorder

Establishing a system for diagnosing dysgraphia that can assist independently or as a psychiatrist’s assistant

Extracting important correlated features associated with dysgraphia

Collecting a rich database from dysgraphic and non-dysgraphic students that will be available for the future studies

The rest of the paper is organized in the following manner. Section 2 describes how the data was collected, introduces raw features, and provides an explanation of how they were extracted and categorized. The results from this work are presented in Section 3.OHF1. In section 4, was attempted to use deep learning to solve the problem. Section 5 discusses the results, comparing them with those from other studies.

2 Experimental Study

2.1 Subjects

A total of 101 students, consisting of 49 dysgraphic students and 52 regular students from the second, third, and fourth grades, participated in the study. They were all right-handed, with 41 students being female and the rest being male. The subjects were from four schools and 15 classes, and both groups matched in age, gender, school, and class. All participants were born in Iran and spoke Persian as their mother tongue.

The HPSQ [27] questionnaire was used to label the participants. It included eleven items completed by a teacher who had been adequately acquainted with the students in the classroom. The teacher gave the student a score between 0 and 4 for each item. A higher score for each question indicated the student’s weakness in writing skills. If the student was scored a total of 14 or higher, they would be considered a dysgraphic child.

2.2 OHF-1 Dataset

The students were asked to copy a couple of verses written on the board on paper placed on a digitizer. For the online collection of the children’s handwriting data, it is necessary to utilize a digitizer with high sensitivity to the movements of the pen, which in addition to the direction of the pen movements, records the hand angle, the amount of the wrist rotation, as well as the pressure applied to the paper. For this purpose, a Wacom digitizer Model Intuos Pro Paper Large we used, which was able to record the following data from the pen almost every five milliseconds (see Fig. 1):

Stroke number

(X, Y, and Z): pen movement coordinates.

Pressure (8,192 pen pressure levels): the amount of pressure that the pen puts on the tablet.

Altitude (30-90 degrees): pen angle to the tablet surface.

Azimuth (0-359 degrees): pen rotation rate.

Time: sampling time.

BTN: a boolean value, 0 for in-air and 1 for on-surface movement.

Fig. 1

X, Y, Z, and Altitude/Azimuth information.

Fig. 2

Handwriting samples from a typical student and a dysgraphic student demonstrating in-air and on-surface trajectory of the pen.

In addition to recording pen information when the pen is in contact with the digitizer screen (on-surface), this digitizer can record the pen data even when the pen is in the air and has not come in contact with the tablet surface. We call it in-air. In-air pen data recording is conducted as far as the pen is not more than 10 mm away from the tablet screen. To get the above information from the pen, we developed software that, after the student finished writing the verses, stored the pen data in a record every five milliseconds. Depending on how long the writing process took, the records were saved.

Our dataset consisted of “pen-tablet raw data”, “On-Surface, In-Air, and Both images”, and “personal information”. We called this dataset OHF-1 (the dataset is available at https://github.com/aminic/OHF-1). Some sample images from this dataset are shown in Fig. 2 and Appendix.

Figure 2 (a) illustrates a handwriting sample from a typical student with an HPSQ score below 14, while Fig. 2 (b) shows a handwriting example from a dysgraphic student with an HPSQ score above 14. More samples from our database are provided in Appendix.

Table 1

Kinematic handwriting features

Feature	Description
length	the path length, is defined as $\sqrt{(x [n + 1] - x [n]^{2} + (y [n + 1] - y [n])^{2}}$
width	amount of displacement along the X
height	amount of displacement along the Y
speed	the length divided by time
velocity	amount of changing the position of the pen over time
acceleration	amount of changing the velocity of the pen over time
jerk	amount of changing the acceleration of the pen over time
horizontal velocity/acceleration/jerk	velocity/acceleration/jerk in the direction of the X-axis
vertical velocity/acceleration/jerk	velocity/acceleration/jerk in the direction of the Y-axis
normalizes jerk	normalized by movement duration
orientation	describe the slope of handwriting
speed of pressure/azimuth/altitude change	The rate at speed change of pressure/azimuth/altitude with time

2.3 Feature extraction

Initially, the following kinematic features were computed for each record of raw pen data. Table 1 provides a kinematic feature description. From the above kinematic features, several other features, e.g. the number of changes in velocity (NCV), the number of changes in acceleration (NCA) as well as temporal features such as the total amount of time that the pen spent in the air on-surface/in-air were extracted. The features extracted at this stage are provided in Table 2. For subsequent processing, the vector data such as velocity to scalar values was converted using statistical functions. The statistical functions we used are as follows:

Arithmetic mean, harmonic mean, geometric mean, trimmed means (5, 10, 20, 30, 40, 50), and mode.

Maximum, Minimum, Max Position, Min Position, Max Relative Position, and Min Relative Position.

Skewness, kurtosis, and moments (1, 2, 3, 4, 5, 6).

Median, percentile (1, 5, 95, 99), quartiles (1, 3), and deciles (1, 2, 3, 4, 5, 6, 7, 8, 9).

Range, interquartile range.

Standard deviation, variance, and Shannon entropy.

Table 2
Temporal features and post-processing on kinematic features

Feature Description

HNCV/VNCV horizontal/vertical NCV

HNCA/VNCA horizontal/vertical NCA

NCJ the number of changes in jerk

HNCJ/VNCJ horizontal/vertical NCJ

relative NCV NCV relative to writing time

relative NCA NCA relative to writing time

relative NCV/NCA NCV/NCA relative to writing time

relative NCV/NCJ NCV/NCJ relative to writing time

relative NCA/NCJ NCA/NCJ relative to writing time

in-air time the amount of in-air time

on-surface time the amount of on-surface time

normalized in-air time in-air time divided by the total time

normalized on-surface time on-surface time divided by the total time

in-air/on-surface in-air/on-surface time

total time on-surface time + in-air time

#strokes the number of Strokes

Feature	Description
HNCV/VNCV	horizontal/vertical NCV
HNCA/VNCA	horizontal/vertical NCA
NCJ	the number of changes in jerk
HNCJ/VNCJ	horizontal/vertical NCJ
relative NCV	NCV relative to writing time
relative NCA	NCA relative to writing time
relative NCV/NCA	NCV/NCA relative to writing time
relative NCV/NCJ	NCV/NCJ relative to writing time
relative NCA/NCJ	NCA/NCJ relative to writing time
in-air time	the amount of in-air time
on-surface time	the amount of on-surface time
normalized in-air time	in-air time divided by the total time
normalized on-surface time	on-surface time divided by the total time
in-air/on-surface	in-air/on-surface time
total time	on-surface time + in-air time
#strokes	the number of Strokes

Finally, to evaluate the impact of the in-air features on dysgraphia diagnosis, all features related to in-air and on-surface data were categorized into two separate groups. The following steps were taken for the three different groups below:

Using merely in-air pen features

Using merely on-surface pen features

Using a combination in-air and on-surface features

2.4 Feature Selection

Dimension reduction methods frequently fall into two categories, i.e. feature selection and feature extraction, each of which has its characteristics. On the one hand, feature extraction methods achieve dimension reduction by combining the main features. Thus, they can build a set of new features, which are usually more compact and more distinctive. On the other hand, feature selection reduces the dimensions by eliminating irrelevant and duplicate features. Feature selection is very useful for applications in which the main features are important for the model interpretation and knowledge extraction, since during this process, the main characteristics of the dataset are preserved. In view of the fact that in this study, we sought to identify the most important features associated with the dysgraphic disorder so that they can be used in the feature studies, the feature selection method was employed to reduce the dimensions. In this context, feature selection methods have become an integral part of the learning process to deal with data of higher dimensions. The proper feature selection can improve an inductive learner from various aspects, including learning speed, generalization capacity, and simplicity of the extracted model.

To reduce the dimensions, the data were first analyzed using the Mann-Whitney U test, and the significance level was set at P < 0.05. Features that gained values equal to or greater than 0.05 in the Human Whitney test were eliminated and identified irrelevant data. Afterward, the features with redundancy were addressed. Reducing data redundancy increased speed, reduces complexity, and speeds up the training phase.

Subsequently, the most important features were acquired based on the Pearson and Spearman correlations. Table 3 lists the most important features when the data exclusively includes in-air pen features. Table 4 shows the most important features when the data has only on-surface pen features. The most important features for the case in which the data consists of a combination of all in-air and on-surface pen features are listed in Table 5. Ultimately, to attain the best subset of features, the recursive feature elimination (RFE) [28] method was used, which recursively eliminates the features and builds the model with the remaining ones.

Table 3
The ten features that are highly correlated with the target feature (in-Air)

Feature Pearson Spearman

Curve Length 0.473286373436767 0.497395562458699

VNCV 0.386140802 0.412121644957688

Distance (Skewness) 0.376430660804278 0.445073898101705

Speed of altitude change (Trim Mean(5)) 0.355979268613931 0.332276543773639

Distance (Range) 0.339179962417097 0.379841693189088

Duration (Skewness) 0.333033930303024 0.344507582194755

Acceleration (Trim Mean(5)) 0.332141139193329 0.298301437048318

Vertical displacement (Range) 0.323922530863548 0.321747071537453

Speed of altitude change (Percentile10) 0.322796135732172 0.322796135732172

Slope (Harmonic Mean) 0.307575099016076 0.2751983644751

Vertical Distance (Skewness) 0.289126190468183 0.301019445586344

Acceleration (Kurtosis) 0.286147513869725 0.216081678773041

Feature	Pearson	Spearman
Curve Length	0.473286373436767	0.497395562458699
VNCV	0.386140802	0.412121644957688
Distance (Skewness)	0.376430660804278	0.445073898101705
Speed of altitude change (Trim Mean(5))	0.355979268613931	0.332276543773639
Distance (Range)	0.339179962417097	0.379841693189088
Duration (Skewness)	0.333033930303024	0.344507582194755
Acceleration (Trim Mean(5))	0.332141139193329	0.298301437048318
Vertical displacement (Range)	0.323922530863548	0.321747071537453
Speed of altitude change (Percentile10)	0.322796135732172	0.322796135732172
Slope (Harmonic Mean)	0.307575099016076	0.2751983644751
Vertical Distance (Skewness)	0.289126190468183	0.301019445586344
Acceleration (Kurtosis)	0.286147513869725	0.216081678773041

Table 4

The ten features that are highly correlated with the target feature (On-Surface)

Feature	Pearson	Spearman
Azimuth (Range)	0.325281047469992	0.265960421379311
Curve Length	0.300715505718915	0.289467909299735
Vertical Jerk (Range)	0.294150718313464	0.338054280788611
Vertical Jerk (Percentile99)	0.29285045325298	0.248853476545365
Speed of Altitude Change (Geometric Mean)	0.289459904049351	0.276557368744112
Speed Of Azimuth Change (Standard Deviation)	0.286244452248491	0.294224424241279
Azimuth (Interquartile Range)	0.274128676632322	0.198833045707795
Vertical Speed (Moment2)	0.254890356338796	0.217440683042054
Altitude (Entropy)	0.254594576052954	0.248018279094843
Altitude (Interquartile Range)	0.253824977540574	0.215519177954509

Table 5

The ten features that are highly correlated with the target feature (All)

Feature	Pearson	Spearman
Curve Length ( in-Air)	0.473286373436767	0.497395562458699
VNCV ( in-Air)	0.386140802	0.412121644957688
Distance (Skewness, in-Air)	0.376430660804278	0.445073898101705
Speed of altitude change (Trim Mean(5), in-Air)	0.355979268613931	0.332276543773639
Distance (Range, in-Air)	0.339179962417097	0.379841693189088
Duration (Skewness, in-Air)	0.333033930303024	0.344507582194755
Acceleration (Trim Mean(5), in-Air)	0.332141139193329	0.298301437048318
Azimuth (Range, On-Surface)	0.325281047469992	0.265960421379311
Vertical displacement (Range, in-Air)	0.323922530863548	0.321747071537453
Speed of altitude change (Percentile10, in-Air)	0.322796135732172	0.322796135732172
Duration (Standard Deviation, On-Surface)	0.322756263400189	0.322084011756043
Vertical Acceleration (Max, On-Surface)	0.310567903385887	0.305452220771292

2.5 Classification

Our purpose was to build a model for diagnosing dysgraphic disorder among students. To this end, three machine learning methods, i.e. SVM, Random forest and AdaBoost were used and compared and the Python programming language was applied along with the scikit-learn library [29] for the implementation.

The fundamental notion of an SVM classifier is to calculate a maximum margin hyperplane, which separates two different classes of data. To train the nonlinear separated functions, the dataset is implicitly mapped utilizing a kernel function, for which the separating hyperplane is situated in the space of higher dimensions. The newly found samples are then grouped based on their hyperplane side. The radial basis functions (RBF) [30] kernel were utilized, which may be defined as $K (x, x_{i}) = e^{\frac{- ∥ x - x_{i} ∥^{2}}{2 γ^{2}}}$ (1)

In the above equation, gamma is a parameter by which the RBF function width is controlled. Using a grid search, the penalty parameter and the gamma kernel parameters were optimized. A search was performed over a specific grid defined by the Cartesian product of the two sets C = [2^-8, 2^-7, . . . , 2⁷, 2⁸], and Gamma = [2^-12, 2^-11, . . . , 2¹¹, 2¹²].

The random forest algorithm [31] is a classification method consisting of a large number of separate decision trees that act as an ensemble. Instead of relying on a single decision tree, a random forest does the prediction based on each and every tree according to the majority of votes and considers the final result as the output. The more the number of trees in the forest, the higher the accuracy and therefore, the problem overfitting is avoided.

The AdaBoost algorithm [32] is an iterative procedure that attempts to approximate by combining a large number of weak classifiers. Starting with an unweighted training sample, it builds a classifier, e.g. a classification tree to generate the class labels. If a training dataset is misclassified, the weight of that training data is increased (it is boosted). A second classifier is then constructed using the new weights. Again, the misclassified training dataset increases its weight and the procedure is repeated.

Cross-validation is a beneficial technique for assessing the performance of machine learning models. This method helps to realize how the machine learning model we have developed may be generalized to an independent dataset. Using cross-validation, we can test a machine learning model in the training phase to assess its performance and get an idea of how to generalize our machine learning model to an independent dataset. The k-fold cross-validation is one of the most common types of cross-validation, which is widely used in machine learning. The primary dataset is divided into k equal subsets. Each subset is called a fold. Here, for the validation of the classifier, the leave-one-out cross-validation method was utilized, which is a type of k-fold cross-validation, in which k is equal to N, i.e., the number of samples in the dataset.

3 The Results experimental study of the OHF-1 dataset

The performance of the trained classifier based on the calculation of accuracy, sensitivity, precision, and F1-score over the experimental samples is defined as follows [33]: $Accuracy = \frac{TP + TN}{TP + TN + FP + FN}$ (2) $Sensivity / Recall = \frac{TP}{TP + FN}$ (3) $Precision = \frac{TP}{TP + FP}$ (4) $Specificity = \frac{TN}{TN + FP}$ (5) $F 1 - Score = 2 \times \frac{Precision \times Recall}{Precision + Recall}$ (6) where TP (true positive) and FP (false positive) indicate the number of dysgraphic individuals correctly identified and the number of individuals incorrectly known as dysgraphic, respectively. Also, TN (true negative) means the total number of individuals correctly identified as non-dysgraphic or control. In contrast, FN (false negative) means the total number of individuals incorrectly identified as non-dysgraphic or control.

The SVM classification criteria for diagnosing dysgraphia using in-air pen features, on-surface pen features, and a combination of both are provided in Table 6. The Confusion Matrix of classifiers for All Features is reported in Table 7 and The ROC curves are shown in Fig. 3 and the loss function curves of SVM classifier illustrated in 4.

Table 6

Performance of classifiers

	Accuracy	Sensitivity	Precision
SVM
In-air	0.80952	0.75	0.9
On-surface	0.71429	0.75	0.6
In-air + On-surface	0.85714	0.81818	0.9
Random Forest
In-air	0.7619	0.72727	0.8
On-surface	0.71429	0.75	0.6
In-air + On-surface	0.80952	0.8	0.8
AdaBoost
In-air	0.7619	0.72727	0.8
On-surface	0.71429	0.7	0.7
In-air + On-surface	0.80952	0.75	0.9

Table 7

Confusion Matrix of classifires for All Features

	Non-Dysgraphic	Dysgraphic
SVM
Non-Dysgraphic	42.86%	9.52%
Dysgraphic	4.76%	42.86%
Random Forest
Non-Dysgraphic	42.86%	9.52%
Dysgraphic	9.52%	38.1%
AdaBoost
Non-Dysgraphic	38.1%	14.29%
Dysgraphic	4.76%	42.86%

Fig. 3

ROC curves.

Fig. 4

LOSS curves.

As for the validation of the proposed method scalability, the model was tested with 10 samples in the first step. Then the test was continued with 20, 30,... up to 101 samples to obtain the accuracy and time spent in each step. The accuracy of SVM classifier with different sample sizes illustrated in Fig. 5. Additionally, Fig. 6 shows the time spent in each step. Also, in order to validate the robustness of the proposed method, noise data was added to the dataset and the proposed method was able to remain stable despite the presence of noise.

Fig. 5

The accuracies with different sample sizes.

Fig. 6

The required times to train with different sample size.

4 More Experiments

It was attempted to use deep learning to solve the problem in a way that will be discussed subsequently. However, due to the low number of training samples, no acceptable results and accuracy were obtained. In addition, the deep learning method works as a black box and eventually, it does not help much in recognizing the handwriting features that affect dysgraphia diagnosis. In order to achieve better results, it is necessary to collect much more data, which was not possible according to the existing conditions, and given the amount of the available data, the SVM method performed better.

In the applied method, which used deep learning, first, each of the features of the pen collected every 5 milliseconds were considered as a time series, and LSTM and CNN methods were used for the automatic classification of dysgraphia. In the CNN method, the time series was first transformed into an image of fixed dimensions through normalization in the time dimension. Each of the pen signals were converted into an image separately. The sum of the signals was obtained in one single image as well.

5 Discussion

A total of 101 students (49 students with dysgraphia and 52 more without dysgraphia) were asked to use an ink light pen to copy a couple of verses written on the board on A4 paper placed on a digitizer. After collecting the raw pen data for each subject, a vast extraction of kinematic, temporal, and spatial features was conducted. The vector values were converted into scalar values with various statistical functions. After the feature selection step using the SVM method, the samples were categorized into three groups depending on the utilization of in-air data, on-surface data, or both types of data. In the case of using in-air data, an accuracy of 80.9% was achieved; in the case of using on-surface data, accuracy of 71.4% was achieved; using both datasets simultaneously, an accuracy of 85.7% was achieved.

Our results revealed that psychiatrists could use handwriting evaluation as a complementary method for diagnosing dysgraphia. We believe that by using new handwriting features, taking more subjects and diverse tasks from students, and further adjustment of machine learning techniques, it is possible to improve the prediction accuracy. Here, it was demonstrated that online handwriting features could reveal the effect of dysgraphia on handwriting and accordingly could be helpful for diagnosis. In addition, a standard questionnaire was used to differentiate between the dysgraphic and non-dysgraphic groups based on the scores that teachers would give the students; factors such as teacher experience and taste could also be effective in this test.

The advantage of using online handwriting features to diagnose disorders and diseases is that the access to handwriting signals in a clinic or at home is relatively easy [10]. The dataset was collected with a pen-tablet connected to a laptop at school. The commercial tablet in the market was used without any hardware changes. The software environment is simple and user-friendly which makes it possible to use it at home, and this is an important advantage over other approaches.

Our findings showed that in-air features had considerable information regarding the diagnosis of dysgraphia. Consequently, they may serve as a significant part of dysgraphia diagnosis. The kinematic features, azimuth, altitude, and the number of changes in acceleration were identified as the most effective features related to dysgraphia, as well. Contrary to our beliefs, although a digitizer with high sensitivity to pressure was used, this feature did not play a significant role in our work.

Two important in-air features were curve length and VNCV. Figure 7 shows that children with dysgraphia hold the pen for a longer time in the air when writing. The results are consistent with previous results [2]; it also appears that children with dysgraphia need more time to decide to write words. Because of higher values of the VNCV (in-air) feature for children with dysgraphia (Fig. 8), it seems that the control and stability among children suffering from dysgraphia are less than those among non-dysgraphic children.

Fig. 7

Curve Lenght (in-air) feature.

Fig. 8

VNCV (in-air) feature.

Figure 9 illustrates the importance of the azimuth feature when writing on the surface, which shows that the hand and wrist rotation in dysgraphic children is different from that in non-dysgraphic children, which may be due to elementary mistraining of how to hold a pen. The second important on-surface feature was the duration feature (Fig. 10), which again shows the importance of time and temporal features concerning dysgraphia.

Fig. 9

Azimuth (on-surface) feature.

Fig. 10

Azimuth (on-surface) feature.

Footnotes

Appendix

In this appendix, a few samples are provided which were located in the boundary region, and the system was unable to diagnose them accurately (Figs.11, 12, 13, and 14).

References

Shen

I.-H.

, Lee

T.-Y.

and Chen

C.-L.

, Handwriting performance and underlying factors in children with attention deficit hyperactivity disorder, Research in Developmental Disabilities 33(4) (2012), 1301–1309.

Rosenblum

and Dror

, Identifying developmental dysgraphia characteristics utilizing handwriting classification methods, IEEE Transactions on Human-Machine Systems 47(2) (2017), 293–298.

Racine

M.B.

, Majnemer

, Shevell

and Snider

, Handwriting performance in children with attention deficit hyperactivity disorder (adhd), Journal of Child Neurology 23(4) (2008), 399–406. PMID: 18401033.

Feder

K.P.

and Majnemer

, Handwriting development, competency, and intervention, Developmental Medicine & Child Neurology 49(4) (2007), 312–317.

Peverly

S.T.

, Vekaria

P.C.

, Reddington

L.A.

, Sumowski

J.F.

, Johnson

K.R.

and Ramsay

C.M.

, The relationship of handwriting speed, working memory, language comprehension and outlines to lecture note-taking and test-taking among college students, Applied Cognitive Psychology 27(1) (2013), 115–126.

Martins

M.R.I.

, Bastos

J.A.

, Cecato

A.T.

, Araujo

M.d.L.S.

, Magro

R.R.

and Alaminos

, Screening for motor dysgraphia in public schools, Jornal de Pediatria 89(1) (2013), 70–74.

Drotár

, Mekyska

, Rektorová

, Masarová

, Smékal

, Faundez-Zanuy

A new modality for quantitative evaluation of parkinson’s disease: In-air movement, in 13thIEEE International Conference on BioInformatics and Bio-Engineering, (2013), pp. 1–4, IEEE.

Rosenblum

, Samuel

, Zlotnik

, Erikh

and Schlesinger

, Handwriting as an objective tool for parkinson’s disease diagnosis,pp, Journal of Neurology 260(9) (2013), 2357–2361.

Drotár

, Mekyska

, Smékal

, Rektorová

, Masarová

, Faundez-Zanuy

Prediction potential of different handwriting tasks for diagnosis of parkinson’s, in 2013 E-Health and Bioengineering Conference (EHB) (2013), pp. 1–4, IEEE.

10.

Drotár

, Mekyska

, Rektorová

, Masarová

, Smékal

and Faundez-Zanuy

, Analysis ofin-air movement in handwriting: A novel marker for parkinson’s disease, Computer Methods and Programs inBiomedicine 117(3) (2014), 405–411.

11.

Graça

, Sarmento e Castro

, Cevada

Parkdetect: Early diagnosing parkinson’s disease, in 2014 IEEE International Symposium on Medical Measurements and Applications (MeMeA) (2014), pp. 1–6, IEEE.

12.

Pereira

C.R.

, Pereira

D.R.

, Da Silva

F.A.

, Hook

, Weber

S.A.

, Pereira

L.A.

, Papa

J.P.

A step towards the automated diagnosis of parkinson’s disease: Analyzing handwriting movements, in 2015 IEEE 28th International Symposium on Computer-based Medical Systems, (2015), pp. 171–176, IEEE.

13.

Drotár

, Mekyska

, Rektorová

, Masarová

, Smékal

and Faundez-Zanuy

, Decisionsupport framework for parkinson’s disease based on novel handwriting markers, IEEE Transactions on NeuralSystems and Rehabilitation Engineering 23(3) (2014), 508–516.

14.

Drotár

, Mekyska

, Rektorová

, Masarová

, Smékal

and Faundez-Zanuy

, Evaluation ofhandwriting kinematics and pressure for differential diagnosis of parkinson’s disease, ArtificialIntelligence in Medicine 67 (2016), pp.39–46.

15.

Pereira

C.R.

, Weber

S.A.T.

, Hook

, Rosa

G.H.

, Papa

J.P.

Deep learning-aided parkinson’s disease diagnosis from handwritten dynamics, in 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI) (2016), pp. 340–346, IEEE.

16.

Chahkandi

, Nejad, O. Khayat and J. Razjouyan, Software development of an intelligent spirography test system for neurological disorder detection and quantification, Journal of Intelligent & Fuzzy Systems 28(5) (2015), 2149–2157.

17.

Khawar

, Kaleem

, Moetesum

, Siddiqi

Feature relevance analysis for handwriting based identification of parkinson’s disease, in Mediterranean Conference on Pattern Recognition and Artificial Intelligence, (2022), pp. 158–171, Springer.

18.

Mekyska

, Faundez-Zanuy

, Mzourek

, Galaz

, Smekal

and Rosenblum

, Identification and rating of developmental dysgraphia by handwriting analysis, IEEE Transactions on Human-Machine Systems 47(2) (2017), pp.235–248.

19.

Hen-Herbst

and Rosenblum

, Which characteristics predict writing capabilities among adolescents with dysgraphia? Pattern Recognition Letters 121 (2019), 6–12.

20.

Rispler

, Luria

, Kahana

and Rosenblum

, Mood impact on automaticity of performance: Handwriting as exemplar, Cognitive Computation 10(3) (2018), 398–407.

21.

Morello

, Ajcevic

, Perrone

and Accardo

, Analysis of kinematic parameters relationships in normal and dysgraphic children, in World Congress on Medical Physics and Biomedical Engineering 2018 (2019), pp. 189–193, Springer.

22.

Gargot

, Asselborn

, Pellerin

, Zammouri

, Anzalone

S.M.

, Casteran

, Johal

, Dillenbourg

, Cohen

and Jolly

, Acquisition of handwriting in children with and without dysgraphia:A computational approach, PLoS One 15 (2020), 1–22.

23.

Drotár

and Dobešs

, Dysgraphia detection through machine learning, Scientific Reports 10(1) (2020), pp.1–11.

24.

Deschamps

, Gaffet

, Aloui

, Boutet

, Brault

and Labyt

, Methodological issues in the creation of a diagnosis tool for dysgraphia, NPJ digital Medicine 2(1) (2019), pp.1–3.

25.

Asselborn

, Gargot

, Kidzi'nski

Ł.

, Johal

, Cohen

, Jolly

and Dillenbourg

, Automated human-level diagnosis of dysgraphia using a consumer tablet, NPJ Digital Medicine 1(1) (2018), pp. 1–9.

26.

Moetesum

, Diaz

, Masroor

, Siddiqi

and Vessio

, A survey of visual and procedural handwriting analysis for neuropsychological assessment, Neural Computing and Applications (2022), pp. 1–18.

27.

Rosenblum

, Development, reliability, and validity of the handwriting proficiency screening questionnaire (hpsq), American Journal of Occupational Therapy 62 (2008), pp. 298–307.

28.

Guyon

, Weston

, Barnhill

and Vapnik

, Gene selection for cancer classification using support vector machines, Machine Learning 46(1) (2002), pp. 389–422.

29.

Pedregosa

, Varoquaux

, Gramfort

, Michel

, Thirion

, Grisel

, Blondel

, Prettenhofer

, Weiss

, Dubourg

et al., Scikit-learn: Machine learning in python,pp, The Journal of Machine Learning Research 12 (2011), pp. 2825–2830.

30.

Han

, Qubo

, Meng

Parameter selection in svm with rbf kernel function, in World Automation Congress 2012, pp. 1–4, IEEE, 2012.

31.

T.K.

, Random decision forests, in Proceedings of 3^rd International Conference on Document Analysis and Recognition 1 (1995), 278–282.

32.

Freund

and Schapire

R.E.

, A decision-theoretic generalization of on-line learning and an application to boosting, Journal of Computer and System Sciences 55(1) (1997), pp. 119–139.

33.

Malik

O.A.

, Faisal

, Hussein

B.R.

Ensemble deep learning models for fine-grained plant species identification, in 2021 IEEE Asia-Pacific Conference on Computer Science and Data Engineering (CSDE) (2021), pp. 1–6, IEEE.