Intelligent gaze tracking approach for trail making test

Abstract

Trail making test is a cognitive impairment test used for understanding the visual attention during the visual search task. The classical paper pencil method measures the completion time of the participant and there was no mechanism for comparison across the participant with similar feature. The psychologist has to observe the reactions of the participants during the trial process and there is no mechanism to capture it. This study made an attempt to resolve the above problem and tried to infer additional parameters which can support psychologist to understand the participant performance in trail making test. The insight provided by the approach is to extract various features which helps a psychologist by providing individual profiling and group profiling of a person and can understand the group of people who show similar cognitive impairment while performing trail Making Test. The proposed Intelligent Gaze Tracking approach could classify the participant into three different groups like low, high and medium cognitive impairment based on the extracted gaze features. The proposed approach has been compared across existing literature survey to significantly show the advantage of the system in terms of identifying the people with similar characteristics in terms of cognitive impairment.

Keywords

Eye tracking cognitive impairment trail making test area of interest scanpath fixation adaptive neuro fuzzy inference system k-means clustering

1 Introduction

The Trail Making Test (TMT) is a neuropsychological test of visual attention and task switching [1]. It is one of the tests most commonly employed to assess attention and executive functions. It assesses mental flexibility, concentration, visual attention, and motor control. In the test the subject is instructed to connect a set of dots as quickly as possible while still maintaining accuracy. Traditional way of conducting test was using paper and pen method. The test can provide information about visual search speed, scanning, speed of processing, mental flexibility, as well as executive functioning. The cognitive impairment associated with dementia, for example, Alzheimer’s disease can be detected using TMT. Researchers started using this test for assessing cognitive dysfunction. Several cognitive screening tools developed for industry started using TMT to detect cognitive impairment.

Nowadays the life of the people has changed a lot and some people may have difficulty in remembering, perceive new things, focus on something, or making decisions that affect their everyday life [2]. Cognitive impairment varies from mild to severe. The mild impairment itself will start affecting the cognitive functions slowly and the people will lose the ability to understand a task and results in inability to live independently when it became severe impairment.

In this paper the feasibility to assess the trail making test using eye movement (oculomotor) measures has been explored. The eye movement measures are captured using remote eye tracking technology [3]. The eye movements have been tracked and various features like gaze coordinates, pupil diameter, blink frequency etc. will be obtained. Various inference can be made from these features. Since the classical paper pencil method measures the reaction time of the participant, the psychologist has to observe the reactions of the participants during the trial process and there is no mechanism to capture it. The study made an attempt to resolve the above problem and tried to infer additional parameters like error rate, scanpath comparison score, attentional blindness which can support psychologist to understand the participant performance in trail making test. The task performed by the participants were to connect the numbers in sequence. The insight provided by the approach could classify the participant into three different groups like low, high and medium cognitive impairment based on the extracted gaze features and helps a psychologist to understand individual profiling and group profiling of a person. The paper could extract some unique features compared to other TMT methods and it could provide valid inferences for understanding the cognitive impairment of the participants.

The paper is organized as follows: Section II explains various ways of conducting Trail Making Test for assessing the cognitive impairment. Section III presents the role of each extracted feature in identifying the cognitive impairment of a person. Section IV shows the results of applying clustering algorithm and fuzzy rule based system for classification of each participant into three different groups. The experiment has been carried in support with HCG Bengaluru.

2 Related work

There are various cognitive tests to monitor the cognitive functioning of a person. Most of the hospitals use the traditional method of test based on paper-pencil method. Usually standard questionnaires are used for understanding the cognitive functioning. The presence of a psychologist is very much required for analysing the conducted tests. The users can also mark the answers incorrectly for the questionnaires. So if the eye movements have been tracked correctly, it can give strong bio markers for assessing the cognitive functioning.

TMT is a cognitive impairment screening test which gives the examiner more insight to a wide range of cognitive skills and can be completed quickly. The TMT has two parts TMT-A and TMT-B. The time taken to complete the test was considered as the score. TMT-A is used to assess visual search speed and tracking while TMT-B is used to assess complex cognitive abilities and processing speed.

The trail making test was used for assessing the intelligence of a person. From 1950 onwards researchers started using it for understanding the cognitive dysfunction. Now it is widely used as one of the diagnostic tools for assessing cognitive impairment. Mostly the completion time is considered as the metric for evaluation of the test and those who take more time to complete is considered as a poor performer. The cognitive malfunctioning or the brain impairment can be the reason for poor performance.

The TMT has been used in different ways to analyse the cognitive impairment. It has been implemented using various methods like 1. Paper-pencil method 2. Digital TMT 3. Eye Tracking version of TMT.

2.1 Paper pencil method

The Traditional method used for identifying the functioning of brain. Paper pencil version of TMT can be used in various applications like identifying cognitively impaired drivers. TMT helps to identify the drivers who are no longer competent to drive because of illness due to dementia [4].

A study on 134 older drivers could prove that the driving competency of certain drivers has declined to an unsafe level by conducting TMT. 87 healthy and 47 cognitive impaired participants were considered for the study. The drivers whose driving competency has declined to unsafe level could make only erroneous completion of TMT-A and TMT-B.

The measure considered for the evaluation of TMT is total time required for the completion and number of errors. There is a restraint in the number of measures which could be obtained using paper pencil version of TMT.

2.2 Digital TMT

Other than paper pencil method, several variants of digital TMT have been developed. Different digital variants include computerized TMT, Tablet based TMT etc. Usage of several digital TMT could provide more useful information than the traditional paper–pencil TMT. Several unique features like number of pauses, pause duration, lifts, lift duration, time inside each circle, and time between circles can be extracted using digital TMT [5].

Most of the research study could prove that there is a significant correlation between the digital version and paper variant of TMT. Certain methods use writing pressure as a measure to evaluate patient’s higher brain dysfunctions level [6]. Use of tablet PC and smart pencil aid to measure the writing pressure. An iPad Pro is used as smart device, and it reproduces the TMT via that screen and stylus pencil. The writing pressure was considered as the performance measure. The line trajectory and its pressure and solution time were recorded. The line was separated into finding phase and moving phase. The finding phase shows a time to find the next maker, and it is related to attention. The moving phase is also related to distance of makers and physical ability of the user. The system could diagnose the possibility of brain dysfunction by comparing the results patient and healthy people.

Though there exist several digital variants of TMT, they may not augment precise data collection.

2.3 Eye tracking version of TMT

Eye tracking is a technology to track the eyes [7]. The eyes can be considered as the mirror of our mind. Though we tend to mark incorrect answers in the questionnaire, our eyes cannot lie. Eye gaze can be one of the measure which helps to indicate the emotions and mental feelings of a person [7, 8]. The eye movement measures like pupil diameter, blink frequency, fixation frequency, fixation duration, saccade frequency, saccade duration, saccade amplitude, saccade velocity etc. can be measured with the help of an eye tracker [9]. There are certain other parameters like saccade length, saccade slope etc. can be derived from those parameters.

There are certain tools for further analysis of eye movement data like area of interest (AOI), attention map, scan path etc. [10]. Scanpath is a graphical representation, which shows how our eye is physically moving through space [11]. When anyone scan through an image unknowingly they will fixate on some locations and moves to another location. This sequence of fixations and saccades together called scanpath.

Some scanpath representations will have numbers which represents the order in which the fixations happened. Scanpath will be represented using AOI string. Each symbol in the string represents either the fixation or dwells in AOI. The scanpath given in Fig. 1 can be represented as AOI String: 612223445187678. In this example each number within the circle is considered as AOI and given the same number as corresponding AOI name. There are 8 AOIs. It indicates that the first fixation of the scanpath is on the AOI6, then on AOI1, then repeatedly three fixations on AOI2 and so on.

Fig. 1

Scanpath.

A mathematical model has been developed to assess the visual selection process during the execution of a TMT [12]. The study explains the adaptation from paper-pencil method to eye tracking version. The participants were asked to connect the targets by their gaze. They try to evaluate how the top down and bottom up approach work together. The participants were instructed to connect the numbers or alphabets in the order so that it makes cognitive intention (top down) and the scene or stimuli perfection makes bottom up way of selection. They had used stimuli with some distractors and high saliency. The features like saccades, fixation, direction of saccades with respect to fixation were evaluated. From these parameters they could observe the time gap between the respective targets and direction error, which is the difference between the targeted direction and observed direction. Those parameters helped to analyse the TMT.

Since ALS patients are unable to communicate by speaking or writing assessment of their cognitive impairment is a challenging task. An eye tracker enabled cognitive tests help to assess the cognitive functioning of those patients [13].

Paper pencil method was a classical method for evaluation of cognitive impairment. Since it was using only completion time as a measure, psychologist has to spend time in analyzing the behavior of each person. Though there exist digital TMT and eye tracking version of TMT, the proposed model Eye tracking based TMT model could infer additional parameters like error rate, scanpath comparison score, attentional blindness which helps the psychologist to understand the participant performance in trail making test.

3 System overview

The study has ethical committee approval from Health Care Global Enterprises Limited, Bangalore. The data is collected from the employees of the same institute. The data is collected from 31 healthy participants aged between 20 to 54 using SMI REDn Professional eye tracker. Written informed consent was signed by all the participants. The participants were not reported with any psychiatric or neurological illness.

The Trail Making Test (TMT) [5] was selected as the stimulus for the study. It had two parts for this test. The first set, TMT-A had only numbers as the targets, but the second set TMT-B had combination of numbers and characters as targets. The TMT-A had two stimuli. The first stimulus in TMT-A simple had number from 1 to 8 distributed randomly and the second stimulus TMT-A complex had numbers from 1 to 25 distributed randomly, shown in Fig. 2.

Fig. 2

Trail Making Test-A stimulus [5].

The second set TMT-B also had two stimuli. The first stimulus in TMT-B simple had numbers from 1 to 4 and alphabet from A to D and all distributed randomly. The second stimulus TMT-B complex had numbers from 1 to 12 and the alphabet from A to L distributed randomly, shown in Fig. 3.

Fig. 3

Trail Making Test-B Stimulus.

The TMT-A stimulus is used for understanding the speed in cognitive processing. The TMT-B stimulus is used for examining the attention and control cognitive flexibility of the user. TMT-A is generally used to test their visual search ability and motor skills. TMT-B is used to test their mental flexibility and high level cognitive skills [14]. Participants were asked to connect the numbers visually and asked to tell the numbers loudly while they visit each number.

Usually the TMT was conducted using paper and pencil method, where the participant will be asked to connect the numbers in ascending order in TMT-A and numbers and the alphabet combination in ascending order like 1A2B3 C etc. in TMT-B using the pencil. When the participant makes an error it will be corrected by the instructor who conducts the experiment and the participant proceeds after correcting it. The total time taken for visiting and connecting all the numbers is considered as the only performance metric. It cannot record the error rate however it was assumed that the error rate will be reflected in the total completion time.

The system architecture is shown in Fig. 4. It is the comparison of proposed Intelligent gaze tracking model with paper pencil version of TMT. In the proposed model, each stimulus was displayed to the user and collected the data using eye tracker. The data is also collected from all the participants using paper pencil method of the TMT. The eye tracking version of the TMT could extract the features like Dwell Time [ms], Normalized Dwell [ms/Coverage], First Fixation Duration [ms], Glances Count, Total Revisits to AOI, Fixation Count, Dwell Time [% ], Fixation Time [ms], Fixation Time [% ] and Average Fixation Duration [ms]. From all these features, more inference could be obtained by calculating i) The error rate, the number of times they missed the target while connecting the numbers, ii) attentional blindness-Participant may be fixating on the target, however mentally they may not perceive it, iii) the intermediate time gap between reaching the targets iv) the total completion time and v) scanpath comparison score. All these features were clustered using k-means clustering algorithm. The validity of the clusters has been analysed using silhouette plot and the ratio of between_SS and total_SS has also been observed and verified.

The clustered data has been given to Adaptive Neuro Fuzzy Inference system (ANFIS) model and generated fuzzy rules. These rules have been used to identify the levels of cognitive impairment which helps the psychologist to understand the personal profiling of a person. With the proposed model the TMT can be performed without a psychologist and can provide their cognitive impairment level. The algorithm of the proposed model is given below.

Display the eye tracking version of TMT stimuli to the participants

Collect the gaze data

Extract the features and select the required features

Extract the derived features which can give more inference to understand the cognitive impairment

Apply k-means clustering algorithm and cluster the data.

Clustered data has been given to ANFIS model and generate fuzzy rules

Based on the generated rules, identify the level of cognitive impairment

Fig. 4

Comparison of architecture of proposed Intelligent gaze tracking model with paper pencil versions of TMT.

The paper pencil method considers only the completion time as a measure and if it is above a specified threshold value, the participant will be classified as a person with high cognitive impairment. While conducting the test the psychologist has to carefully observe the participant for getting other inferences. It needs the presence of a psychologist throughout the conduct of the experiment.

The various inferences observed from the gaze features obtained using eye tracking version of TMT are listed.

i) The error rate – The participant is asked to visit the targets in ascending order. As they visually focus on each target, they will be fixating on that and it makes an eye measure fixation on that target.

While visiting the targets they may miss the targets in the specific order. That is considered as the error and the error rate can be found out by calculating the number of times they missed the target based on the equation Eq. (1). On the collected data from 31 participants, the error rate was calculated and shown in Fig. 5. $Error Rate = \sum Missed Targets$ (1)

Fig. 5

Error rate of each participant.

The participants 15, 19 and 22 had missed target for 20 times, where participants 4 and 8 missed only once while performing the TMT. The traditional paper and pencil method cannot bring out the error rate in the experiment. It can give only the total time taken for completing the entire task as shown in Fig. 5. While viewing the target a visual pattern i.e. scanpath will be generated for each person. This visual pattern will be different for each person. Figure 6(a) and (b) shows the scanpath with more and less error rate.

Fig. 6

(a) Scanpath of participant with more error rate (b) Scanpath of participant with less error rate.

ii) Attentional Blindness-Considering the scanpath, it has been observed that though the participant has visited the target, they are not able to perceive it or mentally aware of it and found to be searching for the target known generally as attentional blindness [15]. Considering the scanpath string 24111233445676787882 obtained from a participant, it has been observed that the participant had fixated on the AOI 6 and since he was mentally unaware of it, he visited AOI 7 and again came back to 6 and continued visiting. The same participant has again fixated on AOI 7 and unknowingly again searching for 7. So after visiting AOI 8, he has come back to AOI 7 and continued visiting.

iii) The intermediate time gap between reaching the targets -The Fixation duration, the time spent on each AOI and time to reach each AOI has been extracted. Based on the time taken to reach each AOI, it could understand which AOI was easily reachable and which was taking more time to reach as shown in Table 1. Considering reaching each AOI as different task and based on the time to reach each AOI, the task can be classified as easy, average and difficult task and how each participant looked at those tasks can be analysed.

Table 1

Time to reach each AOI

Participant	Total reaction time	Time to reach AOI1 (ms)	Time to reach AOI2 (ms)	Time to reach AOI3 (ms)	Time to reach AOI4 (ms)	Time to reach AOI5 (ms)	Time to reach AOI6 (ms)	Time to reach AOI7 (ms)	Time to reach AOI8 (ms)
P12	7096	427.8	333.3	1566.4	366.6	599.9	700.5	232.7	233.4
P13	12556.8	6291	2533	433.5	333	533.3	666.6	166.6	200
P14	5180.5	747.8	1166.4	200	0	333.4	1133.1	399.9	266.7
P15	6874.3	440	866.9	766.5	233	1166.8	866.2	899.9	499.9
P17	11309.9	2477.9	4599.2	366.9	199.6	766.6	1033.4	333.3	233
P18	7327.7	2695	767.1	766.3	333	299.9	1000	133.2	200.1

iv) The total completion time- According to paper pencil version of TMT, total completion time was considered as the important feature to classify the users into cognitively impaired or not. The total completion time is extracted from eye tracking features. Figure 7 shows total time taken by each participant for completing the entire task in millisecond.

Fig. 7

Total time taken by each participant.

v) Scanpath comparison score- The analysis of scanpath of each participant can bring out more inference about the cognitive processing of a person. Scanpath is considered as the visualisation of the viewing pattern [16]. It is the connection of fixations in the order in which the fixations have been formed while viewing any stimulus. It can show various information like fixation, fixation duration, re-fixation, saccade, saccade duration, backtracking, regression etc. This information can give more inference for various applications and study.

Since TMT has been used to detect the cognitive impairment of a person, the various ways of comparison of scanpath can bring more inference to it. The scanpath string for each scanpath is the sequence of fixations on each AOI. Scanpath comparison helps to understand the commonality in viewing behaviour both within and between participants [17]. The eye pattern, an open source software platform had been used for comparison of scanpath of each participant [18]. Sequence comparison has been used in bioinformatics to align DNA and for the comparison of protein sequences. In eye movement research, the sequence comparison has been used to compare pairs of scanpath comprising AOI strings. There are different principles for the comparison of scanpath like string edit distance, scanmatch etc. The string edit distance or Levenshtein distance is calculated as the minimum insert, delete and substitute operations needed to transform one string into another. A smaller distance indicates both the scanpaths are more similar.

Considering the following example,

String1 : 1 2 3 4 5 6 7 8

String2 : 1 2 3 5 4 6 7 8

Here in String2, 5 need to be substituted with 4 and 4 with 5. This gives a total edit distance of two for the comparison of string1 and string2.

Consider string3 as 1 2 8 7 6 5 4 3, last six characters need to be substituted and it makes edit distance as six. So this pair of scanpath is three times more dissimilar than the first pair. The string edit distance has been calculated across all the scanpath strings generated by each participant. The similarity among five participants’ scanpath string is shown in Fig. 8. The Levenshtein similarity matrix shows the edit distance between two scanpath strings.

Fig. 8

Levenshtein similarity matrix.

Consider the scanpath strings given in Table 2. The edit distance calculation between the string S1 and S5 are shown in Fig. 9. Sequence alignment is done for the comparison of scanpath strings with unequal string length. Substitutions and gap insertions are required for the alignment of the scanpath string. Gaps or blank spaces have been introduced at locations where matching between the elements has not been possible. Gap penalty is paid for every such unmatched element.

Table 2

Scanpath string

String name	Scanpath string
S1	612345187678
S3	82872187868714234587678
S4	621434523435458768781
S5	612345678

Fig. 9

Edit distance calculation among the scanpath string.

Considering the participant with scanpath shown in Fig. 6(b) as a perfect visual search pattern, the participant who is having edit distance closer to that can be considered as a participant with no cognitive impairment and who has edit distance far away from that can be considered as a participant with cognitive impairment.

ScanMatch is another method used for comparing fixation sequences based on Needleman Wunsch algorithm [19]. It gives the score by comparing scanpath strings by providing score for letter substitutions and penalty gap.

From the obtained edit distance of each participant, a scanpath comparison score has been generated. The score is calculated based on the distance to a perfect scanpath. The score indicates how much it is closer to a perfect scanpath. The value closer to 1 indicates perfect scanpath i.e. the participant could visually connect the targets without any error.

The extracted features like error rate, attentional blindness and scanpath comparison score using the eye tracking based TMT could give more inference than the digital TMT or paper pencil method. These features could only be extracted using the eye tacking techniques and played a major role in classifying the users into three categories: low, high and medium cognitive impairment. These three features can only be extracted based on the visual pattern generated by participant while viewing the stimuli and it is not possible to extract using digital or paper pencil method.

4 Result analysis

Thirteen gaze features like error rate, Scapath comparison score, total completion time, Dwell Time [ms], Normalized Dwell [ms/Coverage], First Fixation Duration [ms], Glances Count, Total Revisits to AOI, Fixation Count, Dwell Time [% ], Fixation Time [ms], Fixation Time [% ], Average Fixation Duration [ms] etc has been extracted. On the extracted features, k-means clustering algorithm has been applied and classified the participants into three different groups, cluseter1, cluster2 and cluster3 with the participants who have low, high and medium cognitive impairment respectively based on the similarities in their characteristics in terms cognitive impairment. The clustering algorithm is applied for the features extracted for all the four stimuli TMT-A simple, TMT-A complex, TMT-B simple, TMT-B complex.

In clustering, the inter cluster similarity should be less and intra cluster similarity should be more, then it has been considered as a perfect clustering. Within_SS will give the similarity within the cluster and the high score indicates high similarity within the cluster. Between_SS indicate the similarity across the clusters and the low score indicate less similarity across the clusters. In optimal clustering, since the clusters are very different from each other, then most of the total variance is explained by the variance between the groups. And of course, since the variance within each group is very small, it would explain only a small fraction of the total variance in the data. The ratio between_SS/total_SS if it is nearing 100%, then it indicates that it is a perfect clustering. The result of cluster similarity for each stimulus is shown in Table 3. The gaze features obtained from TMT-B complex could cluster the participants into an appropriate group.

Table 3
Cluster validity for each stimuli

Stimulus Between_SS/total_SS

TMT-A simple 80.2

TMT-A complex 84.7

TMT-B simple 82

TMT-B complex 86.2

Stimulus	Between_SS/total_SS
TMT-A simple	80.2
TMT-A complex	84.7
TMT-B simple	82
TMT-B complex	86.2

Silhouette analysis shows the visualization of how much similar are the observations within the cluster relative to other clusters. Values close to 1 indicate that the observations are perfectly suitable to the assigned cluster.

Values close to 0 indicate that the observation has the chance of fitting into any of the cluster. Values close to – 1 indicate that the observations are allotted to the wrong cluster. The Silhouette plot of the obtained cluster is shown in Fig. 10.

Fig. 10

Silhouette plot of the obtained clusters.

The demographic details of all the 31 participants has been obtained and shown in Fig. 11. There were 17 female and 14 male candidates. All the participants’ demographic details were also considered for a better classification.

Fig. 11

Demographic details of each participant.

The details like age, education, gender, vision were considered. The education details of employees were considered as well educated, average and less educated. The participants were classified into 4 groups based on their age like 20–29, 30–39, 40–49, 50–59.

The participants have been clustered into three classes based on the obtained eye tracking features. The impact of demographics details on each cluster has been analysed and shown in Fig. 12.

Fig. 12

Visualization of demographic details on the clustered data.

It has been observed that the education and age of each participant has a major role in classifying them into various clusters. All the well-educated participants except one has been classified into the same cluster. Most of the well-educated participants have been classified as cluster1 i.e. participants with low cognitive impairment, most of the less educated participants have been classified as cluster2 and average educated participants have been classified as cluster3. The participants with age group 50–59 has been classified into cluster2, participants with high cognitive impairment.

Individual profiling of participant in all the three groups were analysed across the stimuli and observed almost same characteristics across the stimuli. The extracted features of participants p1, p19, p3 across various stimuli from cluster 1, cluster 2 and cluster 3 respectively are shown in Fig. 13.

Fig. 13

Individual profiling of participants in each clustered group.

The clustered data has been given as input to ANFIS model [20] and generated fuzzy rules as shown in Table 4. The trapezoidal membership function has been used for generating fuzzy rules. The names of the linguistic terms on each input variable is given as very very large(VVL), very large(VL), large(L), medium(M), small(S) and very small(VS), very very small(VVS). The participants have been classified as three groups low, high and medium cognitive impairment and it could predict all the clustered data correctly [21]. The clustered participants’ details are shown in Fig. 14. The rules assigned for each features Error rate, Scapath comparison score, total completion time (sec), Dwell Time [ms], Normalized Dwell [ms/Coverage], First Fixation Duration [ms], Glances Count, Total Revisits to AOI, Fixation Count, Dwell Time [% ], Fixation Time [ms], Fixation Time [% ], Average Fixation Duration [ms] and the corresponding prediction classes are shown in Table 4 as Error rate, Scan_Com_Score, Total Comp_Time (sec), Dwell Time (ms), Norm_Dwell time (ms/coverage), First Fix_ Duratn (ms), Glan_Count, Revisit, Fix_Count, Dwell Time (%), Fix_Time(ms), Fix_Time (%), Avg_Fix_duratn(ms), cls.

Table 4

Tabular representation of fuzzy rule base system

Error Rate	Scan_Com_Score	Total Comp_Time (sec)	Dwell Time (ms)	Norm_Dwell time (ms/coverage)	First Fix_Duratn (ms)	Glan_Count	Revisit	Fix_Count	Dwell Time (%)	Fix_Time (ms)	Fix_Time (%)	Avg_Fix_duratn (ms)	cls
VVL	VVL	S	S	M	S	M	M	M	VL	S	VL	S	1
V	VS	VS	VS	VVS	S	VVS	VVS	VS	VVL	VS	VVL	S	3
VS	VS	VS	VS	VS	M	VVS	VVS	VS	VVL	VS	VVL	M	3
VS	VS	VS	VS	VS	S	VS	VS	VS	VL	VS	VVL	M	3
L	L	L	L	M	VL	M	M	M	VVL	L	VVL	VL	1
L	L	VS	S	L	L	S	S	S	VL	S	VVL	VL	2
VVS	VVS	VS	VS	VS	M	VS	VS	VS	VVL	VS	VVL	L	3
VVL	VVL	M	S	M	VL	M	M	S	VVL	M	VVL	M	1
L	L	VS	VVL	L	M	M	M	L	VVL	VVL	VVL	L	2
VS	VS	VVS	VS	S	S	VS	S	VS	VL	VS	VVL	M	3
VVS	VVS	VVS	VVS	VS	VVL	VVS	VVS	VVS	VVL	VVS	VVL	VVL	3
M	M	VS	VS	S	S	S	S	S	VL	VS	VL	M	2
VS	VS	M	S	S	M	VS	VS	S	VL	S	VL	S	3
S	S	VS	VS	VS	L	VVS	VVS	VS	VL	VS	VL	L	3
S	S	S	S	S	M	M	S	M	L	S	L	M	3
S	S	S	L	VVL	VL	M	M	M	VVL	L	VVL	VL	2
VVL	VVL	S	VL	VVL	M	VVL	VVL	VVL	VL	VL	VL	M	1
M	M	VS	M	S	M	VS	VS	S	VL	M	VL	VL	2
L	L	S	L	VVL	M	VVL	VVL	VL	L	L	L	M	2
M	S	S	S	VVS	VVS	VS	VS	S	VVS	VS	VVS	VVS	2
VVL	VVL	M	L	L	M	M	M	L	VL	L	VL	L	1
VS	S	VVL	M	S	S	VS	VS	S	M	M	M	L	3
M	M	S	VS	VS	VS	S	S	S	M	VS	M	VS	2
M	M	VS	L	M	VL	S	S	M	VVL	L	VVL	VL	2
M	M	VS	M	M	M	M	M	M	VL	M	VVL	M	2
M	M	M	L	L	L	M	M	L	VL	L	VL	L	2
VS	S	VS	VVL	L	VVL	L	L	L	M	VVL	M	VVL	3
L	L	S	M	S	S	M	M	L	L	M	L	S	2
M	M	M	M	S	M	S	S	M	VVL	M	VVL	S	2
M	M	S	M	S	S	S	S	M	VL	M	VL	S	2
S	S	S	L	L	M	M	M	M	VVL	L	VVL	L	2

Fig. 14

K-means clustering on the gaze features.

The data which is also collected using paper pencil method of TMT is analysed. The completion time was the only data collected using paper pencil method and the completion time of each participant showed good correlation with the same feature extracted using the eye tracking version of TMT.

5 Conclusion and future enhancement

The proposed Intelligent gaze tracking system could infer various additional features like error rate, scanpath comparison score, attentional blindness which can help a psychologist to understand the cognitive impairment of a person while performing trail making test. The data has been collected from 31 participants for TMT-A simple, complex and TMT-B simple and complex. The collected data has been clustered and given as input to an ANFIS model. The ANFIS model could generate the fuzzy rules. It could find each participant’s cognitive impairment level and classify them as low, high and medium cognitive impairment. The model helps the psychologist by providing individual and group profiling of a person compared to the normal paper pencil method.

In future, model can be developed as a screening tool for most of the cognitive impairment diseases like Parkinson’s disease, Alzheimer’s disease etc. which helps a psychologist to understand the cognitive impairment level of a person. Model can be improved by giving more training which helps to achieve better accuracy.

References

MacPherson

S.E.

, Allerhand

, Cox

S.R.

and Deary

I.J.

, Individual differences in cognitive processes underlying Trail Making Test-B performance in old age: The Lothian Birth Cohort 1936, Intelligence75 (2019), 23–32.

Olazaran

, Hoyos-Alonso

M.C.

, Del Ser

, Barral

A.G.

, Conde-Sala

J.L.

, Bermejo-Pareja

and Lopez-Pousa

, Practical application of brief cognitive tests, Neurologia (English Edition)31(3) (2016), 183–194.

Venugopal

, Amudha

and Jyotsna

, Developing an application using eye tracker, International Conference on Recent Trends in Electronics, Information & Communication Technology, pp. 1518–1522, IEEE 2016.

Dobbs

B.M.

and Shergill

S.S.

, How effective is the Trail Making Test (Parts A and B) in identifying cognitively impaired drivers?, Age and ageing42(5) (2013), 577–581.

Fellows

R.P.

, Dahmen

, Cook

and Schmitter-Edgecombe

, Multicomponent analysis of a digital Trail Making Test, The Clinical Neuropsychologist31(1) (2017), 154–167.

Mabuchi

, Kubota

, Takeda

, Manji

and Matsuda

, A writing pressure analysis method for evaluation of trail making test using smart device, 2017 6th International Conference on Informatics, Electronics and Vision & 2017 7th International SymposiuminComputationalMedical and Health Technology (ICIEV-ISCMHT). IEEE, 2017.

Jyotsna

and Amudha

, Eye Gaze as an Indicator for Stress Level Analysis in Students, 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI). IEEE, 2018.

Ren

, Barreto

, Huang

, Gao

, Ortega

F.R.

and Adjouadi

, Off-line and on-line stress detection through processing of the pupil diameter signal, Annals of Biomedical Engineering42(1) (2014), 162–176.

Shojaeizadeh

, Detecting task demand via an eye tracking machine learning system, Decision Support Systems116 (2019), 91–101.

10.

Xia

, Predicting Human Saccadic Scanpaths Based on Iterative Representation Learning, IEEE Transactions on Image Processing28(7) (2019), 3502–3515.

11.

Chandrika

K.R.

and Amudha

, A fuzzy inference system to recommend skills for source code review using eye movement data, Journal of Intelligent & Fuzzy Systems34(3) (2018), 1743–1754.

12.

Veneri

, Pretegiani

, Rosini

, Federighi

, Federico

and Rufa

, Evaluating the human ongoing visual search performance by eye tracking application and sequencing tests, Computer methods and programs in biomedicine107(3) (2012), 468–477.

13.

Shravani

, Ramya Sai

, Vani Shree Amudha

and Jyotsna

, Assistive Communication Application for Amyotrophic Lateral Sclerosis Patients, International Conference On Computational Vision and Bio Inspired Computing, ICCVBIC 2019 (Presented).

14.

Waldecker

, Talbot

and Turner

M.R.

, An eye-tracking version of the trail-making test, PloS one8(12) (2013), e84061.

15.

Berkovitch

, Del Cul

, Maheu

and Dehaene

, Impaired conscious access and abnormal attentional amplification in schizophrenia, NeuroImage: Clinical18 (2018), 835–848.

16.

Blascheck

, Kurzhals

, Raschke

, Burch

, Weiskopf

and Ertl

, State-of-the-art of visualization for eye tracking data, In Proceedings of EuroVis2014 (2014).

17.

Anderson

N.C.

, Anderson

, Kingstone

and Bischof

W.F.

, A comon of scanpath comparison methods, Behavior research methods47(4) (2015), 1377–1392.

18.

West

J.M.

, Haake

A.R.

, Rozanski

E.P.

and Karn

K.S.

, eyePatterns: software for identifying patterns and similarities across fixation sequences. In Proceedings of the 2006 symposium on Eye tracking research & applications (pp. 149–154). ACM, 2006.

19.

Cristino

, Mathot

, Theeuwes

and Gilchrist

I.D.

, ScanMatch: A Novel Method for Comparing Fixation Sequences, Behaviour Research Methods42 (2010), 692–700.

20.

Amudha

, Radha

and Smitha

, Analysis of fuzzy rule optimization models, Int J Eng Technol7(5) (2015), 1564–1570.

21.