Multi gait recognition using Clustering based Faster Regions-Convolutional Neural Network

Abstract

Gait recognition is the process of recognizing a person based on their walking style. Each person’s walking gait is distinctive and cannot be imitated by others. However, the walking motion of a person will be changed based on their behaviour but their walking pattern doesn’t change. In this paper, a novel Clustering based Faster RCNN has been proposed to identify the single, double and multi-gait. The gait images from the publicly available dataset are pre-processed using Multi scale Retinex (MSR) to reduce the noise artifacts. The Faster RCNN is used for extracting the relevant features from the gait images via the two modules namely CNN and RPN. The CNN layers extract the most relevant features as feature maps and RPN is used for creating the bounding boxes for the extracted features. Fuzzy K-means clustering is used to group the features based on their labels, and it specifies the features acquired using CNN and RPN as input. Finally, the Fast RCNN is employed for classifying the gait images into suspicious and non-suspicious walking pattern. The proposed Clustering based Faster RCNN net achieves the high accuracy rate of 98.74% and 99.19% for suspicious and non-suspicious walking pattern respectively. The proposed Clustering based Faster RCNN model was compared with other traditional models like CNN, U-net, Fab net and Fast R-CNN. The proposed Clustering based Faster RCNN model improves the overall accuracy of 8.86% 33.77% 3.12% and 5.48% better than mmGait, LSTM Net, STDNN and RNN respectively.

Keywords

Gait recognition deep learning faster R-CNN fuzzy K-means clustering multi scale Retinex

1 Introduction

A person’s biometrical characteristic can be categorized into behavioral or physiological characteristics. Biometric features, like the iris, ears, passwords, face, tokens and finger prints, require highly exact detection and a well-managed human interaction to be effective [1]. Gait recognition, which refers to recognizing people by their walking style, can be particularly challenging due to variations in the camera’s point of view and people’s face expressions [2]. People’s appearance might also change as a result of changes in their attire, such as donning a hat or coat or carrying a backpack and handbag [3]. Even though research on gait detection has recently increased, learning discriminative temporal representation is still difficult because the distinctions between silhouettes in the spatial domain are extremely slight [4]. In low-resolution recordings, where other biometrics systems might not work because of a lack of pixels, gait recognition has a great deal of potential for identifying people [5].

When a person is walking with other people, their walking behaviour changes, and when several people are walking together, it becomes more difficult for conventional gait recognition methods. This task has mainly used in security surveillance, such as the identification of suspicious people or activities and the authentication of people [6]. Model-free techniques and model-based techniques are the two basic categories that can be used to divide gait recognition from the perspective of computer vision [7]. Tracking or modeling body parts like thighs, arms, legs and limbs allows model-based techniques to get a number of dynamic or static body characteristics. Gait signatures created using these model parameters are used to recognition and identification of people [8].

A faster R-CNN extracts and detects the ordinary in every frame of the video regardless of whether the pedestrian carries anything or not [9]. RPN and CNN are two different variants of Faster RCNN that have been used to analyze the attribute vectors created from different walking patterns in order to identify the walking styles [10]. Gait recognition is a technique for identifying people based on how they walk. Each person’s walking gait is distinctive and cannot be imitated by others. However, it is complicated to find out the person using their silhouette because the reflection may vary based on the light direction. The walking motion of a person will be changed based on their behaviour but their walking pattern doesn’t change. The main purpose of the study is as follows:

A novel Clustering based Faster RCNN has been proposed to identify the single, double and multi-gait.

The gait images from the CASIA-B gait dataset are pre-processed using Multi Scale Retinex is used to reduce the noise artifacts.

The Faster RCNN is used for extracting the relevant features from the gait images via the two modules namely CNN and RPN.

The CNN layers extract the most relevant features as feature maps and RPN is used for creating the bounding boxes for the extracted features.

Consequently, Fuzzy K-means clustering uses the features acquired via CNN and RPN as input to group the features according to their labels.

Finally, the Fast RCNN is employed for classifying the gait images into suspicious and non-suspicious walking gait based on the clusters from the clustering algorithm.

The remainder of the analysis is divided into the following sections: Section II the literature review is thoroughly explained. Section III the proposed method is described in detail. Section IV the results and discussion part are covered. Section V the conclusion is discussed.

2 Literature review

In recent years, many studies for gait images are pre-processing, feature extraction and classification strategies have been discussed by the researchers. In this section, a review of a few recent studies is provided.

In 2021 chen. X., et al., [11] had proposed a false gait sample can be added to existing gait datasets using a Multi-view Gait Generative Adversarial Network, providing useable gait samples for cross-view gait detection methods based on deep learning. The MvGGAN technique improved the ability of gait organization models to be simplified. A comparison of the suggested program’s recognition accuracy when only genuine samples under 90 are offered, both with and without fake samples.

In 2020 Meng. Z., et al. [12] had proposed the gait of 95 participants would be “seen” by two mmWave radars in two different environments over the course of about 30 hours in this first-of-its-kind mmWave gait data collection. In contrast to traditional camera-based methods, mmWave-based gait identification has the distinct benefit of maintaining effectiveness in non-line-of-sight situations, such as in the absence of light in the presence of obstructions. For instances involving a single person, the mmGaitNet can reach 90% accuracy, while for cases involving five coexisting people, it can achieve 88% accuracy.

In 2020, Makihara, Y., et al. [13] had suggested a network to evaluate the phase of the single input image, and the network to reconstruct the gait cycle uses the calculated phase. This reduced an encoded feature’s dependence on the phase of that specific image. An example of this method is a phase-aware gait cycle reconstructor. A good trade-off between recognition and reconstruction accuracy is achieved by concurrently optimizing the recognition and PA-GCR networks.

Fig. 1

The overall process of the proposed methodology.

In 2021 Zhao, X., et al., [14] had suggested PoseGait, a model-based gait recognition technique. Gait recognition is a stimulating yet exciting task in biometrics. RNN/LSTM nor CNN can effectively extract spatio-temporal gait feature with the help of combining two losses. The best recognition rate for the pose feature is 60.92% if there is no difference.

In 2019 Zhang, Z., et al., [15] had proposed the use of an autoencoder-based network and a multi-layer LSTM called GaitNet with innovative loss functions to clearly separate pose features from visual appearance. Frontal-View Gait dataset with an effort on frontal-view walking gait identification. This network outperforms our approach by 81.2% on the all procedure of the FVG dataset, achieving a recognition accuracy of 65.4% TDR at 1% FAR.

In 2019 Singh, J.P., et al., [16] had suggested the dataset is divided into two categories i.e., Single-Gait (SG) same subjects walk alone and Multi-Gait (MG) subjects walk in a group. This dataset’s goal is to examine how a person’s stride changes depending on whether they walk with others or by themselves. So, they created a new database which focused on multi-Gait occlusion situation and the CAISA-B dataset achieved 75% accuracy.

In 2018 Gadaleta, M. and Rossi, M., [17] had unveiled IDNet, a user authentication system that makes advantage of motion data gathered by smartphones. A smartphone with a trustworthy block for extracting the walking cycle independent of direction, walking cycles of these into a multi-stage authentication method, and one-class SVMs to classify the coherent integration. In less than five walking cycles, IDNet produced misclassification rates that were less than 0.15 percent.

In 2018 Tong, S., et al. [18] had suggested a multi-view spatial-temporal deep neural network (STDNN) for recognizing gaits. A temporal feature network and spatial feature network a make up the STDNN (SFN). In order to establish the identity of the person, an innovative approach is used to integrate the recognition scores obtained by SFN and TFN. The highest recognition scores were 95.67% for STDNN.

In 2017 chen, X., et al. [19] had proposed that human graphlets be used in a tracking-by-detection method to derive an individual’s whole silhouette. After that, a latent conditional random field (L-CRF) model creates features that are consistent and discriminatory. A new constriction is added to the model’s training in the latent structural support vector machine architecture in order to enhance the concert of multi-gait recognition on the OU-ISIR, this technique yields a rate of 94.7%.

In 2020 Hasan, M.M. and Mustafa, H.A., [20] had proposed a RNN network to effectively train the network for reliable gait recognition by devising a number of techniques and a GRU architecture. They designed a unique gait feature descriptor that is invariant to covariate factors and achieved comparable performance to techniques that call for expensive 3D poses or the calculation of the gait energy image (GEI). This method achieved average development of approx 10% with CCR of 93.34% from PSTN.

Due to the presence of several covariate elements, such as different perspective angles, changes in clothes, walking speeds, load carriage, etc., the classic approaches for distinguishing individual people from gait remain a difficult topic in computer vision research. The proposed clustering based Faster RCNN method mainly focused on identify the single, double and multi-gait.

3 Clustering based Faster R-CNN methodology

In this section, a novel Clustering based Faster RCNN has been proposed to identify the single, double and multi-gait. In the pre-processing stage, Multi Scale Retinex is used to reduce the noise artifacts. The Faster RCNN is used for extracting the relevant features from the gait images via the two modules namely CNN and RPN. The CNN layers extract the most relevant features as feature maps and RPN is used for creating the bounding boxes for the extracted features. Consequently, Fuzzy K-means clustering uses the features acquired via CNN and RPN as input to group the features according to their labels. Finally, the Fast RCNN is employed for classifying the gait images into suspicious and non-suspicious walking gait based on the clusters from the clustering algorithm.

Table 1
Dataset description

Classes No. of images Total

Normal 450 1364

With coat 447

With bag 467

Classes	No. of images	Total
Normal	450	1364
With coat	447
With bag	467

In this paper, the CASIA-B dataset has been used. This dataset contains a total 124 subjects and 110 sequences per subject. There are 10 sequences for each of the 11 viewpoints (0°, 18°, 180°). The walking condition consists of five sequences of normal walking (N), three sequences of walking with a bag (WB), and two sequences of walking while wearing a coat or jacket (WC) for each subject. For example, there are 11×(5 + 3+2)=110 sequences for each subject. Totally, 1364 images are available in CASIA-B dataset.

3.1 Image pre-processing

Pre-processing is an important role for reducing the noise and enhancing the various changes in gait images. SSR is extended by MSR which is a colour constancy and dynamic range compression-based image processing algorithm that is inspired by human perception. The MSR algorithm, which is a linear weighted summary of the single-scale Retinex method, is expressed exactly in the following formula. $\begin{matrix} N_{j} (y, z) = \sum_{k = 1}^{M} W_{K} \\ {{logJ}_{j} (y, z) - \log [F_{k} (y, z) * J_{j} (y, z)]} \end{matrix}$ (1) where W_k is the weight of various scales, K is a representation of the wrapping function’s scales, satisfying $\sum_{k = 1}^{M} W_{K} = 1$ , and the general value is 3, which represents the wraparound function at scale C_K, represents three scales of large, medium and small F_k (y, z). Generally speaking, guaranteeing with the advantages of high, medium and low three dimensions, the value of K is usually 3, and W1 = W2 = W3 = 1/ 3, in addition, the experimental results show that taking 15, 80,120 respectively. After dealing with the low illumination image enhancement for the original color video image. The surround function is given by: $F_{m} (y, z) = K_{m} e^{- (y^{2} + z^{2}) / c_{n}^{2}}$ (2)

Where the normalization factor K_m = 1/(∑_y∑_zF (y, z)) and c_n are the scales that control the extent of the surround. The number of scales required, the weight values and the scale values, the three obvious questions about MSR. The experiments have given that three scales are enough for most of the weights, and images can be equal. The Common static scales include 15, 80, and 250. As a result of not knowing how an image scales up against actual scenes, these are more experimental than theoretical. The weights can be changed to provide more emphasis on either colour rendition or dynamic range compression.

3.2 Clustering based Faster RCNN model

The proposed model focused on Clustering based Faster RCNN net to identify the single, double and multi-gait. The clustering based Faster RCNN model is an integration of clustering for cluster the features based on their labels and the Faster RCNN is used for extracting the relevant features from the gait images via the two modules namely CNN and RPN.

3.2.1 Faster RCNN

The Faster RCNN is used for extracting the relevant features from the gait images via the two modules namely CNN and RPN. The CNN layers extract the most relevant features as feature maps and RPN is used for creating the bounding boxes for the extracted features. First, image feature maps are retrieved as one of the object detection approaches using a set of basic relu with pooling layers and conv using Faster R-CNN. The fully linked layers and RPN layers have the same feature maps. CNN has both a pooling layer and a convolution layer. The structure of clustering based Faster RCNN is shown in Fig. 2.

Fig. 2

Clustering based Faster RCNN.

3.2.1.1 Convolution Neural Network In Faster-CNN, our proposed network allocates several ROIs of comparable sizes on various feature map levels. The selection of the appropriate ROI to obtain the required information will be complicated by these anchors. Therefore, multi-scale anchors do not need to be specified at each feature level. On each feature map, it is standard procedure to employ a single scale anchor. For example, the size of the anchors on P2, P3, P4, and P5 is 82, 162, 322 and 642, while the multiple aspect ratio is still 1 : 2, 1 : 1, and 2 : 1. In the original image, each level feature map pixel will consequently produce 3 anchors. However, the majority of these anchors continue to be incorrect and have no ROI. The input feature maps and proposals can be obtained by employing a ROI pooling layer.

3.2.1.2 Region Proposal Network The RPN was used to establish regions of interest with anchors for each feature maps. Convolutional feature maps are layered with a small network to build the proposal, which produces a number of rectangular object proposals with an objectness score. To prevent filter or image pyramids, the concept of an anchor box is introduced. The mapping of each region to each reference anchor box that follows enables the detection of objects at various scales. The sliding window is the focus of an anchor that is connected to scales and aspect ratios. In this way, the foreground and background values of the gait image were determined. By using RPN predictions, the anchor chose optimum boundaries for the gait images and fine-tuned their size and position. If many anchors overlap, capture the anchors with the greatest foreground value and discard the others after obtaining the final regional proposal and passing it to ROI Align.

Loss function:

The Faster RCNN is created by combining the Fast RCNN and RPN detector modules. Bounding box regression loss and classification loss make up the total multi-task loss function as defined in Eq.3 with L_cls and L_reg functions defined in Eq. (4) and (5). $L_{cls} (q_{j}, q_{j}^{*}) = - q_{j}^{*} \log (q_{j}) - (1 - q_{i}^{*}) \log (1 - q_{j})$ (3) $L_{reg} (t^{v}, u) = \sum_{y ɛ y, z, w, h} L_{1}^{smooth} (t_{j}^{v} - u)$ (4) $L_{1}^{smooth} (p) = {\begin{matrix} 0.5 p,^{2} if | p | < 1 \\ | p | - 0.5, otherwise \end{matrix}$ (5)

3.2.1.3 Classification The gait images are classified into suspicious and non-suspicious walking gaits using the Fast RCNN and the clusters from the clustering technique. To expedite the training procedure, Fast R-CNN has been used in place of the traditional technique of detecting regions and running CNN. The region proposals are developed using a tiny sliding window that covers the whole convolution feature map of output layer. Another name for the little sliding window is a lower dimensional feature. The classifier and regression layers, which are both completely connected, are split into these features. The faster RCNN is created by combining the RPN and fast RCNN detector modules.

The output of the regressors is a region border with geographic coordinates (q, r, H, W). $M = [\frac{q - q^{a}}{W^{a}}, \frac{r - r_{a}}{H_{a}}, \frac{logW}{W_{a}}, \frac{logH}{H_{a}}]$ (6) $M^{*} = [\frac{q^{*} - q^{a}}{W^{a}}, \frac{r^{*} - r_{a}}{H_{a}}, \frac{{logW}^{*}}{W_{a}}, \frac{{logH}^{*}}{H_{a}}]$ (7)

Where, W^*, H^*, q^*, p^* are the ground truth bounding box centre, width, and height and q_a,p_a, H_a,, W_a are the height, centre, and width of anchor.The argmax switches are utilized by the reverse function of the ROI pooling layer to determine the partial derivative of the loss function with respect to each input variable a_j $\frac{\partial l}{\partial a_{i}} = \sum_{q} \sum_{r} [j = j^{*} (q, r)] \frac{\partial l}{\partial b_{qr}}$ (8)

Where, every mini-batch ROI p and for output unit b_qr, the partial derivative $\frac{\partial l}{\partial b_{qr}}$ is accumulated if ‘i’ is selected as the argmax for b_qr by max pooling. The convolutional procedure can be expressed as, $\begin{matrix} m^{k + 1}, n^{k + 1}, f = \sum_{m = 0}^{128} \sum_{n = 0}^{128} \sum_{f = 0}^{N} F_{m, n, f} \\ \times r_{m}^{k} {k + 1}_{+ m, n} {k + 1}_{+ m, n} \end{matrix}$ (9)

Where, F is the filter bank, q is the image tensor input, f is the filter number, k is the layer number, N number of filters, m and n are the spatial coordinates and r convolution output.

The function of the object identification method was estimated by average precision. $AveragePrecision = 1 / 11 \sum_{recall - n} P ({recall}_{n})$ (10)

Where recall-n = [0, 0.1, 0.2, , 1.0] $\begin{matrix} Loss ({u_{j}}, {v_{j}}) = \frac{1}{N_{c}} \sum {jL}_{c} (u_{j}, u_{j}^{*}) \\ + λ \frac{1}{N_{r}} \sum_{j} q_{j}^{*} L_{r} (v_{j}, v_{j}^{*}) \end{matrix}$ (11) $L_{c} (q_{j}, q_{j}^{*}) = q_{j}^{*} {logq}_{j} - (1 - q_{j}^{*}) \log (1 - q_{j})$ (12)

Where, N_C, N_r normalized weighted parameters and weighted by a balance parameter for the outputs of the r and c layers, u_j j^th an object, v_j vector with four local coordinates for the predicting bounding box and the projected probability of the anchor, I anchor’s index in a mini-batch, L_c classification loss, $v_{j}^{*}$ is a + ve anchor-related ground truth box: the log loss resulting from two classes (object vs not object), $u_{j}^{*}$ is an anchor which indicated by the ground truth label is+ve (-ve) for 1/(0), $u_{j}^{*}$ L_r is a only+ve anchors activate regression loss.

3.2.2 Fuzzy k-means clustering

Fuzzy K-means clustering to group of data based on walking while wear a coat, typical walking and carrying bag. The lower body joints’ gait, however, is primarily changed when walking with other people. Additionally, the maximum accuracy is obtained when all joints are combined. The baseline method, however, does not capture enough dynamic gait information with its 14 joints, resulting in a lower recognition accuracy than our suggested method based on dense trajectories.

In terms of fuzzy K-Means clustering, squared l₂-standard loss is complex due to the huge deviations. Therefore, introduce the proposed clustering model’s adaptive loss function. Given l₁-norm, squared l₂-norm and arbitrary vector m are defined as, $| | g | |_{1} = \sum_{j} | g_{j} | and | | g | |_{2}^{2} = \sum_{j} g_{j}^{2}$ (13)

Generally, the squared l₂-standard loss function is sensitive to large outliers but insensitive to tiny outliers. In contrast, l₁-standard loss is sensitive to minor outliers but insensitive to large outliers. In order to address the aforementioned problem, the adaptive loss function for vector g is introduced. $| | g | |_{σ} = \sum_{j} \frac{(1 + σ) g_{j}^{2}}{| g_{j} | + σ}$ (14) where g_j is the j-th part of the vector x and σ is the adaptive parameter. Figure 3 displays examples of the adaptive loss function under various.

Fig. 3

Example of the adaptive loss function using various.

The suggested approach combines nonnegative spectral clustering and fuzzy K-Means into a single model. To improve the quality of attraction graph, the similarity matrix is created specially by investigating the data and cluster structures. Fuzzy K-Means clustering is described as the process of assigning data points to each cluster with a given level of membership. $z_{1} = 1, min_{z} ⩾ 0, μ_{k} \sum_{j = 1}^{n} \sum_{k = 1}^{c} z_{jk} | | y_{j} - μ_{k} | |_{2}^{2}$ (15)

Where number of clusters is C, fuzzy membership is Z ∈R^n×c. Moreover, μ_K is the clustering centroid matrix and centroid of the K-th cluster is C = [μ₁, μ₂, …μ_C] ∈ R^d×c. The membership degree of the j-th sample’s connection with the j-th group is represented by each item of matrix Z. The square l₂-norm is replaced with the robust loss-norm in order to address the issue of outliers’ sensitivity and enhance clustering performance. Problem is modified as $z_{1} = 1, min_{z} ⩾ 0, μ_{k} \sum_{j = 1}^{n} \sum_{k = 1}^{c} z_{jk} | | y_{j} - μ_{k} | |_{σ}$ (16)

The neutral function in achieves improved strength regardless of large or little losses because to the adaptive -norm, which acts as an interruption between the l₁-norm and l₂-norm.

4 Results and discussion

The experimental setup for this research was created using the deep learning toolbox in MATLAB. In this section accuracy, precision, recall, specificity and F1 score are the different metrics used to evaluate the proposed Clustering based Faster RCNN Network. Moreover, the proposed Clustering based Faster RCNN Network was also compared with other traditional network and state-of-art models.

4.1 Performance analysis

In this work, specificity, accuracy, precision, recall, and F1 score were used as the basis for the performance analysis calculations. $accuracy = \frac{TP + TN}{TP + TN + FP + FN}$ (17) $Specificity = \frac{TN}{TN + FP}$ (18) $Precision = \frac{TP}{TP + FP}$ (19) $recall = \frac{TP}{TP + FN}$ (20) $f_{1} = 2 (\frac{precision * recall}{precision + recall})$ (21)

False-positives, false-negatives, true-positives, and true-negatives are denoted by FP, FN, TP, and TF respectively.

The performance of proposed model for three classes that includes suspicious and non supiscious. The proposed approach achieved higher accuracy of 0.987 and 0.991 for s supiscious and non supiscious. The proposed approach achieved higher specificity of 0.965 and 0.977 for suspicious and non supiscious and precision is 0.961 and 0.954 for suspicious and non-suspicious. The proposed approach achieved higher recall of 0.985 and 0.957 for suspicious and non supiscious and F1 score is 0.934 and 0.969 for suspicious and non supiscious that can be measured via TPR and FPR parameters. The overall accuracy of 0.987 and the overall specificity of 0.962 and precision, recall and F1 score is 0.979,0.929 and 0.941.

Figure 4 shows that the proposed model has achieved high accuracy in both training and testing accuracy, and loss is shows in Fig. 5. The performance is based on the specificity, recall, accuracy, precision and F1 score, and the accuracy obtained by the proposed model is 98.76%.

Table 2

Performance analysis of proposed method

Classes	Accuracy	Specificity	Precision	Recall	F1
					score
Suspicious	0.987	0.965	0.961	0.985	0.934
Non-suspicious	0.991	0.977	0.954	0.957	0.969

Fig. 4

Training and Testing accuracy of proposed method.

Fig. 5

Training and Testing loss of proposed method.

4.2 Comparative analysis

The comparative assessment was conducted among four prior deep learning networks is demonstrated in Table 3.

Table 3
Comparison between traditional deep neural networks

Networks Parameters Flops Complexity

CNN 53891k 719M 717

Google net 7354k 534M 140

Mobile net 81.46k 568M 170

Faster R-CNN 30.27k 51M 51

Networks	Parameters	Flops	Complexity
CNN	53891k	719M	717
Google net	7354k	534M	140
Mobile net	81.46k	568M	170
Faster R-CNN	30.27k	51M	51

According to Table 3, traditional networks like CNN, Google Net, and Mobile Net require a lot of parameters to get those results, which raises the complexity level of each network. But the Faster R-CNN only utilize a smaller number of parameters so the complexity is reduced. Moreover, small number of flops are used in the proposed model that results in the low computational time when compared to the prior methods. Hence, the complexity structure of the proposed model is slightly high when compared to the light weight deep learning networks like shuffle net.

In this division, a comparison between the proposed model and conventional deep learning networks is also conducted. Based on a performance comparison with existing methods, this technique outperforms them in terms of productivity. Performance is evaluated using F1 score, precision, specificity, recall, and accuracy. The proposed model is compared with three different deep learning algorithms in this comparison analysis.

Table 4 illustrates that the results attained in terms of overall accuracy rate. From Table 4 the traditional networks like CNN, U-net, Fab net and Fast R-CNN obtains less accuracy compared to the Faster R-CNN. Faster R-CNN preserves the high accuracy ranges of 98.78%. The accuracy obtained by CNN, U-net, Fab net and Fast R-CNN is 97.68% 96.76% 95.67% and 94.21% respectively. The specificity obtained by CNN, U-net, Fab net, Fast R-CNN and Faster R-CNN is 95.46%96.48%93.64% 91.38% and 97.53%. Precision is obtained by CNN, U-net, Fab net, Fast R-CNN and Faster R-CNN is 94.63% 93.53% 91.45% 92.15% and 95.21% Recall is obtained by CNN, U-net, Fab net, Fast R-CNN and Faster R-CNN is 92.42% 93.37% 87.76% 90.61% and 94.67% F1 score is obtained by CNN, U-net, Fab net, Fast R-CNN and Faster R-CNN is 90.56% 92.34% 86.98% 88.85% and 92.85% The Faster R-CNN net’s accuracy rate is higher than that of the existing models. Thus, it is clearly seen that Clustering based Faster R-CNN model performs better than other techniques.

Table 4

Comparison between traditional deep learning networks

Networks	Accuracy	Specificity	Precision	Recall	F1
					Score
CNN	97.68	95.46	94.63	92.42	90.56
U-Net	96.76	96.48	93.53	93.37	92.34
Fab net	95.67	93.64	91.45	87.76	86.98
Fast RCNN	94.21	91.38	92.15	90.61	88.85
Faster R-CNN	98.78	97.53	95.21	94.67	92.85

According to Table 5, the proposed Clustering based Faster RCNN model improves the overall accuracy of 8.86% 33.77% 3.12% and 5.48% better than mmGait, LSTM Net, STDNN and RNN respectively.

Table 5

Comparison between proposed and the existing models

Author	Methods	Accuracy (%)	Specificity	Precision	Recall	F1 score
Meng, [12]	mmGait Net	90	87.48	89.48	86.26	90.38
Zhang, [15]	LSTM Net	65.4	62.10	69.17	73	76.38
Tong, [18]	STDNN	95.67	91.38	90.13	90.26	89.27
Hasan, [20]	RNN	93.34	89.37	90.89	90.38	83.48
Proposed	Clustering based Faster R-CNN	98.76	94.28	95.39	93.29	91.38

The specificity obtained by mmGait, LSTM Net, STDNN and RNN is 87.48% 62.10% 91.38% and 89.37% The proposed clustering-based faster RCNN model achieved a specificity of 94.28% The precision obtained by mmGait, LSTM Net, STDNN and RNN is 89.48% 69.17% 90.13% and 90.89% The proposed clustering-based faster RCNN model achieved a precision of 95.39% The proposed clustering-based faster RCNN model achieved a Recall of 93.29% The proposed clustering-based faster RCNN model achieved a F1 score of 91.38%

5 Conclusion

In this paper, a novel Clustering based Faster RCNN approach has been proposed to identify the single, double and multi-gait. From CASIA-B gait dataset, we choose single, double and multi-gait images are pre-processed using Multi scale Retinex (MSR) to reduce the noise artifacts. Finally, the Fast RCNN is employed for classifying the gait images into suspicious and non-suspicious walking gait based on the clusters from the clustering algorithm. The proposed Clustering based Faster RCNN net achieves the high accuracy rate of 98.74% and 99.19% for suspicious and non-suspicious walking gait respectively. The proposed Clustering based Faster RCNN model was compared with other traditional models like CNN, U-net, Fab net and Fast R-CNN. The proposed Clustering based Faster RCNN model improves the overall accuracy of 8.86% 33.77% 3.12% and 5.48% better than mmGait, LSTM Net, STDNN and RNN respectively. The complexity structure of the proposed model is slightly high when compared to the light weight deep learning networks, since the complexity can be reduced to achieve stable performance in multi-gait recognition. In future, the accuracy of the proposed model can be improved by using the advanced deep learning networks or optimization algorithms.

Footnotes

Acknowledgments

The authors with a deep sense of gratitude would thank the supervisor for his guidance and constant support rendered during this research.

References

Asif

, Tiwana

M.I.

, Khan

U.S.

, Ahmad

M.W.

, Qureshi

W.S.

and Iqbal

, Human gait recognition subject to different covariate factors in a multi-view environment, Results in Engineering 15 (2022), 100556.

Makihara

, Nixon

M.S.

, Yagi

Gait recognition: Databases, representations, and applications. Computer Vision: A Reference Guide (2020), pp. 1–13.

Sepas-Moghaddam

, Ghorbani

, Troje

N.F.

, Etemad

Gait recognition using multi-scale partial rep-resentation transformation with capsules. In 2020 25th international conference on pattern recognition (ICPR) IEEE, (2021), 8045–8052.

Huang

, Zhu

, Wang

, Yang

, He

, Liu

, Feng

Context-sensitive temporal fea-ture learning for gait recognition. In Proceedings of the IEEE/CVF International Conference on Computer Vision, (2021), 12909–12918.

Chen

, Weng

, Lu

and Xu

, Multi-gait recognition based on attribute discovery, IEEE Transactions on Pattern Analysis and Machine Intelligence 40 (7) (2017), 1697–1710.

Portillo-Portillo

, Leyva

, Sanchez

, Sanchez-Perez

, Perez-Meana

, Olivares-Mercado

, Toscano-Medina

and Nakano-Miyatake

, A view-invariant gait recognition algorithm based on a joint-direct linear discriminant analysis, Applied Intelligence 48 (5) (2018), 1200–1217.

Sharif

, Attique

, Tahir

M.Z.

, Yasmim

, Saba

and Tanik

U.J.

, A machine learning method with threshold based parallel feature fusion and feature selection for automated gait recognition, Journal of Organizational and End User Computing (JOEUC) 32 (2) (2020), 67–92.

Wang

, Fan

, Fang

and Dai

, generalized linear discriminant analysis based on euclidean norm for gait recognition, International Journal of Machine Learning and Cybernetics 9 (4) (2018), 569–576.

Kim

J.H.

, Batchuluun

and Park

K.R.

, Pedestrian detection based on faster R-CNN in nighttime by fusing deep convolutional features of successive images, Expert Systems with Applications 114 (2018), 15–33.

10.

Ghosh

A Faster R-CNN and recurrent neural network- based approach of gait recognition with and without carried objects. Expert Systems with Applications 2022, 117730.

11.

Chen

, Luo

, Weng

, Luo

, Li

and Tian

, Multi-view gait image generation for cross-view gait recognition, IEEE Transactions on Image Processing 30 (2021), 3041–3055.

12.

Meng

, Fu

, Yan

, Liang

, Zhou

, Zhu

, Ma

, Liu

and Yang

, Gait recognition for co-existing multiple people using millimeter wave sensing. In Proceedings of the AAAI Conference on Artificial Intelligence 34 (01) (2020), 849–856.

13.

, Makihara

, Li

, Yagi

, Lu

Gait recognition from a single image using a phase-aware gait cycle reconstruction network. In European Conference on Computer Vision Springer, Cham. 2020, 386–403.

14.

Liao

, Yu

, An

and Huang

, A model-based gait recognition method with body pose and human prior knowledge, Pattern Recognition 98 (2020), 107069.

15.

Zhang

, Tran

, Yin

, Atoum

, Liu

, Wan

, Wang

Gait recognition via disentangled representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition 2019, 4710–4719.

16.

Singh

J.P.

, Arora

, Jain

, Som

U.P.S.

A multi-gait dataset for human recognition under occlusion scenario. In 2019 International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT), 2019, (Vol. 1,pp. 1–6). IEEE.

17.

Gadaleta

and Rossi

, Idnet: Smartphone-based gait recognition with convolutional neural networks, Pattern Recognition 74 (2018), 25–37.

18.

Tong

, Fu

, Yue

and Ling

, Multi-view gait recognition based on a spatial-temporal deep neural network. , IEEE Access 6 (2018), 57583–57596.

19.

Chen

, Weng

, Lu

and Xu

, Multi-gait recognition based on attribute discovery, IEEE Transactions on Pattern Analysis and Machine Intelligence 40 (7) (2017), 1697–1710.

20.

Hasan

M.M.

and Mustafa

H.A.

, Multi-level feature fusion for robust pose-based gait recognition using RNN, Int J Comput Sci Inf Secur (IJCSIS) 18 (4) (2020).

Multi gait recognition using Clustering based Faster Regions-Convolutional Neural Network

Abstract

Keywords

1 Introduction

2 Literature review

Table 1 Dataset description Classes No. of images Total Normal 450 1364 With coat 447 With bag 467

3.2.1 Faster RCNN

4.1 Performance analysis

Table 3 Comparison between traditional deep neural networks Networks Parameters Flops Complexity CNN 53891k 719M 717 Google net 7354k 534M 140 Mobile net 81.46k 568M 170 Faster R-CNN 30.27k 51M 51

Footnotes

Acknowledgments

References

Table 1
Dataset description

Classes No. of images Total

Normal 450 1364

With coat 447

With bag 467

Table 3
Comparison between traditional deep neural networks

Networks Parameters Flops Complexity

CNN 53891k 719M 717

Google net 7354k 534M 140

Mobile net 81.46k 568M 170

Faster R-CNN 30.27k 51M 51