Target recognition of sport athletes based on deep learning and convolutional neural network

Abstract

The main purpose of the various methods of evaluating athlete feature recognition is to monitor the current health of the athletes, thereby providing some feedback on the quality of individual training. Based on deep learning and convolutional neural networks, this paper studies athlete target recognition and proposes a feature vector extraction method based on curvature zero point. Moreover, based on the ideas of deep learning and convolutional neural networks, this paper builds an athlete feature recognition model and optimizes the algorithm. In order to verify the feasibility and efficiency of feature extraction algorithm of the sport athletes proposed by this paper and to facilitate comparison with other algorithms, this paper conducts an algorithm performance test on the sport athlete database. The research results show that the method proposed in this paper has certain advantages in the feature extraction of athletes and can be used in subsequent sports training systems.

Keywords

Deep learning convolutional neural network sports athlete recognition feature extraction

1 Introduction

With the rapid development of computer technology and the Internet, people have given computers more and more missions, so that computers can replace or assist humans in all directions to achieve intricate and human-incapable things. There are various ways for humans to obtain information from the outside world, and humans mainly obtain external information and perceive changes in information from the aspects of sight, hearing, smell, and touch. Among them, vision [1] is the primary way for humans to understand the world from the senses and obtain information. It not only refers to the perception of light source signals, but also includes the acquisition, processing, transmission, storage and understanding of signals. Therefore, giving computer vision ability is the most urgent and most important ability. Multiple disciplines such as machine learning, image processing, pattern recognition, and artificial intelligence are included in computer vision research. Its main scientific research direction is to process and analyze many pictures and videos and other data, so that the computer can use vision to obtain picture information and make corresponding accurate judgments on it [2]. The purpose is to enable computers to have similar self-analysis and judgment skills as humans, and to be able to complete some work instead of humans. Human behavior recognition based on computer vision [3] is one of many research directions that have received much attention in recent years. It combines technologies such as computer vision and behavioral pattern analysis and understanding, and its main purpose is to detect, identify, and understand research objects of interest with video images of human targets. Due to its broad research prospects and application fields, it has been extensively studied by more and more researchers in recent years and has obtained many research results. These are undoubtedly major breakthroughs in the field of human behavior recognition applications. Human behavior recognition technology refers to a technology that combines machine vision and video image processing and analysis to perform in-depth intelligent analysis of the collected image sequence and automatically determine the specific behavior of the human target. According to the detailed information reflected in the data information of the video image, it is analyzed and judged in detail [4].

2 Related work

The literature [5] used an RGBD sensor (Microsoft Kinect) as an input sensor and calculated a set of features based on human posture and motion, and image and point cloud information. Their algorithm used a Markov model based on hierarchical maximum entropy. The model regarded a person’s activities as consisting of a set of self-activities and uses dynamic programming to infer a two-layer graph structure. The experiments proved the effectiveness of the algorithm. The literature [6] provided an effective capture technique to accurately estimate the position of 3D skeleton joints from a single depth map and to identify the human actions in the depth image sequence by constructing a local popular frame of the data features of the skeleton nodes.

The literature [7] proposed a novel feature extraction method. This method extracts semi-local features from the depth image, which is called a random occlusion module (ROP). It can effectively search a very large sampling space, and the sparse coding method was still used to encode these features. The literature [8] proposed a feature expression for RGB.D human motion recognition and used the time pyramid matching method ScTPM based on sparse coding. For RGB video, this paper adopted the spatio-temporal feature descriptor CS.M1tD of the central symmetric motion local ternary mode. For the three-node feature, combined with the ScTPM expression method and feature descriptor CS.M1tp proposed in this paper, the color and depth information were fused at the feature layer and the score layer, respectively. The literature [9] combined bone nodes and depth images for motion recognition and used the angle model of bone nodes and the spatiotemporal features of depth images to construct new descriptors. The proposed descriptor is easy to implement, generates a feature set of relatively small size, and has a fast classification scheme, which is suitable for real-time applications. The literature [10] also tried the fusion of RGB images and depth images. First, the depth difference motion history image (DDMHI) was obtained from the depth image, and the same method was used to obtain the full image difference motion history image (WRDMHI) from the RGB image. After that, the motion features were extracted and fused effectively. The literature [11] mapped the depth image to three two-dimensional Cartesian planes and formed a depth motion map (DMM) to describe the action by the difference between the frames outside the threshold range. Then, the direction gradient histogram (HOG) was extracted on the DMM. Finally, the classic LIBSVM classifier was used for classification, which has achieved very good results.

In recent years, the tremendous increase in computer computing power has enabled researchers to process visual data with advanced deep learning algorithms at an acceptable speed. Therefore, the automatic feature learning method, namely deep learning, has also been widely used in skeleton motion recognition. Among them, the most applied is the structure based on recurrent neural network (RNN). The most widely used structure is the structure based on recurrent neural network (RNN). In recent years, some machine learning methods have been used for behavior recognition with good results. For example, literature [13] proposed to use an improved convolutional neural network to recognize behavior, and literature [14] proposed a 3D convolutional neural network to recognize behavior recognition methods. The literature [15] proposed an end-to-end hierarchical RNN (HBRNN) structure for skeleton action recognition and divided the body into 5 parts based on the structure of the human body. The result achieved better results than the previous manual feature method on the data set MSRAction3D. The literature [16] proposed the Co- occurrence LSTM. The Co-occurrence LSTM network does not need to deliberately group joints, it can automatically select the joint points that have discriminative power to the sequence, and it achieves a result about 10% higher than HBRNN on the data set SUB Kinect interaction In addition, Convolutional Neural Network (CNN) is also applied to skeleton action recognition. The literature [17] proposed three different action recognition models based on deep learning: Time Selection LSTM, Spatio-Temporal CNN and Joints Selection CNN. Moreover, the literature has conducted a large number of experiments on several public databases for each model and obtained effective results.

3 Fourier-curvature space descriptor

The literature [19] applied Fourier descriptors to object shape feature extraction, and the literature [20] also applied Fourier descriptors to human body recognition. The Fourier descriptor is the bilateral Fourier transform coefficient under the mapping of the two-dimensional real coordinate system of the human edge curve to the complex coordinate system and is also the frequency domain analysis result of the human edge curve signal. Because the contour curve of the human body edge can be regarded as a periodic function of a two-dimensional plane, it can also satisfy the Fourier transform condition. The Fourier transform coefficient itself does not have rotation, translation, or scale invariance, and it needs to be normalized further.

For any sampling point s_k (μ_k, v_k) on the edge curve of the human body, the mapping to the complex plane is s_k = μ_k + jv_k. All sampling points {s_k, k = 1, 2, 3, ⋯ , N } mapped on the complex plane are subjected to bilateral Fourier transform [21]: $a (f) = \frac{1}{N} \sum_{k = 1}^{N} s_{k} e^{- j \frac{2 π}{N} fk}, f = 0, 1, 2, 3, \dots, N - 1$ (1)

The one-dimensional vector composed of Fourier coefficients a (f) is the Fourier descriptor. By inverse transforming it and restoring back to the real coordinate system, the edge curve of the original human body contour image can be obtained. Similar to the Fourier transform in the field of signal processing, the low-frequency coefficient of the Fourier transform determines the overall shape of the contour curve of the human body, and the high-frequency coefficient reflects the details of the shape. When retaining the appropriate number of coefficients, the algorithm can achieve good robustness and fully preserve shape details. The length of the Fourier descriptor used in this paper is 100 [22].

If the contour curve of the human body has a certain translation in a certain direction, the coordinate value in the complex plane increases by a constant: s_k = μ_k + Δx + j (v_k + Δy) = s_k + Δxy. According to the nature of the Fourier transform, when the function increases by a constant, the DC component a (0) of the Fourier transform result increases by an amount of impact [23]: $\begin{matrix} a (f) = \frac{1}{N} \sum_{k = 1}^{N} s_{k} e^{- j \frac{2 π}{N} fk} = \frac{1}{N} \sum_{k = 1}^{N} (s_{k} + Δ xy) e^{- j \frac{2 π}{N} fk} \\ = a (f) + Δ xy δ (f) \end{matrix}$ (2)

The translation invariance of the Fourier descriptor can be achieved by discarding the a (0). When the body contour rotates, the edge curve rotates by a certain angle θ, and the complex plane coordinate becomes s_ke^jθ. It can also be seen from the nature of the Fourier transform that changes in rotation introduce changes in phase [24 –26]: $\begin{matrix} a (f) = \frac{1}{N} \sum_{k = 1}^{N} s_{k} e^{- j \frac{2 π}{N} fk} = \frac{1}{N} \sum_{k = 1}^{N} (s_{k} e^{j θ}) e^{- j \frac{2 π}{N} fk} \\ = a (f) e^{j θ} \end{matrix}$ (3)

By selecting the amplitude value of the Fourier transform coefficient and ignoring its phase value, the purpose of rotation invariance can be achieved. Similarly, when scale conversion occurs, the scale conversion factor λ is introduced, and the complex plane coordinate becomes λ_{s
_k}. At this time, the Fourier transform is as follows: $\begin{matrix} a (f) = \frac{1}{N} \sum_{k = 1}^{N} s_{k} e^{- j \frac{2 π}{N} fk} = \frac{1}{N} \sum_{k = 1}^{N} (λ s_{k}) e^{- j \frac{2 π}{N} fk} \\ = λ a (f) \end{matrix}$ (4)

At this time, the Fourier transform result is that each Fourier coefficient is multiplied by the scale conversion factor λ. In order to eliminate the influence of scale transformation on the Fourier descriptor, each Fourier coefficient can be divided by a (1). At this point, the Fourier descriptors for rotation, translation, and scale invariance can be obtained: $\begin{matrix} F . D = {| \frac{a (1)}{a (1)} |, | \frac{a (2)}{a (1)} |, | \frac{a (3)}{a (1)} |, \dots, | \frac{a (f)}{a (1)} |}, \\ f = 1, 2, 3, \dots, N - 1 \end{matrix}$ (5)

The curvature is most often calculated in the form of a curve parameter equation.

Let the plane curve be Γ and its parametric equation be μ = θ (t) , v = ψ (t), define the curve curvature: $\hat{K} (t) = \frac{θ {(t)}^{'} ψ {(t)}^{″} - θ {(t)}^{″} ψ {(t)}^{'}}{{(θ {(t)}^{'}^{2} + ψ {(t)}^{'}^{2})}^{3 / 2}}$ (6)

After introducing Gaussian smoothing, the curve is convolved with one-dimensional Gaussian functions of different scales, that is $μ_{1} = θ (t) * g σ (t), v_{1} = ψ (t) * g σ (t)$

From the differential nature of the convolution operation, the curvature formula becomes (* is the convolution operation): ${\hat{K}}_{1} (t) = \frac{\begin{matrix} (θ (t) * g σ {(t)}^{'}) (ψ (t) * g σ {(t)}^{″}) - \\ (θ (t) * g σ {(t)}^{″}) (ψ (t) * g σ {(t)}^{'}) \end{matrix}}{{(θ (t) * g σ {(t)}^{'})}^{2} + {(ψ (t) * g σ {(t)}^{'})}^{3 / 2}}$ (7)

For the contour curve of the human body, it is a discrete point, so it is impossible to list the parameter equations. Therefore, the curvature should be directly given by the coordinates of the edge neighborhood points: $\hat{K} = \frac{| {\tilde{y}}^{″} |}{{(1 + {\tilde{y}}^{'}^{2})}^{3 / 2}}$ (8)

Among them, ${\tilde{y}}^{″}, {\tilde{y}}^{'}$ represents the second and first order numerical derivatives of the neighborhood of the curvature point, and μ_k, v_k corresponds to the discrete point k on the contour curve: $\begin{matrix} {\tilde{y}}_{k - 1}^{''} = \frac{v_{k + 1} + v_{k - 1} - 2 v_{k}}{(μ_{k + 1} - μ_{k}) + (μ_{k} - μ_{k - 1})} \\ {\tilde{y}}_{k - 1}^{'} = \frac{v_{k + 1} - v_{k - 1}}{μ_{k + 1} - μ_{k - 1}} \end{matrix}$ (9)

With the evolution of the scale σ, the curvature of the curve after Gaussian smoothing also changes. The core idea of the curvature scale space is to record the change process of the zero point of curvature to form a characteristic map of the curvature scale space corresponding to the curve. This paper proposes a feature vector extraction method based on the zero point of curvature.

Each sub-picture in Fig. 1 shows the change of the curvature zero point corresponding to the contour curve of the human body contour after smoothing with different Gaussian scales. The “+” sign in the sub-picture is the curvature zero mark, indicating that the curvature at the change point is zero. It can be found from the figure that as the Gaussian scale continues to increase, the number of zero points of curvature of the smoothed curve continues to decrease. For the given human body contour example, when σ = 5, the corresponding number of curvature zero points is zero. In addition, the point marked by the rectangular frame in the curvature scale space feature map is the maximum value reference point in the traditional recognition algorithm, and the image recognition is completed according to the corresponding position k and relative distance of the reference point.

Fig. 1

Schematic diagram of the spatial characteristics of the curvature scale of the human body edge curve.

The number of curvature zero points at different Gaussian scales is extracted from the histogram in Fig. 1 to form a curvature scale space feature vector of the human body contour image, which is used for human body contour recognition. It can be seen from the table that the feature vector based on the zero point of curvature proposed in this paper is effective in the human body contour recognition task, and it shows no relation with the length of the feature vector. That is, increasing the degree of smoothing (increasing the value of σ) at a certain step size does not affect the final recognition result. Therefore, σ_max = 5, len = 9 can be selected as the feature vector according to the data in the table. In addition, fixing the maximum value of σ and reducing the step size have no obvious effect on the recognition results.

Since image retrieval does not need to strictly output its category, the retrieval rate is often higher than the recognition accuracy rate. According to the citation data, it can be inferred that when the retrieval degree tends to be recognized, the retrieval rate is far below 39.29%, which also shows that the method proposed in this section is reasonable.

4 Construct the multi-feature fusion neural network classifier

Gradient as a vector parameter plays an important role in the numerical analysis of multivariate nonlinear functions, and gradient is also the basis of related algorithms of fully connected neural networks. Therefore, it is necessary to introduce the gradient and its application in neural networks.

We assume that a certain surface z = f (x, y) in three-dimensional space is defined in a certain neighborhood of p₀ (x₀, y₀). The angle between the ray l induced by p₀ and the positive direction of the x axis is φ, The direction of l is used as the reference direction and is denoted as $\vec{l}$ . If the limit: $lim_{t \to 0^{+}} \frac{f (x_{0} + t cos φ, y_{0} + t sin φ) - f (x_{0}, y_{0})}{t}$ (10)

exists, it is the directional derivative of the function z = f (x, y) at point p₀ (x₀, y₀) along the direction $\vec{l}$ . Moreover, t is the offset distance in the $\vec{l}$ direction. If the x-axis or y-axis is selected as the reference direction, the above formula can derive the partial derivative of the function.

If the function z = f (x, y) is differentiable in the neighborhood of p₀ (x₀, y₀), that is, the two sides of the function increment: $\begin{matrix} f (x_{0} + Δ_{x}, y_{0} + Δ_{y}) - f (x_{0}, y_{0}) \\ = \frac{\partial f}{\partial x} Δ_{x} + \frac{\partial f}{\partial y} Δ_{y} + o (t) \end{matrix}$ (11)

are divided by t and taken limit to obtain the following result: $\begin{matrix} lim_{t \to 0^{+}} \frac{f (x_{0} + Δ_{x} y_{0} + Δ_{y}) - f (x_{0}, y_{0})}{t} \\ \frac{\partial f}{\partial x} cos φ + \frac{\partial f}{\partial y} sin φ \end{matrix}$ (12)

The limit form is written as a vector form: $[\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}] \cdot {[cos φ, sin φ]}^{T}$ , and [cos φ, sin φ] ^T is the direction vector of $\vec{l}$ . The direction of $\vec{l}$ is changed to maximize the limit, which is the gradient of the point, and the gradient is also a vector $[\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}]$ . Now, the gradient descent algorithm in the neural network is considered. We assume that the gradient descent algorithm is applied at [x₀, y₀] (vector form) to find a point smaller than f (x₀, y₀). It can be seen from the nature of the gradient that such points appear in the opposite direction of the gradient direction, so the point sought should satisfy: $[x_{1}, y_{1}] = [x_{0}, y_{0}] - ɛ [\frac{\partial f}{\partial x}, \frac{\partial f}{\partial y}]$ (13)

ɛ is an important parameter in machine learning: learning rate (step size). In practical applications, it can be considered that the main purpose of the existence of the gradient vector is to indicate the direction in which the function falls, and the parameter that really affects the next value is actually the learning rate Therefore, each iteration of the gradient descent algorithm is greatly affected by the learning rate. If the learning rate is not set properly, it will cause difficulty in the convergence of the cost function.

Considering that the gradient direction has specified the search direction, the original multivariate function optimization problem is transformed into a line search problem. That is, with the learning rate ɛ as an independent variable, the original target multivariate function $f (\vec{x_{k}})$ becomes a univariate function like $f (\vec{x_{k}} + \vec{ɛ_{k}} \vec{d_{k}})$ . Among them, $\vec{x_{k}}$ is a known point, and $\vec{d_{k}}$ is the search direction (the opposite direction of the gradient). Such a unary line search problem can be given the optimal solution range by Wolfe-Powell criterion. The Wolfe-Powell criterion is given by the following mathematical expression, and $\vec{g_{k}^{T}}$ is the gradient at $f (\vec{x_{k}})$ : $\begin{matrix} f (\vec{x_{k}} + ɛ_{k} \vec{d_{k}}) ⩽ f (\vec{x_{k}}) + ɛ_{k} ρ \vec{g_{k}^{T}} \vec{d_{k}} \\ < f (\vec{x_{k}}), 0 < ρ < 0.5 \end{matrix}$ (14) $\nabla f {(\vec{x_{k}} + ɛ_{k} \vec{d_{k}})}^{T} \vec{d_{k}} ⩾ σ \vec{g_{k}^{T}} \vec{d_{k}}, σ \in (ρ, 1)$ (15)

Since $\vec{d_{k}}$ and $\vec{g_{k}^{T}}$ are in opposite directions, $ɛ_{k} ρ \vec{g_{k}^{T}} \vec{d_{k}} < 0$ and then $f (\vec{x_{k}}) + ɛ_{k} ρ \vec{g_{k}^{T}} \vec{d_{k}} < f (\vec{x_{k}})$ hold. The formula (14) actually specifies the upper limit of the value of ɛ_k in $f (\vec{x_{k}} + ɛ_{k} \vec{d_{k}})$ , which is also a necessary condition for gradient descent.

Equation (15) specifies the gradient constraint of the search point from the gradient angle, that is, the gradient at the search point should be greater than or equal to σ times the initial gradient. According to the nature of gradient descent, the gradient of the selected search point should not be greater than the gradient of the initial point, and σ < 1 constrains the initial point, the above formula holds, and the lower limit of ɛ_k is given. The learning rate satisfying Equations (14) and (15) can be considered as the optimal value. By applying the Wolfe-Powell criterion to the neural network, the adaptive of learning rate ɛ can be achieved.

We assume that an input 264-dimensional feature vector is $\vec{x}$ , the weight parameter vector is $\vec{θ}$ , and the assumed model satisfies: $0 < h_{\vec{θ}} (\vec{x}) < 1$ . Then, according to the related theory in higher mathematics, the logical function n satisfies n, and its expression is: Then, according to the related theory in higher mathematics, the logical function g (z) satisfies 0 < g (z) < 1, and its expression is: $g (z) = \frac{1}{1 {+ e}^{- z}}$ (16)

The assumed model function after using logical function constraints is: $h_{\vec{θ}} (\vec{x}) = g ({\vec{θ}}^{T} \vec{x}) = \frac{1}{1 {+ e}^{- {\vec{θ}}^{T} \vec{x}}}$ (17)

One of the important reasons for choosing this logic function is its derivative characteristic. As known from the foregoing, the gradient descent method depends on the partial differentiation of the objective function, and g (z) is differentiable everywhere in the definition domain and the derivative form is simple: g′ (z) = g (z) (1 - g (z)). The derivation of g (z) completely depends on the forward propagation of the neural network, which helps to improve the calculation efficiency.

The cost function is the objective optimization function in machine learning, and it is also the application object of the gradient descent algorithm. The common cost function idea is to calculate the Euclidean distance or square error function. If the hypothetical model $h_{\vec{θ}} (\vec{x}) = g ({\vec{θ}}^{T} \vec{x})$ defined above is followed by the square error function, the cost function becomes a non-convex function. This means that the cost function has many minimum points, and the gradient descent algorithm cannot be used to find the global minimum, and the gradient descent will converge to different extreme points due to the initial position, resulting in unstable performance of the neural network.

Since the calculation result of $h_{\vec{θ}} (\vec{x})$ is constrained between 0 and 1, it can be regarded as a probability value. By a little modification on the basis of Jeffery distance, the cost function formula under the constraint of logical function can be derived: $\begin{matrix} Cost (h_{\vec{θ}} (\vec{x}), y) = - ylog (h_{\vec{θ}} (\vec{x})) - \\ (1 - y) log (1 - h_{\vec{θ}} (\vec{x})) \end{matrix}$ (18)

The above formula is derived from a single input. Since the neural network needs to put the training set into the network during the training process, the above formula is changed to: $J (\vec{θ}) = - \frac{1}{m} \sum_{i = 1}^{m} [\begin{matrix} y^{i} log (h_{\vec{θ}} ({\vec{x}}^{i})) \\ + (1 - y^{i}) log (1 - h_{\vec{θ}} ({\vec{x}}^{i})) \end{matrix}]$ (19)

In the formula, i is the i-th sample in the training set, and yⁱ is its corresponding theoretical output value. It can be proved that the new cost function is a convex function, which can be converged to the global optimal value by the gradient descent algorithm. The partial differential formula of $J (\vec{θ})$ for the j-th parameter in vector $\vec{θ}$ is given: $\begin{matrix} \frac{\partial J (\vec{θ})}{\partial θ_{j}} = - \frac{\partial}{\partial θ_{j}} \frac{1}{m} \sum_{i = 1}^{m} [\begin{matrix} y^{i} log (h_{\vec{θ}} ({\vec{x}}^{i})) \\ + (1 - y^{i}) log (1 - h_{\vec{θ}} ({\vec{x}}^{i})) \end{matrix}] \\ = \frac{1}{m} \sum_{i = 1}^{m} [h_{\vec{θ}} ({\vec{x}}^{i}) - y^{i}] x_{j}^{i} \end{matrix}$ (20)

In order to solve the problem of overfitting, we introduce a regularization method, and regularization can effectively reduce the parameter size while retaining all the features. The idea of regularization is: When optimizing the cost function $J (\vec{θ})$ , a penalty is introduced (can also be considered as additive noise). Under the constraint of the amount of penalty, in order to make $J (\vec{θ})$ as small as possible, the value $\vec{θ}$ will be reduced accordingly, so as to solve the problem of overfitting. After introducing the penalty, the cost function becomes: $\begin{matrix} J (\vec{θ}) = - \frac{1}{m} \sum_{i = 1}^{m} [\begin{matrix} y^{i} log (h_{\vec{θ}} ({\vec{x}}^{i})) \\ + (1 - y^{i}) log (1 - h_{\vec{θ}} ({\vec{x}}^{i})) \end{matrix}] \\ + \frac{λ}{2 m} \sum_{j = 1}^{n} θ_{j}^{2} \end{matrix}$ (21)

θ_j is the jth element in $\vec{θ}$ , θ₀ is the reserved bias parameter and does not participate in regularization, and λ is the regularization parameter.

In a multi-classifier constructed by a fully connected neural network, each input will have k output values and a weight matrix Θ formed by multiple weight vectors $\vec{θ}$ . By generalizing formula (21) to the actual neural network, we can get: $\begin{matrix} J (Θ) = - \frac{1}{m} \sum_{i = 1}^{m} \sum_{t = 1}^{k} [\begin{matrix} y_{t}^{i} log {(h_{\vec{θ}} ({\vec{x}}^{i}))}_{t} \\ + (1 - y_{t}^{i}) log {(1 - h_{\vec{θ}} ({\vec{x}}^{i}))}_{t} \end{matrix}] \\ + \frac{λ}{2 m} \sum_{l = 1}^{L - 1} \sum_{i = 1}^{s_{l}} \sum_{j = 1}^{s_{l + 1}} {(Θ_{ji}^{l})}^{2} \end{matrix}$ (22)

The back propagation algorithm of formula (22) is now given in matrix form. If it is assumed that the fully connected neural network has a total of 4 layers of neurons, then by formula (20), the partial differential of the output value of the cost function of the entire neural network to the input of the output layer is written as: $δ^{4} = {out}^{4} - y$ (23)

Among them, out⁴ represents the logical function constraint term for the input amount of the output layer. It can be further obtained that the partial differential of the output value of the cost function of the neural network to the input value of the third layer is: $δ^{3} = {(Θ^{3})}^{T} δ^{4} \cdot [{out}^{3} \cdot (1 - {out}^{3})]$ (24)

In the formula, [out³ · (1 - out³)] is the differential term of the aforementioned logic function. Similarly, the partial differential of the output value of the cost function of the neural network to the input of the second layer is: $δ^{2} = {(Θ^{2})}^{T} δ^{3} \cdot [{out}^{2} \cdot (1 - {out}^{2})]$ (25)

The above formulas give the partial differential of the output of the neural network to the input of each layer. From the transfer properties of partial differentials, the partial differential form of the output of the neural network cost function to an element in the weight matrix of the layer 1 neural network is: $\frac{\partial J (Θ)}{\partial Θ_{ij}^{l}} = {out}_{j}^{l} δ_{i}^{l + 1}$ (26)

After formula (26) is applied to the entire neural network, the partial differentiation of the cost function output for each weight value can be obtained. The ultimate goal of the neural network solution is to find the θ value that makes J (Θ) the minimum value according to the gradient descent algorithm.

In most convex optimization problems, the selection of initial parameters is not required. However, for multilayer neural networks, the selection of initial parameters is conditional. If the initial parameters are all zero, it means that all activation units of the second layer of neural network have the same value. Similarly, if the initial parameter is selected to be the same non-zero value, it will also result in the same result. The above two situations will make the entire neural network into a redundant state.

Generally, random parameters that satisfy a uniform distribution are selected. $θ = {rand}_{0, 1} \times (2 \times eps) - eps$ (27)

rand_0,1 means uniform distribution in [0, 1], the above formula is equivalent to uniform distribution in [- epseps]. eps can choose a smaller value, and in this article eps = 0.12.

5 Algorithm performance test

In order to verify the feasibility and efficiency of the sport athletes feature extraction algorithm proposed in this paper and in order to facilitate comparison with other algorithms, this paper conducts an algorithm performance test on the sport athletes library database.

In this section of the experiment, 40 images were randomly selected as the database for this experiment, that is, the experimental database has a total of 40 categories, 12 images per category, and a total of 480 images. In each category, 6 pictures are selected for training and 6 pictures for testing. The experiment is conducted on the Windows 10 platform, the device memory is 12 G, the CPU was Interl CoreTM i54200U, the software environment is PychalTn.2018.3.5 and MATLAB R2018a, and the relevant code is written in Python3 and MATLAB language.

Experiment 1: The first experiment is to verify that the performance of sports ROI images after texture enhancement and fusion to perform feature extraction is better than that of directly using original sports images to perform feature extraction.

In this experiment, a total of 5 tests were conducted, and the average of the 5 results was taken as the final result. Under the conditions of Gabor filter bank scales of 7×7, 9×9, 11×11, 13×13, the accuracy and feature extraction time-consuming of the original sports image, the texture-enhanced image and the fusion image of the two sports images are tested respectively. The results are shown in Table 1 and Figs. 2 and 3.

Table 1
Correct rate and feature extraction time-consuming of different image sets

The original image Texture enhanced image Fused image

Correct rate Time-consuming (s) Correct rate Time-consuming (s) Correct rate Time-consuming (s)

1 0.971 26.604 0.991 27.157 0.966 26.458

2 0.957 26.226 1.000 27.390 0.966 26.476

3 0.947 25.940 0.999 27.374 0.975 26.698

4 0.941 25.781 1.003 27.465 0.968 26.516

5 0.969 26.548 0.992 27.181 0.967 26.491

average value 0.957 26.220 0.997 27.313 0.968 26.528

	The original image	Texture enhanced image	Fused image
1	0.971	26.604	0.991	27.157	0.966	26.458
2	0.957	26.226	1.000	27.390	0.966	26.476
3	0.947	25.940	0.999	27.374	0.975	26.698
4	0.941	25.781	1.003	27.465	0.968	26.516
5	0.969	26.548	0.992	27.181	0.967	26.491
average value	0.957	26.220	0.997	27.313	0.968	26.528

Fig. 2

The accuracy of different image sets.

Fig. 3

The feature extraction time-consuming of different image sets.

The effect of Texture enhanced image is 0.1% higher than that of using the original image directly, which is not a big improvement. The feature extraction takes 0.5 s longer than the original sport athletes image feature extraction, and the extra time is used for texture enhancement of the athlete image. The feature recognition accuracy rate of the fused image of sport athletes is 1.4 percentage points higher than the original image and 1.3 percentage points higher than the texture-enhanced image. The time from image texture enhancement to image fusion to feature extraction is about 0.6 s longer than the original image, and it is only about 0.1 second longer than the image after texture enhancement.

In summary, it is feasible to take the next recognition method after the original image and the texture-enhanced sports image are fused, and the effect is better than using the original image for feature recognition. This method is conducive to improving the reliability of sports feature recognition, feature extraction is not time-consuming, and averages 0.05 s per picture, which can be accepted by the public. To a certain extent, the results prove the effectiveness and feasibility of this feature extraction algorithm based on neural network structure.

Experiment 2: In order to further prove the effectiveness and feasibility of the sport athletes feature extraction method in this paper, this experiment 2 is conducted under the same sport athletes ROI image data set and SVM classifier as experiment 1, and make 5 comparison experiments with the recognition performance of the LBP feature extraction algorithm and the PCA dimensionality reduction algorithm. Moreover, the evaluation index is unchanged, and the average value is taken. The results are shown in Table 2 and Fig. 4. It can be seen from Table 3 that the accuracy rate of the sport athletes feature extraction method in this paper is several percentage points higher than that of LBP and PCA algorithms. Since the feature processing stage requires multiple filtering and sampling at multiple scales, the time for feature extraction is higher than LBP and PCA. However, in terms of accuracy, compared with LBP and PCA feature extraction algorithms, the sport athletes feature extraction algorithm in this paper is more reliable.

Table 2

Test results of different feature extraction algorithms

	Correct rate	Feature extraction time (s)
LBP+SVM	0.942633	3.2825
PCA+SVM	0.904758	0.6464
This article method+SVM	0.977983	25.957

Fig. 4

Test statistical diagram of different feature extraction algorithms.

Table 3

The correct rate of vein recognition of sportsmen fusion images under the same number of scales

	2 scales	4 scales	6 scales	8 scales
1	0.96	0.97	0.95	0.89
2	0.95	0.98	0.96	0.89
3	0.96	0.97	0.94	0.92
4	0.95	0.98	0.95	0.95
5	0.97	0.97	0.94	0.91
average value	0.96	0.97	0.95	0.91

Experiment 3: The number of different scales in the Gabor function filter bank affects the reliability of features and the overall efficiency of the algorithm to some extent. In order to select a suitable filter bank, the accuracy and efficiency of the algorithm in this chapter under different combinations of Gabor function groups and different scales are tested. The size of the scale is 7×7, 9×9, 11×11, 13×13, 15×15, 17×17, 19×19, 21×21, and the number increases in order: 2, 4, 6, 8. This experiment is carried out on the images of sport athletes after fusion. In order to reduce the experimental error, this experiment is carried out a total of 5 times, and the average value after the fusion of the correct rate and feature extraction time of the ROI image of the sports player after fusion is used as the final test result. The results are shown in Table 3, Table 4, Fig. 5, and Fig. 6.

Table 4

The time-consuming of vein extraction of sportsmen fusion images under different scale numbers

	2 scales	4 scales	6 scales	8 scales
1	9.23	24.54	46.48	79.98
2	9.41	25.68	51.81	82.11
3	10.12	24.91	46.99	82.48
4	10.47	26.21	50.86	80.89
5	9.83	27.14	50.88	80.94
average value	9.81	25.70	49.40	81.28

Fig. 5

The correct rate of vein recognition of sportsmen fusion images under the same number of scales.

Fig. 6

Statistical diagram of the time-consuming of vein extraction of sportsmen fusion images under different scale numbers.

As can be seen from Table 3 and Fig. 5, the algorithm proposed in this paper can effectively complete the sport athletes feature recognition task, and the recognition rate under different filter scale combinations are all above 94%. Through the algorithm performance test on the four sets of different scale numbers, it is found that the recognition rate increases first and then decreases, as shown in Fig. 5. When the Gabor function group in the hidden layer 2 in the neural network proposed in this chapter uses 4 Gabor scales (7×7, 9×9, 11×11, 13×13), the recognition rate is the highest.

As can be seen from Table 4 and Fig. 6, as the number of scales increases, the time for feature extraction is almost doubled. The reason is that as the number of kernels of different scales in the Gabor function filter bank increases, more filtering and sampling operations are required, the number of image operations is more, the number of feature maps generated during the extraction process is more, and the final feature value vector has more detailed information. However, since the resolution of the image of sport athletes itself is not very high, proper scale filtering can effectively extract texture features. If the number of scales is too large, as the scale size increases, some details may be ignored, and a certain degree of overfitting may be caused. Therefore, in Fig. 6 the accuracy rate will increase first and then decrease. Although the feature extraction time is higher than 2.5 times the number of 2 scales when the number of 4 scales, considering that the importance of accuracy rate exceeds the time of feature extraction in practice, this paper believes that 4 different Gabor scales should be selected to form a filter.

6 Conclusion

In the field of sports recognition based on machine vision, the target recognition algorithm used in auxiliary training equipment has the characteristics of high time complexity, high hardware equipment requirements, complicated operation and expensive equipment. Moreover, it does not have a machine recognition function and requires manual operation, which reduces training efficiency. Therefore, in response to this problem, this paper has conducted in-depth research on the target recognition algorithm based on machine learning. By conducting experiments on the currently commonly used target detection methods, and comparing the experimental results and analysis, this paper combines a sports scene, and proposes a moving target detection method based on deep learning and convolutional neural networks. The algorithm obtains good results in the application scenario of this paper, and has strong anti-interference and noise capabilities, and effectively avoids the problem of complex background changes, which paves the way for the subsequent feature extraction. By summarizing the results of relevant comparative experiments, it can be concluded that the average time taken by this method to complete an Epoch is lower than that of a convolutional neural network. In the case of a small difference in accuracy, the network is smaller and fewer parameters need to be calculated. Moreover, with higher computing power, the method in this paper tends to be processed in real time. Similarly, in embedded systems with lower computing power, the method in this paper has better computing performance than convolutional neural networks.

References

Shuai

, Li

, Guo

, et al., Motion Capture With Ellipsoidal Skeleton Using Multiple Depth Cameras[J], IEEE Transactions on Visualization and Computer Graphics 23(2) (2016), 1.

Agethen

, Otto

, Mengel

, et al., Using Marker-less Motion Capture Systems for Walk Path Analysis in Paced Assembly Flow Lines[J], Procedia Cirp 54 (2016), 152–157.

Gritsenko

, Dailey

, Kyle

, et al., Feasibility of Using Low-Cost Motion Capture for Automated Screening of Shoulder Motion Limitation after Breast Cancer Surgery[J], PLOS ONE 10(6) (2015), e0128809.

Spector

J.T.

and Lieblich

, Letter regarding ‘Comparison between low-cost marker-less and high-end marker-based motion capture systems for the computer-aided assessment of working ergonomics’ by Patrizi et al. and research reproducibility[J], Ergonomics 60(4) (2017), 597–598.

Cahill-Rowley

and Rose

, Temporal-spatial reach parameters derived from inertial sensors: comparison to 3D marker-based motion capture[J], Journal of Biomechanics 52 (2017), 11–16.

Ichikawa

and Masakura

, Motion Capture Depends Upon the Common Fate Factor Among Elements[J], Perception 46(12) (2017), 301006617720123.

Sourioux

, Bestaven

, Guillaud

, et al., 3-D motion capture for long-term tracking of spontaneous locomotor behaviors and circadian sleep/wake rhythms in mouse[J], Journal of Neuroence Methods 295 (2017), 51.

Ancillao

, Savastano

, Galli

, et al., Three dimensional motion capture applied to violin playing: A study on feasibility and characterization of the motor strategy[J], Computer Methods & Programs in Biomedicine 149 (2017), 19–27.

Komisar

, Novak

A.C.

and Haycock

, A novel method for synchronizing motion capture with other data sources for millisecond-level precision[J], Gait & Posture 51 (2016), 125–131.

10.

Sá

, Marques

, Rocha

N.B.F.

, et al., Kinematic parameters of throwing performance in patients with schizophrenia using a markerless motion capture system[J], Somatosensory Research 32(2) (2015), 77–86.

11.

Prim

G.D.S.

, Gonçalves

B.S.

and Vieira

M.L.H.

, A representação do corpo e do movimento: uma análise da interatividade do motion capture[J], Design & Tecnologia 5(9) (2015), 23.

12.

Pablo ngel-López

and de la Peńa

N.A.

, Kinematic Hand Analysis Using Motion Capture Technology[J], IFMBE proceedings 49 (2015), 257–260.

13.

Kim

M.K.

, Kim

T.Y.

and Lyou

, Performance Improvement of an AHRS for Motion Capture[J], Journal of Institute of Control 21(12) (2015), 1167–1172.

14.

Zhou

Z.M.

and Chen

Z.W.

, A Survey of Motion Capture Data Earning as High Dimensional Time Series[J], International Journal of Multimedia and Ubiquitous Engineering 10(9) (2015), 17–30.

15.

Khan

M.A.

, Multiresolution coding of motion capture data for real-time multimedia applications[J], Multimedia Tools & Applications 76(15) (2016), 1–16.

16.

Puupponen

, Wainio

, Burger

, et al., Head movements in Finnish Sign Language on the basis of Motion Capture data: A study of the form and function of nods, nodding, head thrusts, and head pulls[J], Sign Language & Linguistics 18(1) (2015), 41–89.

17.

Lee

and Yoo

, Low-Cost 3D Motion Capture System Using Passive Optical Markers and Monocular Vision[J], Optik International Journal for Light & Electron Optics 130 (2016), 1397–1407.

18.

Rahman

, Memetic Computing Based Numerical Solution to Troesch Problem, Journal of Intelligent & Fuzzy Systems 37(1) (2019), 1545–1554.

19.

Fry

A.C.

, Herda

T.J.

, Sterczala

A.J.

, et al., Validation of a motion capture system for deriving accurate ground reaction forces without a force plate[J], Big Data Analytics 1(1) (2016), 11.

20.

Rahman

M.M.

, Analysis of Finger Movements of a Pianist Using Magnetic Motion Capture System with Six Dimensional Position Sensors[J], Transactions of the Virtual Reality Society of Japan 15 (2017), 243–250.

21.

Liu

, et al., Indoor Intrusion Detection Based on Fuzzy Membership-aided Dempster-Shaper Theory, Journal of Intelligent & Fuzzy Systems 38(4) (2020), 3687–3696.

22.

Park

S.W.

, Park

H.S.

, Kim

J.H.

, et al., 3D displacement measurement model for health monitoring of structures using a motion capture system[J], Measurement 59 (2015), 352–362.

23.

Giannetti

, Petrella

, Bach

, et al., In vivo Bone Position Measurement Using High-Frequency Ultrasound Validated with 3-D Optical Motion Capture Systems: A Feasibility Study[J], Journal of Medical & Biological Engineering 37(7) (2017), 1–8.

24.

Zhang

, Wang

, Chu

, et al., Application of Optical Motion Capture Technology in Power Safety Entitative Simulation Training System[J], Optics & Photonics Journal 06(8) (2016), 155–163.

25.

Wang

S.K.

, Xie

and Hu

, Research on Protection Property of Running Sportswear Fabrics Based on 3-D Motion Capture System[J], Textiles and Light Industrial Science and Technology 3 (2014), 57–62.

26.

Miura

, Kaiga

, Shibata

, et al., Low-dimensional Feature Vector Extraction from Motion Capture Data by Phase Plane Analysis[J], Journal of Information Processing 25 (2017), 884–887.