Monitoring and analysis of athletes’ local body movement status based on BP neural network

Abstract

At present, the body recognition detection of athletes is mostly technical recognition, and the detection of exercise state is less, and the related research is basically blank. Based on this, based on BP neural network algorithm, this study develops athletes’ motion capture based on wearable inertial sensors, and builds a wireless signal transmission scheme based on sensor system. At the same time, this paper constructs the coordinate system to complete the attitude angle settlement and motion recognition and combines the athlete’s actual situation to establish the athlete’s limb trajectory calculation model and analyzes the athletes’ movement patterns. In addition, this paper combines neural network algorithm to analyze, and builds a neural network based athlete body motion recognition model, and analyzes the model effectiveness through simulation system. Studies have shown that when using time domain features+trajectory features as neural network inputs, the hand recognition rate is somewhat improved compared to the use of only time domain features as neural network inputs. It can be seen that the algorithm model of this study has certain validity and can be used as a reference for subsequent related research gradient theory.

Keywords

BP neural network athletes limb recognition motion capture

1 Introduction

The development of microelectronic technology and digital camera technology has promoted the development of human motion capture and detection technology. The classic optical motion capture technology uses the video device to obtain the image information of the human motion and analyzes the relevant image sequence to obtain the human motion characteristic parameters, thereby identifying the human motion state. However, the video device is large in size and high in power consumption, and is mainly suitable for fixed scenes, and is easily affected by external environmental factors such as illumination and shooting angle and cannot be continuously and widely recognized for human motion in a long time [1]. With the development of MEMS inertial sensors and the maturity of embedded technology and wireless sensor network technology, human motion capture and detection systems based on inertial sensors have become possible. It is generally made into a wearable form, and has the advantages of being light and simple, low in cost, convenient to wear, and strong in real time compared to the conventional optical human motion capture and detection system [2].

Moving object detection is to process the video sequence and extract the motion-generated area from the background, which is based on the detection technology of the underlying video information. At present, the main methods of moving target detection include background difference method, interframe difference method and optical flow method. The background difference method is to distinguish the image sequence from the reference background model to realize the detection of the moving target. The interframe difference method is based on the difference between time-series images and detects moving objects by inter-frame image changes. The optical flow method realizes moving object detection by combining similar vectors by calculating the optical flow vector in the image. In addition to the above three mature and sophisticated detection methods, some people have proposed the minimum energy function method and the learning-based method. In addition, a variety of traditional methods can be combined to achieve the detection of moving targets [3].

The human body motion capture and detection method of the wearable inertial sensor has problems such as low calculation accuracy of the human motion angle and low recognition rate of the human body, and most of the systems use the wired method for data transmission, which limits the movement of the human body to some extent [4]. Based on this, this thesis intends to combine MEMS inertial sensor technology with Zigbee wireless network technology to study high-precision attitude angle solving method and detection system, so as to achieve high-precision measurement and high-efficiency recognition of athletes’ local limb movement state.

2 Related work

The research work on human motion can be traced back to the research work on human perception by psychologist Johansson in 1973 [5]. In the experiment, he attached the highlight to the joint point of the person. Through the sequence of highlights of the movement, people can identify the movement of the person being tested. Since then, people have begun to recognize the importance and application value of motion analysis technology, and more and more universities, research institutes and commercial organizations have been involved in research work in this field. At the same time, some authoritative journals and academic conferences in the world also regard sports analysis as an important research content [6]. Through years of research and efforts, scientists have achieved certain results: In 1997, VSAM, led by the US Defense Advanced Research Projects Agency and the main video surveillance project of Carnegie Mellon University and Massachusetts Institute of Technology, was an intelligent surveillance system developed for military and civilian scenarios. It can centralize the video signals distributed in different locations to the command center and autonomously monitor the anomalies in the video. When an abnormality occurs, it can give the staff an alarm signal, which greatly reduces the labor intensity of the staff. Moreover, it requires only a small number of people to complete a wide range of monitoring tasks [7]; CMU and Samoff jointly developed a real-time video moving target classification and tracking system, which can cooperate with multiple cameras to identify people and vehicles in complex scenes and achieve continuous tracking [8]; Tsinghua University Human Body Information Monitoring Laboratory has developed a system suitable for measuring small offsets to measure the small shaking of the archers’ shoulders and elbows; Literature [9] implements a set of human motion analysis system, which uses manual methods to mark human joint points and uses Kalman filter to track the movements of long jumpers; In literature [10], a 3D active contour motion capture algorithm without target extraction is proposed, which combines motion capture with three-dimensional reconstruction by means of a human body model, and theoretically realizes motion capture of an unmarked human body.

In 1992, Yamato et al. [11] introduced HMM into human motion recognition. Since then, HMM and its improved models have been widely used in the field of human motion recognition, such as Mohiuddin Ahmad et al.’s multi-dimensional hidden Markov model (Multi-dimensional HMM) [12], Oliver et al.’s coupled hidden Markov model (CoupledHMM) [13], Luhr and Nguyen et al’s HierarchicalHMM, and Duong et al’s S-HSMM [14] et al. However, HMM belongs to the Genative Model, which has a strong independence assumption. That is, the observation value at time f depends only on the state at time f and is not particularly suitable for motion sequences with long distance dependence. At present, many domestic and foreign researchers have proved the defects and shortcomings of the generated model in human motion analysis through a large number of reasoning and experiments. In addition, some scholars have begun to use the discriminative model to model human motion. Among them, the most widely used is the Conditional Random Field (CRF) and its improved model [15]. Although the CRF can make good use of the context information in the action sequence for modeling and analysis, most of the current CRF models used for motion modeling assume that there is only one action mode at a time, and it does not consider the case where there are multiple actions at a time. However, in real life, people may have multiple modes of action at the same time. For example, a person can grab a head while walking, and it includes two actions of “walking” and “grabbing”, and the two actions are observed from different scales. According to the literature [16], it can be known that human motion contains multiple scales of motion details. Large scales generally refer to movements related to human trajectories, which reflect the relationship between people or people and the external environment, mesoscale refers to the movement related to the human body posture, and small scale refers to the movement of the human limb. Motion details at different scales have different roles in motion recognition, and there are certain constraints and restrictions between states of each scale. For example, people can “grab the head” while “walking”, but it is impossible for people to “kick their legs while walking”. The reason is that “walking” can be compatible with “scratching”, but it is not compatible with “kicking” [17].

3 Coordinate system and attitude matrix

According to the basic working principle of the system, before the measurement and recognition of the athlete’s arm movement state, it is necessary to first obtain the posture change during the arm movement. The attitude of the moving target is represented by the angle between the motion coordinate system and the reference coordinate system. Therefore, to calculate the attitude change during the arm movement, we must first determine the coordinate system. The coordinate systems involved are as follows [18]:

(1) Inertial coordinate system (i system)

The inertial coordinate system is without considering the earth’s orbit around the sun. It takes the center of the earth as the origin O, and the Z axis points to the Earth’s north pole along the Earth’s axis. The X and Y axes are perpendicular to the Z axis in the Earth’s equatorial plane and do not rotate with the Earth. Moreover, the X axis points to the vernal equinox, and the Y axis and the X axis and the Z axis form a right-hand coordinate system.

(2) Earth coordinate system (e system)

The Earth coordinate system has the Earth’s center as the origin O, and the X axis points to the Greenwich meridian, and the Z axis points to the Earth’s North Pole along the Earth’s axis, which is positive to the north, and the Y axis points to the east longitude 90-degree direction, which forms a right-handed coordinate system with the X-axis and the Z-axis. It rotates at an angular rate ω_ie of the Earth’s rotation relative to the inertial coordinate system.

(3) Geographical coordinate system (n series)

The coordinate origin O of the geographic coordinate system coincides with the centroid of the carrier, the X-axis direction points to the true north, the Y-axis direction points to the east, the Z-axis direction points to the sky, and the O-XYZ constitutes the right-handed rectangular coordinate system. The geographic coordinate system is also known as the O-NED coordinate system. Among them, N represents North (positive north), N axis corresponds to X axis, E represents East (positive east), E axis corresponds to Y axis, and D axis corresponds to Z axis. This paper uses the geographic coordinate system as the reference coordinate system.

(4) Carrier coordinate system (b series)

The carrier coordinate system refers to a coordinate system that moves together with the carrier. The origin O of the carrier coordinate system is at the center of gravity of the carrier, the X-axis is along the direction of motion of the carrier and the positive direction of which is the same as the direction of motion of the carrier, the Y-axis is perpendicular to the X axis and points to the right side of the carrier, and the Z axis and the X axis and the Y axis form a right hand rectangular coordinate system, which is vertically downward along the carrier. The joint coordinate system and sensor coordinate system of the athlete’s arm in this paper are the carrier coordinate system.

The angular position between the carrier coordinate system and the reference coordinate system can be described in three separate angles. It can be seen from Euler’s theorem that the rotation of the carrier coordinate system relative to the reference coordinate system (O - X_nY_nZ_n) can be achieved by three independent rotations. The axes X_nY_nZ_n point to the east, north, and sky directions, respectively, and we assume that the initial carrier coordinate system O - X_bY_bZ_b coincides with the reference coordinate system. The attitude angle of the carrier can be determined in turn as follows: $\begin{matrix} O - X_{n} Y_{n} Z_{n} \frac{Z_{n}}{φ} \to O - X_{1} Y_{1} Z_{1} \frac{X_{1}}{θ} \to \\ O - X_{2} Y_{2} Z_{2} \frac{Y_{2}}{γ} \to O - X_{b} Y_{b} Z_{b} \end{matrix}$

The rotation process is shown in Fig. 1 [19].

Fig. 1

Attitude angle description.

It can be seen that the three rotation angles uniquely determine the angular position of the carrier coordinate system in the reference coordinate system. The three corners in Fig. 1 are a set of Euler angles. Among them, θ is the pitch angle, γ is the roll angle, and φ is the heading angle.

The coordinate transformation matrix of the reference coordinate system to the carrier coordinate system in Fig. 1 is [20]: $\begin{matrix} C_{n}^{b} = C_{2}^{b} C_{1}^{2} C_{n}^{1} = [\begin{matrix} cos γ & 0 & - sin γ \\ 0 & 1 & 0 \\ sin γ & 0 & - cos γ \end{matrix}] \\ [\begin{matrix} 1 & 0 & 0 \\ 0 & cos θ & sin θ \\ 0 & - sin θ & cos θ \end{matrix}] [\begin{matrix} cos φ & - sin φ & 0 \\ sin φ & cos φ & 0 \\ 0 & 0 & 1 \end{matrix}] = \\ [\begin{matrix} cos γ cos φ + sin γ sin φ sin θ \\ sin φ cos θ \\ sin γ cos φ - cos γ sin φ sin θ \end{matrix} \\ \begin{matrix} - cos γ sin φ + sin γ sin φ sin θ \\ cos φ cos θ \\ - sin γ sin φ - cos γ cos φ sin θ \end{matrix} \\ \begin{matrix} - sin γ cos θ \\ sin θ \\ cos γ cos θ \end{matrix}] \end{matrix}$ (1) In the formula, $C_{n}^{b}$ is a unit orthogonal matrix, and there is $C_{n}^{b} = {[C_{n}^{b}]}^{- 1} = {[C_{n}^{b}]}^{T}$ (2)

4 Analysis of the movement of the athlete’s arm

To measure and identify the athlete’s arm movement state, we must first analyze the movement of the arm. The key to arm movement is the movement of the various joints of the arm, so motion analysis of each joint of the arm is required [21].

4.1 Installation of motion state detection module and establishment of its coordinate system

The arm motion state measurement and recognition system consists of three motion state detection modules and is mounted at the middle of the athlete’s chest, upper arm and lower arm, respectively. The motion state detecting module installed on the upper arm and the lower arm is mainly responsible for detecting the motion information of the arm, and the motion state detecting module installed on the chest is mainly used as a reference point. The three motion state detection modules are all terminal devices in the Zigbee network, and the detected data is transmitted to the coordinator connected to the computer through the Zigbee network. Moreover, the coordinator acts as a convergence point in the Zigbee network and uploads data to the computer through the serial port.

The motion state detection module installed at the chest position is used as a reference point O_c. When the arm is moving, the chest reference point position is fixed relative to the shoulder joint, and the coordinate direction of the installation is: The X_c axis points to the direction of the human head, the Z_c axis points to the rear of the athlete, and O_cX_cY_cZ_c constitutes the right-handed rectangular coordinate system. The coordinate systems of the two inertial motion detection modules of the upper and lower arms are: O_pX_pY_pZ_p and O_fX_fY_fZ_f. The two motion state detection modules are mounted in the same manner, both of which are X-axis parallel to the arm pointing forward, the Y-axis perpendicular to X to the outside, and the Z-axis perpendicular to the XY plane.

The coordinate system of the motion state detecting module installed at the chest position is a reference coordinate system (C system), and the coordinate system of the motion state detecting module mounted on the upper arm and the lower arm is a carrier coordinate system (b system).

The arm movement is the mutual rotation between the joints of the arm. Therefore, the shoulder joint, the elbow joint and the wrist joint are selected as the arm movement state, and the coordinate system of each joint is established as shown in Fig. 4. We assume that the shoulder coordinate system is S - X_SY_SZ_S, the elbow joint coordinate system is E - X_EY_EZ_E, and the wrist joint coordinate system is W - X_WY_WZ_W. The coordinate system of the shoulder joint is in the same direction as the reference coordinate system C, the X axis of the elbow joint and the wrist joint are parallel to the arm pointing forward, the Y axis is perpendicular to the X axis to the outside of the arm, the Z axis is directed to the rear of the athlete, and the three joint coordinate systems are in the same plane as the X-Y plane of the chest reference coordinate system C.

In Fig. 2, S is the shoulder joint, E is the elbow joint, and W is the wrist joint. l_p is the geometric length of the shoulder joint to the elbow joint, and l_f is the geometric length of the elbow joint to the wrist joint.

Fig. 2

Reference coordinate system of the arm position.

It can be seen from Fig. 2 that the detection system of the present invention only has the motion state detecting module installed on the upper arm and the lower arm, so the motion track of the end of the hand cannot be calculated. Therefore, this paper calculates the trajectory of the wrist joint as the trajectory of the end of the arm.

4.2 Motion trajectory solution

After calculating the change of the attitude angle of each joint during the arm movement, the trajectory of the athlete’s wrist can be calculated according to the geometric size of the arm and the position of the reference point. The attitude angle change calculated by the motion state detecting module (b system) mounted on the arm is relative to the geographic coordinate system (n system). Therefore, to obtain the motion trajectory of the wrist relative to the reference coordinate system (C system), coordinate transformation is required. Then, the transformation matrix of the elbow joint coordinate system (E system) to the C system can be derived as [22]: $C_{E}^{C} = C_{n}^{C} C_{E}^{n} = C_{n}^{C} {(C_{n}^{E})}^{T}$

In the formula, $C_{n}^{E}$ is a transformation matrix from n to E, and $C_{n}^{C}$ is a transformation matrix from n to c. Moreover, the transformation matrix of the wrist coordinate system (W series) to the C system is: $C_{W}^{C} = C_{n}^{C} C_{W}^{n} = C_{n}^{C} {(C_{n}^{W})}^{T}$

In the formula, $C_{n}^{W}$ is the transformation matrix from n to w.

It can be seen from Fig. 4 that the shoulder coordinate system is in the same direction as the reference coordinate system, and the coordinate position of the shoulder joint S point in the C system is: $p_{s} = {[x_{s}, y_{s}, 0]}^{T}$ (3)

Fig. 3

Comparison of normalized eigenvalues.

Fig. 4

Learning curve.

In the formula, x_s is the offset of the S joint of the shoulder joint with respect to the reference point O_c in the X-axis direction, and y_s is the offset of the S joint of the shoulder joint with respect to the reference point O_c in the Y-axis direction.

Since the X-axis of the elbow joint is parallel to the arm pointing forward, only the elbow joint has a component relative to the reference point in the X-axis direction. From the Equation (1), the coordinate position of the elbow joint E at the C system is: $p_{E} = l_{p} C_{E, 1}^{C} + p_{s}$ (4)

In the formula, $C_{E, 1}^{C}$ is the first column vector of the coordinate transformation matrix $C_{E}^{C}$ .

Similarly, the coordinate position of the wrist W point in the C system is: $p_{W} = l_{f} C_{W, 1}^{C} + p_{E}$ (5)

In the formula, $C_{W, 1}^{C}$ is the first column vector of the coordinate transformation matrix $C_{W}^{C}$ .

For each type of arm movement, 25 experiments were performed, and a total of 100 sets of arm motion state measurement data were obtained. The data of each group of arm movements is solved to obtain the motion trajectory of each group of wrists. In the calculation of the wrist motion trajectory, we take l_p = 0.3m, l_f = 0.25m, x_s = 0.2m, y_s = -0.2m.

4.3 Error analysis of motion trajectory

When calculating the trajectory of the wrist, there are many factors that affect the accuracy of the trajectory calculation, such as the installation accuracy of the sensor, the measurement accuracy of the sensor, and the accuracy of the solution angle of each joint. The factors causing the trajectory calculation error are analyzed, and the trajectory calculation error model is established to determine the influence degree of different factors on the trajectory calculation error, which can provide guidance for improving the trajectory calculation accuracy, and thus improve the arm movement state recognition rate.

According to formula (4) and formula (5), the calculation result of the trajectory at the wrist are related to the elbow joint attitude angle solution results, wrist joint attitude angle settlement results, chest posture angle settlement results, the geometric length “2” of the shoulder joint to the elbow joint, the geometric length f of the elbow joint to the wrist joint, and the installation position of the chest measuring unit p_s and other parameters. Among them, the calculation error of the trajectory caused by l_P, l_f and p_s is the trajectory calculation error caused by the installation error of the measurement unit. Moreover, the remaining factors are the trajectory calculation error caused by the error of the joint attitude angle calculation and the trajectory error caused by the attitude angle calculation error, which can be regarded as the sensor measurement error. Therefore, there are two main types of trajectory errors calculated in this paper: installation error and measurement error.

The main installation errors are: Δl_p, Δl_f, Δx_s, Δy_x,

The measurement errors are mainly:

$Δ θ_{E}, Δ γ_{E}, Δ φ_{E}, Δ θ_{W}, Δ γ_{W}, Δ φ_{W}, Δ θ_{C}, Δ γ_{C}, Δ φ_{C}$

It can be seen that the calculated trajectory error has 4 installation errors and 9 measurement errors, and reducing these errors can effectively improve the trajectory calculation accuracy. Here, we assume that the installation error is zero, and only analyze the influence of the attitude angle error of each joint on the calculation accuracy of the trajectory. Since the human torso remains motionless during the experiment, the chest posture angle is fixed. Therefore, it can be set that Δθ_C, Δγ_C, Δφ_C is zero. The following is an analysis of the trajectory calculation error caused by the elbow joint attitude angle error Δθ_E, Δγ_E, Δφ_E, and the wrist joint attitude angle error Δθ_W, Δγ_W, Δφ_W.

After the total differential is carried for the equation (4), the following formula can be obtained: $Δ P_{E} = l_{p} C_{n, 1}^{C} J_{E} Δ δ_{E}$ (6)

In the formula, ΔP_E is the position error of the elbow joint, ΔP_E = (Δx_EΔy_EΔz_E) ^T; $C_{n, 1}^{C}$ is the first row vector of the coordinate transformation matrix $C_{n}^{C}$ ; J_E is the Jacobian matrix of the first row vector of the coordinate transformation matrix $C_{n}^{E}$ ; Δδ_E is the error vector of the elbow joint attitude angle. Δδ_E = [Δθ_EΔγ_EΔφ_E] ^T. $C_{n, 1}^{C}$ and J_E are shown in Equations (7) and (8), respectively. $\begin{matrix} C_{n, 1}^{c} = [\begin{matrix} sin γ_{C} cos φ_{C} + sin γ_{C} sin φ_{C} sin θ_{C} \\ - cos γ_{C} sin φ_{C} + sin γ_{C} cos φ_{C} sin θ_{C} \\ - sin γ_{C} cos θ_{C} \end{matrix}] \\ = [\begin{matrix} sin γ_{E} sin φ_{E} cos θ_{E} \\ sin γ_{E} cos φ_{E} cos θ_{E} \\ sin γ_{E} sin θ_{E} \end{matrix} \\ \begin{matrix} cos γ_{E} sin φ_{E} \sin θ_{E} - sin γ_{E} cos φ_{E} \\ cos γ_{E} cos φ_{E} sin θ_{E} + sin γ_{E} sin φ_{E} \\ - cos γ_{E} cos θ_{E} \end{matrix} \\ \begin{matrix} sin γ_{E} cos φ_{E} sin θ_{E} - \cos γ_{E} \sin φ_{E} \\ - sin γ_{E} sin φ_{E} sin θ_{E} - cos γ_{E} cos φ_{E} \\ 0 \end{matrix}] \end{matrix}$ (7)

After equation (5) is taken to be total differentiated, the following formula [25 –27] can be obtained: $Δ P_{w} = l_{f} C_{n, 1}^{C} J_{w} Δ δ_{w} + Δ P_{E}$ (8)

In the formula, ΔP_w is the position error of the wrist joint.

ΔP_W = (Δx_WΔy_WΔz_W) ^T. J_w is the Jacobian matrix of the first row vector of the coordinate transformation matrix $C_{n}^{W}$ , and Δδ_w is the error vector of the wrist joint attitude angle Δδ_w = (Δθ_WΔγ_WΔφ_W) ^T, J_w is shown in formula (9): $\begin{matrix} J_{W} = [\begin{matrix} sin γ_{W} sin φ_{W} cos θ_{W} \\ sin γ_{W} cos φ_{W} cos θ_{W} \\ sin γ_{W} sin θ_{W} \end{matrix} \\ \begin{matrix} cos γ_{W} sin φ_{W} \sin θ_{W} - sin γ_{W} cos φ_{W} \\ cos γ_{W} cos φ_{W} sin θ_{W} + sin γ_{W} sin φ_{W} \\ - cos γ_{W} cos θ_{W} \end{matrix} \\ \begin{matrix} sin γ_{W} cos φ_{W} sin θ_{W} - \cos γ_{W} \sin φ_{W} \\ - sin γ_{W} sin φ_{W} sin θ_{W} - cos γ_{W} cos φ_{W} \\ 0 \end{matrix}] \end{matrix}$ (9)

Equation (8) is the error model of the motion trajectory at the wrist.

5 Identification algorithm model analysis

5.1 Feature extraction

In the process of athlete’s limb recognition, sensors are needed for information transmission, and the acceleration, angular velocity and magnetic field information during limb movement are measured. Therefore, features can be extracted directly from the data measured by each sensor to reflect the motion state of the arm. For a set of time-varying signals {x₁, x₂, . . . , x_n}, its statistical parameters are generally extracted as eigenvalues. Mean, variance, peak, crest factor, skewness, waveform factor, pulse factor, margin factor, and kurtosis are common statistical feature parameters. The sensor measurement signals extracted in this paper have more time domain features. The mean ( $M = \frac{1}{n} \sum_{i = 1}^{n} x_{i}$ ) reflects the average intensity of the data; the variance $(var = \frac{1}{n} \sum_{i = 1}^{n} {(x_{i} - \bar{x})}^{2})$ indicates the degree to which the data deviates from the mean; the skewness $(k_{s} = \frac{n \sum_{i = 1}^{n} {(x_{i} - \bar{x})}^{3}}{(n - 1) (n - 2) d_{s}^{3}})$ measures the direction and extent of the data skew. In the formula, d_s is the sample standard deviation; the maximum (x_Max = x (max)) and minimum (x_Min = x (min)) describe the maximum and minimum amplitude of the signal.

Speed and three-axis magnetic field information. Although the time domain feature can be extracted for each axis direction of each sensor, the use of the three-axis total feature quantity to represent the characteristics of the sensor not only simplifies the feature quantity, but also avoids the influence of signal instability of a single axis, which reduces the requirement for the wearing mode of the sensor. This article contains two measurement modules for the upper arm and the lower arm, and the characteristics of the two modules are also integrated. The formula for calculating the comprehensive feature quantity is as follows:

$F = \sqrt{F_{E, x}^{2} + F_{E, y}^{2} + F_{E, z}^{2} + + F_{W, x}^{2} + F_{W, y}^{2} + F_{W, z}^{2}}$

In the formula, F is the comprehensive feature quantity. The experiment found that the magnetic field signal only has a significant change in the skewness characteristic. Therefore, the magnetic field signal only extracts the skewness feature, while the acceleration and angular velocity extract all the features, and a total of 11 time domain feature quantities are extracted. Because the magnitudes of the four types of arm motion eigenvalues are different, it is not convenient to compare the differences between the same features of different arm movements and considering the input arm motion recognition algorithm input value, this paper normalizes all eigenvalues to the [0,1] interval. The normalization function is $x_{io} = \frac{x_{i} - x_{min}}{x_{max} - x_{min}}$

In the formula, x_i and x_io are the normalized before and after values of the i-th feature, and x_max and x_min are the maximum eigenvalue and the minimum eigenvalue of the same feature of different arm movements, respectively. The time domain normalized eigenvalues of the four motion states are obtained. We assume that the recognition target vectors for the four arm movements are represented as follows: $k_{1} = \frac{y_{m} - y_{1}}{x_{m} - x_{1}}$ $k_{2} = \frac{y}{x} \frac{(max) - y (min)}{x (max) - x (min)}$ $S = (y (max) - y (min)) \cdot (x (max) - x (min))$ $P = \frac{S}{2 (y (max) - y (min)) + (x (max) - x (min))}$ $vk = \frac{σ_{x}}{σ_{y}} = \frac{\sqrt{\frac{1}{n} \sum_{i = 1}^{m} {(x_{i} - \bar{x})}^{2}}}{\sqrt{\frac{1}{n} \sum_{i = 1}^{m} {(y_{i} - \bar{y})}^{2}}}$

Among them, k₁, k₂, S, P, vk represent the endpoint slope, endpoint slope, area, area to perimeter ratio, and variance ratio, respectively. N indicates the number of intersections between the graph and the center line.

According to the above method, the trajectory characteristics of the wrist are extracted, and the trajectory characteristics of the wrists of the four types of arms are obtained, as shown in Table 1. As can be seen from the table, the difference in the same eigenvalue size of the four types of actions is obvious. The same kind of motion was repeatedly tested, and the reproducibility of the extracted six sets of eigenvalues was also better, indicating that the feature quantity selection in this paper is suitable. It is normalized, and the results are shown in Fig. 1.

Table 1
Four types of trajectory characteristics of the wrists

Horizontal line Vertical line Slash Closed line

k ₁ 0.04 7.32 3.91 2.32

k ₂ 0.15 6.85 3.90 2.33

vk 0.01 43.92 17.61 8.44

S 22.38 198.52 275.97 2281.57

P 1.21 2.41 36.36 13.19

N 2 2 2 4

	Horizontal line	Vertical line	Slash	Closed line
k ₁	0.04	7.32	3.91	2.32
k ₂	0.15	6.85	3.90	2.33
vk	0.01	43.92	17.61	8.44
S	22.38	198.52	275.97	2281.57
P	1.21	2.41	36.36	13.19
N	2	2	2	4

5.2 Neural network

In order to realize the recognition of the athlete’s arm movement state, this paper designs a three-layer BP neural network. It can be seen from the above that each type of action extracts 11 kinds of time domain feature quantities and 6 kinds of hand motion track feature quantities of the original signal, so the number of input layer units of the BP neural network is 17. There are four types of arm movements that need to be identified, so the number of output layer units of the BP neural network is four.

The choice of the number of hidden layer units is related to the number of input and output units and the requirements of the problem. If the number of hidden layer units is too large, it will lead to problems such as long learning time, complicated network structure, difficulty in convergence, and the like, and the accuracy of the final network becomes low. If the number of hidden layer units is too small, the network generalization ability is weak, the fault tolerance is poor, and the performance of the network is not good. Therefore, it is necessary to find an optimal number of hidden layer units. The following three formulas are reference formulas for selecting the optimal number of cells in the hidden layer. $\sum_{i = 0}^{n} C_{m}^{i} > N$

In the formula, m is the number of hidden layer units, n is the number of input layer units, and N is the number of samples. If there is i > m, then $C_{m}^{i} = 0$ is obtained. $m = \sqrt{n + l} + a$

In the formula, l is the number of neurons in the output layer, and a is a constant between [1, 10]. m = log ₂n.

In this paper, 25 sets of data were detected for each type of arm movement, so there were 100 sets of sample data, 30 sets of data were used for BP neural network training, and 70 sets of data were tested by BP neural network. According to the optimal number of hidden layer units, the reference formulas (4.10) to (4.12) are selected, and 4 < m<14 can be obtained. The value of m is sequentially brought into the network for training, and the training result of Table 2 is obtained. The transfer function of the first layer network of network training is tansig, the transfer function of the second layer network is logsig, the training function is runningdx, the maximum number of training is 1000, the training target error is 0.001, and the learning rate is 0.1%.

Table 2
Identification target vectors for four arm movements

Arm movement Target vector

Horizontal line (1, 0, 0, 0)

Vertical line (0, 1, 0.0)

Slash (0, 0, 1,0)

Closed line (0, 0, 0, 1)

Arm movement	Target vector
Horizontal line	(1, 0, 0, 0)
Vertical line	(0, 1, 0.0)
Slash	(0, 0, 1,0)
Closed line	(0, 0, 0, 1)

When the number of hidden layers is 8, the number of network training steps is the least, and the network training error is similar to the minimum error. Therefore, the number of hidden layer units is selected to be eight. The learning curve obtained on this basis is shown in Fig. 4.

After 109 steps of training and network convergence, the set target error value of 0.001 is reached. In this paper, 70 sets of data are used to test the trained BP neural network, and the test output error E of the network can be calculated. When the test output error E of the network is less than the target value, the recognition is passed, otherwise the recognition effect is not achieved. Figure 5 is the magnitude of the mean square error E of the actual output of each set of data when the 70 sets of data are tested using the time domain feature and the time domain feature+track feature, respectively.

Fig. 5

Network output error of the test sample.

As can be seen from Fig. 5, when only the time domain features are used for identification, 11 sets of test samples are not recognized, including 4 sets of horizontal line test samples, 4 sets of vertical line test samples, 2 sets of diagonal line test samples and 1 set of closed line test samples. When using the time domain feature+track feature to identify, there are 2 sets of test samples that are not recognized, including 1 set of vertical line test samples and 1 set of closed line test samples. Therefore, the recognition rate of the four kinds of motions of the BP neural network under the input of different feature quantities can be obtained, as shown in Table 3, and it is drawn into a statistical graph as shown in Fig. 6.

Table 3

Identification rate statistics table

	Time domain feature	Time domain feature+trajectory feature
Draw horizontal line	80%	100%
Vertical line	81%	98%
Slash	90%	100%
Drawing closed line	97%	97%
average value	87%	100%

As can be seen from Table 3 and Fig. 6, when the time domain feature+trajectory feature is used as the neural network input, the recognition rate of the arm motion is increased by 13% compared with the use of only the time domain feature as the neural network input. In general, the more BP neural network training input samples, the more sample categories it contains, the higher the fault tolerance rate, and the higher the training recognition rate.

Fig. 6

Statistical graph of recognition rate.

6 Analysis of results

This study introduces the algorithm for the recognition of athletes’ arm movement status. Firstly, this paper extracts the time domain features of the sensor signals in the arm movement and the trajectory characteristics of the wrist and determines the feature quantity to be extracted by the arm motion state recognition algorithm. Then, this paper uses two recognition algorithms, BP neural network and support vector machine, to identify and classify the motion states of the four types of arms. In addition, this paper compares the results of time domain feature input and time domain feature+track feature input under two algorithms. The calculation results show that the input recognition rate of the time domain feature+trajectory feature proposed in this paper is higher than the input recognition rate of the time domain feature.

At the same time, this paper studies the high-precision attitude angle solving method. In this paper, the Kalman filter algorithm is used to effectively combine the inertial system and the attitude reference system, which improves the accuracy of attitude angle calculation. Moreover, according to the magnitude of the motion acceleration of the carrier, the magnitude of the measured noise variance of the Kalman filter is adjusted in time, so as to reduce the influence of the motion acceleration on the attitude accuracy of the Kalman filter. The method not only overcomes the influence of carrier motion acceleration, but also compensates for the cumulative error of the inertial system. Compared with the classical Kalman filter algorithm, it can obtain higher attitude angle resolution accuracy.

In addition, this paper analyzes the arm movement mode and establishes the arm movement trajectory calculation model. Through the actual test of four typical arm movement states, such as drawing horizontal line, vertical line, oblique line and closed line, the motion track of the corresponding wrist is obtained by solving the problem. Moreover, by comparing with the theoretical trajectory, it is verified that the correctness of the model is verified. In addition, this paper establishes the trajectory calculation error model and analyzes the influence of the attitude angle of each joint on the trajectory calculation accuracy. It is found that the heading angle error of each joint is the main reason that affects the accuracy of the trajectory calculation.

The more input samples the neural network trains, the more sample categories it contains, the higher the fault tolerance rate, and the higher the training recognition rate. If necessary, it can be studied in conjunction with a support vector machine.

7 Conclusion

In this paper, the sensor is used to capture and detect the movement state of the human body, and the high-precision attitude angle calculation method is studied, and the detection and high-efficiency recognition of the athlete’s local limb movement state are realized. According to the basic working principle of the system, to measure and recognize the athlete’s arm movement state, it is necessary to first obtain the posture change during the arm movement. Combining the physiological structure of the athlete’s arm with the arm training in general rehabilitation training, this article simplifies the athlete’s arm into a five-degree-of-freedom robotic arm, including the upper arm, lower arm, and hand. The five degrees of freedom are the rotational freedom of the three shoulders, the freedom of rotation of the elbows, and the freedom of rotation of the wrist. In order to realize the recognition of the athlete’s arm movement state, this paper designs a three-layer BP neural network. In this paper, the time domain feature + trajectory feature is input as a neural network, and the recognition rate of arm motion is increased by 13% compared with the use of only time domain features as neural network input. In addition, the more BP neural network training input samples, the more sample categories it contains, the higher the fault tolerance rate, and the higher the training recognition rate.

References

Wang

, Li

, Gao

, et al., Action recognition from depth maps using deep convolutional neural networks, IEEE Transactions on Human-Machine Systems 46(4) (2016), 498–509.

, Hou

, Wang

, et al., Joint distance maps based action recognition with convolutional neural networks, IEEE Signal Processing Letters 24(5) (2017), 624–628.

Paul Ijjina

and Krishna Mohan

, Human action recognition using genetic algorithms and convolutional neural networks, Pattern Recognition, 2016:S0031320316000169.

Hou

, Li

, Wang

, et al., Skeleton optical spectra-based action recognition using convolutional neural networks, IEEE Transactions on Circuits and Systems for Video Technology 28(3) (2018), 807–811.

Tacchetti

, Isik

and Poggio

, Spatio-temporal convolutional neural networks explain human neural representations of action recognition, Plos Computational Biology 13(12) (2016), 77–84.

, Cheng

, Su

, et al., Stratified pooling based deep convolutional neural networks for human action recognition, Multimedia Tools and Applications 76(11) (2017), 13367–13382.

Wang

, Zhao

, Gong

, et al., Parallelizing convolutional neural networks for action event recognition in surveillance videos, International Journal of Parallel Programming 45(4) (2017), 734–759.

Aouaidjia

, Bin

, Po

, et al., Deep convolutional neural networks for human action recognition using depth maps and postures, IEEE Transactions on Systems, Man, and Cybernetics: Systems 54(12) (2018), 1–14.

Wang

, Li

, Gao

, et al., Depth pooling based large-scale 3d action recognition with convolutional neural networks, IEEE Transactions on Multimedia 56(12) (2018), 91–95.

10.

, Tong

and Tang

, Modelling human body pose for action recognition using deep neural networks, Arabian Journal for Science and Engineering 56(2) (2018), 45–56.

11.

Yangyang

, Jun

, Lei

, et al., Ensemble one-dimensional convolution neural networks for skeleton-based action recognition, IEEE Signal Processing Letters 25(7) (2018), 1044–1048.

12.

Chen

, Weng

, Lu

, et al., Deep manifold learning combined with convolutional neural networks for action recognition, IEEE Transactions on Neural Networks and Learning Systems 29(9) (2017), 1–15.

13.

Chaitra

B.H.

, Anupama

H.S.

and Cauvery

N.K.

, Human action recognition using image processing and artificial neural networks, International Journal of Computer Applications 80(9) (2014), 31–34.

14.

Vadapalli

and Vadapalli

, Facial action unit recognition from video streams with recurrent neural networks, International Journal of Computer Applications 96(19) (2014), 31–39.

15.

Francisco

Ordó?ez

and Daniel

, Deep convolutional and lstm recurrent neural networks for multimodal wearable activity recognition, Sensors 16(1) (2016), 115.

16.

Ordóñez , Javier

, Roggen , et al., Sensors, Vol. 16, Pages 115: Deep Convolutional and LSTM Recurrent Neural Networks for Multimodal Wearable Activity Recognition, Sensors 16(1) (2016).

17.

Ronao

C.A.

and Cho

S.B.

, Human activity recognition with smartphone sensors using deep learning neural networks, Expert Systems with Applications 59(C) (2016), 235–244.

18.

Wang

, Cao

, Shen

, et al., Temporal pyramid pooling based convolutional neural networks for action recognition, IEEE Transactions on Multimedia 27(12) (2015), 2613–2622.

19.

Ijjina

E.P.

and Mohan

C.K.

, Hybrid deep neural network model for human action recognition, Applied Soft Computing, 2015:S1568494615005268.

20.

Jung

E.Y.

, Personalized urination activity recognition based on a recurrent neural network using smart band, International Neurourology Journal 22(Suppl 2) (2018), S91–100.

21.

Kautz

, Groh

B.H.

, Hannink

, et al., Activity recognition in beach volleyball using a deep convolutional neural network, Data Mining and Knowledge Discovery 45(23) (2017), 45–57.

22.

Kavi

, Kulathumani

, Rohit

, et al., Multiview fusion for activity recognition using deep neural networks, Journal of Electronic Imaging 25(4) (2016), 043010.

23.

Mohanty

Sachi Nandan

, et al., Optimal rough fuzzy clustering for user profile ontology based web page recommendation analysis, Journal of Intelligent & Fuzzy Systems 37(1) (2019), 205–216.

24.

Zae

, Young-Seob

, Hyung

, et al., Investigating the impact of possession-way of a smartphone on action recognition, Sensors 16(6) (2016), 812–826.

25.

, An

, Bennamoun

, et al., SkeletonNet: mining deep part features for 3D action recognition, IEEE Signal Processing Letters 23(9) (2017), 71–80.

26.

Sheng

Yajun

, Xu

Ming

and Jin

Zhenlin

, Research on evaluating brand marketing of leisure tourism city with fuzzy information, Journal of Intelligent & Fuzzy Systems 37(2) (2019), 1679–1686.

27.

Liu

, Shu

, Tang

, et al., Computational model based on neural network of visual cortex for human action recognition, IEEE Transactions on Neural Networks and Learning Systems 23(6) (2017), 1–14.

Monitoring and analysis of athletes’ local body movement status based on BP neural network

Abstract

Keywords

1 Introduction

2 Related work

3 Coordinate system and attitude matrix

4.1 Installation of motion state detection module and establishment of its coordinate system

5.1 Feature extraction

Table 1 Four types of trajectory characteristics of the wrists Horizontal line Vertical line Slash Closed line k 1 0.04 7.32 3.91 2.32 k 2 0.15 6.85 3.90 2.33 vk 0.01 43.92 17.61 8.44 S 22.38 198.52 275.97 2281.57 P 1.21 2.41 36.36 13.19 N 2 2 2 4

Table 2 Identification target vectors for four arm movements Arm movement Target vector Horizontal line (1, 0, 0, 0) Vertical line (0, 1, 0.0) Slash (0, 0, 1,0) Closed line (0, 0, 0, 1)

7 Conclusion

References

Table 1
Four types of trajectory characteristics of the wrists

Horizontal line Vertical line Slash Closed line

k ₁ 0.04 7.32 3.91 2.32

k ₂ 0.15 6.85 3.90 2.33

vk 0.01 43.92 17.61 8.44

S 22.38 198.52 275.97 2281.57

P 1.21 2.41 36.36 13.19

N 2 2 2 4

Table 2
Identification target vectors for four arm movements

Arm movement Target vector

Horizontal line (1, 0, 0, 0)

Vertical line (0, 1, 0.0)

Slash (0, 0, 1,0)

Closed line (0, 0, 0, 1)