Video analysis method of basketball training assistant based on deep learning theory during COVID-19 spread

1 Introduction

Since its onset, the COVID-19 pandemic has spread to almost all countries of the world. Social and physical distancing measures, lockdowns of businesses, schools and overall social life, which have become commonplace to curtail the spread of the disease, have also disrupted many regular aspects of life, including sport and physical activity. In order to keep basketball training during COVID-19 spread, the basketball video analysis is used.

Deep learning is an ant colony algorithm based on the machine learning theory of neural network, which is used to optimize the technical and tactical decision of basketball project [1]. The function of the model is to optimize the data adaptively through deep learning in the mass of technical and tactical data, so as to achieve the purpose of adapting to the technical and tactical decision-making of sports. Basketball is an advantageous sport in China, and there are many researches on the theory of technology and tactics. It is a very complex combinatorial optimization problem that can accurately and flexibly use technology, change tactics and choose correct tactics decision in basketball match. It has the characteristics of non-linear, multi constraint and discreteness [2]. Based on the characteristics of ant colony optimization algorithm and deep learning adaptive self-organizing learning ability, combined with the technical and tactical characteristics of basketball project, this paper studies the optimization algorithm model suitable for winning technical and tactical decision-making of basketball project, which can effectively improve the technical and tactical decision-making ability of basketball project and guide the application of technical and tactical decision-making of the project.

Shooting is the total name of all kinds of action methods used by the players who hold the ball to put the ball into the basket from the top of the basket [3]. The cumulative number of shooting scores determines the outcome of the game. Therefore, shooting becomes the focus of competition between offensive and defensive sides, and is the core technology of basketball. With the development of basketball and the improvement of athletes’ form and function quality, the shooting technology is developing continuously, the shooting position is from low to high, and the shooting speed is from slow to fast [4]. There are more and more ways of shooting and the rate of shooting is increasing. The evolution and development of shooting technology marks the leap of basketball from one stage to another

Students’ daily training includes physical training, shooting, rebounding, defense, fast break, etc [5]. Shooting training is the longest running in process of basketball skills. Shooting training also includes free throws, jump shots, three-point shots and other training. As the basic shooting training, penalty basket training is the beginning of the technical training. In this stage, once players develop any irregular technical action, it will directly affect the instability of their shooting percentage, and cannot further improve their shooting percentage [6]. Therefore, in the training stage of the penalty basket, it is particularly important to cultivate the shooting skills and rhythm habits of the penalty basket. At the same time, due to the high intensity and high antagonism of the game, the penalty basket brought by fouls is the most simple and effective means of scoring, which often affects the trend of the whole game, even the outcome of the game in the later stage of the game [7–10].

In the game, there are often “chop and kill” tactics against the strength players who are not good at the penalty basket. It is a kind of tactics to get the ball right and then get the initiative attack through the foul of such players, especially when the remaining time of the game is less than 1 minute, it is most frequently used. Due to the lack of flexibility of the strength players, the rhythm and technical steps of the penalty basket are often wrong, which leads to the shooting rate of the penalty basket of this type of players is always lower than 50% [11]. At the same time, with the development of the competition, the physical strength of the athletes has declined seriously [12]. A good penalty basket technique should be the most labor-saving way to get the best corner technique. This requires athletes to analyze their speed and angle of release in the daily training of penalty basket, and find out the closest penalty basket technique to the theoretical speed and angle of release in combination with their physical conditions

Deep learning theory is combined with this paper. This paper studies an analysis method of free throw based on basketball video teaching.

2 Deep learning research

With the improvement of computer performance and image computing ability, it is more and more common to apply video image related technology to the scene of sports analysis [13]. Multiple cameras are used to cover the whole football game, and the actions and tracks of each player on the field are captured. In this paper, a tracking algorithm is proposed to get the track of football players in the game [14]. By building an individual model based on characteristics and athletes’ movement trend, the algorithm can identify the target well and help the coach to analyze the players’ tactical position and movement amount in the game. In the study of Li et al, a human body fitting model based on morphological technology is proposed to locate the four key joints of the body, extract the angle information of the body and the angle of entering the water, and realize the purpose of assisting the diver to correct the air movement and the position of entering the water in daily training. Ren Hongyu, from China University of mining and technology, proposed a non-invasive synchronization method based on image features without interfering with normal training, and studied the data acquisition and synchronization method of shot put. In this method, an algorithm is proposed to automatically obtain shot putting frame. The algorithm can pre judge shot frame, filter out the image sequence without shot frame, then get shot position through Hough transform, further verify and filter shot frame, and eliminate the false shot frame before shot frame [15]. After getting the shot frame, on this basis, the track change of shot in the adjacent frame is analyzed, and the shot angle and speed are obtained. Andrea fossati et al. used monocular camera to extract the three-dimensional motion posture of human body, and applied it to identify and detect golf players’ swing. By comparing the detected posture information of human skeleton with that of professional athletes, the corresponding guidance is given.

The concept of deep learning is based on the theory of artificial neural network, which was put forward in 2006. Based on the artificial neural network, the deep learning structure is composed of multi hidden layers and multi perceptron. The original attribute class or feature representation can form a more abstract high-level by combining the low-level features, so as to discover the distributed feature representation of data. Deep confidence network (DBN) proposes a layer by layer unsupervised greedy training algorithm to solve more deep structure related optimization problems. At the same time, a real multi-layer structure learning algorithm based on convolutional neural network is proposed, which reduces the number of parameters to improve the training performance. There are two kinds of deep learning methods: supervised learning and unsupervised learning, which construct different learning models. Among them, the first convolutional neural network (CNN) is a deep supervised learning machine learning model; the second deep confidence network (DBNs) is an unsupervised learning machine learning model

The essence of deep learning optimization is to learn more useful features in the data by building machine learning model with high-level hidden layer and massive training data, and finally improve the accuracy of classification and prediction, including deep learning model with multi-level hidden layer, as shown in Fig. 1. In depth learning, different from the traditional artificial neural network learning, it is reflected in depth model structure and feature learning: (1) the depth of model structure is increased from three layers of shallow hidden layer to five layers, six layers and even one. There are more hidden layer nodes in the layer; (2) clear feature learning, through layer by layer feature training, learning samples in the original feature into new feature learning, making classification and prediction easier.

OPEN IN VIEWER

Fig. 1

Deep learning model with multiple hidden layers.

Basketball teaching project is an advantage project in our country’s competitive sports. There are many theoretical researches on the technology and tactics of this project. From the theoretical study of event group, we can get the winning factors of basketball events are fast, ruthless, complete, accurate and flexible. In basketball, one of the important winning factors of free throw ability in the game is to change the technical and tactical decision-making in time and surprise in the process of the game to control the whole game and win the final game. Most of these winning factors are mainly reflected in the technical and tactical features. The quantity and quality of video in basketball project influence each other, the quantity is flexible and changeable can help to improve the quality of tactics, and the use of high-quality tactics can make up for the shortage of tactics quantity.

The technical route of this study is shown in Fig. 2. Research questions were raised. The content of the study was determined. The research methods are selected and the data of basketball technical and tactical cases are collected. Statistical analysis was carried out. The data is optimized by deep learning, and the ant colony algorithm model is designed. The optimization model is tested by practical application, and the expected results are achieved

P ij k = { r _ 1 j a ( t ) , η ij b ( t ) ∑ r _ 1 j a ( t ) , η ij b ( t ) 0 , oterwise , j ∈ allowedk , s ∈ allowedk(1)

OPEN IN VIEWER

Fig. 2

Technology roadmap.

Based on the above description of the characteristics of deep learning theory, an ant colony algorithm of deep learning optimization for basketball technical and tactical decision-making is established to solve the optimization problem of actual technical and tactical decision-making. The optimization process is as follows: (1) set the number of iterations and initialization of pheromone data. (2) Record the information of each confrontation process, including the time and the combination of technology and tactics to build the taboo table of technology and tactics decision of both sides of badminton vs. the data of taboo table is self-organizing and deep learning by using deep learning, and the decision-making combination taboo table of output results is used to judge whether it is in line with the optimal technology and tactics decision-making route. (3) The task of each ant is to find the best decision-making route. (4) Update the data in the tabu list to determine whether the best win has been achieved, otherwise, turn to step (3) to start again, and step (5) will continue to reach the optimal; and (5) to complete the confrontation process when the technical and tactical decision-making is optimal. If the number of iterations reaches M 2, the output optimal solution will be completed; otherwise, I = + 11, go to step (2) for the next optimization.

In order to overcome the limitation of manual playback of each frame in the traditional auxiliary training mode, the system proposed in this paper is automatic without manual intervention. Shot frame detection is the first step of automatic video analysis. The algorithm of the assistant system takes the shot frame as the starting point, and then obtains the frame set of the penalty basket and the shot state information of the basketball.

In order to extract the features of the joint points of the shooting arm, the first step of the algorithm is to mark the joint points of the shooting arm. Then, the foreground is extracted by Gaussian background modeling, and on this basis, the contour of the marked point is further obtained by marking features, and the coordinate information of the joint point is obtained by statistical analysis of the contour [40]. Finally, the mathematical characteristics of the hand frame are analyzed by the changes of the vertical coordinates of the marked joint points and the distance between the ball and the wrist joint points. And through the solution of the mathematical model to get the hand frame accurately.

In the whole process, the arm can be approximately considered to be composed of two rigid bodies, so in the study, the arm’s penalty action is analyzed by recording the coordinate track information of the joint points of the arm. By marking the joint point of the shooting arm, we can replace the action of the arm in the process of penalty shooting. Because basketball players in the game and daily training, will take wrist or elbow protectors, so in the corresponding protective devices on the color mark, the impact on the player’s penalty action can be minimized. As shown in Fig. 3.

OPEN IN VIEWER

Fig. 3

Arm marking diagram.

As shown in Fig. 3, point a represents the mark point close to shoulder joint, point B represents the mark point of elbow joint, and point C represents the mark point of wrist joint.

In the research, we first need to extract the foreground of the input video unit, on this basis, we only traverse the foreground pixels, which can improve the efficiency of the algorithm, and then compare the corresponding RGB values, set the pixel points that meet the range of the color of the mark to 255, the other pixel points to 0, and get the feature binary image containing the mark points. In order to remove the possible irrelevant characteristic points, carry out corresponding corrosion and expansion. Through the above method, the binary graph containing only the marked points is obtained. Then the contour of binary image is extracted [42], and the contour that can contain each mark point is obtained. Through the statistical analysis of the pixel coordinates of the contour block, the statistical centroid of the pixel block is obtained as the joint point coordinates. Through the geometric position relationship of three marked points, three points are distinguished. In this way, the two-dimensional coordinates of three joint points in each frame are obtained, and the mathematical characteristics of the shot frame and the shot arm in the penalty basket technique are analyzed through the track changes of the coordinates.

Video input: through the camera to collect the players’ penalty action, each time 20 groups of penalty as a basic video analysis unit. Foreground extraction: select the method of Gaussian background modeling to extract the foreground. Before the mixed Gaussian foreground extraction, the Gaussian filtering of each frame is beneficial to the foreground extraction and can reduce the noise interference in the background. Mark feature segmentation: through traversal and comparison of RGB values of pixel points, the mark points matching the mark are obtained. At the same time, before RGB traversal and comparison, the foreground frame will be filtered to smooth the color range of marker points. After traversing and comparing, the image is corroded and expanded to reduce noise interference. Feature contour extraction to obtain coordinates: extract the contour of the pixel block [, and use statistical analysis to obtain the coordinates of the marked points. Since the same color mark is used, the obtained coordinates need to be distinguished. Coordinate differentiation: because the origin of the pixel coordinates of the computer stored pictures starts from the upper left corner, downward is the positive direction of the Y axis (row coordinates), and right is the positive direction of the X axis (column coordinates). Therefore, point is distinguished by the smallest X-axis coordinate corresponding to point a on the shoulder, and point B and point C are distinguished by the distance between other two points and point a. In the algorithm, point B represents the adjacent point (elbow joint), point C represents the far point (wrist joint) coordinate storage: because the subsequent track fitting is needed, the acquired data needs to be stored in the document, and the coordinates are stored in the auxiliary system through the database. As shown in Fig. 4 below is an experimental diagram of image preprocessing algorithm.

OPEN IN VIEWER

Fig. 4

Experimental graph of image preprocessing algorithm.

The first line is the foreground extraction graph of video continuous frame, the second line is the binary graph of mark feature segmentation, and the third line is the minimum circle fitting contour graph.

The above image preprocessing is mainly aimed at the extraction of marked joint points. As for the positioning of basketball, this paper uses the improved depth learning algorithm to obtain.

The data used in this paper is: choose a technical standard basketball coach and two ordinary students, let three people each 80 groups of free throws, each 20 groups as a video unit. Through image preprocessing algorithm for more than 15000 frames of three video units, the corresponding track change characteristics are obtained. It can be found that the vertical coordinate track of joint points of three subjects has their own periodic repetition.

The key of penalty basket technique is to make the ball have a better shot state, so as to be able to score accurately in the basket. Therefore, the research and analysis of the movement state of the basketball and the flight path of the ball in the air are inseparable from the positioning analysis of the ball. Now the problem is that under the existing equipment conditions, the fast moving object such as the ball can’t focus effectively to get a better ball outline, so in view of the characteristics of the video, this paper uses the method of foreground extraction to effectively set the ball’s foreground position, on this basis, through the improvement of the algorithm to get the relatively accurate position of the ball, and puts forward the basketball positioning detection algorithm.

In this paper, because the basketball track in the video is detected, in the relationship between the front and back frames, the foreground information of the motion can be obtained through Gaussian background modeling, which includes the person and the ball. Through the first stage of motion foreground extraction, the range of circle detection is reduced to less than half of the original region. On this basis, the foreground image is corroded, and then the uncorroded foreground image is subtracted from the corroded image to obtain the image containing only the foreground edge information, so that the number of pixels to be calculated to construct Hough array is minimized.

After preprocessing the video frame, Hough transform is performed. However, when shot is used to detect the circle of an image directly, it is necessary to map the pixel points of the whole image to the parameter space, and then judge point by point. The efficiency and accuracy of the algorithm are affected. Therefore, based on the Standard Hough transformation, this paper proposes an improved method, the basic idea is to reduce the accumulator dimension and replace multiple loops with multidimensional arrays. At this time, we need to know the radius range of the circle to be tested, that is, the radius r must fall within the range of (Rmin, Rmax).

Penalty basket is an orderly process of body coordination and force, which is carried out according to certain technical steps, each step has its corresponding technical indicators. How to distinguish the connecting points of each action is the premise of the analysis of technical action. According to the corresponding time point algorithm of penalty step, the action is divided into stages, and the technical parameters of each step are extracted quantitatively. Through the analysis of a large number of body movements of the athletes with the technical standard of penalty basket, the arm track model of each technical step is synthesized to help guide the technical movements of the athletes.

Hand frame detection algorithm is based on the above-mentioned hand frame model to automatically obtain the shot frame of the penalty basket. In the experiment, the track of the basketball coach and two ordinary students are analyzed to obtain the shot frame, respectively, to verify the effectiveness of the algorithm.

Because of the technical action standards of professional coaches, the frame sequence numbers of t0 (start), T1 (arm lifting), T2 (squatting) and T3 (hand) can be extracted clearly from their tracks.

From the data in the table, it can be analyzed that the number of frames between each step of the technical standard athletes in each penalty basket is also maintained at about 10 frames, with standard and stereotyped penalty basket technology.

In this paper, the prediction results of the shot frame of one video unit of two students are analyzed. Because there is no professional track that can clearly see the corresponding step stage of penalty basket technology, here only T3 (shot frame) is shown. It also verifies that the characteristics of the hand frame match the algorithm model.

The experimental results show that the algorithm is reliable and universal. It is worth noting that in more complex scenes, such as fast shot or the presence of serious noise background, the marked joint points cannot be obtained clearly, which makes the accuracy of the extraction results of coordinate values decline. In this case, the performance of the algorithm below may be reduced.

In the T3 frame, in order to save more effort to put the ball into the basket, save the theoretical minimum release speed and the corresponding best release angle. As the height and arm extension of the athletes under test are known, the corresponding release height can be basically determined. Through many experiments, we take the average release height as H = 2.38M, and the corresponding penalty distance as l = 4.225 m. Then the best release angle θ M = 49.51 °, and the minimum release speed VM = 6.80 m/s can be calculated by theoretical formula. Divide VM into vertical velocity vy = 5.17 m/s and VX = 4.42 m/s. Note: at this time, the coefficient k = 0.0175.

The professional player’s penalty basket hit rate is more than 90%. This time, 18 out of 20 penalty baskets were hit. Obviously, the professional athlete’s hand angle is always near the best hand angle, which is also consistent with the conclusion in the paper.

At present, in the basketball penalty basket assistant training, the main method is to record the training process by camera, and then judge the gain and loss of the penalty basket technology subjectively by watching the video, so as to guide the improvement of the player’s penalty basket technology. This method has the following problems. First, when the training video is increased, if only through video fast forward and playback of artificial positioning and penalty basket technology key frame, the efficiency is low. Secondly, although it is intuitionistic, it relies on the coach’s own experience, the guidance given by the coach is subjective and lack of attention to the athlete’s own conditions. Finally, the quality of the penalty basket depends on the hit result in the final analysis. In this way, the lack of data analysis on the ball’s release information and flight path is not conducive to the adjustment of the player’s release to approach the best theoretical release data of the individual, the formation of a stable and labor-saving penalty basket technology, and the stability of the player’s penalty basket in the competition process.

Therefore, in view of the above problems, the system can automatically locate the shot frame, obtain the video frame set of the penalty basket, analyze the technical action of the penalty basket, extract the shot data of the ball, and compare with the good technical template of the penalty basket, and put forward corresponding technical correction and adjustment suggestions of the shot state.

The auxiliary training system of this paper is to analyze the video of the athletes’ penalty basket training by using the related video image processing algorithm, obtain the relevant technical data of the penalty basket for unified induction, store them in the database for saving, and form the personal database of the players, provide the professional training data analysis for the athletes, and prepare for the adjustment of the training focus. The back-end algorithm of the system, through the statistical analysis of a large number of professional athletes’ penalty basket movement data, combined with the theoretical experience of sports authoritative journals and teaching materials, quantifies the key indicators of the technical action of the penalty basket, analyzes and summarizes the standard steps of the penalty basket technique and the technical indicators of the relevant actions, as the template of the penalty basket technique. Finally, through the analysis and comparison of the video of the players’ penalty basketball, the corresponding auxiliary training suggestions are given.

The purpose of the system is to acquire the relevant technical features and store them by analyzing each video unit of the penalty basket. These technical features include coordinate track information, hand frame information and key frame information of arm joint point. And give constructive advice to the athletes’ technical action, help them improve their own technology, gradually improve their own penalty basket technology, and improve the stability. So the system menu can be divided into four parts: user login, relevant information input, technical advice and data view.

The video playback menu provides two kinds of video viewing, namely, the playback of the penalty basket frame set of one video unit 20 groups and the playback of the corresponding wrist flick video. Because we can analyze the technical frame set and the shot frame set of each penalty basket through the information of the athletes’ penalty basket track, we can filter out the irrelevant frames and directly display the most important information to the users. Under the technical guidance menu, it provides a general evaluation of technical completion, as well as detailed opinions on hand type, arm lifting, squatting and stretching, which can help athletes guide the technical action of the penalty basket.

During the period of covid-19, computer, image processing, database and other technologies are applied in basketball penalty basket assistant training system, which is more and more in line with the era of intelligence and data. In this paper, the video of the players’ free throw basket is taken as the original data. Through the analysis of the arm track in the process of the free throw basket, the technical steps and essentials of the free throw basket are studied. In this paper, by reading the relevant literature, the author theoretically analyzes and summarizes the penalty basket technique, and deduces that there is an optimal shooting speed and shooting angle at the corresponding height. Then, through a large number of analysis on the video data of free throws of professional athletes and coaches, the correctness of the theory is verified, and through the track analysis, the relevant technical features are digitized to extract the track template of the arm in the process of free throws. The details are as follows:

The realization of the frame detection algorithm of penalty basket. Through the proposed mathematical model of hand frame, it can automatically and accurately locate the hand frame. And on this basis, get the speed and angle of basketball. Finally, the effectiveness and robustness of the algorithm are demonstrated by experiments.

The time nodes of each step are obtained by the algorithm of time points of each step. According to the track characteristics and technical parameters of boom, squat and extension, a large number of data are statistically analyzed to get the corresponding template track. In the auxiliary training system, through the comparison with the template track, the corresponding technical guidance is given. At the same time, the corresponding technical parameters of the subjects were calculated and compared with the standard parameters to achieve the purpose of assisting athletes training. It provides a new method for basketball training during covid-19.