Application of image content feature retrieval based on deep learning in sports public industry

Abstract

The image content retrieval can effectively promote the development of the entire industry. At present, sports competition is becoming more and more fierce, and the requirements for image content retrieval are getting higher and higher. In this paper, research has been carried out on image descriptor generation, image feature quantization and coding, accurate nearest neighbor cluster center fast search, multi-dimensional inverted index construction and fast retrieval. Moreover, based on deep learning, this paper constructed an effective detection algorithm for the characteristics of sports images, and compared the image shape and color as examples. It can be seen from the comparative study that the research method of this paper can effectively reduce the size of the candidate set of query results without affecting the accuracy of the query, which is of great significance for improving the speed of image query and has certain significance for promoting the development of sports public industry.

Keywords

Deep learning image content retrieval feature extraction sports industry image feature analysis

1 Introduction

Computer vision refers to the replacement of visual organs with various imaging systems as input-sensitive means and the replacement of the brain by computers for processing and interpretation. The ultimate research goal of computer vision is to enable computers to visually observe and understand the world like humans. It has the ability to adapt to the environment independently. It is a goal that needs long-term efforts. Therefore, before the ultimate goal is achieved, the medium-term goal of people’s efforts is to establish a visual system that can accomplish certain tasks according to the degree of visual sensitivity and intelligent feedback [1]. For example, an important application area of computer vision is the visual navigation of autonomous vehicles. However, there is currently no condition for a system that can recognize and understand any environment like humans and complete autonomous navigation. Therefore, the current research goal of people is to realize a visual aid driving system that has road tracking capability on the highway and can avoid collision with the vehicle in front. The point to be pointed out here is that the computer replaces the human brain in the computer vision system, but it does not mean that the computer must complete the processing of visual information according to the human visual method [2]. Computer vision should process visual information based on the characteristics of the computer system. However, the human visual system is by far the most powerful and sophisticated visual system known to people. Therefore, using computer information processing methods to study the mechanism of human vision and establishing computer theory of human vision is also a very important and interesting research field. This research is called Computational Vision. Computational vision can be considered a field of research in computer vision. There are many disciplines whose research objectives are similar to or related to computer vision. These disciplines include image analysis, pattern recognition or image recognition, scene analysis, image understanding, etc. [3].

The current image content retrieval is more used in sports, transportation and other industries. Especially in the sports industry, the image content retrieval can effectively promote the development of the entire industry. At present, sports competition is becoming more and more fierce, and the requirements for image content retrieval are getting higher and higher. Therefore, based on deep learning, this study analyzes the image content retrieval in the sports industry and promotes the application of image content retrieval in the sports industry.

2 Related work

In view of the high efficiency of content-based image retrieval technology, since the 1990 s, scholars at home and abroad have been devoted to the research of various CBIR technologies such as content feature extraction, feature vector matching, and related feedback. With the deepening of research, many efficient retrieval systems have appeared at home and abroad [4].

The earliest application of content-based image retrieval technology was the QBIC system developed by IBM, which was used to query paintings. In addition to the QBIC system, Picunter developed by the University of Illinois, VisualSeek of Columbia University, Photobook of the MIT Multimedia Lab, Blobworld of Berkeley University, and Simplicity of Penn State University are all well-known. Previous research on image retrieval has been developed for more than 20 years, and traditional search engine companies include Google. Baidu also provides some content-based image retrieval products [5].

QBIC: It was the first system software to go into commercial applications and was developed in 1993 by IBM Labs. QBIC can process the multi-dimensional features separately, and it can extract the color features, texture features and shape features of the target image separately, and then process them according to different user query methods. The QBIC system has been able to perform image retrieval in read-only databases and extended standard DBMSs. In addition, QBIC uses R.tree to solve high-dimensional index problems, and has significant results in medical imaging, art photography retrieval, and commercial image retrieval [6].

VisualSeek: It is an image retrieval tool developed by Columbia University. The system performs similarity matching by the spatial positional relationship of different color regions in the image. The system automatically extracts and indexes the highlighted color regions in the image, and then the system finds the most similarly arranged images of similar regions. The VisualSeek system is powerful, and efficiently retrieves the corresponding images according to the size of the region, the color information and the relative position of the space and realizes the image retrieval function on the line [7].

Photobook: It is a tool developed by the Massachusetts Institute of Technology for image browsing and querying. Its three components are mainly responsible for extracting texture, face and shape features. The user can select one of these three types for image retrieval according to his or her preference. In the subsequent version of FourEyes, user interactive image annotations were added, which achieved good results [8].

BDLP(Berkeley Digital Library Project): It was developed by the University of California at Berkeley. Users can enter keywords, locations, images, photographers, etc., and the system quantizes the color of each image into 13 color bins, and the user can choose these 13 colors or color matching methods. All features will be placed in a relational database, and the retrieved images will be presented to the user in an unordered manner with the author of the photographic photos, serial number, location and other related information [9].

Google Similar Images: It was developed by Google’s Radhika Malpani. It uses image recognition technology to filter search results and help users find similar images instead of identical images. Google Image Search is mainly used to find similarity pictures.

Baidu maps: The traditional image search technology is to search for relevant pictures on the Internet by inputting keywords. However, Baidu maps use the user-uploaded pictures or url address links to search for similarity pictures and attach information related to this picture [10].

In the past two decades, the field of CBIR has attracted a lot of attention. For characterization, color and texture features have been widely used. Moreover, color is the most widely used visual content applied to image retrieval and it is robust to scale transformation, direction and noise. The representation of feature descriptors corresponding to different color spaces is also very different, and the commonly used color spaces include RGB, LAB, LUV, HSV, YCrCb, etc. [11]. B. Ojeda-Magaña proposed a method of color histograms, and the method performs image matching by drawing a histogram and then calculating the distance of each bin [12]. Next, Yuan L et al. proposed the theory of color indexing, which not only describes the color distribution of the entire image, but also contains the main color features of the image without expressing all global color components [13]. Traditional color histograms are widely used in CBIR systems, but such forms of color features cannot express relative spatial information of objects in an image. Therefore, the researchers proposed several color feature descriptors that can describe the spatial information of the target object [14].

As another important way of image description, various methods of describing image texture features have also been proposed. Further, researchers have proposed combining multiple underlying features to replace the content of the entire image. Wang F B introduces a salient structural histogram (SSH) to describe the color and edge information of each graph [15]. Seghir R proposed the color difference histogram (CDH) descriptor, which combines the advantages of the symbiotic histogram [16]. These methods combine independent descriptors into one long descriptor, which improves retrieval accuracy but increases computational complexity. In addition, these methods are only a limited breakthrough. Because they do not select the most recognizable features, they cause problems such as redundant information and high feature dimensions.

3 Theoretical analysis

3.1 Problem analysis

The image retrieval method based on the inverted index structure improves the query speed by only searching part of the data in the database. When constructing the inverted index structure, the k-means algorithm is used to train the cluster center of k numbers on a training set: C ={ c₁, c₂, ⋯ c_k }, so that an index structure containing an inverted list of k numbers can be constructed. The inverted list of L ={ l₁, l₂, ⋯ l_k } numbers is equivalent to one cluster, and the cluster center is the index key corresponding to the inverted list [17].

When inserting the image features in the database into the inverted index structure, for the image feature x_i, the Euclidean distance of the cluster centers of k numbers in c to x_i is first calculated, and then the nearest cluster center cx_i is found. Its calculation method is as shown in formula (1): $c_{x_{i}} = arg min_{c_{j} \in c, j = 1, \dots, k} d (x_{i}, c_{j})$ (1)

In the formula, c_{x
_i} is the cluster center closest to x_i, and d (x_i, c_j) is the Euclidean distance between x_i and each visual word in the index structure.

Finally, x_i is inserted into the inverted list corresponding to the cluster center c_{x
_i}. The information stored in the inverted list includes the image feature ID of x_i and other metadata of x_i, such as PQ, ERVQ, RVQ, and TC. The main purpose of encoding the feature vector is to reduce the storage space requirement and increase the speed of the distance between the calculated features. The metadata of x_i used in this chapter is RVQ code.

After finding the closest cluster center c_{x
_i} and calculating the RVQ encoding, the image feature x_i is inserted into the inverted list corresponding to c_{x
_i} [18].

In the image feature retrieval stage, for the query image feature q, the Euclidean distance of the cluster center of k numbers in c to q is first calculated. Then, the nearest 1 or w cluster centers are found, and the visual features in the index list of these cluster centers are used as the query results. As in the query feature q in Fig. 1, in the case of w = 3, all the feature points in the inverted list corresponding to the three cluster centers of the visual words b, c, and d are used as the query results. Finally, if the final query result only needs knn number of similar results, the RVQ-coded asymmetric distance calculation method is used to calculate the Euclidean distance between the query feature and the quantized copy of the database feature, and the query results are sorted to obtain the final knn number of most similar query results.

Fig. 1

Schematic diagram of traditional image retrieval method based on inverted index.

At present, the commonly used retrieval methods are all the features in the inverted list corresponding to the cluster center of the w-number closest to the query feature as the query result and used for sorting. However, in the actual case, similar to the query feature is only the feature points whose spatial locations are located around the query feature. Without affecting the accuracy of the query, if the query results of the traditional search method can be effectively filtered, only the features located around the query feature are used for sorting, and the scale of the sorted set is reduced, which is of great significance for improving the query speed. Therefore, based on the retrieval method based on inverted index structure, an adaptive retrieval method is proposed to retrieve only the feature points around the spatial position of the query feature points, and reduce the number of feature points used for sorting. The purpose is to improve the query speed without affecting the accuracy of the query. Schematic diagram of fully filtered adaptive search as show Fig. 2.

Fig. 2

Schematic diagram of fully filtered adaptive search.

3.2 Fully filtered adaptive retrieval

It is based on a retrieval method based on a standard inverted index structure. After retrieving the nearest inverted list of w numbers, the fully filtered adaptive search is used to filter the feature points in the w inverted lists to obtain feature points located near the query feature points in spatial position and used for sorting.

The key to fully filtering adaptive retrieval is to determine the appropriate radius for the location of the query feature points. By querying the feature point as the center of the sphere, a hypersphere is constructed in the feature space, and the feature points outside the range of the hypersphere in the query result are filtered out, and only the feature points located inside the hypersphere are sorted.

An inverted index structure L ={ l₁, l₂, ⋯ l_k } is given, and the center of the cluster corresponding to the inverted list is C ={ c₁, c₂, ⋯ c_k }. For the query feature point q, the fully filtered adaptive retrieval method is as follows:

Step 1: The distance d ={ d₁, d₂, ⋯ d_k } between q and the cluster center corresponding to all inverted lists is calculated separately.

Step 2: The distance value of the smallest front w number is selected from d, d ={ d_q,1, d_q,2, ⋯ d_q,k } and all corresponding feature points of w numbers in the inverted list are used as the query result RS_q ={ y₁, y₂, ⋯ y_m }.

Step 3: In the feature space, a hypersphere with q as the center of the sphere is constructed, and the corresponding hypersphere radius is R_q. It is calculated according to the distance from the query feature to the nearest cluster center {c_q,1, ⋯ c_q,w } of the w number, and its calculation formula is as follows: $R_{q} = λ \times \frac{1}{w} \sum_{i = 1}^{w} d {(q, c_{q, i})}^{2} = λ \times \frac{1}{w} {∥ q - c_{q, i} ∥}^{2}$ (2)

In the formula, λ is the scale factor, which is used to adjust the radius of the hypersphere to obtain the smallest radius. In this way, the feature points similar to the query feature points in the query result are located inside the hypersphere without affecting the query accuracy.

Step 4: Using the asymmetric distance method, the Euclidean distance between the features q and RS_q is calculated, and only the feature points satisfying the formula (3) are retained, thereby filtering the dissimilar features: ${∥ q - y_{i} ∥}^{2} ⩽ R_{q} (i = 1, 2, \dots, m)$ (3)

New query results ${R S}_{q new} = {y_{1}^{'}, y_{2}^{'}, ..., y_{b}^{'}}$ are obtained.

Step 5: According to the distance between the feature and the query feature in RS_qnew, it is sorted, and the feature point with the smallest distance of knn number is the final query result.

2 is a schematic diagram of fully filtering adaptive retrieval on a two-dimensional feature space when w = 3, and a, b, c, d,..., k are respectively cluster centers corresponding to the inverted list. For the query feature point q, after determining the nearest three cluster centers b, c, and d, the corresponding radius R can be calculated by formula (2) to determine the range of the circle, thereby filtering the features that are not similar to the query feature. This method only sorts the feature points located in the circle, thereby reducing the number of feature points to be sorted and improving the query speed.

The fully filtered adaptive retrieval method is identical to the standard inverted index based retrieval method on the inverted index structure. The difference is that in the feature query process, the fully filtered adaptive retrieval method increases the filtering mechanism of the query results, filters out the dissimilar query results, and only retains the query results near the query feature points.

4 Incomplete filtering adaptive retrieval

The fully-filtered adaptive retrieval method constructs a hypersphere with the query feature as the sphere in the feature space and filters out the feature points outside the hypersphere in the query result, thereby reducing the number of sorted feature points and improving the query speed. However, fully filtering adaptive retrieval requires calculating the Euclidean distance between the query features and all features in the query results and comparing them to the corresponding hypersphere radius. The time cost of this process is proportional to the number of feature points in the query results. For this reason, without affecting the accuracy of the query, in the process of filtering the query result, the incomplete filtering adaptive retrieval method reduces the number of calculations of the distance between the query feature and the feature in the query result and the comparison between the distance and the radius of the hypersphere, thereby reducing the time required for filtering the non-similar features and further improving the query speed.

The incompletely filtered adaptive retrieval method is based on the standard inverted index structure, and it reduces the number of distance calculations in the feature filtering process and the number of comparisons between the distance and the radius of the hypersphere by adding an inverted index to each inverted list.

In the training phase, the hierarchical k-means algorithm (HKM) is used to train the cluster centers required for the two-level inverted index structure on a training sample set: First, the k-means algorithm is used to train the original training set to obtain the clustering center C ={ c₁, c₂, ⋯ c_{k
₁} } of the k₁-number of the first layer. Then, the k-means algorithm is used again to train the training sample set falling in each cluster center c_i (i = 1, 2, ⋯ , k_i), thereby obtaining the k₂ number of cluster center C_{c
_i} ={ c_i1, c_i2, ⋯ c_{ik
₂} } of the second layer in c_i.

For the inverted index structure corresponding to the incomplete filtering adaptive retrieval, since each cluster center c_i in the cluster center set C corresponds to inverted list L = { l_i1, l_i2, ⋯ l_{ik
₂} } (i = 1, 2, ⋯ , k₁) of k₂ numbers, there are a total of k₁ × k₂ inverted lists in the index structure.

When the feature points in the feature database are inserted into the inverted index structure, for the feature point x, the Euclidean distances of all cluster centers in C to x are first calculated and the closest cluster center c_i is found by using formula (1). Then, the distances of all feature points in the second-level cluster center set C_{c
_i} corresponding to c_i to x are calculated. Similarly, formula (1) is used to find the nearest cluster center c_ij. Finally, the ID of x and its RVQ encoding are inserted into the inverted list corresponding to c_ij.

Query phase: For query feature points q, the method of incomplete filtering adaptive retrieval is as follows:

Step 1: D₁ ={ d₁, d₂, ⋯ d_{k
₁} } is obtained by calculating the Euclidean distance from q to all cluster centers C ={ c₁, c₂, ⋯ c_{k
₁} } in the first layer index.

Step 2: The smallest distance value {d_q,1, d_q,2, ⋯ d_q,w } of the first w numbers is selected from D₁, and the corresponding cluster center is C_{c
_q} ={ c_q,1, c_q,2, ⋯ c_q,w }.

Step 3: The cluster centers corresponding to the first layer cluster centers of w numbers in the second layer index are merged to form a set $C_{c_{q}}^{'} = ⋃_{i = 1}^{w} C_{c_{q, i}} = ⋃_{i = 1}^{w} ⋃_{j = 1}^{k_{2}} C_{c_{q, i} j}$ .

Step 4: All feature points in the inverted list corresponding to C_{c
_q,i
j} obtained in Step 3 are used as the initial query result ${RS}_{qo} = ⋃_{i = 1}^{w} ⋃_{j = 1}^{k_{2}} l_{c_{q, i} j}$ of feature q.

Step 5: As with fully filtered adaptive retrieval, a hypersphere S_q with q as the center of the sphere is constructed in the feature space, and the corresponding hypersphere radius is R_q. It is calculated by the distance {d_q,1, d_q,2, ⋯ d_q,w } of the query feature to the cluster center of the nearest w number in the first layer index and Formula (2).

Step 6: The Euclidean distance between the query feature q and all cluster centers C_{c
_q,i
j} in $C_{c_{q}}^{'}$ is calculated.

Step 7: According to formula (4), the inverted list corresponding to the dissimilar cluster centers is filtered.

$\begin{matrix} {∥ x - C_{c_{q, i} j} ∥}^{2} \\ ⩽ R_{q} (i = 1, 2, \dots, w, j = 1, 2, \dots, k) \end{matrix}$ (4)

Since each cluster center C_{c
_q,i
j} in the obtained cluster set $C_{c_{q}}^{'}$ corresponds to one inverted list and each inverted list is equivalent to one cluster, all feature points in the cluster are considered to be similar to the cluster center C_{c
_q,i
j}. Therefore, C_{c
_q,i
j} can be used to represent all feature points in the corresponding inverted list and used for dissimilar feature filtering. If C_{c
_q,i
j} is inside the hypersphere S_q, it is considered that all the feature points in the corresponding inverted list are also located inside S_q, which is similar to the query feature.

Step 8: After the step 7’s dissimilar feature filtering, the feature set ${RS}_{q} = ⋃_{i = 1}^{w} ⋃_{j = 1}^{k_{2}} l_{c_{q, i} j}$ to be sorted is obtained. In the formula, C_{c
_q,i
j} is located inside the hypersphere S_q.

Step 9: Using the asymmetric distance method, the approximate Euclidean distance between the query features q and the features in RS_q is calculated and sorted, and the knn number of feature points with the smallest distance are used as the final query results.

The query feature q is given. After determining the three nearest cluster centers b, c, d in the first layer index, the radius R is calculated using Equation (2) and the range of circles applicable to the current query feature is determined. Taking the cluster corresponding to b as an example, since in the second layer index corresponding to b, only the cluster centers b1 and b2 are in the circle, and b3, b4, b5, and b6 are all outside the circle, only the feature points in the inverted list corresponding to b1 and b2 are used for sorting. The dissimilar feature filtering process for clusters c and d is also in accordance with the above method.

Compared with the full filtering adaptive retrieval, in each inverted list, the incomplete filtering adaptive retrieval classifies the feature points falling into the inverted list again to form several clusters. Moreover, it uses these sub-cluster centers to represent corresponding sub-clusters and is used to filter non-similar features in the query results, thereby greatly reducing the number of distance calculations when filtering non-similar features. Under the premise of retrieving the inverted list of the nearest w numbers and obtaining the initial number of features in the initial query result, if the number of feature points used for sorting after fully adaptive filtering is n′, and the number of feature points is sorted by a fast sorting algorithm, the time complexity of fully filtering adaptive retrieval is O (n × d + n′ log n′). If the number of feature points used for sorting after incomplete adaptive filtering is n″, and the number of feature points is sorted by quick sorting, the time complexity of incomplete filtering adaptive retrieval is O ((w × k₂ + n″) × d + n″ log n″). Among them, d is the vector dimension of the image feature, and O (n × d) and O ((w × k₂ + n″) × d) are the Euclidean distance time complexity that need to be calculated for the fully filtered adaptive retrieval method and the incompletely filtered adaptive retrieval method, respectively. O (n′ log n′) and O (n″ log n″) are the time complexity of sorting the query results by two adaptive search methods: full filtering and incomplete filtering.

5 Sports object retrieval based on shape features

In this experiment, three categories of character, site and equipment are randomly selected in the public stadium, and each category has 100 images, and the image is processed into a standard image of 640×480 (for the convenience of the following comparative experiments). 10 images are randomly selected from the images of each class in the experimental image library as an example, and 300 images in the image library are regarded as target images. This study uses Matlab as a plat-form for experiments. Each search is input with one image, and each type of image is performed 10 times, so that a total of 30 search experiments are performed. In the search result, the same image as the sample image is the correct image, and the top 20 images are output and displayed. Then, the number of correctly retrieved images is counted, and the average precision rate and recover rate ratio of the images of each category are calculated, as shown in Table 1. The examples obtained by processing are shown in Figs. 4 –6.

Table 1
Average recover rate and precision rate

Average recover rate Average precision rate

Character 0.423 0.44

Site 0.451 0.47

Equipment 0.472 0.48

	Average recover rate	Average precision rate
Character	0.423	0.44
Site	0.451	0.47
Equipment	0.472	0.48

Fig. 3

Schematic diagram of incomplete filtering adaptive retrieval.

Fig. 4

Character image output based on shape features.

Fig. 5

Site image output based on shape features.

Fig. 6

Image output of sports equipment based on shape features.

As can be seen from Figs. 4 –6 shown in the experimental results section, among the first 20 images of the character image output, 13 of them are correct images, and among the first 20 images of the field image output, 14 of them are correct images, and among the first 20 images of the site, 14 of them are correct images. The experimental results show that the method has better recover rate and precision rate, and because the extracted contour feature points occupy a very small part of the contour, the calculation amount is greatly reduced, and the retrieval efficiency is effectively improved.

To compare with the following, we set the system to return 50 images at a time. The average re-call and average precision rate obtained are shown in Fig. 7. Average precision rate image based on shape features as shown in Fig. 8.

Fig. 7

Average recover rate image based on shape features.

Fig. 8

Average precision rate image based on shape features.

6 Color features are used for shape-based image retrieval

Under normal circumstances, the photos taken by people in the stadium are basically a combination of people and landscapes, a combination of objects and scenery, and simple scenery. Moreover, the subject of concern is a part of the picture, people, things, or scenery. However, whether it is people, things, or scenery, it is part of a picture. Compared with the rest of the picture, there is a large color difference between the two parts, and people rely on these differences to distinguish the image. If we use these differences to describe the outline of the object, it is more in line with the human visual point of view, and it is relatively simple. Character image output based on color features as shown in Fig. 9. Site image output based on color features as shown in Fig. 10. Image output of sports equipment based on shape features as shown in Fig. 11.

Fig. 9

Character image output based on color features.

Fig. 10

Site image output based on color features.

Fig. 11

Image output of sports equipment based on shape features.

This experiment still uses three categories of characters, site and equipment, each of which has 100 images, processes the image into 640×480 standard images, and then divides each image into 20×20 small blocks, according to the above method. 10 images are randomly selected from the images of each class in the experimental image library as an example, and 300 images in the image library are regarded as target images. This study uses Matlab as a platform for experiments. Each search enters an image, so that the three types of images need to perform a total of 30 search experiments. In the search result, the same image as the sample image is the correct image, and the top 20 images are output and displayed. Then, the number of images correctly retrieved is counted, and the average image precision rate and recover rate ratio of each category are calculated, as shown in Table 2.

Table 2

Average precision rate and recover rate

	Average recover rate	Average precision rate
Character	0.462	0.47
Site	0.458	0.46
Equipment	0.476	0.5

It can be seen from the graphs shown in the experimental results that 15 of the first 20 images detected by the character image are correct, and 14 of the first 20 images detected by the site image are correct, and 15 of the first 20 images detected by the equipment image are correct. It can be seen that the retrieval ability of this method is slightly poor for an image in which the difference between the target object and the background color in the image is not obvious.

The system is set to return 50 pictures at a time, and the average recover rate and average precision rate are calculated as shown in Figs. 12 and 13.

Fig. 12

Average recover rate image based on color features.

Fig. 13

Average precision rate image based on color features.

The results are then compared to the methods used previously. By comparison, shape-based image retrieval with color features has a good precision and recall rate. This method is more effective than the previous two methods, and has good adaptability to changes in image size, rotation, etc., and has high application value in practice.

7 Analysis and discussion

The rapid development of the Internet has led to an explosive growth in image representation of multimedia data from Weibo, mobile phones, social networking sites, news sites, and multimedia sharing sites. However, while bringing people rich image resources, there is a problem that needs to be solved urgently. Faced with massive amounts of image data, how to help people quickly get the images they really need, especially those that are difficult to express clearly with traditional text descriptions, is an urgent problem to solve. This is especially true in the sports industry. Based on this, this study analyzes public sports applications through image retrieval based on deep learning.

Based on the retrieval method of standard in-verted index, two retrieval methods of fully filtered adaptive retrieval and incompletely filtered adaptive retrieval are proposed. Instead of using the entire initial query result for sorting, the fully filtered adaptive search is for the query feature, by adaptively calculating the radius and constructing a hypersphere that queries the feature as the center of the sphere. Moreover, it only sorts the features located inside the hypersphere, thereby filtering out non-similar features and improving the query speed. Incomplete filtering adaptive retrieval reduces the time overhead of query result filtering and further improves the query speed by di-viding each inverted list into several subclasses and using the corresponding cluster centers for non-similar query result filtering. In addition, these two adaptive retrieval methods can be easily extended to other retrieval methods for inverted index structures.

Content-based image indexing and retrieval has become a research hotspot in the field of image retrieval at home and abroad, and it is also a re-search difficulty. This paper has carried out re-search on image descriptor generation, image feature quantization and coding, accurate nearest neighbor cluster center fast search, multi-dimensional inverted index construction and fast retrieval. It lays a preliminary theoretical foundation for some technical key points of large-scale image indexing and retrieval based on content. In the future, based on the existing research, we will carry out perfect and in-depth research from the following aspects:

In this paper, low-level visual features are used in the calculation of image descriptors and visual features that are similar in Euclidean distance are considered similar. However, these similar visual features have dissimilar possibilities in the semantic content of the image. Therefore, for a specific application field, combining the underlying visual features of the image with the relevant semantic information to calculate descriptors that more accurately describe the image content is a problem that needs to be studied in the next step.

The training of the enhanced residual quantization method designed by the thesis is completed in the original space of the visual features of the image. Thus, the time required for the training phase in-creases as the visual feature dimension of the image increases. Therefore, in the next stage of re-search, while mapping image visual features from high-dimensional space to low-dimensional space to complete enhanced residual quantization code-book training, we will find ways to minimize the impact of mapping errors. That is to say, under the premise of not affecting the accuracy of the ap-proximate representation of the visual features of the image, the quantization error and the mapping error are simultaneously considered, and the training efficiency is improved.

The current update of the inverted index is implemented by reconstructing the inverted index when the data changes to a certain extent. Although this is a general processing method in the field of image retrieval, the time consumption is also very large for large-scale data sets. Therefore, designing an index structure that supports local updates is an issue that needs further study.

At present, this paper mainly focuses on the performance optimization of image feature indexing and retrieval in feature indexing, indexing and retrieval, and does not involve how to apply these algorithms to parallel computing environment. Nevertheless, some of the methods presented in this paper can be easily handled in parallel. Taking the retrieval of image visual features as an ex-ample, since the Euclidean distance calculation of the clustering center corresponding to all the in-verted lists is independent of each other and there is no mutual dependence, this process can be processed in parallel. Similarly, before sorting the query results, the distance calculation between the features and the query results can be queried for parallel pro This kind of calculation will further improve the performance of the proposed method. Therefore, applying the method in the paper to the distributed parallel computing platform (such as: hadoop, etc.) is the next important research work.

8 Conclusion

At present, image content retrieval is widely used in sports, transportation and other industries, especially in the sports industry. Image content retrieval can effectively promote the development of the entire industry. Moreover, the current sports competition is becoming more and more fierce, and the requirements for image content retrieval are getting higher and higher. Therefore, based on deep learning, this study analyzes the image con-tent retrieval in the sports industry and promotes the application of image content retrieval in the sports industry. This paper has carried out re-search on image descriptor generation, image feature quantization and coding, accurate nearest neighbor cluster center fast search, multi-dimensional inverted index construction and fast retrieval, which lays a preliminary theoretical foundation for some technical key points of large-scale image indexing and retrieval based on con-tent. In the experiment, the corresponding results of the research method were obtained by randomly selecting the characters, equipment and the public stadium. On this basis, the effectiveness of the algorithm is analyzed. The research shows that the proposed algorithm has certain effects and can provide theoretical reference for subsequent related research.

Footnotes

Acknowledgments

This paper was supported by (1) Project Title: Practice, Problems and Strategies of Local Governments Purchasing Public Sports Services-Taking Nanchang City as an Example 2018009; and (2) Fund Project: Guangzhou international marathon risk management research, GDSS2014106.

References

Chen

, Li

, Liu

, et al., Improving Deep Visual Representation for Person Re-identification by Global and Local Image-language Association[J]. 2018.

Zhang

, Wu

, Cheng

, et al., Image-to-Video Person Re-Identification with Temporally Memorized Similarity Learning[J], IEEE Transactions on Circuits and Systems for Video Technology (2017), 1.

Balamurali

, Chandrasekar

, Multiple parameter algorithm approach for adult image identification[J], Cluster Computing, 2018.

Yin

, Zheng

W.S.

, Wu

, et al., Adversarial Attribute-Image Person Re-identification[J]. 2017.

Wang

J.N.

, Chen

X.L.

, Hou

X.W.

, et al., Construction, implementation and testing of an image identification system using computer vision methods for fruit flies with economic importance (Diptera: Tephritidae)[J], Pest Management Science, 2016.

Zhu

, Jing

X.Y.

, You

, et al., Image to Video Person Re-identification by Learning Heterogeneous Dictionary Pair with Feature Projection Matrix[J], IEEE Transactions on Information Forensics and Security (2017), 1.

Zuiderbaan

, Harvey

B.M.

, Dumoulin

S.O.

, Image identification from brain activity using the population receptive field model[J], PLoS ONE12(9) (2017), e0183295.

Wang

, Yang

Y.L.

, Liu

, Study of traffic characteristics based on internet web real-time traffic conditions by image identification technology[J], The Imaging Science Journal (2017), 1–8.

Senthooran

, Chathuranga

M.T.

, Kartheeswaran

, E-Sign Detector: Image Steganography-Based Employee Identification System[J], Social Science Electronic PublishingXII(2) (2017), 33–44.

10.

Pauli

, Knowledge-based adaptive identification of 2-D image structures[J], Proceedings of SPIE - The International Society for Optical Engineering (2017), 1395.

11.

Chen

, Hassanpour

, Survey: How good are the current advances in image set based face identification? – Experiments on three popular benchmarks with a na?ve approach[J], Computer Vision and Image Understanding (2017), S1077314217300577.

12.

Ojeda-Magaña

, Quintanilla-Domínguez

, Ruelas

, et al., Improvement of the Image Sub-Segmentation for Identification and Differentiation of Atypical Regions[J], International Journal of Pattern Recognition and Artificial Intelligence, 2017.

13.

Yuan

, Chen

, Zeng

L.L.

, et al., Gender Identification of Human Brain Image with A Novel 3D Descriptor[J], IEEE/ACM Transactions on Computational Biology and Bioinformatics (2015), 1.

14.

, Yang

, Zhu

, et al., Identification of Microplankton in South China Sea with Image-Matching Individual PCR[J], Journal of Ocean University of China18(1) (2019), 219–226.

15.

Wang

F.B.

, Tu

, Wu

, et al., Femtosecond laser ablation power level identification based on the ablated spot image[J], International Journal of Advanced Manufacturing Technology94(4) (2017), 1–8.

16.

Seghir

, Pierron

, Viscoelastic properties identification through innovative Image-Based DMTA strategy[J]. 2017.

17.

Kumar

B.A.

, Aishik

, Kumar

B.A.

, et al., Script Identification in Natural Scene Image and Video Frames using an Attention based Convolutional-LSTM Network[J], Pattern Recognition, (2018), S0031320318302590.

18.

Xie

, Li

, Zhong

, et al., Image-to-Video Person Re-Identification by Reusing Cross-modal Embeddings[J]. 2018.

Application of image content feature retrieval based on deep learning in sports public industry

Abstract

Keywords

1 Introduction

2 Related work

3 Theoretical analysis

3.1 Problem analysis

Table 1 Average recover rate and precision rate Average recover rate Average precision rate Character 0.423 0.44 Site 0.451 0.47 Equipment 0.472 0.48

8 Conclusion

Footnotes

Acknowledgments

References

Table 1
Average recover rate and precision rate

Average recover rate Average precision rate

Character 0.423 0.44

Site 0.451 0.47

Equipment 0.472 0.48