Fast detection of moving objects based on sequential images processing

Abstract

Nowadays, moving object detection in sequence images has become a hot topic in computer vision research, and has a very wide range of practical applications in many fields of military and daily life. In this paper, fast detection of moving objects in complex background is studied, and fast detection methods for moving objects in static and dynamic scenes are proposed respectively. Firstly, based on image preprocessing, aiming at the difficulty of feature extraction of moving targets in low illumination at night, Gamma change is used to process. Secondly, for the fast detection of moving objects in static scenes, this paper designs a detection method combining background difference and edge frame difference. Finally, aiming at the fast detection of moving objects in dynamic scenes, a feature matching detection method based on the SIFT algorithm is designed in this paper. Simulation experiments show that the method designed in this paper has good detection performance.

Keywords

Keywords: Sequence image fast detection of moving objects Gamma change SIFT algorithm

1 Introduction

The automatic detection technology of moving objects has always been a hot and challenging problem in the field of computer vision. This technology combines image processing, automatic control, information science and other technologies organically, and becomes the bottom key technology of many computer vision systems in practical engineering applications. Such as precision strike weapon guidance system, computer-aided driving system, missile terrain and map matching system, industrial pipeline automatic monitoring system, intelligent room, intelligent robot, medical image system, intelligent transportation system [1, 2] and so on.

As the visual basis of human perception of the world, image is an important means for humans to obtain information, express information and transmit information. Image technology [3] is a general term for various image-related technologies. Its main content is a series of work on digital images using computers and other electronic devices. Image technology is divided into three levels according to the processing method, degree of abstraction and amount of data: image processing, image analysis [4] and image understanding.

Image-based target detection is a complex process, and the difference in the application environment and algorithm scheme will greatly affect the target detection effect [5]. With the further development of image technology and the need for Engineering applications, the effective detection of interesting objects in sequence images by using advanced digital image technology has become the difficulty of image detection technology research. The purpose of moving object detection based on a sequence image is to separate the moving area from the background area in the image sequence. Since the development of the theory of moving object detection in sequential images, many different methods of target detection have been proposed. Common algorithms include optical flow method [6, 7], image difference method [8], extended EM algorithm [9, 10], wavelet transform-based method [11], motion energy detection method [12], and artificial neural network-based method [13, 14]. There are three classical detection algorithms: optical flow method, inter-frame difference method [15] and background difference method [16].

Based on the above background, this paper uses Gamma change to process the sequence image based on image preprocessing, and then proposes corresponding fast detection methods for moving objects in static and dynamic environments respectively. The specific contributions of this paper are as follows.

Based on image preprocessing, because of the difficulty of feature extraction of moving targets in low illumination at night, Gamma change is used to process.

Aiming at the fast detection of moving objects in static scenes, a detection method combining background difference and edge frame difference is designed in this paper.

Aiming at the fast detection of moving objects in dynamic scenes, this paper designs a detection method based on SIFT algorithm feature matching, which has good performance.

2 Fast detection method of moving object

2.1 Sequence image preprocessing

2.1.1 Grayscale transformation of image

After graying, each pixel of the color image has only 256 gray values. In the process of using the algorithm and processing, it can reduce the storage space, reduce the amount of calculation, improve the speed of image processing and ultimately improve the speed of moving object detection. The color image can be transformed into a gray image according to formula (1). $Y = 0.299 R + 0.587 G + 0.114 B$ (1)

Among them, Y is calculated according to the relationship between R, G, B color components and brightness signal Y in YUV. R, G and B represent red, green and blue components respectively. The weighting coefficients of three colors, red, green and blue, are affected by the color component signals.

2.1.2 Binarization

The specific method is as follows. If the input image is f (x, y) and the output image is F (x, y), a threshold T is usually selected. The binary transformation function expression of the image is shown in formula (2). $F (x, y) = {\begin{matrix} 1, f (x, y) ⩾ T \\ 0, f (x, y) < T \end{matrix}$ (2)

The threshold T divides the image F (x, y) into two parts. In this paper, the pixels larger than T are set to white (target object) and the areas smaller than T are set to black (background).

2.1.3 Image denoising

The definition of one-dimensional median filtering is shown in formula (3), pp. $Z_{k} = med (x_{k - N}, x_{k - N + 1}, \dots, x_{k}, \dots, x_{k + N})$ (3)

Among them, med means median operation on the pixels in the image. In the above formula, 2 N + 1 pixel are sorted, and the final output pixel is the median of the pixel sequence.

Similarly, two-dimensional pixel matrices represent two-dimensional images, and two-dimensional median filtering is suitable for two-dimensional pixel matrices. The definition of two-dimensional median filtering is given in the following formula: $f (x, y) = \underset{(i, j) \in S}{median} [f_{S} (i, j)]$ (4)

2.1.4 Low illumination processing

In this paper, Gamma transform enhancement is used as the method of low illumination processing: the average value of the original image is adjusted to 0, and the variance is adjusted to 1. According to the distribution histogram distribution characteristics, the current image is judged to be of high light or low light type, and the initial fine adjustment of the gray level is performed according to the corresponding type, and the image is corrected by Gamma. Because there are few cases of excessive light intensity in the natural traffic environment, this paper only considers the dark situation. The transformation formula is shown in formula (5), pp. $D (x, y) = 255 - \frac{(D (x, y) - 255)^{2}}{255}$ (5)

The method used in this paper uses a mapping method for all points of the image, so there will be no local over-darkness or over-brightness.

2.2 Fast detection method of moving target in static scene

2.2.1 Background difference detection method

Our method is used to segment the background and foreground. Formula (6) is used to process each pixel of a differential image Sub _ image. $M_{t} (x, y) = {\begin{matrix} 1, (Sub_image (x, y) > TH) \\ 0, else \begin{matrix} \begin{matrix} \begin{matrix} \end{matrix} \end{matrix} \end{matrix} \end{matrix}$ (6)

For the Ostu method, the probability distribution w₀ = 0 at the gray value of “0” should be considered. Threshold TH divides the image pixels into C₀ = (0, 1, ⋯ , TH) C₁ = (TH + 1, TH + 2, ⋯ , L - 1) and categories, representing the target and background respectively.

After moving target detection, there may be false targets. To solve this problem, the left, right, upper and lower parts of each connected domain are saved in memory. For each domain, the following comparisons are made with other domains:

$flag_connection (i) = {\begin{matrix} \begin{matrix} false, if (position (i, 1) > position (j, 1) & & (flag_connection (j)! = false \\ \begin{matrix} \end{matrix} position (i, 2) > position (j, 2) & & (flag_connection (i)! = false \\ \begin{matrix} \end{matrix} position (i, 3) > position (j, 3) & & (position (i, 4) < position (j, 4))) \end{matrix} \\ \begin{matrix} \begin{matrix} \begin{matrix} \begin{matrix} \begin{matrix} \begin{matrix} \begin{matrix} \begin{matrix} \end{matrix} \end{matrix} \end{matrix} \end{matrix} true, & else \end{matrix} \end{matrix} \end{matrix} \end{matrix} \end{matrix}$ (7)

2.2.2 Edge frame difference detection method

The specific steps are as follows:

1) Gauss filtering: The image is filtered by a two-dimensional Gauss function. The variance δ of two-dimensional Gauss function depends on experience. The two-dimensional Gauss function is: $Gaussian (x, y) = \frac{1}{2 π δ} exp (\frac{x^{2} + y^{2}}{2 δ^{2}})$ (8)

For a gray image, the filtering process is equivalent to the convolution process, which is described as follows: $Filter (x, y) = \frac{\sum_{x, y \in M} f (x, y) * Gaussian (x, y)}{\sum_{x, y \in M} Gaussian (x, y)}$ (9)

2) x, y direction gradient: image Filter (x, y) is a 2-D discrete matrix, and then the first derivative of x and y direction is obtained. That is to say, the first-order differential is solved in the discrete. Based on this, the gradient in the direction of x and y is solved as follows: $\begin{matrix} G_{x} = \frac{1}{2} (| Filter (x + 1, y) - Filter (x, y) | \\ + | Filter (x + 1, y + 1) - Filter (x, y + 1) |) \\ G_{y} = \frac{1}{2} (| Filter (x, y + 1) - Filter (x, y) | \\ + | Filter (x + 1, y + 1) - Filter (x + 1, y) |) \end{matrix}$ (10)

3) Total gradient at (x, y): To simplify the computational inconvenience caused by the two gradients in the x y and directions, it is necessary to take the total gradient of the pixel (x, y) as shown in formula (11), pp. $G = \sqrt{G_{x}^{2} + G_{y}^{2}}$ (11)

2.2.3 Detection method combining background subtraction and edge inter-frame subtraction

Assuming that there are moving objects in the monitoring scene, the motion contour information of moving objects can be obtained by the difference between adjacent frames. To obtain more reliable contour information, it is undoubtedly a feasible method to use the difference between the adjacent three frames. At the same time, to avoid the influence of noise on the detection results, it is possible to use the edge information of the image and the three-frame difference method to realize the detection of moving objects in the monitoring scene, that is, to form a three-frame difference method based on the edge information. The edge segmentation results obtained by the three-frame difference method based on edge information and the results of moving object detection based on background difference are calculated reasonably, that is to say, the problem of clothing dividing the human body into multiple or multiple moving objects is solved to a certain extent. This scheme not only ensures the rapidity of detection, but also reduces the false detection rate of moving objects in complex monitoring scenarios. The specific implementation process is shown in Fig. 1.

Fig. 1

Flow chart of detection method combining background subtraction and edge inter-frame subtraction.

2.3 Fast detection method of moving target in dynamic scene

The details of this method are described in detail below.

1. Feature point extraction

Based on the good performance of the SIFT algorithm, this paper uses the SIFT algorithm to extract features. The process can be divided into four steps: detecting scale-space extremum points, accurately locating extremum points, specifying direction parameters for each key point and generating feature point descriptors: detection of extremum points in scale space, accurate positioning of extremum points, specify orientation parameters for each key point, generation of feature point descriptors

2. Feature matching

In this method, a feature-based matching method is used. When the SIFT feature vectors of two images are generated, the Euclidean distance of the key point feature vectors is used as the similarity measure of the key points in two images.

3. Correction of matching pairs of feature points

This paper proposes a strategy of eliminating pseudo-feature points.

In the first step, all feature matching pairs extracted are substituted into the least squares formula to obtain the affine parameter (A₁, B₁).

In the second step, the coordinate (x, y) of the feature points in the reference frame is matched and the estimated coordinate position (x″, y″) of the feature points in the current frame is calculated according to the affine parameter (A₁, B₁). If the Euclidean distance between the coordinates (x′, y′) (x″, y″) and of the feature points matched with (x, y) in the current frame exceeds a certain threshold T₁, the matching pair is considered to contain false feature points, and the matching pair is eliminated.

In the third step, the least-squares affine parameter solution (A₂, B₂) is calculated by using the feature matching pairs left behind in the second step. Repeat the second step, but the threshold is T₂.

In the fourth step, the least-squares affine para-meter solution (A₃, B₃) is calculated by using the remaining feature matching pairs.

By reasonably adjusting the values of T₁ T₂ and, the purpose of eliminating false feature points can be achieved.

4. Motion compensation

Because bilinear interpolation has high accuracy, can achieve smoothness between pixels, there will be no mosaic phenomenon, so this paper uses the bilinear interpolation method.

5. Background subtraction

The variation of illumination and the estimation of affine parameters may also cause errors. In practice, there is a large amount of noise in the differential image, so it is necessary to threshold the difference image to eliminate the noise.

3 Summary of article methods

Aiming at the sequence images collected in this paper, based on fully studying the algorithms of moving object detection summarized by predecessors, appropriate detection methods are proposed based on static and dynamic environments respectively to achieve accurate detection of moving objects.

The main steps of this method are as follows:

Aiming at moving object detection in a static environment, this paper designs a detection method combining background difference and edge frame difference, and its implementation process is shown in Fig. 2.

Aiming at moving target detection in a dynamic environment, this paper designs a detection method based on SIFT algorithm feature matching, and its implementation process is shown in Fig. 3.

Fig. 2

Moving target detection flow chart in a static environment.

Fig. 3

Moving object detection flow chart in a dynamic environment.

4 Method performance analysis

Figure 4 shows the sequence images collected in this paper. Before detecting moving targets, the images need to be preprocessed to avoid interference of equipment and environmental factors on subsequent moving target detection.

Fig. 4

Original image.

Firstly, the image is grayed. The result of gray processing is shown in Fig. 5. From the figure, we can see that the gray processing can reduce the dimension of the three-dimensional image to two dimensions, but the contour of the image and the moving object has not changed.

Fig. 5

Grayscale processing.

After grayscale processing, the overall illumination of the image is still very low. Therefore, Gamma transform is used to process the image. The processing results are shown in Fig. 6. From the figure, we can see that the overall illumination of the image has been improved, and the illumination of the target has been improved obviously, which effectively reduces the impact of low illumination on the subsequent image processing.

Fig. 6

Low illumination processing.

In the process of sequence image acquisition, there is inevitably noise generated by equipment and environmental influences, so it is necessary to denoise the image. Figure 7 is the image with salt and pepper noise, and Fig. 8 is the image after median filtering. It can be seen that the filtering process significantly reduces the interference of salt and pepper noise to the image [17].

Fig. 7

Image with noise.

Fig. 8

Filtered image.

The image includes target, background and noise. After noise filtering, it is necessary to separate the target from the background to make the moving object more prominent. Binarization is a method to separate the foreground target pixel from the background pixel in the image. The result of binarization is shown in Fig. 9. As can be seen from the figure, most of the background and moving objects are distinguished successfully by binary processing, so it can be known that binary processing is an important step in the process of moving object detection.

Fig. 9

Binarized image.

After preprocessing, this paper first tests the combined background difference and edge inter-frame difference motion detection method in the static environment designed in this paper, and analyzes the performance of the method. The method of this paper is compared with the classical background difference method and the interframe difference method. The comparison results are shown in Table 1.

Table 1

Performance analysis of moving target detection method in a static environment

Method	Correct detection rate (%)	False detection rate (%)	Processing time (s)
Background subtraction method	86.1	6.8	10.9
Interframe difference method	81.6	3.5	11.3
Method in this article	96.3	1.3	12.2

From Table 1 and Fig. 10, we can see that the proposed motion detection method combining background difference and edge frame difference can combine the advantages of background difference and frame difference, and enhance their respective advantages, so that the two methods can be well combined. Compared with the background subtraction method and the inter-frame subtraction method, the correct detection rate is 96.3%, and the false detection rate is 1.3%.

Fig. 10

Comparison of method performance before and after improvement under static environment.

Secondly, the moving target detection method based on SIFT feature matching in the dynamic environment designed in this paper is tested, and the performance of this method is analyzed. Comparing the proposed method with the block matching method and the wavelet transform method, the comparison results are shown in Table 2.

Table 2

Performance analysis of moving target detection methods in a static environment

Method	Correct detection rate (%)	False detection rate (%)	Processing time (s)
Based on Block Matching	83.2	3.7	9.7
Based on Wavelet Transform	85.7	4.9	10.6
Method in this article	92.7	1.9	7.2

Combining Table 2 and Fig. 11, we can see that compared with block-matching and wavelet transform, the performance of the proposed method has improved significantly in three aspects: correct detection rate, false detection rate and processing time. Among them, the correct detection rate of this method is 92.7%, the false detection rate is 1.9%, and the processing time is 7.2 seconds.

Fig. 11

Comparison of method performance before and after improvement in dynamic environment.

5 Conclusions

Nowadays, as the key technology of computer vision and intelligent video surveillance systems, moving object detection has become more and more important. It has been widely used in many fields and contains tremendous energy. It has also attracted the attention and research of many scholars. But up to now, there are still many problems that cannot be solved well in moving object detection in sequence images. To detect moving targets quickly and accurately, this paper proposes corresponding methods for static and dynamic environments in a complex background. Gamma transform is added to image preprocessing to process low illumination images, which can effectively improve the quality of images. Then a detection method combining background subtraction and edge inter-frame subtraction is proposed for a static environment. Because of the dynamic environment, a method of feature matching detection based on the SIFT algorithm is proposed. Finally, the proposed method is tested separately. By comparing with other methods, the good performance of the proposed method is illustrated, which proves that the proposed method can detect moving targets effectively and timely in their respective environments.

Funding

This work was supported by Chongqing Big Data Engineering Laboratory for Children, Chongqing Electronics Engineering Technology Research Center for Interactive Learning, the Science and Technology Research Project of Chongqing Municipal Education Commission of China (KJ1601401), the Science and Technology Research Project of Chongqing University of Education (KY201725C), Basic Research and Frontier Exploration of Chongqing Science and Technology Commission (CSTC2014jcyjA40019), Project of Science and Technology Research Program of Chongqing Education Commission of China. (KJZD-K201801601)

References

Zhou

, Wang

and Li

, Applying the Technology of Moving Target Detection in Missile Training Equipment, International Journal of Pattern Recognition & Artificial Intelligence 31(6) (2017), 17.

Gadicha

and Gadicha

, Implicit Authentication Approach by Generating Strong Password through Visual Key Cryptography, Journal of Cybersecurity and Information Management 1(1) (2020), 5–16.

Zou

, Rapid Detection of Cucumber Leaves Pigments Based on Near Infrared Hyper-spectral Image Technology, Transactions of the Chinese Society for Agricultural Machinery 43(5) (2012), 152–156.

Celebi

M.E.

, Mendonca

and Marques

J.S.

, Dermoscopy Image Analysis [Book review, IEEE Transactions on Medical Imaging 35(4) (2016), 1147–1148.

Suwa

, Yamamoto

, Tsuchida

, et al., Image-Based Target Detection and Radial Velocity Estimation Methods for Multichannel SAR-GMTI, IEEE Transactions on Geoscience & Remote Sensing 55(99) (2017), 1–14.

Liu

Y.J.

, Zhang

J.K.

, Yan

W.J.

, et al., A-Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition, IEEE Transactions on Affective Computing 7(4) (2016), 299–310.

Alnamoly

M.H.

, Alzohairy

A.M.

and El-Henawy

I.M.

, A survey on gel images analysis software tools, Journal of Intelligent Systems and Internet of Things 1(1) (2020), 48–60.

Murashko

F.V.

, Ryzhkova

E.A.

and Vlasenko

O.M.

, Search for an Object in an Image by Image Difference Method to Find Contours of a Natural Leather Blank in Pattern Cutting Process, Fibre Chemistry 50(1) (2018), 38–41.

Balakrishnan

, So

H.Y.

and Man

H.L.

, EM Algorithm for One-Shot Device Testing With Competing Risks Under Weibull Distribution, IEEE Transactions on Reliability 65(2) (2016), 973–991.

10.

Mullai

, Sangeetha

, Surya

, Kumar

G.M.

, Jeyabalan

and Broumi

, A Single Valued Neutrosophic Inventory Model with Neutrosophic Random Variable, International Journal of Neutrosophic Science 1(2) (2020), 52–63.

11.

Wei

, Cao

and Chen

, Seismic Time–Frequency Analysis via Empirical Wavelet Transform, IEEE Geoscience & Remote Sensing Letters 13(1) (2016), 28–32.

12.

Verma

and Singh

, Throughput maximization by alternative use of single and double thresholds based energy detection method, Optik – International Journal for Light and Electron Optics 127(4) (2016), 1635–1638.

13.

Folkes

S.R.

, Lahav

and Maddox

S.J.

, An artificial neural network approach to the classification of galaxy spectra, Monthly Notices of the Royal Astronomical Society 283(2) (2018), 651–665.

14.

Moghaddam

A.H.

, Moghaddam

M.H.

and Esfandyari

, Stock market index prediction using artificial neural network, Journal of Economics Finance & Administrative Science 21(41) (2016), 89–93.

15.

Chang

R.F.

and Chen , et al., Interframe difference quadtree edge-based side-match finite-state classified vector quantization for image sequence coding, Circuits & Systems for Video Technology IEEE Transactions on 6(1) (1996), 32–41.

16.

Aronniemi , Sainio and Lahtinen , Chemical state quantification of iron and chromium oxides using XPS: the effect of the background subtraction method, Surface Science 578(1) (2005), 108–123.

17.

Chen

Hanxin

, Fang

, Fan

Dong Liang

, Huang

Wenjian

, Huang

Jinmin

, Cao

Chenghao

, Yang

Liu

, He

Yibin

and Zeng

, Particle Swarm Optimization Algorithm with Mutation Operator for Particle Filter Noise Reduction in Mechanical Fault diagnosis, International Journal of Pattern Recognition and Artificial Intelligence, 2020.