Dynamic tracking algorithm suitable for intelligent video surveillance

Abstract

In order to improve the shortcomings of the existing algorithm and aiming at the problems of classical algorithm in moving object detection and tracking, a moving object detection algorithm combined with improved texture and chromatic information is proposed. The texture and color two kinds of features insensitive to shadow are used to describe the background, and then the background description is applied to the Gauss mixture model to put forward a new method for moving target detection, which can better resist the influence of shadow and background illumination changes. At the same time, the Camshift tracking algorithm jointing LBP texture and color information and target tracking algorithm based on Kalman filter and Blob matching method are put forward, and the actual effect of the algorithm is tested. The test results showed that the algorithm reduces the computational complexity and improves the robustness of target tracking under the complex scene. It summed up that the tracking algorithm has good performance.

Keywords

Intelligent video surveillance dynamic tracking local two-element mode filter

1. Introduction

Computers have been widely used in the processing of information such as text, sound, and image. Because of the large amount of data and slow manual processing, video information is urgently required to be edited and analyzed by computer. An important research direction of video processing by computer is the intelligent video surveillance [1]. Functionally speaking, video surveillance system can be used for security protection, information acquisition and traffic scheduling and command and so on, providing services such as public place security, special place security, medical real-time monitoring, distance education and so on [2]. From the application, the video surveillance system have been widely used in various industries, such as traffic flow monitoring, highway toll system, traffic violation investigation, remote monitoring of important dangerous road bridge and other places [3]; detection of factory workshop product line and control as well as appearance detection; the Public Security Department streets monitoring system, the court places monitoring, entertainment places monitoring, drug places monitoring and prison monitoring; real-time monitoring and alarm of important warehouse, confidential record room, financial institutions and other regional sites; unmanned monitoring of office, parking lot, public square, airport, station, residential district, intelligent buildings and so on [4].

At present, video surveillance has been widely used in various commercial applications, but it often does not give full play to its real time active supervision. In a word, the equipment with a certain intelligence video monitoring system has been the urgent demand of many industries [5]. Therefore, the research of intelligent video surveillance is of great social and economic value, and it has also become a hot-spot in the field of computer vision [6]. Intelligent video surveillance is in no need of manual intervention cases, using the method of computer vision, digital image processing and video image analysis, for automatic analysis of the video camera image sequences, realizing detection, tracking and identification of moving target in the video scene [7]. On this basis, we analyze and judge the behavior of the target, give the description of the behavior and action of the moving target, find some suspicious cases automatically, and realize the identification and automatic alarm function of abnormal cases in the system scene, so as to guide and plan the action [8].

Intelligent video surveillance is a multidisciplinary research field, and it is also a hot research topic in the field of computer vision and image processing [9]. The intelligent visual monitoring system is the most effective means to solve the financial department, the machine department, and the sensitive public occasion real-time, automatic and all-weather monitoring [10]. Because of the wide application value and prospect, intelligent video surveillance technology has attracted the interests of many research institutions, enterprises and research workers. Many researches have been carried out [11].

2. Methodology

2.1 Composition of intelligent video surveillance system

In intelligent video surveillance system, video capture, image processing, moving object detection, moving object tracking, moving object classification and behavior description and understanding are included. In addition, an intelligent video surveillance system often involves the same monitoring of multiple scenes captured by multiple cameras, so sometimes it involves information fusion. The overall framework of an intelligent video surveillance system can be represented as Fig. 1.

Figure 1.

The whole processing framework of intelligent video surveillance system.

As shown in Fig. 1, intelligent video surveillance system mainly includes video and image acquisition module, image preprocessing module, moving object detection module, moving object classification module, moving target tracking module, behavior understanding and description module, and alarm and execution module. The above seven modules are the components of the intelligent video surveillance system. So far, the intelligent video surveillance system rely on the above modules that it can complete the traditional monitoring system passive monitoring function. It can also complete some new functions as follows, including intrusion detection, line detection, abandoned objects detection, lost object detection, abnormal changes of camera scene, moving target count, fire and smoke detection, defocus detection, target active tracking and human abnormal behavior detection [12]. Of course, with the deepening of the application, many new requirements will be put forward according to the different requirements of the user, all of which require the progress of the research.

2.2 Moving target tracking technology

According to the environment in which the moving target and target are, it is a key problem to select the appropriate feature of the moving target tracking algorithm. At present, the characteristics of the sports target are divided into three main categories: local characteristics, global characteristics and relationship characteristics. Global characteristics refer to some integral characteristics of moving objects, such as perimeter, area, invariant moments, auto regressive model and Fourier transform frequency spectrum. The computation of these features depends on the location, gray value and spatial relationship of all pixels in the target area or on the boundary of the target area, so the global feature is robust to the random noise appearing on some pixels. However, because the global features are the overall statistical characteristics based on the pixels in the moving area, when the moving object is occluded, the occlusion part has great influence on its computation, and the tracking effect is not ideal. Local features refer to some local information of a moving target, such as some special points on a moving target, a particular edge or some special texture. When the motion target is obscured, the target tracking uses the local feature more effectively than the global feature, but the local feature is easily disturbed by the noise. The relationship features refer to the relationship information between some local significant features, such as the prominent angle between two edges and the distance between the two local feature points, etc. The selection of a good feature as a matching feature is the prerequisite for a good tracking algorithm for moving targets. For tracking complex targets, it is the most direct and effective way to track targets with multiple features because of the interference of the shape and posture of the target. At present, the commonly used methods are shown in Fig. 2.

Figure 2.

Commonly used moving target tracking methods classification (left).

2.3 Camshift tracking algorithm combining LBP texture and chromaticity information

Continuous adaptive mean shift (Camshift) algorithm is a non-parametric method of gradient estimation of probability density function. It can adjust the size of tracking window adaptively according to the size of the target being tracked. The algorithm is simple and has good real-time performance. The traditional Camshift algorithm is target tracking based on the color histogram. When the target and background color are similar, or the interference target and the tracked target color are similar, the correctness of tracking is greatly affected. This paper uses the combination of LBP texture and chromaticity to describe the moving objects, expecting to overcome these problems. At the same time, because shadows exist widely in the image, the brightness of the shadow area is reduced compared with the background. The shadow will also affect the correctness of tracking.

In this study, when the LBP texture and chroma are used to describe the moving objects, the two features are insensitive to the shadow, and can overcome the effect of shadow on the tracking results. For regions lacking texture, the performance of LBP texture is not good, but color is often able to achieve good results. But for regions with the same color of moving targets and backgrounds, LBP texture often achieves some results. The moving target tracking algorithm based on texture and color consists of the following parts. First of all, the algorithm is initialized, namely manually specifying or getting the moving object region through the target detection, the calculation of joint histogram of the LBP texture and color in the target area. And then, the probability model of candidate matching is determined, and the initial position of the target and the number of iterations are set.

The full name of the Camshift algorithm in English is the continuously adaptive mean shift algorithm. The core of the Camshift algorithm is the Meanshift algorithm. Camshift algorithm is an iterative process, and it puts all the operations of Meanshift in the frame of video sequence. The target center of search window bandwidth of the last frame is used as that of the next Meanshift algorithm. The iteration is continuing and thus it can complete tracking of moving target. The principle of the Camshift algorithm can be summarized as:

For all pixels $\left\{{x_{i}|i=1,2,\ldots,n}\right\}$ , the probability distribution $\widehat{q_{u}}$ of their color is calculated and used as the matching model of moving targets in the specified or detected elliptical or rectangular box area with moving objects. The probability density of the color feature is as follows:

$\displaystyle\widehat{q_{u}}=C\mathop{\sum}\limits_{i=1}^{n}k\left({||x_{i}||^% {2}}\right)\delta\left[{b\left({x_{i}}\right)-u}\right]$ (1)

In the above formula, $n$ suggests the number of points in the target area, $k\left(x\right)$ is a kernel function, and the closer the distance to the search window center, the greater its value. $b\left({x_{i}}\right)$ refers to the color feature corresponding to the pixel, and when it satisfies the feature $u$ , the value of the pulse function $\delta\left[{b\left({x_{i}}\right)-u}\right]$ is 1; otherwise, it is 0. $C$ indicates the normalized function, and if the number of the eigenvector $u$ is $m$ , $C$ can be derived by $\sum_{u=1}^{m}\widehat{q_{u}}=1$ , that is:

$\displaystyle C=1/\mathop{\sum}\limits_{i=1}^{n}k\left({||x_{i}||^{2}}\right)$ (2)

Secondly, we read the next image from the video sequence. We calculate the joint histogram of LBP texture and chroma information in the rectangle range with half width of the smallest circumscribed rectangle, and determine the candidate model. Then, we calculate the weight of the mean value, the location of the possible target in the next frame and the number of iterations. Finally, when the iteration stops in the current frame, the target position is at the center of the current frame. Otherwise, we adjust the target position and continue the current iteration. The flow chart of the algorithm is shown in Fig. 3.

Figure 3.

Flow chart of tracking algorithm based on joint LBP texture and chromaticity information (right).

2.4 Moving target tracking algorithm based on Kalman filter and Blob matching method

The Blob matching method uses the shape features of moving targets to match candidate targets. This method has better tracking effect on rigid objects. The shape features include target size, contour shape and spatial moment. In the case of less moving objects, the Blob matching method is easy to match, so the calculation speed is very fast. However, when the number of moving objects is more, each Blob is matched with the other, the amount of calculation increases rapidly, but the calculation speed will become slower. At the same time, the Blob matching method uses the shape of the moving target as the feature, so it is not suitable for the tracking of non-rigid moving objects.

The Kalman filter can predict the possible position of the moving target in the next frame based on the information of the current frame. If the Kalman filter is combined with the Blob matching method, the Kalman filter is used to predict the approximate location of the moving target. Then, Blob matching is carried out near the prediction location so that the computation amount of matching can be greatly reduced. Moving target tracking algorithm based on Kalman filter and Blob matching method is shown as follows. Firstly, the two-value images in the moving target area look for each connected region in two-value image and each region is denoted as a Blob. For each Blob, we calculate the center location, spatial moments and contour size as features. These features and the number of each Blob are stored as a record. Secondly, according to the historical status of the central location of the moving target before the current frame, we predict the location of every Blob in the next frame using the Kalman filter algorithm. Again, according to the size, contour shape and space moment features of every Blob, we match near the prediction position. If we find Blob matching with it, then, we mark it as the same number, indicating that the prediction is successful; otherwise, it is assigned a new number, storing a new Blob record. The flow chart of the algorithm is shown in Fig. 4.

Figure 4.

Flow chart of moving target tracking algorithm based on Kalman filter and Blob matching method.

In summary, through the analysis of the video images sequence, moving target tracking algorithm can match the candidate moving target detected by different characteristics, and calculate out the corresponding position of the moving object in each frame image. That is the location belonging to the same moving target in the video image sequence. In this study, moving target tracking is expounded. Then, Camshift moving target tracking algorithm based on LSP texture and chroma and moving target tracking based on Kalman filter and Blob information are proposed.

3. Results analysis and discussion

3.1 Experimental results of Camshift tracking algorithm for combining LBP texture and chromaticity information

This algorithm is implemented on the PC machine of Intel i3-350 and 2G memory by adding Visual C++6.0 to OpenCV1.0. Experiments are carried out by using the algorithm discussed here, the traditional algorithm and automatic Camshift tracking algorithm jointing multi features to track the two videos of the video surveillance standard test video set PETS2000 and PETS2001 DATASETI Camera2. The experimental results show that the 3 frames of PETS2000 are in turn 125, 162 and 180, and the 3 frames of PETS2001’s DATASET 1 are in turn 556th, 584th and 676th frames. After experiments, for the LBP operator, when $P=$ 8, then $R=$ 1; for chromatic, when $\alpha=$ 15, the experiment can achieve better results in tracking accuracy and speed. At this time, the joint histogram of LBP texture and chroma has 34 levels.

In the experiment, we can also find that the three methods can track the red vehicle accurately, because the color of red vehicle is prominent in the whole video, and there is no other interference. But this method is more accurate in target location. When the screen is dark, tracked vehicle and pavement background color contrast is small. And in the process of tracking, there is pedestrian interference, and the traditional algorithm has inaccurate positioning. Automatic Camshift tracking algorithm jointing multi feature can track the moving target, but the positioning is not accurate and this method has a good effect. In terms of time consumption, the algorithm discussed here has not great difference with automatic Camshift tracking algorithm jointing multi feature in the average number of iterations per frame. But the tracking speed has been improved, and the tracking speed is not very different from the other two, as shown in Table 1 for details.

Table 1
Performance comparison of three algorithms

Algorithms	Number of	Average number of	Tracking
	elements	iterations per frame	speed (fps)
Camshift algorithm	4096	2.6	18.9
Automatic Camshift tracking algorithm	64	2.1	14.1
jointing multi features
The algorithm discussed here	34	2.15	17.5

3.2 Experimental results of moving target tracking based on Kalman filter and Blob matching method

The algorithm proposed in this paper is compared with the Meanshift moving object detection algorithm, and the Meanshift algorithm has mistaken detection. However, the algorithm proposed in this paper can accurately track moving targets. The proposed method can reach 25 frame second processing speed in the hardware and software environment, which meets the real-time requirement.

After experiments, when there is not much overlap in the moving target area, the algorithm proposed in this paper can better solve the interference of moving background to moving targets. In addition, the tracking accuracy of the moving target is improved by combining the multiple features of Blob with the Kalman filter.

However, this method also has some shortcomings. For example, after moving targets are crossed and separated to continue moving, there may be two moving target as a moving target and can be regarded as two new goals. That is to say, sometimes, it cannot solve the occlusion problem between the moving targets well. After analysis, it is found that although the Kalman filter makes full use of the motion information related to motion object and predicts the possible location in the next frame, the final result is still matched by the Blob matching method. Therefore, even when the target is occluded, the Kalman filter predicts the location of the moving target correctly, but at this time, the Blob cannot match with the candidate Blob that has been stored and updated so that the moving target cannot be tracked correctly.

4. Conclusion

In this study, aiming at the application of moving object detection and tracking algorithm, the LBP (Local Binary Pattern) operator is improved, and the speed of moving object detection and tracking using LBP texture is improved at the time of feature matching. In the meanwhile, a moving target detection algorithm combined with improved LBP texture and chromaticity information is proposed. This method can well resist the influence of shadow and background light. In addition, a Camshift tracking algorithm for combining LBP texture and chromaticity information is proposed. Compared with other algorithms, this method can achieve better results for moving targets similar to background colors and interference targets similar to the tracked targets color. Finally, a target tracking method based on Kalman filter and Blob matching method is proposed. The Kalman filter is used to predict the location of moving targets in the next frame. Blob matching is applied in the prediction area, which reduces the computation and improves the robustness of target tracking in complex scenes.

References

Ren

and Smith

J.S.

, Multi-view and multi-plane data fusion for effective pedestrian detection in intelligent visual surveillance, Multidimensional Systems Signal Processing27(4) (2016), 1–23.

Alshaqaqi

and Boumehed

, New technique for extraction moving object based on active contours for intelligent visual surveillance applications, International Review on Computers Software10(7) (2015), 660.

Ramamoorthy

and Banu

U.S.

, Intelligent video surveillance system using background subtraction technique and its analysis, International Journal of Applied Engineering Research10(2) (2015), 4849–4863.

Bouchrika

Carter

J.N.

and Nixon

M.S.

, Towards automated visual surveillance using gait for identity recognition and tracking across multiple non-intersecting cameras, Multimedia Tools Applications75(2) (2016), 1201–1221.

Lan

Jiang

and Fan

, Real-time automatic obstacle detection method for traffic surveillance in urban traffic, Journal of Signal Processing Systems82(3) (2015), 1–15.

Zhang

Wang

and Huang

, Multiple pedestrian tracking based on couple-states Markov chain with semantic topic learning for video surveillance, Soft Computing19(1) (2015), 85–97.

Morozov

A.A.

, Development of a method for intelligent video monitoring of abnormal behavior of people based on parallel object-oriented logic programming, Pattern Recognition Image Analysis25(3) (2015), 481–492.

Gouet-Brunet

and Lameyre

, Object recognition and segmentation in videos by connecting heterogeneous visual features, Computer Vision & Image Understanding111(1) (2015), 86–109.

Bloisi

D.D.

Previtali

and Pennisi

, Enhancing automatic maritime surveillance systems with visual information, Transactions on Intelligent Transportation Systems18(4) (2017), 824–833.

10.

Zhang

Muramatsu

and Zhou

, Blind image quality assessment using the joint statistics of generalized local binary pattern, Signal Processing Letters22(2) (2015), 207–210.

11.

Zhang

Hou

and Ma

, Efficient video frame insertion and deletion detection based on inconsistency of correlations between local binary pattern coded frames, Security Communication Networks8(2) (2015), 311–320.

12.

Hsieh

J.W.

et al., Abnormal scene change detection from a moving camera using bags of patches and spider-web map, IEEE Sensors Journal15(5) (2014), 2866–2881.

Dynamic tracking algorithm suitable for intelligent video surveillance

Abstract

Keywords

1. Introduction

2. Methodology

2.1 Composition of intelligent video surveillance system

3.1 Experimental results of Camshift tracking algorithm for combining LBP texture and chromaticity information

Table 1 Performance comparison of three algorithms

4. Conclusion

References

Table 1
Performance comparison of three algorithms