Fast motion tracking based on moth-flame optimization and kernel correlation filter

2.1 Trackers based on KCF

Recently, Kernel correlation filters for visual tracking have drawn much attention because of its computational efficiency. Tracking algorithms based on kernel correlation filters have made considerable progress. Jeong et al. [9] employ a scale space filter and multi-block scheme based on a kernelized correlation filter tracker to address the problems associated with scale variation and occlusion. Uzkent et al. [10] design an ensemble of KCF to address the variations in scale and translation of moving objects which deploy each of KCFs in turn and develop a particle filter to guarantee run-time performance and reduce any potential drifts between individual KCFs’ transition respectively. Gao et al. [11] introduce the spatial regularization component into the ridge regression model used by KCF to overcome traditional KCF does not consider the prior spatial constraint of the feature distribution of the target which simultaneously keeps the real-time and improve the tracking performance. Li et al. [12] present an online tracking framework which combines shallow convolutional neural networks with KCF. The dataset shows that the method achieves excellent performance. Zhang et al. [13] propose an output constraint transfer (OCT) method for mitigating the drifting problem of KCF by modeling the distribution of correlation response in a Bayesian optimization framework. Ding et al. [14] propose a scalable visual tracking algorithm based on kernelized correlation filters, referred to as quadrangle kernelized correlation filters (QKCF) to handle the scale variation of object. Li et al. [15] propose a new scale adaptive kernelized correlation filter tracker (SKCF), which estimates an accurate scale and models the distribution of correlation response to address the template drifting problem. Zhou et al. [16] make some things to improve KCF covers fused features HOG, color-naming, and HSV to boost the tracking performance;enlarge the scale space from the countable integer space into uncountable float space for handling the scale variations; propose adaptive learning rate and occlusion detection mechanism to update the target appearance model for occlusion problem. Wang et al. [17] propose a multi-scale superpixels and color feature guided kernelized correlation filters (MSSCF-KCF) to deal with fast motion and scale variation. Zhang et al. [18] introduce Simulated Annealing into KCF and a unified framework was designed for object tracking of abrupt motion.

2.2 Trackers based on swarm intelligence

In a sense, tracking procedure can be treated as optimizing problem to search optimal solution in the search space. Among the many optimization methods, swarm intelligence optimization algorithm has attracted more and more attention and has been applied to tracking problems successfully. Zhang et al. [19] incorporate the temporal continuity information into the Particle Swarm Optimization (PSO) for forming a multilayer importance sampling in the framework of the particle filter. In this case, the tracker got better performance especially when the object has an arbitrary motion or undergoes large appearance changes. Kang et al. [20] propose a hybrid gravitational search algorithm (HGSA) method which combines GSA’s gravitational update component with the cognitive and social components of PSO to increase the utilization of particle information and to facilitate exploitation. Wang et al. [21] propose an ant colony optimization (ACO) based iterative particle filter in which ACO is incorporated into the particle filtering framework in order to overcome the well-known problem of particle impoverishment. Nguyen et al. [22] presente a modified bacterial foraging optimization (BFO) algorithm, and design a visual tracking system based on the bacterial foraging optimization to handle some challenges. Gao et al. [23] propose BA-based (bat algorithm) tracker. A BA-based tracking architecture is given and the results showed it is a potentially powerful tracking method.

Although the above methods are successful in tracking, but a considerable number of them predict the position of a target under the assumption that the target is moving smoothly. For efficiency, these methods typically use a smaller local search region. However, these methods will no longer work when the target is in rapid motion. Compared with the above methods, in this paper, we use the optimization algorithm to dynamically adjust the tracker’s search mode so that it can adapt to both fast motion and smooth motion scenarios.

3.1 The base KCF tracker

Using the ridge regression as a filtering model, KCF aims to find a function f (x _i ) = w ^T x that minimizes the squared error over samples x _i and their objective function y _i ,

min w ∑ i ( f ( x i ) - y i ) 2 + λ ∥ w ∥ 2(1)

Where w is kernelized filtering template, λ is a regularization parameter.

The kernel trick is used to further improve performance by allowing the classification on a diversified high-dimensional feature space. The inputs are mapped to the feature space using φ (x _i ), defined by the kernel k (x _j , x _i ) = φ (x _j ) φ (x _i ) = x _j x _i . Generally, w can be expressed as w = ∑ _i α _i φ (x _i ). Then

f ( x i ) = w T x = ∑ i n α i κ ( x i , x j )(2)

Let kernel matrix K = κ (x _i , x _j ). If the kernel matrix K is circulant, the dual space coefficients α can be learnt as below: α ˆ = y ˆ k ˆ xx + λ(3)

Where k ˆ xx is the kernel correlation between x and itself.

In the image, the target can be obtained by the trained parameter α and the base image patch x. Suppose a new sample as z, a confidence map y will be calculated by: y = F - 1 ( k ˆ xz ) ⊙ α ˆ(4)

Where ⊙ is the element-wise product, k ^xz is the kernel correlation between sample x and z, the position of the maximum value of y is regarded as the new position of the predicted tracking object.

Inspired by the navigation method of moths in nature called transverse orientation, Mirjalili [24] proposes Moth-Flame Optimization (MFO) algorithm. The key components in the MFO algorithm are moths and flames, which are considered to be a solution, however, they differ in the way they are handled and updated. The moth is the actual search body that moves in the search space, and the flame is the best solution that the moth has acquired so far.

Spiral flight of moth.

KCF based approaches have shown impressive performance on the tracking benchmarks. However, there are still some deficiencies. Because of the localized search strategy, KCF may fail when the target undergoes fast motion or abrupt motion. In addition, the direct expansion of the search area will bring problems such as difficulty in updating the filter and background interference. We note the advantages of the optimization approach in global exploration. In this paper, MFO algorithm is mentioned to solve the problem of abrupt motion tracking. Through providing possible rough target positioning through MFO algorithm, the shortcomings of traditional KCF can be compensated to some extent. The efficiency of KCF also provide the possibility for this method mentioned above. Based on this, we built the MFOKCF fast motion tracking framework, as shown in Fig. 1.

OPEN IN VIEWER

Fig.1

The flowchart of the MFOKCF tracker.

The MFOKCF tracking framework consists of two parts: the basic KCF module and the MFO module. In frame t, the tracker first determines the search window based on the target location of the previous frame. Then, feature extraction and filtering operations are performed on the extracted candidate samples to obtain the target response. Preliminary target is identified according to the response. We test the reliability of the preliminary target to see if it is lost due to fast motion. The MFOKCF tracker used a conservatively updated correlation filter to maintain the long-term memory of the target. This memory generator adopts a smaller patch to reduce the introduction of background in the training process. It is updated only when the target is in smooth motion. Threshold Thr is defined to decide the necessity of model update. If the response of the memorizer is larger than Thr, the predicted target is considered to be consistent with the target memory. Otherwise, the target is considered to have escaped the search range of the backbone tracker. For this, MFO module is used to search for estimated target locations in a global context, and then the search patch from MFO is provided to KCF for more accurate target location.

Compared with the traditional correlation filter, our method uses the optimization method to improve the tracker’s global exploration ability. When fast motion occurs,tracking drifts will not only cause the tracking failure, but also introduce a lot of background information when the correlation filter is updated, which makes it difficult to recover the follow-up tracking. We first use a conservatively updated correlation filter to capture the occurrence of fast motion. The MFO algorithm is then introduced to provide the target with a new candidate region from which to recover the target. Moreover, the framework avoids introducing more errors in the process of correlation filtering to some extent.

4.1 Experimental setup

To verify the performance of the proposed tracing method,our MFOKCF tracker is validated on three data sets, including self-made fast motion data sets OTB-2013 and OTB-2015 respectively. The proposed method was tested in MATLAB R2016a. The experiments were executed on the PC with Intel Core i5 2.50GHz and 8GB RAM. Threshold Thr determines whether fast motion occurs, we adopted an adaptive threshold to adjust the timing of MFO invocation. After testing, we set the threshold Thr to be 0.38 times of the average memorizer response.The setting of the basic KCF module can be referred to [5]. In order to get anywhere globally, flames number set to 2 and moths number set to 500. The parameters of the tracking method are consistent in each experiment.

4.2 Fast motion sequences

We use 10 challenging sequences to prove that the proposed method can adapt to the visual tracking, especially fast motion. The sequences are divided into two groups according to the motion displacement of the target in the adjacent image (Listed in Table 1). One of the both contains the CARDARK, MOUNTAINBIKE, TRELLIS, MAN and DEER sequences, whose motion displacement is less than 50 pixels. And the displacement of the other group is more than 50 pixels, including the BLURCAR3, ZT, BLURFACE, FACE2 and FLEETFACE sequences. FACE2 and ZT are our own. Other sequences are from the website http://visualtracking.net. In addition, in order to mimic the large motion displacement formed by the fast motion of the target, the frames 306-310 in BLURFACE sequence and 401- 410 in FLEETFACE sequence and the frames 26-35 in MAN sequence are extracted.

We compared our tracker with other state-of-the-art trackers, including, High Speed Tracking with Kernelized Correlation Filters(KCF) [5], Accurate Scale Estimation for Robust Visual Tracking (DSST) [25], Fast Compressive Tracking(FCT) [26], Fast Tracking via Spatio-Temporal Context Learning (STC) [27], Least soft-threshold squares tracking(LSST) [28] and Context-Aware Correlation Filter Tracking(CACF) [29]. The same set of parameters of MFO is applied to all sequences.

4.2.1 Qualitative analysis

The slight motion group

In CARDARK sequence, as shown in Fig. 2 (row #1), there is a car moving quickly in a dark road scene with background clutter and varying lighting conditions. In the sequence, the target has a relatively smooth motion. Most of trackers have a good performance except FCT that fails at frame #95 due to the illumination changes and the low contrast between foreground and background.

In MOUNTAINBIKE sequence, as shown in Fig. 3 (row #2), object undergoes in-plane and out-of-plane rotations in a cluttered background. LSST fails to locate the target after out-of-plane rotation occurs at frame #85. From frame #85, #158 and #195 other trackers are able to accurately track the object throughout the entire sequences.

The TRELLIS sequence, as shown in Fig. 2 (row #3), it is captured in an outdoor environment where the object appearance changes significantly as a result of cast shadows, motion and head pose variation. FCT, STC and LSST start to fail in tracking the face after the #412 frame.

In MAN sequence, as shown in Fig. 2 (row #4), the larger illumination changing and displacement from frame #25 to #26. As a consequence, all trackers drift or even fail to different degrees. In contrast to these competing trackers, our tracker is able to successfully track the face till the end of the sequence.

In DEER sequence, as shown in Fig. 2 (row #5), it describes several deer run and jump in a river. The target undergoes large motion and some frames are blurred, for example frame #32. Meanwhile, there are similar targets at frame #43 that interfere with tracking. DSST, CACF and KCF lose the target from frame #26. Our tracker generates accurate results even with heavy motion blur and water occlusion.

The large motion group

In BLURCAR3 sequence, as shown in Fig. 3 (row #1), the target is severely blurred in the cluttered background due to fast motion of the camera or the fast motion of the target itself. From the results we can see that our tracker can handle these challenges well. KCF, LSST and FCT, fail to locate the target at frame #119 while STC break down at frame #274.

In FACE2 sequence, as shown in Fig. 3 (row #2), the occurrence of fast motion results in a motion displacement up to 88 pixels between adjacent frames, accompanied by a target blur. Most trackers lost their targets in frames #0153 and #0160. However, our method and CACF achieve the best tracking results.

In FLEETFACE sequence, as shown in Fig. 3 (row #3), the displacement reaches 125 pixels. At frame #0365, the FCT begins to deviate significantly from the target. At frame #411, the sequence experienced the largest motion displacement. Overall, combined with the results of the frame #0411 and #0635, only our tracker can complete the entire sequence.

In BLURFACE sequence, as shown in Fig. 3 (row #4), there are the severe motion blur at frame #150 and #272, which reduces the discriminative information in feature vectors, it is difficult to predict their locations. At the beginning, all trackers perform well in sequences. However, only our tracker obtains the target when it undergoes fast motion at frame #311. Compared with these trackers, our tracker outperform the other trackers.

In ZT sequence, as shown in Fig. 3 (row #5), a human face which moves left and right rapidly and motion displacement between consecutive image frames up to 256 pixels. At frame #0011, FCT, LSST and STC deviate from the target first. Immediately thereafter, KCF, STC and DSST failed to track at frame #0033. CACF and our tracker have better tracking performance, and our tracker is slightly better than CACF.

OPEN IN VIEWER

Fig.2

The tracking results with the slight motion.

OPEN IN VIEWER

Fig.3

The tracking results with the large motion.

It can be intuitively seen from the tracking results that in the small motion group, our tracking methods can maintain stable tracking, which indicating that our methods can adapt to smooth or small target displacement. Other comparison algorithms can maintain the tracking in most cases, but there is still some drift or even loss when displacement occurs. When the target displacement increases, in the large motion group, most of the algorithms have serious drift, which leads to the tracking failure. Our algorithm has good tracking performance because it introduces MFO algorithm to estimate the possible position of the target in whole frame. Especially in the FACE sequence, only our method can complete the tracking due to the serious displacement of simulation. Overall, our method can be well adapted to smooth and large displacement tracking scenes.

4.2.2 Quantitative analysis

Fig. 4 respectively shows the Distance Precision and Overlap Precision of 10 sequences. Table 2 and Table 3 report MFOKCF tracker compared to DSST, LSST, KCF, STC, FCT and CACF. Table 2 involves average overlap rate, and Table 3 is related to average error center rate. In the tables, red and blue fonts represent the two best results. It is clear from Fig. 4, Table 2 and Table 3 that MFOKCF tracker performs better than the other 6 trackers when there is a larger movement displacement between continuous images. The MFOKCF tracker also shows better performance in other image sequences. Overall, the method has the advantage of fast motion compared to the 6 trackers.

OPEN IN VIEWER

Fig.4

Precision plots and success plots of OPE.

OPEN IN VIEWER

Table 2

Average overlap rate

Sequences	MFOKCF	KCF	CACF	DSST	FCT	LSST	STC
CARDARK	0.64	0.61	0.64	0.81	0.14	0.87	0.73
MOUNTAINBIKE	0.71	0.71	0.71	0.73	0.65	0.31	0.58
TRELLIS	0.63	0.63	0.79	0.77	0.32	0.27	0.46
MAN	0.82	0.40	0.40	0.24	0.18	0.18	0.18
DEER	0.74	0.62	0.63	0.64	0.66	0.71	0.04
BLURCAR3	0.82	0.82	0.82	0.84	0.25	0.25	0.20
FACE2	0.78	0.76	0.78	0.75	0.37	0.36	0.36
FLEETFACE	0.65	0.40	0.40	0.47	0.38	0.46	0.36
BLURFACE	0.82	0.51	0.51	0.53	0.23	0.30	0.48
ZT	0.82	0.59	0.84	0.65	0.09	0.09	0.35

OPEN IN VIEWER

Table 3

Average center error rate

Sequences	MFOKCF	KCF	CACF	DSST	FCT	LSST	STC
CARDARK	5	6	6	2	47	1	3
MOUNTAINBIKE	8	8	8	8	12	152	7
TRELLIS	7	7	7	3	45	45	34
MAN	3	22	22	30	32	71	63
DEER	5	21	23	17	11	7	510
BLURCAR3	4	4	3	3	180	117	57
FACE2	6	8	6	14	138	116	107
FLEETFACE	26	108	114	131	81	81	113
BLURFACE	8	85	112	75	116	162	90
ZT	18	128	16	54	642	685	100

To test the overall performance of our tracker, we compared our MFOKCF tracker and backbone KCF in OTB-2013 [30] and OTB-2015 [31]. The object tracking benchmark (OTB) is one of the most important datasets in the tracking community. Among them, OTB-2013 contains 50 sequences with different challenges, and OTB-2015 contains 100 sequences. For comparison, the tracking results include the results of some baselines trackers, including HCFT [33], SRDCF [34], Staple [35], MEEM [36], LCT [37], SAMF [38], DCFnet [39], KCF [5] and DSST [25]. Different from the basic KCF tracker based on HOG features, in this section, we use VGGnet [32] deep feature to make the MFOKCF tracker and perform better.

4.3.1 OTB-2013 results

The one-pass evaluation (OPE) result on OTB2013 with the distance precision plot(DPR) and the overlap success plots(OSR) are shown in Fig. 5. It can be seen that the proposed MFOKCF tracker has achieved optimal or suboptimal performance in both metrics. Compared to the basic KCF tracker, the proposed MFOKCF tracker obtain 16.8% gain in DRP and 12.7% gain in OSR. This shows that it is more accurate to obtain the basic samples by MFO algorithm than to apply the localized search window directly. The ability to respond to fast or uncertain motion of the target makes our trackers perform better. Compared to deep feature based tracker DCFnet, the proposed MFOKCF tracker obtain a 9.6% gain in DRP. Moreover, compared to the HCFT tracker using the multilayer convolution feature, the MFOKCF tracker achieves HCFT similar performance on DSP and 2.6% gain on OSR. This shows that our basic sample selection strategy implemented by MFO can help KCF improve tracking performance. In addition, LCT with long-term tracking ability can cope with the fast motion of the target to a certain extent. Compared with LCT, our MFOKCF obtain a 4.3% gain in DRP.

OPEN IN VIEWER

Fig.5

Average overall performance on OTB-2013(Left: Precision Plot Right: Success Plot).

4.3.2 OTB-2015 results

Based on otb-2013, OTB2015 increased the total number of sequences to 100. In the overall performance, our tracker also got the best results.The OPE result on OTB2015 with the DRP and the OSR are shown in Fig. 6. Compared with HCFT, in more complex tracking scenarios, our MFOKCF tracker obtain 0.05% gain in DRP and 3.9% gain in OSR. This shows that our method has better generalization ability. Meanwhile, compared with the basic KCF tracker, our method is obtained 15.5% gain in DRP and 12.9% gain in OSR. Compared with LCT with long time tracking ability, our method still achieves better performance.

OPEN IN VIEWER

Fig.6

Average overall performance on OTB-2015(Left: Precision Plot Right: Success Plot).

The OTB2015 contain 11 different attributes, including out-of-view (OV), deformation (DEF), background clutters (BC), in-plane rotation (IPR), illumination variation (IV), motion blur (MB), fast motion (FM), scale variation (SV), low resolution (LR), out-of-plane rotation (OPR) and occlusion (OCC). We evaluate our tracker in OTB2015 with these 11 different attributes. Our framework improves tracking performance in most scenarios.And There are certain categories that benefit more than others. The most significant improvement is achieved in the cases of motion blur, out-of-plane rotation, background clutter and fast motion as shown in Fig. 7. In particular, if the object experiences fast motion, our framework is very beneficial. Furthermore, our framework also improves significantly for videos with motion blur. This is largely due to the fact that adding the MFO allows for a modified search region. When the target is lost, the MFOKCF retrieves the lost target through a global search. It is also notable that our method has the best performances in background clutter.

OPEN IN VIEWER

Fig.7

Average performance on OTB-100 for 4 attributes.

Combining with MFO algorithm, we propose a fast motion tracking framework based on correlation filtering. MFO algorithm can identify potential targets from the whole framework and provide search patches, so that the KCF tracker has more recognition of specific sequences. In fast motion sequences, our tracker performs well in all test sequences. As the displacement increases, only our method can adapt to the fast motion of the target. This is because the introduced MFO algorithm can effectively capture fast moving targets globally. Because of global sampling, our method also has some adaptability to the uncertain motion of the target (for example, BLURFACE and FLEETFACE). Based on the correlation filter of conservative training, our method can track both smooth and fast motion sequences. Experimental results on otb-2013 data set show that the proposed tracking algorithm has better performance than existing methods. In the 11 challenges of the otb-2015 data sets, our approach performed well for fast motion, motion blur, out-of-field and background complexity. MFO algorithm not only provides more accurate search window, but also enables KCF to use more accurate target template in the process of filter training, thus further improving the accuracy. To sum up, our method can achieve the target fast or smooth tracking.