A Scale Adaptive Mean-Shift Tracking Algorithm for Robot Vision

Abstract

The Mean-Shift (MS) tracking algorithm is an efficient tracking algorithm. However, it does not work very well when the scale of a tracking target changes, or targets are occluded in the movements. In this paper, we propose a scale-adaptive Mean-Shift tracking algorithm (SAMSHIFT) to solve these problems. In SAMSHIFT, the corner matching is employed to calculate the affine structure between adjacent frames. The scaling factors are obtained based on the affine structure. Three target candidates, generated by the affine transformation, the Mean Shift and the Mean Shift with resizing by the scaling factors, respectively, are applied in each iteration to improve the tracking performance. By selecting the best candidate among the three, we can effectively improve the scale adaption and the robustness to occlusion. We have evaluated our algorithm in a PC and a mobile robot. The experimental results show that SAMSHIFT is well adaptive to scale changing and robust to partial occlusion, and the tracking speed is fast enough for real-time tracking applications in robot vision.

1. Introduction

Real-time target tracking is a critical task in robot vision. For example, a service robot needs to track a target person to provide services; a surveillance robot may be required to track a suspected object to get more information. In such real-time target tracking applications, the scale change and occlusion of a target are the most challenging issues besides the time performance. Among various tracking algorithms, the Mean-Shift tracking [1] is one of the most efficient tracking algorithms for real-time applications. But the Mean-Shift tracking algorithm has poor performance when the scale change of a target occurs or targets are occluded. In this paper, we focus on solving these problems of the Mean-Shift tracking algorithm for real-time target tracking in robot vision.

Many techniques have been proposed to solve these problems in the previous work. In [1], a method is proposed to reduce the interference of background near the target area in Mean-Shift tracking by using the background-weighted histogram (BWH). In this method, the radius of the kernel is modified by ±10%. The Mean-Shift tracking algorithm is independently applied three times with different kernel scales. The scale which yields the largest Bhattacharyya coefficient is selected as the kernel scale. Ning et al. [2] proved that BWH is equivalent to the common target representation. No new information in BWH is introduced to improve the performance of Mean-Shift tracking. Then a corrected BWH (CBWH) was proposed by transforming the original target model but not the target candidates. The CBWH scheme significantly reduces the background interference in a target area by enhancing prominent features of a target model and reducing the impact of similar image features shared by the target and background. In [3], an algorithm is proposed based on CBWH. The algorithm performs well when targets become smaller. However, when targets become larger, especially when the scale change exceeds the size of tracking window, it cannot work very well as the tracking window shrinks because the Bhattacharyya coefficient always converges to the local maximum value in a smaller searching window [4].

An automatic bandwidth selection method is proposed in [5]. This method applies backward tracking and object centroid registering to improve the performance of Mean-Shift tracking. This method is based on the theorem that the changes of the target scale and target position within the kernel will not impact the tracking accuracy of the Mean-Shift tracking algorithm. However, this method cannot be employed when the scale change of a target becomes smaller. In [6], a method called CAMSHIFT is proposed to address this problem based on invariant moments. This method is not suitable for real-time tracking due to the complexity of the calculation of the second order central moment. Hu et al. [7] presented an enhanced Mean-Shift tracking method similar to CAMSHIFT algorithm by using joint spatial-color feature space and a novel similarity measure function. Corresponding eigenvalues are used to monitor the scale of the object. These algorithms have great deviation in tracking and might lose the target when the target is occluded. Affine projection is also introduced to solve the problem of scale adaptation [8 –10]. However, simply introducing affine transformation in these methods cannot get a satisfied scale adaptation. The interest point matching in these methods is not efficient to obtain correct affine structures. Furthermore, tracking failure occurs when the target is occluded.

In this paper, we proposed a scale-adaptive Mean-Shift tracking algorithm (SAMSHIFT) to solve these problems. In our algorithm, we provide three target candidates that are obtained by an efficient affine transformation based on the corner point matching, the Mean-Shift tracking algorithm, and the Mean-shift algorithm with resizing by the scaling factors calculated from the affine structure, respectively. Then the best candidate among the three is selected in such a way that we can effectively improve the scale adaptation and robustness to occlusion.

We evaluate our algorithm in a PC and a mobile robot, respectively. Our algorithm is compared with several representative methods including the traditional Mean-Shift tracking algorithm [1], the traditional Mean Shift with 10% adaption [1], and the spatial colour Mean-Shift tracking algorithm [7]. The experiments show that our algorithms can effectively improve the scale adaptation and occlusion robustness both in the PC and mobile robot. Furthermore, the tracking speed of our algorithm is fast enough for real-time tracking applications in robot vision.

The rest of this paper is organized as follows: in Section 2, the Mean-Shift tracking algorithm is outlined, and its limitation in scale change is analysed. In Section 3, our scale-adaptive Mean-Shift tracking algorithm is presented. In Section 4, we present and analyse the experimental results. The conclusions are given in Section 5.

2. Background

In this section, we first introduce the Mean-Shift tracking algorithm and then analyse existing problems.

The Mean-Shift tracking algorithm is a semiautomatic tracking algorithm. The target window in the first frame is initialized manually or by other recognition algorithms. The Mean-Shift tracking algorithm is an iterative scheme, in which the RGB colour histogram of the original target in the first frame is iteratively compared with that of the target candidate regions in the following frames. The objective function is to maximize the correlation between two colour histograms.

Let S = {x_i}_{i = 1, …, n} be a sample of independent and identically distributed random variables drawn from some distribution with an unknown density f(x). The size of the target is M × N pixels. The kernel density estimation used in the traditional Mean-Shift tracking is defined as follows:

f (x) = \frac{1}{M \times N} \sum_{i = 1}^{M \times N} k_{h} ({| \frac{x - x_{i}}{h} |}^{2}),

(1)

where h is the bandwidth, and h > 0.

Target candidates with the same scale are established one by one around the point which is mapped from the target centre in previous frame. Then the Bhattacharyya coefficient is applied to calculate the similarity between each candidate in current frame and the target in previous frame. The Bhattacharyya coefficient is defined as follows:

d (y) = \sqrt{1 - ρ (y)},

(2)

where

ρ (y) \equiv ρ (p (y), q) = \sum_{b = 1}^{B} \sqrt{p_{b} (y) q_{b}} .

(3)

In (3), q represents the target model, ρ(y) represents the candidate model, and B is the number of bins used to calculate the model.

The kernel function $k_{h} (| (x - x_{i}) / h |^{2})$ is symmetrically centered at point x, so the Mean-Shift tracking algorithm is robust to rotation. The statistical property of the kernel density estimation makes it insensitive to partial occlusion. The kernel bandwidth h plays an important role in the Mean-Shift tracking algorithm. It not only determines the sample weights but also reflects the shape and size of a target. In the Mean-Shift tracking algorithm, the kernel scale is initialized by the first tracking window and fixed through the whole tracking process. Due to its fixed kernel scale, the Mean-Shift tracking algorithm has poor performance when the target scale changes a lot.

Figure 1 shows the performance of the Mean-Shift tracking algorithm when the target scale changes. The red solid square is the target we want to track. The red pie is an object in the background. The blue square frame is the tracking window which is fixed in the Mean-Shift tracking algorithm. The surface charts are the Bhattacharyya coefficient surface between the target and the target candidates. These surface charts are centred within a zone with 45 × 45 pixels around the true target location.

Figure 1:

The performance analysis of the Mean-Shift algorithm with fixed kernel scales.

In the Mean Shift tracking algorithm, the deviation generated in each iteration is actually the gradient vector at a certain point on the Bhattacharyya coefficient surface. The process of locating a target is equivalent to the process of searching the peak along the gradient vector on the Bhattacharyya coefficient surface.

In Figure 1(a), the tracking window is smaller than the target size. The shifts move within a flat zone around the true location. This may lead to locating the target with a great shift because there is no extreme point. In Figure 1(b), the size of the tracking window is the same as the target size. Accordingly there is a single peak near the target location in the Bhattacharyya coefficient surface. This may lead to the accurate target location. As shown in Figure 1(c), a large tracking window will lead the Mean-Shift iteration to converge to an area which is mixed with the target and the background.

Hence, the defaults of fixed kernel scales can be summarized as follows:

When the target becomes smaller, the tracking window contains lots of background information. The background in the tracking window affects the color-histogram distribution seriously. It is prone to cause the tracking centre to shift.

When the target becomes larger, the target will exceed the tracking window. The color-histogram distribution of the tracking window is similar if only the tracking window is within the target area. So the tracking window always drifts within the target area.

3. Proposed Algorithm SAMSHIFT

In this section, we present our scale-adaptive Mean-Shift tracking algorithm called SAMSHIFT. We first provide an overview of SAMSHIFT in Section 3.1 and then introduce its two important functions, Corner Points Matching and Affine Transformation, in Sections 3.2 and 3.3, respectively. Finally, we discuss the robustness to partial occlusion and the time complexity of SAMSHIFT in Section 3.4.

3.1. Overview

Algorithm SAMSHIFT is proposed to improve the scale adaptation and robustness to occlusion of the Mean-Shift tracking algorithm. The targets in adjacent frames in a video have slightly different views for the same object. If we can build a correct model to describe different views including scale changing, then we can more precisely track the target object in videos. In SAMSHIFT, we apply the affine transformation because it is often used to model different views for rotation, transformation and scaling of a rigid object. However, feature point matching is required for affine transformation. We use corner points as interest feature points in SAMSHIFT.

The workflow of SAMSHIFT is illustrated in Figure 2, in which W_i is the Mean-Shift tracking window in the ith frame, C_i is the centre of W_i, and C_i′ is the target centre in the ith frame which is mapped from the target centre in the (i – 1) th frame obtained by the Mean-Shift tracking algorithm. W_i′ is W_i resized with s_x and s_y. W_i′′ is the tracking window centred at C_i′ and resized with s_x and s_y. N_i is the number of matching pairs between W_i and W_{i – 1}.

Figure 2:

The workflow of SAMSHIFT.

As shown in Figure 2, in SAMSHIFT, we first get a tracking window by the Mean-Shift tracking algorithm. Then we detect the corner points in the tracking window to match with the corner points in the previous frame (the detailed procedure is described in Section 3.2). Based on these matched corner point pairs, the affine transformation is calculated to get a new target candidate and the scaling factors (the detailed procedure is shown in Section 3.3). The scaling factors are applied to resize the target candidate obtained by the traditional Mean-Shift tracking algorithm. Finally, three target candidates, obtained by the affine transformation, the Mean-Shift tracking algorithm, and the Mean-Shift tracking algorithm with resizing by the scaling factors, respectively, are compared with the target model to select the final target. The candidate which yields the largest Bhattacharyya coefficient is selected as the final target.

Figure 3 shows an example for the proposed method. The remote control toy car is the target in the video. The first target candidate is obtained by the Mean-Shift tracking algorithm. Then, based on the first candidate, the second target candidate was calculated with the affine transform, and the third candidate is obtained by resizing the first target candidate mapped from a previous frame. The final target is chosen among these three candidates by comparing their Bhattacharyya coefficients.

Figure 3:

An example application of SAMSHIFT.

3.2. Corner Points Matching

To apply the affine transformation, it is necessary to find interest feature point pairs to calculate the affine structure. In practice, the interest points with some unchanged image features are used to find the relationship between images. False feature point pairs matching may cause incorrect affine structures. Then it will obtain incorrect target scales.

Corner points are formed from two or more edges. The edges are usually used to define the boundary between two different objects or parts of an object. So corner points can be used to calculate the affine structure. In addition, corner points can also be used to distinguish targets from the background with similar colour-histogram distribution. It is known that the Mean-Shift tracking algorithm is based on statistical data comparison, that is, colour-histogram comparison. Therefore, it is sensitive to the difference between the target and the background. However, the target centre calculated by the Mean-Shift algorithm often drifts away from the true location if the background has the similar colour-histogram distribution to the target. Corner points matching is more accurate than such statistic data comparison.

Harris Corner Detector [11] is used in SAMSHIFT because it is suitable for real-time applications. First, corner features are detected in W_i. Suppose I_x and I_y is the grayscale gradients of point I(x, y) at horizontal direction and vertical direction, respectively. Harris Corner Detector is defined as the following:

M = [\begin{bmatrix} A & C \\ C & B \end{bmatrix}],

(4)

where

\begin{matrix} A = I_{x}^{2} \otimes ω, \\ B = I_{y}^{2} \otimes ω, \\ C = I_{x} I_{y} \otimes ω . \end{matrix}

(5)

In (5), ⊗ denotes the convolution operation, ω denotes a Gaussian window centered at I(x, y). we used a 5 × 5 Gaussian window. A point is a corner point if the corner measure is larger than a threshold. Corner measure R is defined as thefollowing:

R = \det (M) - k \times {(trace (M))}^{2},

(6)

where k = 0.04.

Subsequently, zero-mean normalized cross correlation is employed to seek the best matching pairs among corner features of the ith frame and the (i – 1) th frame. The similarity between the feature point pairs is defined as follows:

C (x, x^{'}) = \frac{1}{{(2 N + 1)}^{2} σ_{I_{0}} (x) σ_{I_{1}} (x^{'})} \times \sum_{i = - N}^{N} \sum_{j = - N}^{N} [I_{0} (x - i, y - j) - {\overset{—}{I}}_{0}] \times [I_{1} (x^{'} - i, y^{'} - j) - {\overset{—}{I}}_{1}] .

(7)

In (7), ${\overset{—}{I}}_{0}$ and ${\overset{—}{I}}_{1}$ are the average greyscales of the windows with the side of (2N + 1) and centred at x and x′, respectively. σ_{I
₀}(x) and σ_{I
₁}(x′) are the standard deviations of the greyscales in the windows centred at x and x′, respectively. The value range of C(x, x′) is [−1, 1]. It is easy to select a unified threshold for all feature points. A feature point pair with the similarity larger than the threshold is considered as a matched pair.

3.3. Affine Transformation

Once the corner points in adjacent frames are detected, the corner matching is used to calculate the affine structure. In 2-dimensional cases, three matched point pairs are required between successive frames to calculate the affine structure [12]. If there are no less than three corner matches, it is possible to determine the affine structure A_i of these matching pairs. According to the principle of affine structure, the interest points satisfy

(\begin{pmatrix} x_{i} \\ y_{i} \end{pmatrix}) = A_{i} (\begin{pmatrix} x_{i - 1} \\ y_{i - 1} \end{pmatrix}) + b_{i} = (\begin{pmatrix} s_{x} & 0 \\ 0 & s_{y} \end{pmatrix}) (\begin{pmatrix} x_{i - 1} \\ y_{i - 1} \end{pmatrix}) + (\begin{pmatrix} e_{x} \\ e_{y} \end{pmatrix}) .

(8)

In (8), $(\begin{smallmatrix} x_{i} \\ y_{i} \end{smallmatrix})$ and $(\begin{smallmatrix} x_{i - 1} \\ y_{i - 1} \end{smallmatrix})$ reflect positions of the matched points in the ith frame and the (i – 1) th frame, respectively. A_i is a 2 × 2 matrix. s_x and s_y are stretch amplitudes in horizontal and vertical directions, respectively. b_i is a 2 × 1 matrix. e_x and e_y represent the translation in horizontal and vertical directions, respectively.

Since $(\begin{smallmatrix} x_{i} \\ y_{i} \end{smallmatrix})$ and $(\begin{smallmatrix} x_{i - 1} \\ y_{i - 1} \end{smallmatrix})$ are known in the above transforming, we can calculate s_x and s_y. In SAMSHIFT, s_x and s_y will be used as the scaling factors in the horizontal and vertical directions, respectively. The target candidate obtained by the Mean Shift will be resized by these scaling factors.

However, mismatches will obtain incorrect affine structure. Hence, RANSAC (RANdom SAmple Consensus) is employed to eliminate the false matching and calculate the final affine structure.

3.4. Discussion

In this section, we discuss the robustness to partial occlusion and the time complexity analysis of the proposed algorithm, respectively.

3.4.1. Robustness to Partial Occlusion

Corner point matching is more accurate than the statistic feature matching. It can reduce the mismatch caused by similar color histogram between the target and background, which may occur in Mean-Shift tracking.

However, corner feature matching on the occluder may obtain incorrect scaling factors. Then the tracking window will be larger or smaller than the true target scale. When the target is occluded, there may be corner matching pairs on the occluder or the background. It will cause the target candidate to be stretched or compressed, or even lost. As shown in Figure 4(a), the moving car is occluded by the tree. The corner feature matches on the tree. The tracking window is compressed.

Figure 4:

Balance to scale adaptation and occlusion robustness.

However, the statistic feature matching in the traditional Mean-Shift tracking algorithm is robust to partial occlusion. Therefore, in SAMSHIFT, the original target candidate obtained by the Mean Shift is also used as one of the target candidates. Together with the resized target candidates, the one which yields the largest Bhattacharyya coefficient is selected as the final target in current frame. By using this scheme, the robustness for partial occlusion of the traditional Mean Shift tracking algorithm is retained. Figure 4(b) shows the tracking performance of our method for the same video. It is robust to partial occlusion and is adaptive to scale changing as well.

3.4.2. Time Complexity Analysis

There are four main components which contribute to the computational cost of SAMSHIFT: the calculation of the gradient information, the Mean Shift tracking, Harris corner detection, and the computation of the affine structure. Let n_h be the pixel number in the tracking window. The time complexity of the proposed algorithm can be expressed as O(n_h). Time cost of target tracking in each frame is related to the size of the target. The bigger the target is, the more time it will cost.

4. Experiments

In this section, we present and analyse the experimental results. To evaluate the performance of our SAMSHIFT algorithm, we implement it in a PC and a mobile robot. SAMSHIFT is compared with three representative tracking algorithms: the traditional Mean-Shift algorithm, the Mean-Shift algorithm with 10% adaption, and the spatial color Mean-Shift algorithm. Next, we first introduce the experimental environment and the performance metrics and then present and discuss the experimental results.

4.1. Experimental Environments

SAMSHIFT is evaluated in both a PC and a mobile robot. In a PC with Pentium (R) 3.0 GHz, we implement the above four algorithms and use a video in VIRAT Ground Video Dataset to test them. In the test video, the image size is converted to 320*240 pixels.

We also test the four algorithms in a mobile robot. The mobile robot is an automatic target tracking car (ATTC) which is shown in Figure 5(a). STM32F407 ARM+DSP processor is used as the main processor in the controller board as shown in Figure 5(b). The ATTC is equipped with an OV7670 video camera and a compass. The ATTC can be used to track a target in specific environments which are difficult for humans to reach, for example, a very small and narrow space. The tracking algorithms are used to guide the ATTC to track a target. As shown in Figure 6, the target is a remote control toy car.

Figure 5:

An automatic target tracking car (ATTC) based on SAMSHIFT.

Figure 6:

The ATTC is tracking a remote control toy car.

4.2. Performance Metrics

The following three performance metrics are used to evaluate the four algorithms. First, the tracking results by images are recorded to show the visual tracking effects in the environments with scale changing and occlusion. We also compare the performance of the target centre position errors and relative scale errors. The target centre position error is the relative position error between the centre of the tracking result and that of the true target. Relative scale is scale normalized by the true scale. It is defined as the following:

S = \frac{W \times H}{W_{t} \times H_{t}} .

(9)

In (9), W and H are the width and height of the current target scale obtained by a tracking algorithm, respectively. W_t and H_t are the width and height of the true target scale, respectively. In a perfect tracking, the target centre position error is close to 0, and the relative scale is close to 1 [13]. Finally, we also give the time cost of SAMSHIFT.

4.3. Results and Discussion

In this section, we present and discuss the experimental results. We first show the results in the PC environment in Section 4.3.1 and then present the results in the mobile robot in Section 4.3.2. Finally, we discuss the time cost in Section 4.3.3.

4.3.1. Experiments in the PC

As we mentioned, the four algorithms have been implemented in the PC. For each algorithm, it is used to track a black car whose size shrinks over time and is occluded by other cars in the parking area. Figure 7 shows the comparison of the tracking results from the four algorithms.

Figure 7:

Tracking results' comparison of the four algorithms in PC.

Figure 7(a) illustrates the tracking results from the traditional Mean-Shift algorithm with the fixed kernel scale. The tracking window drifts around the target area when the size of the target becomes smaller, and the target centre is inaccurate. When the target car is occluded by the white column, the tracking windows drift down largely. After the target car is occluded by the white car, a similar black car parked opposite to the white car is tracked as the target. Figure 7(b) shows the tracking results from the Mean-Shift algorithm with 10% scale adaptation. It can be observed that the tracking results are almost the same as these of the Mean-Shift tracking. Figure 7(c) shows the tracking results by using the spatial color Mean-Shift algorithm. The performance is not good either. When the target is occluded by the white column, the scale of tracking window is much smaller than the true target scale. After the target car is occluded by the white car, the target is lost by this tracker. The tracking results obtained by our SAMSHIFT are shown in Figure 7(d), in which the target is precisely tracked in terms of the scale and the centre position even if the target is occluded by the white column and the white cars.

We manually label the centre position and scales of the moving target for 250 frames. The moving black car, as shown in Figure 7, is subjected to occlusion when passing behind the white column and the white cars.

Figure 8 shows the centre position errors of the target obtained by the four tracking algorithms. As indicated in quantitative comparison in Figure 8, when the black car is occluded by the white car in frame 170, both the Mean Shift and the Mean-Shift with 10% adaption lose the tracking and never recover. The spatial color Mean Shift has a better object representing than the Mean Shift; it loses the tracking at frame 241 and never recovers. SAMSHIFT performs extremely well when the black car is occluded by the white car. In SAMSHIFT, the target candidate generated by the Mean Shift with resizing by the scaling factors loses efficacy, but the target candidate generated by the affine transformation can track the target effectively.

Figure 8:

Centre position error comparison.

Figure 9 shows the relative scale errors of the target obtained by the four tracking algorithms. As indicated in quantitative comparison in Figure 9, SAMSHIFT adapts to the scale changing very well. In the first 100 frames, the scale of the target is almost fixed, and the target scale obtained by the Mean Shift and the Mean Shift with 10% adaption is close to the true scale before 100 frames. When the target is occluded or the scale of the target is changed, the errors become greater. The target scales obtained by the spatial color Mean Shift are always smaller than the true scale. The target windows become much smaller, and target scales have more errors when the target is subjected to occlusion.

Figure 9:

Relative scale comparison.

4.3.2. Experiments in the Mobile Robot

In the mobile robot mentioned before, we conduct experiments to test the scale-adaptive tracking performance. The target moves far away from a nearby location. The scale changes from the large size to the small size. The tracking results of the four tracking algorithms are shown in Figure 10.

Figure 10:

Tracking results' comparison of the four algorithms in ATTC.

As shown in Figures 10(a) and 10(b), the scales of the tracking windows do not change when we use the traditonal Mean-Shift tracking algorithm and the traditional Mean Shift with 10% adaption. The scales of the tracking windows are incorrect when the spatial color Mean-Shift tracking algorithm is used to track the toy car. Furthermore, the drift becomes larger when the target becomes smaller. Our SAMSHIFT algorithm can adapt to the scale changing very well.

We also manually label the centre position and scales of the moving toy car in Figure 10 for 100 frames. The scale of the moving toy car in Figure 10 changes frequently. Because the video is taken by the ATTC which follows the toy car, the scale of the moving toy car changes from the large size to the small size or from the small size to the large size.

Figure 11 shows the centre position errors of the target toy car obtained by the four tracking algorithms. As indicated in quantitative comparison in Figure 11, SAMSHIFT performs extremely well and has the lowest number of errors. The Mean Shift and the Mean Shift with 10% adaption have more errors. The spatial color Mean Shift has the most errors, which means the tracking windows drift away seriously.

Figure 11:

Centre position error comparison.

Figure 12 shows the relative scale errors of the target obtained by the four tracking algorithms. As indicated in quantitative comparison in Figure 12, SAMSHIFT adapts to the scale changing very well. The target scales obtained by the other three tracking algorithms have more errors after 20 frames, and there are more and more errors as the target moves. Because the true target scale changes suddenly from the large size to the small size, the cumulative errors of the Mean Shift, the Mean Shift with 10% adaption, and the spatial color Mean Shift increase.

Figure 12:

Relative scale comparison.

4.3.3. Time Cost of SAMSHIFT

Figure 13 illustrates the time cost of SAMSHIFT. With the target size becoming larger, the time cost increases. However, the curve shows that the time cost is less than 40 ms when the number of the pixels in the target is less than 2 × 10⁴ pixels. That means that SAMSHIFT can be used for real-time tracking applications if there are 25 frames in a second in the video when the target is less than 2 × 10⁴ pixels.

Figure 13:

Time cost versus target size.

5. Conclusion

We proposed an efficient and effective scale-adaptive tracking algorithm in this paper. In this algorithm, the traditional Mean-Shift tracking algorithm is used to obtain the original target location, and the corner feature matching is used to calculate the affine structure and scaling factor between adjacent frames. In each iteration, three target candidates are generated by the affine transformation, the Mean Shift, and the Mean Shift with resizing by the scaling factors, respectively. Then the best candidate is selected. The algorithm has been evaluated in a PC environment and a mobile robot. The experimental results show that our algorithm can effectively adapt to scale changing and is robust to partial occlusion. Furthermore, the tracking speed of our algorithm is fast enough for real-time tracking applications in robot vision.

References

Comaniciu

Ramesh

, and Meer

, “Kernel-based object tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp. 564–577, 2003.

Ning

Zhang

, and Wu

, “Robust mean-shift tracking with corrected background-weighted histogram,” IET Computer Vision, vol. 6, no. 1, pp. 62–69, 2012.

Mohammadi

S. A.

Amoozegar

Jolfaei

, and Mirghadri

, “Enhanced adaptive bandwidth tracking using mean shift algorithm,” in Proceedings of the IEEE 3rd International Conference on Communication Software and Networks (ICCSN '11), pp. 494–498, May 2011.

Collins

R. T.

, “Mean-shift blob tracking through scale space,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. II-234–II-240, June 2003.

Peng

Yang

Liu

, and Zhang

, “Automatic selection of kernel-bandwidth for mean-shift object tracking,” Ruan Jian Xue Bao, vol. 16, no. 9, pp. 1542–1550, 2005.

Bradski

G. R.

, “Computer vision face tracking for use in a perceptual user interface,” in Proceedings of the 4th IEEE Workshop on Applications of Computer Vision (WACV '98), pp. 214–219, 1998.

Juan

, and Wang

, “A spatial-color mean-shift object tracking algorithm with scale and orientation estimation,” Pattern Recognition Letters, vol. 29, no. 16, pp. 2165–2173, 2008.

McLauchlan

P. F.

and Malik

, “Vision for longitudinal vehicle control,” in Proceedings of the IEEE Conference on Intelligent Transportation Systems (ITSC '97), pp. 918–923, November 1997.

Zhu

and Zhu

, “Algorithm of target tracking based on Mean shift with adaptive bandwidth of kernel function,” Opto-Electronic Engineering, vol. 33, no. 8, pp. 11–16, 2006.

10.

Zhao

Knight

, and Reid

, “Target tracking using mean-shift and affine structure,” in Proceedings of the 19th International Conference on Pattern Recognition (ICPR '08), pp. 1–5, December 2008.

11.

Harris

and Stephens

, “A combined corner and edge detector,” in Porceedings of the 4th Alvey Vision Conference, p. 50, 1988.

12.

Hwang

J. W.

and Lee

H. S.

, “Adaptive image interpolation based on local gradient features,” IEEE Signal Processing Letters, vol. 11, no. 3, pp. 359–362, 2004.

13.

Yang

Yuan

, and Wu

, “Spatial selection for attentional visual tracking,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'07), pp. 1–8, June 2007.