Coupling one-shot learning and online discriminative learning for robust object tracking

Abstract

Visual tracking is a very challenging task in computer vision. In this paper, we present a general-purpose framework for robust tracking. We propose to couple one-shot learning and online discriminative learning together to address the fundamental stability-plasticity issue for tracking. A one-shot learner through offline training on large-scale datasets is used as a stable detector which does not suffer model drift while an online discriminative learner is adopted as the tracker which is adaptive to significant appearance changes. Based on the directive framework, we design a baseline tracking model to verify its effectiveness. In practice, a deep Siamese network trained offline plays as the one-shot learner which can re-detect the target in case of tracking drift and failure. A correlation classifier which incorporates a translation model and a scale model plays as the online learner. Through the coupling of the offline and online learning, the simple baseline tracker achieves a good balance between stability and adaptivity without time-consuming optimization. Experimental results on the large-scale benchmark dataset demonstrate the effectiveness of the proposed framework within which the designed baseline tracker outperforms many state-of-the-art methods both in precision and robustness.

Keywords

Visual tracking one-shot learning online learning deep learning

1 Introduction

Visual object tracking is a challenging research topic in computer vision with a variety of practical applications including human-computer interaction and intelligent surveillance. Although great progress has been achieved in recent years, visual tracking still remains a very challenging task due to drastic appearance changes caused by illumination variation, deformation, heavy occlusion and so on. To address these issues, numerous methods have been proposed but a perfect tracker that can deal with all these challenging factors does not exist [1].

To enhance the robustness of tracking, most of modern state-of-the-art tracking researches can be categorized into two types. The first type is to construct new appearance models which aims at increasing the discrimination of the target against background noises. Many methods fall into this type, such as subspace learning [2], compressive sensing model [3], color feature [4] and sparse coding [5]. Recently, representation learning methods have shown great advantages over hand-crafted features [6]. As for the tracking community, trackers based on the features extracted from auto-encoder network [7] and convolutional neural network [8] have been proposed. Though effective in some aspects, powerful feature representations still often fail in some situations, e.g. heavy occlusion and out-of-view.

Another important line of research is to introduce state-of-the-art machine learning methods into the tracking community. These approaches are often referred as tracking-by-detection or tracking-by-classification. Representative methods include multiple instance learning [9, 10], boosting methods [11], ensemble learning [12], support vector machine [13], random forest [14], correlation classifier [15] and so on. Thanks to the advances of machine learning, these trackers show more robustness than traditional motion estimation methods [16]. Since most of these trackers label samples by themselves during online running, however, they are liable to be fooled by false samples and thus suffer the well-known drift problem.

In this paper, we want to treat the tracking problem from a new perspective. We take the starting point from the definition of tracking: given the bounding box defining the object of interest in the first video frame, the tracking task is to automatically output the bounding box of the target in the subsequent frames. From the definition, we can have two important derivations to treat the essentials of tracking. First, tracking is a one-shot learning problem since only one labeled sample (the target patch in the first frame) is offered. All the samples collected from the next frames are all unreliable samples because all of them are labeled by the tracker itself according to the overlap rate between the samples and the tracked regions. If a tracked region has a deviation from the ground truth, small errors will accumulate. This is a positive feedback process which will enlarge the errors until tracking drift and failure. Thus to alleviate this problem, a stable one-shot learner which just utilizes the only one reliable example in the first video frame is a potential solution to robust tracking. In fact, we surely need some intelligent agent to keep the "stable information" during the tracking procedure. A one-shot learner may be the best candidate to play this role. It should be noted that the one-shot learning itself is also a challenging topic. How can a learner find and recognize the target object just based on only one labeled sample? According to the No Free Lunch Theorem, offline training of the one-shot learner to assimilate more auxiliary information is quite necessary.

The second derivation from the tracking definition is, different from classic one-shot learning problem, the tracking process is to recognize and locate the target object in successive images with temporal and spatial continuity. The one-shot learner may not be able to explore the "dynamic information" lies in the temporal and spatial continuity, thus fails to locate the target when it undergoes drastic appearance changes in certain frames during tracking. As a compensation for the one-shot learner trained offline, an online discriminative learner is needed to exploit more information during online tracking and adapt to significant appearance variations.

Based on the analysis above, the tracking problem can be effectively addressed by delicately coupling two types of intelligent agents. The first agent is based on one-shot learning which is used to explore the stable information of the target in a video sequence. The second agent is based on online learning which is used to explore dynamic information lies in consecutive frames. Note that this strategy is actually fusing the offline learning and online learning together. These can be general-purpose guidelines for designing a robust tracker.

According to the guidelines derived above, we have realized a baseline tracker to check its tracking performance. As for the one-shot learner, we adopt the Siamese neural network which is trained offline on a large number of image pairs from video sequences. The fully connected layer of the Siamese network plays the role of a learned metric function instead of a manually defined one, which shows better robustness. Through the whole tracking process, the labeled example of the target in the first frame is stored in the network, namely, the stable information has been kept and explored. As for the online learner, we adopt an enhanced version of correlation classifier with a translation model and a scale model. The online learner is trained with samples collected during online running to adapt to target appearance changes. When a tracking failure has been detected, the one-shot learner can re-detect the target, clear the error accumulation and reset the online learner. Through the coupling of these two learners, robust tracking can be achieved. Experimental results on large-scale tracking benchmark have demonstrated the competitiveness of the baseline tracker, thus proved the advantage of the proposed directive framework and guidelines for tracker design.

In summary, this paper makes the following contributions:

A novel general-purpose framework to tackle the visual object tracking problem is proposed with useful guidelines derived from the fundamental definition.

A baseline tracker is designed and tested on challenging benchmark video sequences with high precision and robustness.

2 Related work

Visual object tracking is one of the most challenging problems in computer vision. The key issue is how to make a tracker robust to various noises in a video sequence and less sensitive to the tracking drift problem. To achieve robust tracking, some methods with certain cooperative mechanisms have been proposed.

In [17], the authors propose to combine the template of target in the frame and an updated template to reduce drift. The TLD model [14] adopts a revised version of Lucas-Kanade for motion estimation, and a random fern detector trained online as a compensation. This strategy is also adopted in [18, 19]. In [20], the authors propose to fuse online classifiers in different time spans to keep more historical information, which works well for alleviating drifting. However, all these methods just rely on information from the online running and cannot tackle the effect of false samples which may contaminate the model itself.

To exploit extra information, some methods adopt pre-trained models for tracking. These methods can be roughly categorized into two types. The first type is to adopt networks without fine-tuning online. In [21], the authors use convolutional neural network (CNN) to learn a saliency map to locate the target. In [8, 22], hierarchical convolutional features are adopted to equip correlation filters to construct a robust ensemble framework. In [23, 24], metric learning via CNN is explored for tracking. All the networks in the above work are trained offline and used as either feature extractor or matching function without online optimization. These methods do not suffer time-consuming optimization for online tracking but may lose some adaptivity to some extent. The other type is fine-tuning the network online to increase its adaptivity. In [7], the authors train an auto-encoder network offline on a large scale of images and use the samples collected online to fine-tune the network to adapt to appearance changes. In [25], a light-weight CNN is proposed which is totally trained during online running. These methods are time-consuming and liable to be affected by false samples, which show inferior tracking performance.

Our method is significantly different from the work present above in that we focus on exploring both of online and offline information and coupling them in a compact manner. Our model incorporates general feature representations learned from large-scale image datasets as well as temporal consistencies between video frames. We name our tracker as OOT (abbreviation for Offline-Online Tracker or One-shot-Online Tracker).

3 Approach

3.1 General framework

We observe that, as for robust tracking, the information that needs to be explored should be decoupled into two parts: the stable information and the dynamic information. The stable information is just extracted from the only one labeled sample in the first frame. A learner should utilize the stable information and never be updated online to keep a fixed false positive detection rate. The stable learner will never suffer error accumulation and model drift, but it may give a false positive estimation. The dynamic information lies in the temporal and spatial continuity between consecutive frames in a sequence. A learner should utilize the dynamic information and be updated online to adapt for the appearance changes that have never been seen before by the tracker. The dynamic learner is adaptive to brand new candidates, but it may suffer the drift problem which leads to tracking failure.

By decoupling the information into two types which are explored by two independent learners, a robust tracker can be constructed by delicately coupling these two learners together. The illustration of the proposed framework is shown in Fig. 1. In practice, here are the guidelines for realizing a tracker based on the proposed framework: (1) it is a good choice to adopt one-shot learning as the stable learner which can incorporate both information from the labeled sample of the target and extra training images; (2) design a robust online learning algorithm as the dynamic learner which is updated properly to adopt to appearance changes.

Fig.1

Illustration of the proposed framework for robust object tracking. Two independent learners explore stable and dynamic information respectively to construct a robust tracking system.

3.2 Baseline model

We develop a baseline model for tracking based on the proposed framework to check its effectiveness. A correlation classifier with a translation model and a scale model is adopted as the online learner. During tracking, the classifier is trained online to adapt for appearance changes of the target. Since correlation classifiers can realize dense sampling with a very efficient manner and alleviate the label ambiguity, the online learner is quite robust. To further reduce the risk of tracking drift, we design a conservative update strategy to train the online learner.

As for the one-shot learner, we adopt the Siamese network [26] which has two branches. The network is trained offline with image pairs on a large-scale dataset. The fully connected layer is learned as a metric function which outputs the similarity between two images. The network has frozen weights all through the tracking procedure and is able to re-detect the target when the online learner drifts.

3.3 One-shot learner

We exploit the Siamese CNN architecture as the one-shot learner. The network has two branches: the exemplar branch and the instance branch. Each branch takes a AlexNet-like structure [27] and can extract hierarchical features from image patches. The exemplar branch takes images with the size of 127 × 127, while the instance branch takes incoming images with the size of 255 × 255.

We adopts a similar curated dataset and training strategy as [24] to train our network. The dataset is from the ImageNet Video dataset [28] which includes 800,000 objects from 2820 videos. The image pairs fed to the two-stream network are from two frames of a video with a distance of N frames. Both of the exemplar image and the corresponding search image are centered at the target and normalized to the input scale of the network. Pairs with the highest similarity are given the ground-truth label " +1 " while lowest similarity has " -1 ".

After the labeled target patch is given in the first frame, it is fed to the exemplar branch and gets its embedding φ (z). Note that the computation is taken only once and the embedding φ (z) is kept as the stable information all through the tracking process. When a new video frame arrives, a patch x centered the tracking result of the last frame is cropped and fed into the instance branch. The one-shot learner finally outputs a score map which indicates the similarity between φ (z) and φ (x) as: $S_{stable} = φ (z) * φ (x) + b$ (1) where b is a bias item. The position of peak value in S_stable is determined as the estimated location of target in the new frame, which can be calculated as: $p^{*} = \underset{p}{argmax} (S_{stable}) .$ (2)

The one-shot learner keeps frozen weights during tracking which avoids error accumulation and model drift in nature. However, tracking failure may still occur due to similar objects in the background. It has been observed that the one-shot learner may switch its bounding box wrongly to an object with similar appearance of the target, especially when the true target undergoes drastic appearance variations. This can be explained by that the one-shot learner does not explore enough temporal information which is just the key strength of the online learner.

3.4 Online learner

The online learner is used for modeling the temporal and spatial relationship of the target appearance in consecutive frames. We use the correlation classifier (also referred as correlation filter) as the online learner. Two important issues need to be treated carefully: feature representation of the samples and the online training strategy. To enhance the discrimination, we use the convolution layers of the Siamese network in the one-shot learner as the feature extractor. The training follows an incremental way on reliable frames.

Let x^k denote the feature map extracted from the k-th convolution layer and y be the 2D gaussian shape label matrix with zero mean and standard deviation proportional to the target size. Let X^k = scripfontF (x^k) , Y = scripfontF (y), where scripfontF (·) denotes the fast Fourier transformation (FFT). Then the correlation classifier W is trained by the minimization problem as: $min_{W} \sum_{i} ∥ Y - W \cdot X^{k} ∥^{2} + λ ∥ W ∥^{2}$ (3) where λ ≥ 0 is a regulation parameter to prevent over-fit. This problem equals to learning a multi-channel correlation filter with a closed form solution in the Fourier domain as: $W_{*, *, c}^{k} = \frac{Y}{X^{k} \cdot X^{k} + λ} ⊙ {\bar{X}}_{*, *, c}^{k}$ (4)

During online running, the correlation classifier is trained in an incremental way using new samples X in a new frame with a learning rate η as:

$T_{t}^{k} = \frac{Y}{X_{t}^{k} ⊙ X_{t}^{k} + λ} ⊙ {\bar{X}}_{t}^{k}$ (5) $W_{t}^{k} = (1 - η) W_{t - 1}^{k} + η T_{t}^{k}$ (6)

The online learner is applied to candidate regions in the new frame. Let z^k denote the output of the k-th layer of the Siamese network, then the response map is computed as: $S^{k} = F^{- 1} (Z^{k} \cdot W)$ (7) where F^-1 is the inverse Fourier transform. The position of the target in this new frame is then estimated by searching the position of maximal value in S^k.

To reduce the risk of drift, the weight W is trained online with a robust way. In practice, the maximal value of S^k is taken as the confidence score. If the confidence is higher than a threshold, training is performed. Otherwise, a low confidence indicates that the online learner may suffer significant noises, then training is forbidden to prevent the learner from contamination. Mathematically, the online learning rate in (6) is calculated as: $η = {\begin{matrix} η^{0}, & if max (S^{k}) > ξ; \\ 0, & else \end{matrix}$ (8) where ξ is a threshold set in advance.

After the translation location has been determined, we construct a target patch pyramid centered at that location for scale estimation. Let M × N be the target size and S = {s₁, s₂,. . . , s_n} denote the set of scale factors. For s_i ∈ S, we extract the features of a patch with the size sM × sN. Let y_{s
_i} denote the correlation response map for the target with size factor s_i, then the optimal scale s^* is estimated as: $s^{*} = \underset{s}{argmax} (max (y_{s_{1}}), . . ., max (y_{s_{n}})) .$ (9)

3.5 Tracking inference

After the ground-truth patch of the target in the first frame is given, it is fed to the one-shot learner and its embedding is kept as the stable information through the entire tracking process. Meanwhile the online learner begins its running with the labeled sample as its initial status.

The one-shot learner never suffers the drift problem since it does not utilize information from online tracking at all. Having been trained on a large number of images, the one-shot learner is still robust to certain distortions of the target based on the only one labeled sample. However, the one-shot leaner may fail to track the target and switch to very similar objects in the background. This is because it lacks the capability of temporal exploration. The online learner is good at modeling temporal information and is adaptive to drastic appearance variations. However, the online learner is inevitable to suffer the drift problem. Their characteristics are shown in Fig. 2.

Fig.2

Robust tracking via the coupling of the one-shot learner and the online leaner. The one-shot learner is stable but may switch to similar objects during tracking. The online learner is adaptive but may suffer tracking drift. By the complementarity of these two agents, the whole tracking model can be both robust and adaptive, thus achieves high precision. Red: success, Yellow: failure.

Based on the pros and cons of these two learners, robust tracking is performed through the coupling and complementarity of them as shown in Fig. 2. The online learner plays as the main tracker since it can explore temporal information between consecutive frames and gives its estimation according to (7). To deal with the drift issue, the one-shot learner plays as a re-detection partner which can re-detect the target in case of tracking failure.

In normal scenarios, the online learner is quite robust with high confidence scores. When the target suffers significant noises from background, the confidence score drops down. If the score is lower than a threshold, the one-shot learner is activated to re-detect the target and reset the tracker.

We take the video sequence named jogging-1 as an example to depict the tracking inference process in detail. As shown in Fig. 3, in normal conditions, the tracker based on the online learner is capable of giving precious estimation with high confidence scores. With the update strategy, the online learner can be quite adaptive to the target appearance changes. From the 66th video frame, however, the target being tracked begins to undergo very heavy occlusion and finally disappears for a few frames. This out-of-view situation poses a very tough challenge to the tracker. Without the stable information which is kept by the one-shot learner, the adaptive learner gradually drifts away from the object with low confidence scores, thus switches to the occlusion itself and tracking fails.

Fig.3

Tracking confidence scores with and without the coupling of the one-shot learner which can play a role of re-detection in case of tracking drift or failure. If without the one-shot learner, the confidence score decreases to low values when suffering heavy occlusion and out-of-view. By exploring the stable information via the one-shot learner, the target can be re-captured and the confidence score increases to a relatively high level and the tracker is more clear about its estimation.

With the coupling of the one-shot learner, the target can be re-detected when it appears again after the heavy occlusion. The confidence score of the tracker increases to a relatively high level, which indicates that the tracker is less affected by noises from background and it is confident about the tracking result again.

4 Experiment

4.1 Setup

For tracking performance evaluation, we carry out experiment on the well-known Online Visual Tracking benchmark [29], which includes 50 challenging video sequences. All of the sequences involve representative challenging factors for visual tracking, such as illumination variation, occlusion, background clutter and fast motion.

For comparison we run several state-of-the-art tracking algorithms with the same initial position of the target. These algorithms include MEEM [20], KCF [15], DSST [30], TLD [14], Struck [13] and ASLA [5]. The KCF tracker and Struck tracker are based on online correlation filter and support vector machine (SVM) respectively without complementary self-correction mechanism. The MEEM tracker and TLD tracker adopt ensemble strategy to enhance the overall robustness. The DSST and ASLA tracker can handle affine transformation well. For fair comparisons, publicly available source or binary codes of these trackers on the benchmark are adopted.

As for the online learner, we set the regularization parameter λ in (8) to 10^-4. The learning rate η in (8) is set to 0.01. The update threshold ξ is set to 0.35. As for the one-shot learner, the threshold value to activate it for re-detection is set to 0.15. The algorithm is implemented in MATLAB on an Intel i7 PC with 16G memory. Note that the parameters of all trackers for comparison are kept the same throughout the experiments.

4.2 Evaluation metric

For quantitative evaluation of tracking performance, there are two widely-used metrics: center location error (CLE) and overlap rate (OR). The center location error is defined as the Euclidean distance (in pixels) between the center of the tracking result and the ground truth for each frame. The overlap score is defined as $OR = \frac{area (B_{T} \cap B_{G})}{area (B_{T} \cup B_{G})}$ , where B_T denotes the bounding box of the tracking result for each frame and B_G is the corresponding ground truth. Note that a smaller average center location error or a larger overlap rate indicates a better result.

Based on these two metrics presented above, we follow the evaluation criteria in the benchmark [29] where success plot and precision plot are adopted for evaluating each tracker. The success plot indicates the ratios of successful frames over one entire video. A tracked frame is regarded as having been successfully tracked if its overlap rate is larger than a given threshold. By varying the threshold gradually from 0 to 1, it generates a plot of the success rate against the overlap threshold for each tracker. The precision plot shows the percentage of frames whose center location error is within a given threshold. Different tracking algorithms are ranked based on the Area Under Curve (AUC) for the success plot and center location error at a specific threshold (20 pixels) for the precision plot.

4.3 Overall performance

The overall tracking performance in terms of precision plot and success plot of the trackers are shown in Fig. 4. The trackers are ranked according to their performance scores in the legend of each plot.

Fig.4

Overall performance of the proposed tracker and several state-of-the-art trackers on the Online Visual Tracking Benchmark. The trackers are ranked by their performance scores shown in the legend.

It can be seen from Fig. 4 that the proposed OOT tracker outperforms the other state-of-the-art tracking methods both in the success plot and the precision plot. The good average performance of our tracker on the benchmark has demonstrated the effectiveness of the proposed tracking framework. The coupling strategy has made a good balance between stability and adaptivity, which has enhanced the overall tracking robustness. Since the OOT tracker is just a simple baseline implementation without further optimization, better precision can be achieved with more advanced machine learning techniques for one-shot learning and online learning or more powerful feature representations.

4.4 Attribute-based performance

To further evaluate the performance on different conditions, we also compute the success plots and precision plots of each tracker on different challenging factors. In the tracking benchmark [29], all the video sequences are annotated with 11 different attributes which are typical challenging factors for visual tracking. We report the results on four representative attributes in terms of success plot and precision plot as shown in Fig. 5. Our OOT tracker ranks first in all of these attributes. The results show that the OOT tracker is quite adaptive to significant appearance changes (rotations) and robust to background noises (occlusion, clutter). This is attributed to the coupling of the two independent learners with complementary roles.

Fig.5

Tracking performance of seven state-of-the-art trackers on four representative attributes of the benchmark videos. (a) Tracking performance scores based on success plot. (b) Tracking performance scores based on precision plot.

4.5 Qualitative evaluation

For qualitative analysis, we also plot the tracking results in terms of bounding boxes in some key frames from several challenging video sequences as shown in Fig. 6. The state-of-the-art trackers include: KCF, MEEM, Struck, TLD. From Fig. 6, it can be observed that our OOT tracker performs well in motion blur (couple), background-clutter (football), occlusion (lemming), out-of-view (jogging-2) and illumination variation (shanking and singer2). The KCF tracker cannot handle out-of-view issues since no self-correction component is equipped. The TLD tracker can re-detect objects but often mistakes similar objects for the target itself. In addition, it does not adopt online learning for the main tracker. The MEEM tracker is a quite good long-term tracker which can restore a drifted tracker to its uncontaminated status by storing its historical snapshots. However, it is still not able to explore the stable information adequately and merely rely on online learning. Overall, our tracker shows better robustness.

Fig.6

Tracking results of OOT tracker and several state-of-the-art trackers on some challenging video sequences. These video sequences include: couple, football, jogging-2, lemming, shaking and singer2.

5 Conclusions

In this paper, we propose a general-purpose framework for designing a robust tracker. We first have a deep insight into the nature of tracking and then derive some guidelines for tracker design. We propose to couple one-shot learning and online learning together to tackle the tracking difficulties. Based on the guidelines, we develop a baseline model to construct a robust tracker. Our analysis and the experimental results have shown that combining offline and online leaners together to explore both of stable information and dynamic information is of great importance for robust tracking. The proposed framework welcomes more powerful and efficient one-shot learning methods and online learning algorithms to be integrated into.

Footnotes

Acknowledgements

This work was supported by The Fundamental Research Funds for the Central Universities (Grant No. 500418754).

References

, Lim

and Yang

M.-H.

, Object tracking benchmark, IEEE Trans Patt Anal Mach Intell 37(9) (2015), 1834–1848.

Ross

D.A.

, Lim

, Lin

R.-S.

and Yang

M.-H.

, Incremental learning for robust visual tracking, International Journal of Computer Vision 77(1–3) (2008), 125–141.

Zhang

, Zhang

and Yang

M.-H.

, Real-Time Compressive Tracking, in: Proc Eur Conf Comput Vis, 2012, pp. 864–877.

Danelljan

, Khan

F.S.

, Felsberg

and van de Weijer

, Adaptive Color Attributes for Real-Time Visual Tracking, in: Proc IEEE Conf Comput Vis Pattern Recognit, 2014, pp. 1090–1097.

, Lu

and Yang

M.-H.

, Visual Tracking Via Adaptive Structural Local Sparse Appearance Model, in: Proc IEEE Conf Comput Vis Pattern Recognit, 2012, pp. 1822–1829.

Bengio

, Courville

and Vincent

, Representation learning: A review and new perspectives, IEEE Trans Patt Anal Mach Intell 35(8) (2013), 1798–1828.

Wang

and Yeung

D.Y.

, Learning a Deep Compact Image Representation for Visual Tracking, in: Adv in Neural Inf Process Syst, 2013, pp. 809–817.

, Huang

J.B.

, Yang

and Yang

M.H.

, Hierarchical Convolutional Features for Visual Tracking, in: IEEE Int Conf Comput Vis, 2015, pp. 3074–3082.

Babenko

, Yang

M.H.

and Belongie

, Visual Tracking with Online Multiple Instance Learning, in: Proc IEEE Conf Comput Vis Pattern Recognit, 2009, pp. 983–990.

10.

Wang

, Yoon

, Xie

S.J.

, Lu

and Park

D.S.

, Visual Tracking with semi-supervised online weighted multiple instance learning, The Visual Computer 32(3) (2016), 307–320.

11.

Grabner

L.C.

and Bischof

H.H.

, Semi-Supervised on-Line Boosting for Robust Tracking, in: Proc Eur Conf Comput Vis, 2008, pp. 234–247.

12.

Avidan

, Ensemble tracking, IEEE Trans Patt Anal Mach Intell 29(2) (2007), 261–271.

13.

Hare

, Saffari

and Torr

P.H.S.

, Structured Output Tracking with Kernels, in: Proc IEEE Int Conf Comput Vis, 2011, pp. 2096–2109.

14.

Kalal

, Mikolajczyk

and Matas

, Tracking-learning-detection, IEEE Trans Patt Anal Mach Intell 34(7) (2012), 1409–1422.

15.

Henriques

J.F.

, Caseiro

, Martins

and Batista

, Highspeed tracking with kernelized correlation filters, IEEE Trans Patt Anal Mach Intell 37(3) (2015), 583–596.

16.

Baker

and Matthews

, Lucas-kanade 20 years on: A unifying framework, Int J Comput Vis 56(3) (2004), 221–255.

17.

Matthews

, Ishikawa

and Baker

, The template update problem, IEEE Trans Pattern Anal Mach Intell 26(6) (2004), 810–815.

18.

Supancic

J.S.

and Ramanan

, Self-Paced Learning for Longterm Tracking, in: Proc IEEE Conf Comput Vis Pattern Recognit, 2013, pp. 2379–2386.

19.

, Yang

, Zhang

and Yang

M.H.

, Long-Term Correlation Tracking, in: Proc IEEE Conf Comput Vis Pattern Recognit, 2015, pp. 5388–5396.

20.

Zhang

, Ma

and Sclaroff

, Robust Tracking Via Multiple Experts Using Entropy Minimization, in: Proc Eur Conf Comput Vis, 2014, pp. 188–203.

21.

Hong

, You

, Kwak

and Han

, Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network, in: Proc of 32rd Int Conf Mach Learn, 2015, pp. 597–606.

22.

, Zhang

, Qin

, Yao

, Huang

, Lim

and Yang

M.H.

, Hedged Deep Tracking, in: IEEE Conf Comput Vis Pattern Recognit, 2016, pp. 4303–4311.

23.

Tao

, Gavves

and Smeulders

A.W.

, Siamese Instance Search for Tracking, in: IEEE Conf Comput Vis Pattern Recognit, 2016, pp. 1420–1429.

24.

Bertinetto

, Valmadre

, Henriques

J.F.

, Vedaldi

and Torr

P.H.

, Fully-Convolutional Siamese Networks for Object Tracking, in: Proc Eur Conf Comput Vis, 2016, pp. 850–865.

25.

, Li

and Porikli

, DeepTrack: Learning Discriminative Feature Representations by Convolutional Neural Networks for Visual Tracking, in: Proc of British Machine Vision Conference, 2014.

26.

Bromley

, Guyon

, LeCun

, SÃd’ckinger

and Shah

, Signature Verification Using a “Siamese” Time Delay Neural Network, in: Adv in Neural Inf Process Syst, 1994, pp. 737–744.

27.

Krizhevsky

, Sutskever

and Hinton

, ImageNet Classification with Deep Convolutional Neural Networks, in: Adv In Neural Inf Process Syst, 2012, pp. 1097–1105.

28.

Russakovsky

, Deng

, Su

, Krause

, Satheesh

, Ma

, Huang

, Karpathy

, Khosla

, Bernstein

, Berg

A.C.

and Fei-Fei

, Imagenet large scale visual recognition challenge, Int J Comput Vis 115(3) (2015), 211–252.

29.

, Lim

and Yang

M.-H.

, Online Object Tracking: A Benchmark, in: Proc IEEE Conf Comput Vis Pattern Recognit, 2013, pp. 2411–2418.

30.

Danelljan

, HÃd’ger

, Khan

and Felsberg

, Accurate Scale Estimation for Robust Visual Tracking, in: Proc of British Mach Vis Conf, 2014, pp. 1–5.