Driving in nighttime or poor illumination is much more dangerous than in daytime or rich illumination conditions so it is important to develop driver assistance systems to support safe driving in nighttime. This paper presents a pedestrian tracking algorithm using a thermal infrared camera. The proposed algorithm first computes response maps with normalized grayscale and Felzenszwalb's histogram of oriented gradient in the framework of correlation filtering. Subsequently, according to response maps, this paper proposes an approach to estimate weight ratios in fusion. The fused response map is ultimately used to complete target detection in the framework of correlation filtering. In order to test the performance of the proposed tracker, extensive comparison and tests have been performed using open-source data sets. By testing through 10 different challenges in image processing and tracking, it is evident that the performance of proposed algorithm is superior to seven representative tracking algorithms with a quite promising real-time performance.
BenglerKDietmayerKFarberB, et al.Three decades of driver assistance systems: review and future perspectives. IEEE Intell Transp Syst Mag2014; 6: 6–22.
2.
RanftBStillerC. The role of machine vision for intelligent vehicles. IEEE Trans Intell Veh2016; 1: 8–19.
3.
ZakloutaFStanciulescu B. Real-time traffic-sign recognition using tree classifiers. IEEE Trans Intell Transp Syst2012; 13: 1507–1514.
4.
Ishida S and Gayko JE. Development, evaluation and introduction of a lane keeping assistance system. In: Intelligent vehicles symposium, Parma, Italy, 14–17 June 2004, pp.943–944. Piscataway: IEEE.
5.
Gandhi T and Trivedi MM. Pedestrian collision avoidance systems: a survey of computer vision based recent studies. In: Intelligent transportation systems conference, Toronto, Canada, 17–20 September 2006, pp.976–981. Piscataway: IEEE.
6.
BertozziMBroggiAFascioliA, et al.Pedestrian detection for driver assistance using multiresolution infrared vision. IEEE Trans Veh Technol2004; 53: 1666–1678.
7.
GeronimoDLopezAMSappaAD, et al.Survey of pedestrian detection for advanced driver assistance systems. IEEE Trans Pattern Anal Mach Intell2010; 32: 1239–1258.
8.
ZhaoXHeZZhangS, et al.Robust pedestrian detection in thermal infrared imagery using a shape distribution histogram feature and modified sparse representation classification. Pattern Recognit2015; 48: 1947–1960.
9.
DavisJWSharmaV. Background-subtraction in thermal imagery using contour saliency. Int J Comput Vis2006; 71: 161–181.
10.
GadeRMoeslundTB. Thermal cameras and applications: a survey. Mach Vis Appl2014; 25: 245–262.
11.
Ko BC, Kwak JY and Nam JY. Online learning based multiple pedestrians tracking in thermal imagery for safe driving at night. In: Intelligent vehicles symposium, Gothenburg, Sweden, 19–22 June 2016, pp.78–79. Piscataway: IEEE.
12.
Liu Q, He Z, Li X, et al. PTB-TIR: A Thermal Infrared Pedestrian Tracking Benchmark, IEEE Transactions on Multimedia. Epub ahead of print 14 August 2019. DOI: 10.1109/TMM.2019.2932615.
13.
YangHShaoLZhengF, et al.Recent advances and trends in visual tracking: a review. Neurocomputing2011; 74: 3823–3831.
14.
DingMYaoYHWeiL, et al.Visual tracking using locality-constrained linear coding and saliency map for visible light and infrared image sequences. Signal Process Image Commun2018; 68: 13–25.
15.
BabenkoBYangMHBelongieS. Robust object tracking with online multiple instance learning. IEEE Trans Pattern Anal Mach Intell2011; 33: 1619–1632.
16.
Grabner H and Bischof H. On-line boosting and vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, New York, USA, 17–22 June 2006, pp.260–267. Piscataway: IEEE.
17.
KalalZMikolajczykKMatasJ. Tracking-learning-detection. IEEE Trans Pattern Anal Mach Intell2012; 34: 409–1422.
ZhouHYuanYZhangY, et al.Non-rigid object tracking in complex scenes. Pattern Recognit Lett2009; 30: 98–102.
20.
NingJZhangLZhangD, et al.Robust object tracking using joint color-texture histogram. Int J Pattern Recognit Artif Intell2009; 23: 1245–1263.
21.
Sun S, Guo Q, Dong F, et al, On-line boosting based real-time tracking with efficient hog, In: Proceedings of the IEEE international conference on acoustics, speech and signal processing, Vancouver, BC, Canada. 26–31 May 2013, pp.2297–2301. Piscataway: IEEE.
22.
Dalal N and Triggs B. Histograms of oriented gradients for human detection. In: 2005 IEEE computer society conference on computer vision and pattern recognition, San Diego, CA, USA, 20–25 June 2005, pp.886–893. Piscataway: IEEE.
23.
LiPXWangDWangLJ, et al.Deep visual tracking: review and experimental comparison. Pattern Recognit2018; 76: 323–338.
24.
HenriquesJCaseiroRMartinsP, et al.High-speed tracking with kernelized correlation filters. IEEE Trans Pattern Anal Mach Intell2015; 37: 583–596.
25.
LiuFGongCHuangX, et al.Robust visual tracking revisited: from correlation filter to template matching. IEEE Trans Image Process2018; 27: 2777–2790.
26.
ZhaoZFengPGuoJ, et al.A hybrid tracking framework based on kernel correlation filtering and particle filtering. Neurocomputing2081; 297: 40–49.
27.
Bolme DS, Beveridge JR, Draper BA, et al. Visual object tracking using adaptive correlation filters. In: IEEE computer society conference on computer vision and pattern recognition, San Francisco, USA, 13–18 June 2010, pp.2544–2550. Piscataway: IEEE.
28.
Henriques J, Caseiro R, Martins P, et al. Exploiting the circulant structure of tracking-by-detection with kernels. In: 12th European conference on computer vision and pattern recognition, Florence, Italy, 7–13 October 2012, pp.702–715. Berlin: Springer.
29.
Danelljan M, Hager G, Khan SF, et al. Learning spatially regularized correlation filters for visual tracking. In: IEEE international conference on computer vision and pattern recognition, Santiago, Chile, 7–13 December 2015, pp.4310–4318. Piscataway: IEEE.
30.
Danelljan M, Hager G, Khan SF, et al. Convolutional features for correlation filter based visual tracking. In: IEEE international conference on computer vision and pattern recognition, Santiago, Chile, 7–13 December 2015, pp.621–629. Piscataway: IEEE.
31.
Danelljan M, Robinson A, Khan FS, et al. Beyond correlation filters: learning continuous convolution operators for visual tracking. In: European conference on computer vision, Amsterdam, the Netherlands, 11–14 October 2016, pp.472–488. Cham: Springer.
32.
Danelljan M, Bhat G, Khan FS, et al. ECO: efficient convolution operators for tracking. In: Proceedings of the 2017 IEEE conference on computer vision and pattern recognition, Honolulu, USA, 21–26 July 2017, pp.6931–6939. Piscataway: IEEE.
KoBCKwakJYNamJY. Human tracking in thermal images using adaptive particle filters with online random forest learning. Opt Eng2013; 52: 113105.
35.
DaiCZhengYLiX. Pedestrian detection and tracking in infrared imagery using shape and appearance. Comput Vis Image Underst2007; 106: 288–299.
36.
FelzenszwalbPFGirshickRBMcAllesterD, et al.Object detection with discriminatively trained part-based models. IEEE Trans Pattern Anal Mach Intell2009; 32: 1627–1645.
37.
Danelljan M, Hager G, Khan SF, et al. Accurate scale estimation for robust visual tracking. In: The British machine vision conference, Nottingham, UK, 1–5 September 2014, pp.65.1–65.11. Birmingham: BMVA Press.
38.
Wu Y, Lim J and Yang MH. Online object tracking: a benchmark. In: IEEE conference on computer vision and pattern recognition, Portland, USA, 23–28 June 2013, pp.2411–2418. Piscataway: IEEE.
39.
ZhangKZhangLYangMH. Fast compressive tracking. IEEE Trans Pattern Anal Mach Intell2014; 36: 2002–2015.
40.
Bao C, Wu Y, Ling H, et al. Real time robust L1 tracker using accelerated proximal gradient approach. In: IEEE conference on computer vision and pattern recognition, Providence, USA, 16–21 June 2012, pp.1830–1837. Piscataway: IEEE.
41.
Qi Y, Zhang S, Qin L, et al. Hedged deep tracking. In: 2016 IEEE conference on computer vision and pattern recognition, Las Vegas, USA, 27–30 June 2016, pp.4303–4311. Piscataway: IEEE.
42.
LiuQLuXHeZ, et al.Deep convolutional neural networks for thermal infrared object tracking. Knowledge Based Syst2017; 134: 189–198.
43.
Li Y and Zhu J. A scale adaptive kernel correlation filter tracker with feature integration. In: 14th European conference on computer vision, Zurich, Switzerland, 6–7 and 12 September 2014, pp.254–265. Cham: Springer.
