In this paper we describe a fully integrated system for detecting, localizing, and tracking pedestrians from a moving vehicle. The system can reliably detect upright pedestrians to a range of 40 m in lightly cluttered urban environments. The system uses range data from stereo vision to segment the scene into regions of interest, from which shape features are extracted and used to classify pedestrians. The regions are tracked using shape and appearance features. Tracking is used to temporally filter classifications to improve performance and to estimate the velocity of pedestrians for use in path planning. The end-to-end system runs at 5 Hz on 1,024 × 768 imagery using a standard 2.4 GHz Intel Core 2 Quad processor, and has been integrated and tested on multiple ground vehicles and environments. We show performance on a diverse set of datasets with groundtruth in outdoor environments with varying degrees of pedestrian density and clutter. In highly cluttered urban environments, the detection rates are on a par with state-of-the-art but significantly slower systems.
Abd-Almageed, W., Hussein, M., Abdelkader, M. and Davis, L. ( 2007). Real-time human detection and tracking from mobile vehicles . IEEE Intelligent Transportation Systems Conference .
2.
Arndt, R., Schweiger, R., Ritter, W., Paulus, D. and Lohlein, O. ( 2007). Detection and tracking of multiple pedestrians in automotive applications. IEEE Intelligent Vehicles Symposium .
3.
Bajracharya, M., Moghaddam, B., Howard, A. and Matthies, L. ( 2008). Detecting personnel around UGVs using stereo vision. Unmanned Systems Technology X (Proceedings of SPIE, Vol. 6962) . Bellingham WA , SPIE.
4.
Bertozzi, M., Broggi, A., Rose, M.D., Felisa, M., Rakotomamonjy, A. and Suard, F. ( 2007). A pedestrian detector using histograms of oriented gradients and a support vector machine classifier. IEEE Intelligent Transportation Systems Conference.
5.
Beymer, D. and Konolige, K. ( 1999). Real-time tracking of multiple people using continuous detection. International Conference on Computer Vision (ICCV).
6.
Bodt, B.A. ( 2008). Detecting and tracking moving humans from a moving vehicle. Unmanned Systems Technology X (Proceedings of SPIE, Vol. 6962). Bellingham WA , SPIE.
7.
Dalal, N. and Triggs, B. ( 2005). Histograms of oriented gradients for human detection . IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
8.
Ess, A., Leibe, B. and Gool, L.V. ( 2007). Depth and appearance for mobile scene analysis. International Conference on Computer Vision (ICCV).
9.
Ess, A., Leibe, B., Schindler, K. and van Gool, L. ( 2008). A mobile vision system for robust multi-person tracking . IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
10.
Eveland, C., Konolige, K. and Bolles, R.C. ( 1998). Background modeling for segmentation of video-rate stereo sequences. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 601-608.
11.
Fuerstenberg, K.C., Dietmayer, K., and Willhoeft, V. ( 2002). Pedestrian recognition in urban traffic using a vehicle based multilayer laserscanner. In IEEE Intelligent Vehicles Symposium.
12.
Gavrila, D.M. and Munder, S. ( 2007). Multi-cue pedestrian detection and tracking from a moving vehicle. International Journal of Computer Vision, 73(1): 41-59.
13.
Goldberg, S., Maimone, M. and Matthies, L. ( 2002). Stereo vision and rover navigation software for planetary exploration. IEEE Aerospace Conference.
14.
Howard, A. ( 2008). Real-time stereo visual odometry for autonomous ground vehicles. IEEE/RSJ Conference on Intelligent Robots and Systems (IROS).
15.
Howard, A., Matthies, L., Huertas, A., Bajracharya, M. and Rankin, A. ( 2007). Detecting pedestrians with stereo vision: safe operation of autonomous ground vehicles in dynamic environments. International Symposium ofRobotics Research.
16.
Liebe, B., Cornelis, N., Cornelis, K. and Gool, L.V. ( 2007). Dynamic 3D scene analysis from a moving vehicle. IEEE Conference Computer Vision and Pattern Recognition (CVPR) .
17.
Ma, G., Park, S., Ioffe, A., Muller-Schneiders, S. and Kummert, A. ( 2007). A real time object detection approach applied to reliable pedestrian detection. IEEE Intelligent Vehicles Symposium.
18.
Navarro-Serment, L.E., Mertz, C. and Hebert, M. ( 2008). LADAR-based pedestrian detection and tracking. Workshop on Human Detection from Mobile Platforms, IEEE International Conference on Robotics and Automation (ICRA).
19.
Sabzmeydani, P. and Mori, G. ( 2007). Detecting pedestrians by learning shapelet features. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .
20.
Seeman, E., Fritz, M. and Schiele, B. ( 2007). Towards robust pedestrian detection in crowded image sequences. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
21.
Shashua, A., Gdalyahu, Y. and Hayun, G. ( 2004). Pedestrian detection for driving assistance systems: single frame classification and system level performance. IEEE Intelligent Vehicles Symposium.
22.
Sotelo, M., Parra, I., Fernandez, D. and Naranjo, E. ( 2006). Pedestrian detection using svm and multi-feature combination . IEEE Intelligent Transportation Systems Conference .
23.
Thornton, S.M., Hoffelder, M. and Morris, D.D. ( 2008). Multi-sensor detection and tracking of humans for safe operations with unmanned ground vehicles. Workshop on Human Detection from Mobile Platforms, IEEE International Conference on Robotics and Automation (ICRA).
24.
Tomiuc, C., Nedevschi, S. and Meinecke, M.M. ( 2007). Pedestrian detection and classification based on 2d and 3d information for driving assistance systems. IEEE Intelligent Computer Communication and Processing Conference.
25.
Tuzel, O., Porikli, F. and Meer, P. ( 2007). Human detection via classification on Riemannian manifolds . IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
26.
Viola, P., Jones, M.J. and Snow, D. ( 2003). Detecting pedestrians using patterns of motion and appearance . IEEE International Conference on Compuer Vision .
27.
Wu, B. and Nevatia, R. ( 2007). Simultaneous object detection and segmentation by boosting local shape feature based classifier. IEEE Conference on Computer Vision and Pattern Recognition (CVPR).
28.
Zhao, T., Nevatia, R. and Wu, B. ( 2008). Segmentation and tracking of multiple humans in crowded environments. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30(7): 1198-1211.
