A study on ego-motion estimation based on stereo camera sensor and 2G1Y inertial sensor with considering vehicle dynamics

Abstract

The vision sensor on a vehicle platform with dynamic motion is affected severely by disturbances that occur owing to vehicle dynamic movement, which results in the degradation of outputs from vision sensor such like the distance between ego-vehicle to objects. To get around this phenomenon, it is necessary to estimate vehicle ego-motion, that is, tri-axis linear velocities and angular velocities that can be utilized to make up for the contaminated outputs from vision sensor. This study proposes a novel method for vehicle ego-motion estimation by using the optical and disparity flows from vision sensor and 2G1Y (longitudinal/lateral acceleration and yaw-rate) measurements from the inertial sensor in vehicle. The approach considers vehicle platform characteristics to estimate ego-motion more accurately. To the end, vehicle kinetics are augmented into the evolving equation represented in a form of state space equation for applying Kalman filter and the vanishing point is to be estimated online to enhance the information of optical flows because the optical flow radiates from a vanishing point that might vary owing to vehicle dynamic motion. To show the effectiveness of the proposed algorithm, experiments have been carried out on real vehicle environments and the results confirm its effectiveness and accuracy.

Keywords

Ego-motion ego-path extended Kalman filter stereo-vision vehicle lateral dynamics Longet–Higgins equation vanishing point

Get full access to this article

View all access options for this article.

References

Stein

Mano

Shashua

. A robust method for computing vehicle ego-motion. In: Proceedings of the IEEE intelligent vehicles symposium, Dearborn, MI, 5 October 2000, pp.362–368. New York: IEEE.

Yamaguchi

Kato

Ninomica

. Vehicle ego-motion estimation and moving object detection using a monocular camera. In: 18th international conference on pattern recognition, Hong Kong, 20–24 August 2006, vol. 4, pp.610–613. New York: IEEE.

Pink

Moosmann

Bachmann

. Visual features for vehicle localization and ego-motion estimation. In: Proceedings of the IEEE intelligent vehicles symposium, Xi’an, China, 3–5 June 2009. New York: IEEE.

Raudies

Neumann

. An efficient linear method for the estimation of ego-motion from optical flow. In: Proceedings of the 31st DAGM symposium, Jena, 9–11 September 2009, pp.11–20. Berlin: Springer.

Bak

Bouchafa

Aubert

. Detection of independently moving objects through stereo vision and ego-motion extraction. In: Proceedings of the 2010 IEEE intelligent vehicle symposium, San Diego, CA, 21–24 June 2010, vol. 4, pp.863–870. New York: IEEE.

Kitt

Ranft

Lategahn

. Detection and tracking of independently moving objects in urban environments. In: Proceedings of the 13th international IEEE conference on intelligent transportation systems (ITSC), Funchal, 19–22 September 2010, pp.1396–1401. New York: IEEE.

Rajamani

. Vehicle dynamics and control. New York: Springer-Verlag, 2005.

Rabe

Franke

Gehrig

. Fast detection of moving objects in complex scenarios. In: Proceedings of the 2007 IEEE intelligent vehicle symposium, Istanbul, Turkey, 13–15 June 2007, pp.398–403. New York: IEEE.

Franke

Rabe

Badino

et al . 6D-vision: fusion of stereo and motion for robust environment perception. In: Kropatsch

Sablatnig

Hanbury

(eds) Pattern recognition. DAGM 2005. Lecture notes in Computer Science, vol. 3663. Heidelberg: Springer.

10.

Mallet

Lacroix

Gallo

. Position estimation in outdoor environments using pixel tracking and stereovision. In: Proceedings of the 2000 IEEE international conference on robotics and automation, San Francisco, CA, 24–28 April 2000, pp.3519–3524. New York: IEEE.

11.

Lin

. Three-dimensional relative positioning and tracking using LDRI. US patent no. 6,677,941, 2004.

12.

Oumer

Panin

. 3D point tracking and pose estimation of a space object using stereo images. In: Proceedings of the 21st international conference on pattern recognition, Tsukuba, Japan, 11–15 November 2012. New York: IEEE.

13.

Badino

Franke

Rabe

et al . Stereo vision-based detection of moving objects under strong camera motion. In: Proceedings of the first international conference on computer vision theory and applications (VISAPP), Setúbal, Portugal, 25–28 February 2006, pp.25–28.

14.

Longuet-Higgins

Prazdny

. The interpretation of a moving retinal image. Proc R Soc Lond B Biol Sci 1980; 208: 385–397.

15.

Keller

Enzweiler

Rohrbach

et al . The benefits of dense stereo for pedestrian detection. IEEE Trans Intell Transport Syst 2011; 12: 1096–1106.