Fast Ego-motion Estimation with Multi-rate Fusion of Inertial and Vision

Abstract

This paper presents a tracking system for ego-motion estimation which fuses vision and inertial measurements using EKF and UKF (Extended and Unscented Kalman Filters), where a comparison of their performance has been done. It also considers the multi-rate nature of the sensors: inertial sensing is sampled at a fast sampling frequency while the sampling frequency of vision is lower. the proposed approach uses a constant linear acceleration model and constant angular velocity model based on quaternions, which yields a non-linear model for states and a linear model in measurement equations. Results show that a significant improvement is obtained on the estimation when fusing both measurements with respect to just vision or just inertial measurements. It is also shown that the proposed system can estimate fast-motions even when vision system fails. Moreover, a study of the influence of the noise covariance is also performed, which aims to select their appropriate values at the tuning process. The setup is an end-effector mounted camera, which allow us to pre-define basic rotational and translational motions for validating results.

Keywords

vision and inertial multi-rate systems sensor fusion tracking

Get full access to this article

View all access options for this article.

References

Abuhadrous, I. , Nashashibi, F. , and Laurgeau, C. (2003). 3D land vehicle localization: a real-time multi-sensor data fusion approach using RT MAPS. International Conference on Advanced Robotics, pp. 71—76.

Albertos, P. (1990). Block multirate input-output model for sampled-data control systems. IEEE Transactions on Automatic Control, AC-35(9): 1085—1088.

Alves, J. , Lobo, J. , and Dias, J. (2003). Camera-inertial sensor modelling and alignment for visual navigation. International Conference on Advanced Robotics, pp. 1693—1698.

Armesto, L.I. (2007). An asynchronous multi-rate approach to probabilistic self-localisation and mapping. Accepted in Robotics and Automation Magazine.

Armesto, L. and Tornero, J. (2003). Dual-rate high order holds based on primitive functions. American Control Conference, pp. 1140—1145.

Armesto, L. and Tornero, J. (2004). SLAM based on Kalman filter for multirate fusion of laser and encoder measurements. IEEE International Conference on Intelligent Robots and Systems, pp. 1860—1865.

Armesto, L. , Chroust, S. , Vincze, M. , and Tornero, J. , (2004). Multi-rate fusion with vision and inertial sensors . International Conference on Robotics and Automation , pp. 193— 199.

Bradski, G. (2000). An open-source library for processing image data . Dr Dobbs Journal, November.

Carpenter, J. , Clifford, P. , and Fernhead, P. (1997). An improved particle filter for non-linear problems. Technical Report, Department of Mathematics, Imperial College .

10.

Chai, L. , H., Hoff , W.A. , and Vincent , T. (2002). Three-dimensional motion and structure estimation using inertial sensors and computer vision for augmented reality. Presence: Teleoperators and Virtual Environments, 11(5): 474— 492.

11.

Chou, J. (1992). Quaternion kinematic and dynamic differential equations. IEEE Transactions on Robotics and Automation, 8(1): 53—64.

12.

Chroust, S. and Vincze, M. (2003). Fusion of vision and inertia data for motion and structure estimation. Journal of Robotics Systems, 21(2): 73—83.

13.

Dissanayake, M. , Newman, P. , Clark, S. , Durrant-Whyte, H. , and Csorba, M. (2001). A solution to the simultaneous localization and map building (SLAM) problem. IEEE Transactions on Robotics and Automation, 17(3): 229—241.

14.

Doucet, A. , de Freitas, N. , Murphy, K. , and Russell, S. (2000). RaoBlackwellised particle filtering for dynamic Bayesian networks. In Uncertainty in Artificial Intelligence .

15.

Doucet, A. , Gordon, N. , and Krishnamurthy, V. (2001). Particle filters for state estimation of jump Markov linear systems. IEEE Transactions on Signal Processing, 49(3): 613— 624.

16.

Gemeiner, P. , Armesto, L. , Montes, N. , Tornero, J. , Vincze, M. , and Pinz, A. (2006). Visual tracking can recapture after fast motions with the use of inertial sensors. Digital Imaging and Pattern Recognition, OAGM/AAPR, pp. 141—150.

17.

Goddard, J. and Abidi, M. (1998). Pose and motion estimation using dual quaternion-based extended Kalman filtering. SPIE, 3313: 189—200.

18.

Goodwin, G. and Feuer, A. (1992), Linear periodic control: A frequency domain viewpoint . Systems and Control Letters, 19: 379—390.

19.

Gordon, N. , Salmond, D. , and Smith, A. (1993). Novel aproach to nonlinear/non-gaussian Bayesian state estimation. IEE-Proceedings-F, 140(2): 107—113.

20.

Grewal, M.S. , Weill, L.R. , and Andrews, A.P. (2001). Global Positioning Systems, Inertial Navigation, and Integration. John Wiley and Sons, Canada.

21.

Gurfil, P. and Kasdin, N. (2002). Two-step optimal estimator for three dimensional target tracking. American Control Conference, pp. 209—214.

22.

Huster, A. and Rock, S. (2003). Relative position sensing by fusing monocular vision and inertial rate sensors. International Conference on Advanced Robotics, pp. 1562— 1567.

23.

Huster, A. and Rock, S. (2001). Relative position estimation for interventioncapable AUVS by fusing vision and inertial measurements. International Symposium on Unmanned Unthethered Submersible Technology.

24.

Jekeli, C. (2001). Inertial Navigation Systems with Geodetic Applications . Walter de Gruyter. John Wiley & Sons, Canada.

25.

Julier, S. and Uhlmann, J. (2002), Reduced sigma points filters for the propagation of means and covariances through non-linear transformations. American Control Conference, Vol. 2, pp. 887—892.

26.

Khargonekar, P. , Poolla, K. , and Tannenbaum, A. (1985). Robust control of linear time-invariant plants using periodic compensation. IEEE Transactions on Automatic Control, AC-30: 1088—1985.

27.

Kranc, G. (1957). Input-output analysis of multirate feedback systems . IEEE Transactions on Automatic Control, AC-3: 21—28.

28.

Lobo, J. and Dias, J. (1998). Integration of inertial information with vision . IEEE Industrial Electronic Society, pp. 1263— 1267.

29.

Longhi, S. (1994). Structural properties of multirate sampled systems . IEEE Transactions on Automatic Control, 39(3): 692—696.

30.

Panerai, F. , Metta, G. , and Sandini, G. (2000). Visuoinertial stabilization in space-variant binocular systems. Robotics and Autonomouns Systems, 30(1-2): 195— 214.

31.

Rehbinder, H. and Ghosh, B. (2001). Multi-rate fusion of visual and inertial data . International Conference on Multi-Sensor Fusion and Integration for Intelligent Systems, pp. 97—102.

32.

Smith, A. and Gelfand, E. (1992). Bayesian statistics without tears: A samplingresampling perspective. American Statistician, 2: 84—88.

33.

Tornero, J. (1985). Non-conventional sampled-data systems modelling . University of Manchester (UMIST), Control System Center Report, 640/1985.

34.

Tornero, J. , Albertos, P. , and Salt J. (1999). Periodic optimal control of multirate sampled data systems. 14th World Congress of IFAC, China , pp. 211—216.

35.

Tornero, J. , Gu, Y. , and Tomizuka, M. (1999). Analysis of multi-rate discrete equivalent of continuous controller. American Control Conference , pp. 2759—2763.

36.

Tornero, J. and Tomizuka, M. (2000). Dual-rate high order hold equivalent controllers . American Control Conference, pp. 175—179.

37.

Tornero, J. and Tomizuka, M. (2002). Modeling, analysis and design tools for dual-rate systems. American Control Conference, pp. 4116—4121.

38.

Ude, A. ( 1999). Filtering in a unit quaternion space for model-based object tracking. IEEE Transactions Robotics and Autonomous Systems, 28(2—3): 163—172.

39.

Vincze, M. , Ayromlou, M. , Ponweiser, W. , and Zillich, M. (2001). Edge projected integration of image and model cues for robust model-based object tracking. International Journal of Robotics Research, 7(20): 533—552.

40.

Zhang, Z. (1998). A flexible new technique for camera calibration. Technical Report, Microsoft Research, Microsoft Corporation .

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB