Real-time Quadrifocal Visual Odometry

Abstract

In this paper we describe a new image-based approach to tracking the six-degree-of-freedom trajectory of a stereo camera pair. The proposed technique estimates the pose and subsequently the dense pixel matching between temporal image pairs in a sequence by performing dense spatial matching between images of a stereo reference pair. In this way a minimization approach is employed which directly uses all grayscale information available within the stereo pair (or stereo region) leading to very robust and precise results. Metric 3D structure constraints are imposed by consistently warping corresponding stereo images to generate novel viewpoints at each stereo acquisition. An iterative non-linear trajectory estimation approach is formulated based on a quadrifocal relationship between the image intensities within adjacent views of the stereo pair. A robust M-estimation technique is used to reject outliers corresponding to moving objects within the scene or other outliers such as occlusions and illumination changes. The technique is applied to recovering the trajectory of a moving vehicle in long and difficult sequences of images.

Keywords

visual odometry direct dense 3D tracking stereo multi-view geometry

Get full access to this article

View all access options for this article.

References

Avidan, S. and Shashua, A. ( 2001). Threading fundamental matrices. Pattern Analysis and Machine Intelligence, 23(1): 73-77.

Baker, S. and Matthews, I. ( 2001). Equivalence and efficiency of image alignment algorithms . Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recognition.

Benhimane, S. and Malis, E. ( 2004). Real-time image-based tracking of planes using efficient second-order minimization. IEEE International Conference on Intelligent Robots Systems, Sendai, Japan.

Burt, P. ( 1984). The pyramid as structure for efficient computation. Multiresolution Image Processing and Analysis. Berlin, Springer, pp. 6-35.

Chiuso, A. , Favaro, P. , Jin, H. and Soatto, S. ( 2002). Structure from motion causally integrated over time. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(4): 523-535.

Comport, A. , Malis, E. and Rives, P. ( 2007). Accurate quadrifocal tracking for robust 3D visual odometry . IEEE International Conference on Robotics and Automation, Rome, Italy.

Comport, A. , Marchand, E. and Chaumette, F. ( 2006a). Statistically robust 2D visual servoing. IEEE Transactions on Robotics, 22(2): 415-421.

Comport, A. , Marchand, E. , Pressigout, M. and Chaumette, F. ( 2006b). Real-time markerless tracking for augmented reality: the virtual visual servoing framework. IEEE Transactions on Visualization and Computer Graphics, 12(4): 615- 628.

Davison, A.J. and Murray, D.W. ( 2002). Simultaneous localisation and map-building using active vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24: 865-880.

10.

Faugeras, O. and Mourrain, B. ( 1995). On the geometry and algebra of the point and line correspondences between n images. IEEE International Conference on Computer Vision, Cambridge, MA, pp. 951-956.

11.

Hager, G. and Belhumeur, P. ( 1998). Efficient region tracking with parametric models of geometry and illumination. IEEE Transactions on Pattern Analysis and Machine Intelligence, 20(10): 1025-1039.

12.

Hartley, R. ( 1995). Multilinear relationships between coordinates of corresponding image points and lines. Proceedings of the Sophus Lie Symposium, Nordjordeid, Norway.

13.

Hartley, R. and Zisserman, A. ( 2001). Multiple View Geometry in computer vision. Cambridge, Cambridge University Press.

14.

Heckbert, P. ( 1989). Fundamentals of Texture Mapping and Image Warping. Master’s thesis, CS Division, U.C. Berkeley.

15.

Heyden, A. and Astrom, K. ( 1997). Algebraic properties of multilinear constraints. Mathematical Methods in the Applied Sciences, 20(13): 1135-1162.

16.

Howard, A. ( 1983). Scale-space filtering. 8th International Joint Conference on Artificiel Intelligence, Karlsruhe, Germany, pp. 1019-1022.

17.

Howard, A. ( 2008). Real-time stereo visual odometry for autonomous ground vehicles. IEEE/RSJ International Conference on Intelligent Robots and Systems, Nice, France. Huber, P.-J. (1981). Robust Statistics. New York, Wiley .

18.

Irani, M. and Anandan, P. ( 2000). About direct methods. IEEE International Conference on Computer Vision: Proceedings of the International Workshop on Vision Algorithms. London, Springer, pp. 267-277.

19.

Konolige, K. and Agrawal, M. ( 2008). Frameslam: from bundle adjustment to realtime visual mapping. IEEE Transactions on Robotics, 24(5): 1066-1077.

20.

Lucas, B. and Kanade, T. ( 1981). An iterative image registration technique with an application to stereo vision. International Joint Conference on Artificial Intelligence, Vancouver, Canada, Vol. 2, pp. 674-679.

21.

Malis, E. ( 2004). Improving vision-based control using efficient second-order minimization techniques. IEEE International Conference on Robotics and Automation, New Orleans, LA, Vol. 2, pp. 1843-1848.

22.

Mouragnon, E. , Lhuillier, M. , Dhome, M. , Dekeyser, F. and Sayd, P. ( 2006). Real-time localization and 3D reconstruction. IEEE Conference of Vision and Pattern Recognition, New York.

23.

Nistér, D. , Naroditsky, O. and Bergen, J. ( 2004). Visual odometry. IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Washington, DC, Vol. 1, pp. 652-659.

24.

Nister, D. , Naroditsky, O. and Bergen, J. ( 2006). Visual odometry for ground vehicle applications. Journal of Field Robotics, 23: 2006.

25.

Odobez, J.-M. and Bouthemy, P. ( 1995). Robust multiresolution estimation of parametric motion models. Journal of Visual Communication and Image Representation , 6(4): 348- 365.

26.

Ogale, A. and Aloimonos, Y. ( 2005). Shape and the stereo correspondence problem. International Journal of Computer Vision, 65: 147-162.

27.

Scharstein, D. , Szeliski, R. and Zabih, R. ( 2001). A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Proceedings of the IEEE Workshop on Stereo and Multi-Baseline Vision, Kauai, HI.

28.

Shashua, A. and Wolf, L. ( 2000). On the structure and properties of the quadrifocal tensor . European Conference on Computer Vision, pp. 710-724.

29.

Silveira, G. , Malis, E. and Rives, P. ( 2008). An efficient direct approach to visual SLAM. IEEE Transactions on Robotics, 20(5): 969-979.

30.

Simond, N. and Rives, P. ( 2008). What can be done with an embedded stereo-rig in urban environments? Robotics and Autonomous Systems, 56: 777-789.

31.

Triggs, B. ( 1995). The geometry of projective reconstruction. I: Matching constraints and the joint image. IEEE International Conference on Computer Vision, Cambridge, MA, pp. 338-343.

32.

van der Mark, W. and Gavrila, D. ( 2006). Real-time dense stereo for intelligent vehicles. IEEE Transactions on Intelligent Transportation Systems, 7(1): 38-50.

33.

Zhang, Z. and Faugeras, O. ( 1992). Three dimensional motion computation and object segmentation in a long sequence of stereo frames. International Journal of Computer Vision, 7(3): 211-241.

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB