Here, we report the design of two deterministic observers that exploit the capabilities of a home-made divergent trinocular visual sensor to sense depth data. The three-dimensional key points that the observers can measure are triangulated for visual odometry and estimated by an extended Kalman filter. This work deals with a four-wheel-drive mobile robot with four passive suspensions. The direct and inverse kinematic solutions are deduced and used for the updating and prediction models of the extended Kalman filter as feedback for the robot’s position controller. The state-estimation visual odometry results were compared with the robot’s dead-reckoning kinematics, and both are combined as a recursive position controller. One observer model design is based on the analytical geometric multi-view approach. The other observer model has fundamentals on multi-view lateral optical flow, which was reformulated as nonspatial–temporal and is modeled by an exponential function. This work presents the analytical deductions of the models and formulations. Experimental validation deals with five main aspects: multi-view correction, a geometric observer for range measurement, an optical flow observer for range measurement, dead-reckoning and visual odometry. Furthermore, comparison of positioning includes a four-wheel odometer, deterministic visual observers and the observer–extended Kalman filter, compared with a vision-based global reference localization system.
JeongJMulliganJCorrellN. Trinocular visual odometry for divergent views with minimal overlap. In: Proceedings of the 2013 IEEE workshop on robot vision (WORV), Clearwater Beach, FL, 15–17 January 2013, pp.229–236. New York: IEEE.
2.
ClippBKimJFrahmJet al. Robust 6DOF motion estimation for non-overlapping, multi-camera systems. In: Proceedings of the IEEE workshop on applications of computer vision (WACV08), Copper Mountain, CO, 7–9 January 2008. New York: IEEE.
3.
EsquivelSWoelkFKochR. Calibration of a multi-camera rig from non-overlapping views. In: EsquivelSWoelkFKochR (eds) Lecture notes in computer science. Berlin: Springer, vol. 4713 (DAGM), 2007, pp. 82–91.
4.
FrahmJKöserKKochR.Pose estimation for multi-camera systems. In: FrahmJKöserKKochR (eds) Lecture notes in computer science. Berlin: Springer, vol. 3175 (DAGM), pp.286–293, 2004.
5.
HavlenaMPajdlaTCornelisK. Structure from omnidirectional stereo rig motion for city modeling. In: Proceedings of the third international conference on computer vision theory and applications (VISAPP), Funchal, 22–25 January 2008, pp.407–414. INSTICC.
6.
HowardA. Real-time stereo visual odometry for autonomous ground vehicles. In: Proceedings of the IEEE/RSJ international conference on intelligent robots and systems (IROS), Nice, 22–26 September 2008, pp.3946–3952. New York: IEEE.
7.
KimJHartleyRFrahmJet al. Visual odometry for non-overlapping views using second-order cone programming. In: Proceedings of the 8th Asian conference on computer vision, Tokyo, Japan, 18–22 November 2007, pp.353–362. Berlin: Springer.
8.
LebralyPAit-AiderORoyerEet al. Calibration of non-overlapping cameras—application to vision-based robotics. In: Proceedings of the British machine vision conference (BMVC), Aberystwyth, 31 August–3 September 2010.
9.
MuhleDAbrahamSHeipkeCet al. Estimating the mutual orientation in a multi-camera system with a non-overlapping field of view. In: Proceedings of the conference on photogrammetric image analysis, Munich, 5–7 October 2011, p.1324. Berlin: Springer.
10.
OskiperTZhuZSamarasekeraSet al. Visual odometry system using multiple stereo cameras and inertial measurement unit. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Minneapolis, MN, 17–22 June 2007, pp.1–8. New York: IEEE.
11.
PlessR.Using many cameras as one. In: Proceedings of the IEEE Computer Society conference on computer vision and pattern recognition, Madison, WI, 18–20 June 2003, pp.1–7. New York: IEEE.
12.
SteweniusHAstromK. Structure and motion problems for multiple rigidly moving cameras. In: Proceedings of the European conference on computer vision, Prague, 11–14 May 2004, pp.252–263. Berlin: Springer.
13.
JingXLiCYinXet al. Calibration and stereo image correction for binocular vision system with divergent optical axes arrangement. In: Proceedings of 6th international congress on image and signal processing (CISP), Hangzhou, China, 16–18 December 2013, pp.729–734. New York: IEEE.
14.
MoriHSumiyaEMashitaTet al. A wide-view parallax-free eye-mark recorder with a hyperboloidal half-silvered mirror and appearance-based gaze estimation. IEEE T Vis Comput Gr2011; 17: 900–912.
15.
Longuet-HigginsH.A computer algorithm for reconstructing a scene from two projections. Nature1981; 293(10): 133–135.
16.
ComportAMalisERivesP. Accurate quadrifocal tracking for robust 3D visual odometry. In: Proceedings of the IEEE international conference on robotics and automation, Roma, 10–14 April 2007, pp.40–45. New York: IEEE.
17.
HartleyRZissermanA.Multiple view geometry in computer vision. 2nd ed.Cambridge: Cambridge University Press, 2004.
18.
PrettoAMenegattiEPagelloE. Omnidirectional dense large-scale mapping and navigation based on meaningful triangulation. In: Proceedings of the IEEE international conference on robotics and automation, Shanghai, China, 9–13 May 2011, pp.3289–3296. New York: IEEE.
19.
LiangBPearsN. Visual navigation using planar homographies. In: Proceedings of the IEEE international conference on robotics and automation (Cat. No.02CH 37292), Washington, DC, 11–15 May 2002, pp.205–210. New York: IEEE.
20.
GuerreroJMartinez-CantinRSaguesC.Visual map-less navigation based on homographies. J Robot Syst2005; 22(10): 569581.
Gwo-LongLChi-ChengC.Acquisition of translational motion by the parallel trinocular. Inform Sci2008; 178: 137–151.
23.
KimJEChungMJChoiBT.Recursive estimation of motion and a scene model with a two-camera system of divergent view. Pattern Recogn2010; 43(6): 2265–2280.
24.
JaeheonJMulliganJCorrellN. Trinocular visual odometry for divergent views with minimal overlap. In: Proceedings of the IEEE workshop on robot vision, Clearwater Beach, FL, 15–17 January 2013, pp.229–236. New York: IEEE.
25.
KaessMDellaertF.Probabilistic structure matching for visual slam with a multi-camera rig. Comput Vis Image Und2010; 114(2): 286–296.
26.
CorreaJSotoA.Active visual perception for mobile robot localization. J Intell Robot Syst2010; 58(3): 339–354.
27.
LimYCLeeMLeeCHet al. Improvement of stereo vision-based position and velocity estimation and tracking using a stripe-based disparity estimation and inverse perspective map-based extended Kalman filter. Opt Laser Eng2010; 48(9): 859–886.
28.
YinJTianGLiG. Object localization and tracking based on multiple sensor fusion in intelligent home. In: Proceedings of the 26th Chinese control and decision conference, Changsha, China, 31 May–2 June 2014, pp.5266–5270. New York: IEEE.
29.
LuoRCLaiCC.Multisensor fusion-based concurrent environment mapping and moving object detection for intelligent service robotics. IEEE T Ind Electron2014; 61(8): 4043–4051.
30.
TribouMJHarmatAWangDWLet al. Multi-camera parallel tracking and mapping with non-overlapping fields of view. Int J Robot Res2015; 34(12): 1480–1500.
31.
ScaramuzzaDFraundorferF.Visual odometry: part ii: matching, robustness, optimization, and applications. IEEE Robot Autom Mag2011; 19: 78–90.
32.
MouWWangHSeetG.Efficient visual odometry estimation using stereo camera. In: Proceedings of the IEEE international conference on control & automation (ICCA), Taichung, Taiwan, 18–20 June 2014, pp.1399–1403. New York: IEEE.
33.
BadinoHYamamotoAKanadeT.Visual odometry by multi-frame feature integration. In: Proceedings of the IEEE international conference on computer vision workshops, Sydney, NSW, Australia, 2–8 December 2013, pp.222–229. New York: IEEE.
34.
NuñezPVázquez-MartínRBanderaA.Visual odometry based on structural matching of local invariant features using stereo camera sensor. Sensors2011; 11(7): 7262–7284.
35.
LiangZXuX. Improved visual odometry method for matching 3D point cloud data. In: Proceedings of the 33rd Chinese control conference, Nanjing, China, 28–30 July 2014, pp.8474–8478. New York: IEEE.
36.
ParraISoteloMLlorcaDFet al. Robust visual odometry for vehicle localization in urban environments. Robotica2010; 28: 441–452.
37.
WeichselbaumJZinnerCGebauerOet al. Accurate 3D-vision-based obstacle detection for an autonomous train. Comput Ind2013; 64(9): 1209–1220.
38.
Fuentes-PachecoJRuiz-AscencioJRend on-ManchaJM.Visual simultaneous localization and mapping: a survey. Artif Intell Rev2015; 43(1): 55–81.
39.
HernandezEIbarraJMNeiraJet al. Visual slam with oriented landmarks and partial odometry. In: Proceedings of the 21st international conference on electrical communications and computers, San Andres Cholula, 28 February–2 March, pp.39–45. New York: IEEE.
40.
HongSYeC. A pose graph based visual slam algorithm for robot pose estimation. In: Proceedings of the world automation congress, Waikoloa, HI, 3–7 August 2014, pp.917–922. New York: IEEE.
41.
WilliamsBReidI. On combining visual slam and visual odometry. In: Proceedings of the IEEE international conference on robotics and automation, Anchorage, AK, 3–7 May 2010, pp.3494–3500. New York: IEEE.
42.
ScaramuzzaDFraundorferF.Visual odometry part I: the first 30 years and fundamentals. IEEE Robot Autom Mag2011; 18: 80–92.
43.
AlcantarillaPFBergasaLMDellaertF. Visual odometry priors for robust EKF-SLAM. In: IEEE international conference on robotics and automation, Anchorage, AK, 3–7 May 2010, pp.3501–3506. New York: IEEE.
44.
YangJLSuDTShiaoYSet al. Path-tracking controller design and implementation of a vision-based wheeled mobile robot. J Syst Control Eng2009; 223(6): 847–862.
45.
LoRCHeCYChiuCJet al. Obstacle detection using combination of disparity and optical flow under around view for driver assistance system. In: Proceedings of international conference on connected vehicles and expo (ICCVE), Las Vegas, NV, 2–6 December, pp.451–458. New York: IEEE.
46.
PantilieCDNedevschiS.Real-time obstacle detection in complex scenarios using dense stereo vision and optical flow. In: Proceedings of the13th international IEEE conference on intelligent transportation systems, Funchal, 19–22 September 2010, pp.439–444. New York: IEEE.
47.
YanSHongJ.Stereo and optical flow calculation using recursive filter. In: Proceedings of the international IEEE conference on information engineering and computer science, Wuhan, China, 19–20 December 2009, pp.14. New York: IEEE.
48.
CaiJ.Integration of optical flow and dynamic programming for stereo matching. Image Process2012; 6(3): 205–212.
49.
ParkCWLeeCY.Sonar sensor data processing based on optical flow in robot navigation. J Syst Control Eng2011; 225(1): 1–7.
50.
WangYTChi YingCTYing-ChiehF.Robot mapping using local invariant feature detectors. Eng Comput2014; 31(2): 297–316.
51.
HilsenbeckSMollerAHuitlRet al. Scale-preserving long-term visual odometry for indoor navigation. In: Proceedings of international conference on indoor positioning and indoor navigation (IPIN), Sydney, NSW, Australia, 13–15 November 2012, p.110. New York: IEEE.
52.
ChenHLiuCRLiC. Data association algorithm for passive sensor system. In: Proceedings of the first international workshop on education technology and computer science, Wuhan, China, 7–8 March 2009, pp.970–974. New York: IEEE.
53.
KenariLRArvanMR. Comparison of nearest neighbor and probabilistic data association methods for non-linear target tracking data association. In: Proceedings of the second RSI/ISM international conference on robotics and mechatronics (ICRoM), Tehran, Iran, 15–17 October 2014, p.4752. New York: IEEE.
54.
WangXHLiPF. Improved data association method in binocular vision-slam. In: Proceedings of the international conference on intelligent computation technology and automation, Changsha, China, 11–12 May 2010, pp.502–505. New York: IEEE.
55.
WuXXuWShiZet al. Feature selection for reliable data association in visual slam. Mach Vision Appl2013; 24(4): 667–682.
56.
Martínez GarcíaEARiveroJVillanuevaNet al. Aparato poligonal de control y multi-sensado distribuido. Patent MX/a/2013/000881, Spain, 2013.
57.
YuanMZhaoRHuaXet al. Research on visual segmentation and localization of multi-robots under different lighting conditions. J Syst Control Eng2015; 229(9): 838–850.
58.
Shi-MinWLi-ChunLChia-JuWet al. A vision-based method for localization of mobile robots. J Syst Control Eng2007; 221(1): 49–59.
59.
KarasfiBTangSHSamsudinKet al. Unsupervised place recognition for assistive mobile robots based on local feature descriptions. J Syst Control Eng2011; 225(8): 1068–1085.
60.
Martinez-GarciaEATorres-MendezLAMohanRE.Multi-legged robot dynamics navigation model with optical flow. Int J Intel Unman Syst2014; 2(2): 121–139.
61.
Martínez-GarcíaEALerinETorres-CordobaR.A multi-configuration kinematic model for active drive/steer four-wheel robot structures. Robotica2015; 33: 2309–2329.
62.
Martinez-GarciaEAMarOTorres-CordobaT. Dead-reckoning inverse and direct kinematic solution of a 4W independent driven rover. In: Proceedings of the IEEE ANDESCON, Bogota, Colombia, 15–17 September 2010.
63.
Martinez-GarciaEATorres-CordobaT., 4WD skid-steer trajectory control of a rover with spring-based suspension analysis. In: Proceedings of the international conference on intelligent robotics and applications, Shanghai, China, 10–12 November 2010.
64.
Sadeghzadeh-NokhodberizNPoshtanJ.Interconnected maximum likelihood estimator and extended Kalman filter for inertial measurement unit calibration fusing three-dimensional camera information. J Syst Control Eng2014; 229(2): 106–117.
65.
MaYSoattoSKoseckaJet al. An invitation to 3D vision: from images to geometric models. Berlin: Springer, 2010.
