In this paper, a fisheye-based visual servo regulation method is proposed for wheeled mobile robots, which utilizes motion planning algorithms to provide trajectories for driving the robot to desired poses in an aligned pattern. To satisfy the wide field-of-view (FOV) requirement in robotic systems, the image projection model of fisheye is analyzed, and derivation and estimation of homography for fisheye vision are implemented to facilitate robotic pose calculation. Compared with traditional visual regulation schemes, motion planning and trajectory tracking techniques are combined to make mobile robots reach given poses in the frontal aligned style, which guarantees the robot to move smoothly and is extendable such as for car-like robots. Desired image trajectories are calculated reversely by the fisheye model with respect to the planned geometric path, and then a tracking controller is employed for maneuvering the mobile robot. Simulation and comparative experimental results are collected to verify effectiveness of the proposed strategy.
AroraPPapachristosC (2020) Mobile manipulator robot visual servoing and guidance for dynamic target grasping. In: 15th international symposium on visual computing, San Diego, CA, 5–7 October, pp. 223–235. Cham: Springer.
2.
BakerSNayarSK (1998) A theory of catadioptric image formation. In: Proceedings of the 1998 IEEE 6th international conference on computer vision, Bombay, India, 4–7 January, pp. 35–42. New York: IEEE.
3.
BarretoJP (2006) A unifying geometric representation for central projection systems. Computer Vision and Image Understanding103(3): 208–217.
4.
ChenJDixonWEDawsonM, et al. (2006) Homography-based visual servo tracking control of a wheeled mobile robot. IEEE Transactions on Robotics22(2): 406–415.
5.
CherubiniAChaumetteF (2013) Visual navigation of a mobile robot with laser-based collision avoidance. International Journal of Robotics Research32(2): 189–205.
6.
FangFLiuXZhangX (2012) Adaptive active visual servoing of nonholonomic mobile robots. IEEE Transactions on Industrial Electronics59(1): 486–497.
7.
FangYSuHWangC, et al. (2018) Adaptive and sliding mode tracking control for wheeled mobile robots with unknown visual parameters. Transactions of the Institute of Measurement and Control40(1): 269–278.
8.
FinkJRibeiroAKumarV (2012) Robust control for mobility and wireless communication in cyber-physical systems with application to robot teams. Proceedings of the IEEE100(1): 164–178.
9.
FitzgibbonAW (2001) Simultaneous linear estimation of multiple-view geometry and lens distortion. In: 2001 IEEE computer society conference on computer vision and pattern recognition, Kauai, HI, 8–14 December, pp. I125–I132. New York: IEEE.
10.
GansNRHuGShenJ, et al. (2012) Adaptive visual servo control to simultaneously stabilize image and pose error. Mechatronics22(4): 410–422.
11.
GaoFWuWLinY, et al. (2018) Online safe trajectory generation for quadrotors using fast marching method and Bernstein basis polynomial. In: 2018 IEEE international conference on robotics and automation, Brisbane, QLD, Australia, 21–25 May, pp. 344–351. New York: IEEE.
12.
GuoDLeangK-K (2020) Image-based estimation, planning, and control of a cable-suspended payload for package delivery. IEEE Robotics and Automation Letters5(2): 2698–2705.
13.
HatakeyamaTNumataTIwataniY (2014) Indoor hovering flight of a nano-scale helicopter by markerless visual servo control. In: 2014 IEEE international conference on robotics and biomimetics, Bali, Indonesia, 5–10 December, pp. 1880–1885. New York: IEEE.
14.
Janabi-SharifiFDengLWilsonWJ (2011) Comparison of basic visual servoing methods. IEEE/ASME Transactions on Mechatronics16(5): 967–983.
15.
JiPZengHSongA, et al. (2018) Virtual exoskeleton-driven uncalibrated visual servoing control for mobile robotic manipulators based on human-robot-robot cooperation. Transactions of the Institute of Measurement and Control40(14): 4046–4062.
16.
KangZZouWMaH, et al. (2019) Adaptive trajectory tracking of wheeled mobile robots based on a fish-eye camera. International Journal of Control, Automation and Systems17(9): 2297–2309.
17.
KannalaJBrandtSS (2006) A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses. IEEE Transactions on Pattern Analysis Machine Intelligence28(8): 1335–1340.
18.
KeshmiriMXieW-FMohebbiA (2014) Augmented image-based visual servoing of a manipulator using acceleration command. IEEE Transactions on Industrial Electronics61(10): 5444–5452.
19.
KumarVWangQMinghuaW, et al. (2018) Computer vision based object grasping 6DoF robotic arm using picamera. In: 2018 4th international conference on control, automation and robotics, Auckland, New Zealand, 20–23 April, pp. 111–115. New York: IEEE.
20.
LiBZhangXFangY, et al. (2018) Visual servo regulation of wheeled mobile robots with simultaneous depth identification. IEEE Transactions on Industrial Electronics65(1): 460–469.
21.
LiuSDongJ (2020) Robust online model predictive control for image-based visual servoing in polar coordinates. Transactions of the Institute of Measurement and Control42(4): 890–903.
22.
LiuSAtanasovNMohtaK, et al. (2017) Search-based motion planning for quadrotors using linear quadratic minimum time control. In: 2017 IEEE/RSJ international conference on intelligent robots and systems, Vancouver, BC, Canada, 24–28 September, pp. 2872–2879. New York: IEEE.
23.
LiuSWattersomMMohtaK, et al. (2017) Planning dynamically feasible trajectories for quadrotors using safe flight corridors in 3-D complex environments. IEEE Robotics and Automation Letters2(3): 1688–1695.
24.
LucaADOrioloGGiordanoPR (2008) Feature depth observation for image-based visual servoing: Theory and experiments. International Journal of Robotics Research27(10): 1093–1116.
25.
MaLChenYMooreK (2006) Analytical piecewise radial distortion model for precision camera calibration. IEE Proceedings: Vision, Image and Signal Processing153(4): 468–474.
26.
MeiCRivesP (2007) Single view point omnidirectional camera calibration from planar grids. In: 2007 IEEE international conference on robotics and automation, Rome, 10–14 April, pp. 3945–3950. New York: IEEE.
27.
MellingerDKumarV (2011) Minimum snap trajectory generation and control for quadrotors. In: 2011 IEEE international conference on robotics and automation, Shanghai, China, 9–13 May, pp. 2520–2525. New York: IEEE.
28.
MellingerDKushleyevAKumarV (2012) Mixed-integer quadratic program trajectory generation for heterogeneous quadrotor teams. In: 2012 IEEE international conference on robotics and automation, Saint Paul, MN, 14–18 May, pp. 477–483. New York: IEEE.
29.
MidorikawaNOhnoKTadokoroS, et al. (2007) Development of small-size multi camera system for snake-like robot, and display of wide view-angle image. In: IEEE international workshop on safety, security and rescue robotics, Rome, 27–29 September, pp. 1–6. New York: IEEE.
30.
NayarSK (1997) Catadioptric omnidirectional camera. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, San Juan, PR, 17–19 June, pp. 482–488. New York: IEEE.
31.
PangZFanLSunJ, et al. (2021) Detection of stealthy false data injection attacks against networked control systems via active data modification. Information Sciences546: 192–205.
32.
QiuYLiBShiW, et al. (2019) Visual servo tracking of wheeled mobile robots with unknown extrinsic parameters. IEEE Transactions on Industrial Electronics66(11): 8600–8609.
33.
RichterCBryARoyN (2016) Polynomial trajectory planning for aggressive quadrotor flight in dense indoor environments. Springer Tracts in Advanced Robotics114: 649–666.
34.
SchafleTRTokuiAUchiyamaN (2018) A hybrid systems approach with input-output linearization for automotive parking control of a nonholonomic mobile robot. In: 2018 3rd international conference on advanced robotics and mechatronics, Singapore, 18–20 July, pp. 340–345. New York: IEEE.
35.
SunNWuYFangY, et al. (2018) Nonlinear antiswing control for crane systems with double-pendulum swing effects and uncertain parameters: Design and experiments. IEEE Transactions on Automation Science and Engineering15(3): 1413–1422.
36.
TahaEI-MFaragA (2003) Statistically robust approach to lens distortion calibration with model selection. IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops8(1): 150–156.
37.
YükselT (2019) An intelligent visual servo control system for quadrotors. Transactions of the Institute of Measurement and Control41(1): 3–13.
38.
YingXHuZ (2004) Can we consider central catadioptric cameras and fisheye cameras within a unified imaging model. In: 8th European conference on computer vision, Prague, 11–14 May, pp. 442–455. Berlin: Springer.
39.
ZhangKChenJLiY, et al. (2018) Unified visual servoing tracking and regulation of wheeled mobile robots with an uncalibrated camera. IEEE/ASME Transactions on Mechatronics23(4): 1728–1739.
40.
ZhangXFangYMaB, et al. (2008) A fast homography decomposition technique for visual servo of mobile robots. In: 27th Chinese control conference, Kunming, China, 16–18 July, pp. 404–409. New York: IEEE.
41.
ZhangXWangRFangY, et al. (2019) Acceleration-level pseudo-dynamic visual servoing of mobile robots with backstepping and dynamic surface control. IEEE Transactions on Systems, Man, and Cybernetics: Systems49(10): 2071–2081.
42.
ZongGWangYKarimiHR, et al. (2022) Observer-based adaptive neural tracking control for a class of nonlinear systems with prescribed performance and input dead-zone constraints. Neural Networks147: 126–135.
