This paper proposes a new method to design a reinforcement learning-based integrated kinematic and dynamic tracking control algorithm for a non-holonomic wheeled mobile robot without knowledge of the system’s drift tracking dynamics. The actor critic structure in the control scheme uses only one neural network to reduce computational cost and storage resources. A novel tuning law for a single neural network is designed to learn an online solution of a tracking Hamilton–Jacobi–Isaacs (HJI) equation. The HJI solution is used to approximate an H∞ optimal tracking performance index function and an intelligent tracking control law in the case of the worst disturbance. The laws guarantee closed-loop stability in real time. The convergence and stability of the overall system are proved by Lyapunov techniques. The simulation results on a non-linear system and wheeled mobile robot verify the effectiveness of the proposed controller.
Abu-KhalafMLewisFL (2005) Nearly optimal control laws for nonlinear systems with saturating actuators using a neural network HJB approach. Automatica41(5): 779–791.
2.
ChenBSUangHJTsengCS (1998) Robust tracking enhancement of robot systems including motor dynamics: a fuzzy-based dynamic game approach. IEEE Transactions on Fuzzy Systems6(4): 538–552.
3.
ChenHMaMWangH. (2009) Moving horizon H∞ tracking control of wheeled mobile robots with actuator saturation. IEEE Transactions on Control Systems Technology17(2): 449–457.
4.
ChwaD (2010) Tracking control of differential-drive wheeled mobile robots using a backstepping-like feedback linearization. IEEE Transactions on Systems, Man, and Cybernetics—Part A: Systems and Humans40(6): 1285–1295.
5.
DierksTJagannathanS (2010) Optimal control of affine nonlinear continuous-time systems using an online Hamilton–Jacobi–Isaacs formulation. In: 49th IEEE Proceedings of the CDC2010 (pp. 3048–3053).
6.
FierroRLewisFL (1998) Control of a nonholonomic mobile robot using neural networks. IEEE Transactions on Neural Networks4: 589–600.
7.
FinlaysonBA (1990) The Method of Weighted Residuals and Variational Principles. New York: Academic Press.
8.
HornikKStinchcombeMWhiteH (1990) Universal approximation of an unknown mapping and its derivatives using multilayer feedforward networks. Neural Networks3: 551–560.
9.
KhoshnamSAlirezaMSAhmadrezT (2011) Adaptive feedback linearizing control of nonholonomic wheeled mobile robots in presence of parametric and nonparametric uncertainties. Robotics and Computer-Integrated Manufacturing27(1): 194–204.
10.
LewisFLJagannathanSYesildirekA (1999) Neural Network Control of Robot Manipulators and Nonlinear Systems. London: Taylor & Francis.
11.
LinWSYangPC (2008) Adaptive critic motion control design of autonomous wheeled mobile robot by dual heuristic programming. Automatica44: 2716–2723.
12.
LuyNT (2012) Reinforcement learning-based optimal tracking control for wheeled mobile robot. In: Proceeding of the IEEE International Conference on Cyber Technology in Automation, Control, and Intelligent Systems, pp. 371–376.
13.
LuyNTThanhNDThanhNT. (2010) Robust reinforcement learning-based tracking control for wheeled mobile robot. In IEEE Proceedings of the ICCAE2010, Vol. 1, pp. 171–176.
14.
MarvinKBSimonGFLiberatoC (2009) Dual adaptive dynamic control of mobile robots using neural networks. IEEE Transactions on Systems, Man, and Cybernetics—part b: Cybernetics39(1): 129–141.
15.
MiyasatoY (2008) Adaptive H∞ control of nonholonomic mobile robot based on inverse optimality. In: Proceedings of the American Control Conference, Seattle, WA, pp. 3524–3529.
16.
MohareriODhaouadiRRadAB (2012) Indirect adaptive tracking control of a nonholonomic mobile robot via neural networks. Neurocomputing88: 54–66.
17.
VamvoudakisKGLewisFL (2010) Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica46: 878–888.
VamvoudakisKGVrabieDLewisFL (2011) Online learning algorithm for zero-sum games with integral reinforcement learning. Journal of Artificial Intelligence and Soft Computing Research1(4): 315–332.
20.
Van Der ShaftAJ (1992) L2-gain analysis of nonlinear systems and nonlinear state feedback H∞ control. IEEE Transactions on Automatic Control37(6): 770–784.
21.
VrabieDPastravanuOLewisFL. (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica45(2): 477–484.
22.
ZenonHMarcinS (2011) Discrete neural dynamic programming in wheeled mobile robot control. Communications in Nonlinear Science & Numerical Simulation16. 2355–2362.
