In this paper, an adaptive fault-tolerant attitude tracking controller based on reinforcement learning is developed for flying-wing unmanned aerial vehicle subjected to actuator faults and saturation. At first, the attitude dynamic model is separated into two dynamic subsystems as slow and fast dynamic subsystems based on the principle of time scale separation. Secondly, backstepping technique is adopted to design the controller. For the purpose of attitude angle constraints, the control technique based on Barrier Lyapunov is used to design controller of slow dynamic subsystem. Considering the optimization of the fast dynamic subsystem, this paper introduces an adaptive reinforcement learning control method in which neural network is used to approximate the long-term performance index and lumped fault dynamic. It is shown that this control algorithm can satisfy the requirements of attitude tracking subjected to the control constraints and the stability of the system is proved from Lyapunov stability theory. The simulation results demonstrate that the developed fault-tolerant scheme is useful and has more smooth control effect compared with fault-tolerant controller based on sliding mode theory.
BlanchiniF (1999) Set invariance in control. Automatica35(11): 1747–1767.
2.
CongBWangLX, et al. (2017) Crosswind take-off and landing characteristics of flying wing. Journal of Beijing University of Aeronautics and Astronautics43(5): 1023–1030.
3.
DingSLiuLParkJH, et al. (2018) A novel adaptive nonsingular terminal sliding mode controller design and its application to active front steering system. International Journal of Robust and Nonlinear Control29(12): 4250–4269.
4.
DuHChenX, et al. (2018) Discrete-time fast terminal sliding mode control for permanent magnet linear motor. IEEE Transactions on Industrial Electronics65(99): 9916–9927.
5.
GuoXYanWCuiR (2020) Integral reinforcement learning-based adaptive NN control for continuous-time nonlinear MIMO systems with unknown control directions. IEEE Transactions on Systems, Man, and Cybernetics50(11): 4068–4077.
6.
HePJagannathanS (2005) Reinforcement learning-based output feedback control of nonlinear systems with input constraints. IEEE Transactions on Systems, Man and Cybernetics35(1): 150–154.
7.
HuQShaoXGuoL (2018) Adaptive fault-tolerant attitude tracking control of spacecraft with prescribed performance. IEEE/ASME Trans Mechatron23(1): 331–341.
8.
HuQTanXAkellaMR, et al. (2017) Finite-time fault-tolerant spacecraft attitude control with torque saturation. Journal of Guidance, Control, and Dynamics40(10): 2524–2537.
9.
JiangBGaoZShiP, et al. (2010) Adaptive fault-tolerant tracking control of near-space vehicle using takagi-sugeno fuzzy model. IEEE Transactions on Neural Networks and Learning Systems18(5): 1000–1007.
10.
KogisoKHirataK (2009) Reference governor for constrained systems with time-varying references. Robotics and Autonomous Systems57(3): 289–295.
11.
LeeJYParkJBChoiYH (2015) Integral reinforcement learning for continuous-time input-affine nonlinear systems with simultaneous invariant explorations. IEEE Transactions on Neural Networks and Learning Systems26(5): 916–932.
12.
LiJGChenX (2017) Flying-wing UAV control design study of nonlinear method and flight validation. Journal of Harbin Institute of Technology49(9): 151–158.
13.
MaYJiangBTaoG (2015) Uncertainty decomposition-based fault-tolerant adaptive control of flexibles spacecraft. IEEE Transactions on Aerospace and Electronic Systems51(2): 1053–1068.
14.
MayneDQRawlingsJBRaoCV (2000) Constrained model predictive control: Stability and optimality. Automatica36(6): 789–814.
15.
MengYJiangBQiR (2018) Fault-tolerant anti-windup control for hypersonic vehicles in reentry based on ISMDO. Journal of the Franklin Institute355(5): 2067–2090.
16.
ModaresHLewisFJiangZ (2016) Optimal output-feedback control of unknown continuous-time linear systems using off-policy reinforcement learning. IEEE Transactions on Cybernetics46(11): 2401–2410.
17.
NgoKBMahonyRJiangZ (2005) Integrator backstepping using barrier functions for systems with multiple state constraints. In: 44th IEEE Conference on Decision and Control, Seville, Spain, 12–15 December 2005, pp: 8306–8312. Piscataway: IEEE.
18.
ShaoXHuQ, et al. (2020) Fault-tolerant prescribed performance attitude tracking control for spacecraft under input saturation. IEEE Transactions on Control Systems Technology28(2): 1–9.
19.
SongRLewisFWeiQ, et al. (2016) Off-policy actor-critic structure for optimal control of unknown systems with disturbances. IEEE Transactions on Cybernetics46(5): 1041–1050.
20.
SongYHuangX, et al. (2017) Robust adaptive fault-tolerant PID control of MIMO nonlinear systems with unknown control direction. IEEE Trans Ind Electron64: 4876–4884.
21.
SuRZongQ (2015) Comprehensive design of disturbance observer and non-singular terminal sliding mode control for reusable launch vehicles. IET Control Theory Appl9(12): 1821–1830.
22.
SunHLiS, et al. (2013) Adaptive fault-tolerant controller design for airbreathing hypersonic vehicle with input saturation. IEEE Trans Ind Electron65: 4135–4144.
23.
SunJLiuCYeQ (2017) Robust differential game guidance laws design for uncertain interceptor-target engagement via adaptive dynamic programming. International Journal of Control90(5): 990–1004.
24.
TangTQiRJiangB (2019) Adaptive nonlinear generalized predictive control for hypersonic vehicle with unknown parameters and control constraints. Journal of Aerospace Engineering233(2): 510–532.
25.
TeeKPGeSS, et al. (2011a) Control of nonlinear systems with partial state constraints using a barrier Lyapunov function. International Journal of Control84(12): 2008–2023.
26.
TeeKPRenBB, et al. (2011b) Control of nonlinear systems with time-varying output constraints. Automatica47(11): 2511–2516.
27.
VrabieDPastravanuO, et al. (2009) Adaptive optimal control for continuous-time linear systems based on policy iteration. Automatica Oxford45(2): 477–484.
28.
WangDHeHLiuD (2017) Adaptive critic nonlinear robust control: a survey. IEEE Transactions on Cybernetics47(10): 3429–3451.
29.
WangHLiJ (2011) Airfoil and wing layout design for a certain flying-wing UAV. Journal of Northwestern Polytechnical University29(5): 789–793.
30.
XuBYangCShiZ (2014) Reinforcement learning output feedback NN control using deterministic learning technique. IEEE Transactions on Neural Networks and Learning Systems25(3): 635–641.
31.
YanJLiuLTangS, et al. (2015) Reinforcement learning design-based adaptive tracking control with less learning parameters for nonlinear discrete-time MIMO systems. IEEE Transactions on Neural Networks & Learning Systems26(1): 165–176.
32.
YangQChenM (2016) Adaptive neural prescribed performance tracking control for near space vehicles with input nonlinearity. Neurocomputing174(B): 780–789.
33.
YuXFuYZhangY (2017) Aircraft fault accommodation with consideration of actuator control authority and gyro availability. IEEE Transactions on Control Systems Technology26(4): 1–15.
34.
YuXLiPZhangY (2018) The design of fixed-time observer and finite-time fault-tolerant control for hypersonic gliding vehicles. IET Control Theory Appl65: 4135–4144.
35.
YinSYangHGaoH, et al. (2017) An adaptive NN-based approach for fault-tolerant control of nonlinear time-varying delay systems with unmodeled dynamics. IEEE Trans Neural Networks Learn Systems28(8): 1902–1913.
36.
ZhangHWeiQLiuD (2011) An iterative adaptive dynamic programming method for solving a class of nonlinear zero-sum differential games. Automatica47(1): 207–214.
37.
ZhangHGZhangLuoYH (2013) An overview of research on adaptive dynamic programming. Acta Automatica Sinica39(4): 303–311.
38.
ZhaoZHYangJLiS, et al. (2017) Continuous output feedback TSM control for uncertain systems with a DC-AC inverter example. IEEE Transactions on Circuits and Systems II Express Briefs65(1): 71–75.
