Vehicle steering control is crucial to autonomous vehicles. However, unknown parameters and uncertainties of vehicle steering systems bring a great challenge to its control performance, which needs to be tackled urgently. Therefore, this paper proposes a novel model free controller based on reinforcement learning for active steering system with unknown parameters. The model of the active steering system and the Brushless Direct Current (BLDC) motor is built to construct a virtual object in simulations. The agent based on Deep Deterministic Policy Gradient (DDPG) algorithm is built, including actor network and critic network. The rewards from environment are designed to improve the effectiveness of agent. Simulations and testbench experiments are implemented to train the agent and verify the effectiveness of the controller. Results show that the proposed algorithm can acquire the network parameters and achieve effective control performance without any prior knowledges or models. The proposed agent can adapt to different vehicles or active steering systems easily and effectively with only retraining of the network parameters.
FagnantDJKockelmanK. Preparing a nation for autonomous vehicles: opportunities, barriers and policy recommendations. Transp Res Part A Policy Pract2015; 77: 167–181.
2.
ZhaoWZLinYShiWG. Control strategy of a novel electric power steering system integrated with active front steering function. Sci China Technol Sci2011; 54(6): 1515–1520.
3.
SongJ. Active front wheel steering model and controller for integrated dynamics control systems. Int J Automot Technol2016; 17(2): 265–272.
4.
WangRZhangHWangJ. Linear parameter-varying controller design for four-wheel independently actuated electric ground vehicles with active steering systems. IEEE Trans Control Syst Techno2014; 22(4): 1281–1296.
5.
XuYHAhmadianM. Study on the performance of active front steering system. Int J Automot Technol2013; 14(4): 595–603.
6.
WanzhongZYijunLIChunyanW. Robust control of hand wheel torque for active front steering system. Sci China Technol Sci2015; 58(1): 107–116.
7.
MinakiRHoriY. Experimental verification of driver-friendly reactive torque control based on driver sensitivity to active front steering. In: 2009 35th annual conference of IEEE industrial electronics, Porto, Portugal, 3–5 November 2009, pp.3077–3082. IEEE.
8.
NohTKimJChoiJ, et al. A control strategy to compensate the reaction torque of active front steering system. SAE paper 2007-01-3659, 2007.
9.
TanakaYYamadaNTsujiT, et al. Vehicle active steering control system based on human mechanical impedance properties of the arms. IEEE Trans Intell Transp Syst2014; 15(4): 1758–1769.
10.
BernardiniyDCairanozSDBemporadA, et al. Drive-by-wire vehicle stabilization and yaw regulation: a hybrid model predictive control design. In: IEEE conference on decision & control, Shanghai, China, 15–18 December 2009, pp.7621–7626. IEEE.
11.
KadirZAMazlanSAZamzuriH, et al. Adaptive fuzzy-pi control for active front steering system of armoured vehicles: outer loop control design for firing on the move system. Stroj Vestn J Mech Eng2015; 61(3): 187–195.
12.
ZhengBAnwarS. Yaw stability control of a steer-by-wire equipped vehicle via active front wheel steering. Mechatronics2009; 19(6): 799–804.
13.
CairanoSDTsengHE. Driver-assist steering by active front steering and differential braking: design, implementation and experimental evaluation of a switched model predictive control approach. In: 49th IEEE conference on decision and control (CDC), Atlanta, GA, 15–17 December 2010, pp.2886–2891. IEEE.
14.
Di CairanoSTsengHEBernardiniD, et al. Vehicle yaw stability control by coordinated active front steering and differential braking in the tire sideslip angles domain. IEEE Trans Control Syst Technol2013; 21(4): 1236–1248.
15.
FalconePBorrelliFTsengHE, et al. Linear time-varying model predictive control and its application to active steering systems: stability analysis and experimental validation. Int J Robust Nonlinear Control2010; 18(8): 862–875.
16.
LiuJGaoLZhangJ, et al. Super-twisting algorithm second-order sliding mode control for collision avoidance system based on active front steering and direct yaw moment control. Proc IMechE, Part D: J Automobile Engineering2020; 25(1): 43–54.
17.
FarazandehAAhmedARakhejaS. An independently controllable active steering system for maximizing the handling performance limits of road vehicles. Proc IMechE, Part D: J Automobile Engineering2014; 229(10): 1291–1309.
18.
ShimTAdireddyGYuanH. Autonomous vehicle collision avoidance system using path planning and model-predictive-control-based active front steering and wheel torque control. Proc IMechE, Part D: J Automobile Engineering2012; 226(6): 767–778.
19.
FalconePBorrelliFAsgariJ, et al. Predictive active steering control for autonomous vehicle systems. IEEE Trans Control Syst Technol2007; 15(3): 566–580.
20.
FukaoTMiyasakaSMoriK, et al. Active steering systems based on model reference adaptive nonlinear control. Veh Syst Dyn2004; 42(5): 301–318.
21.
ZhangJYKimJWLeeKB, et al. Development of an active front steering (afs) system with qft control. Int J Automot Technol2008; 9(6): 695–702.
22.
JianWUYouqunZXuewuJI, et al. Generalized internal model robust control for active front steering intervention. Chin J Mech Eng2015; 28(2): 285–293.
23.
WuJLiuYWangF, et al. Vehicle active steering control research based on two-DOF robust internal model control. Chin J Mech Eng2016; (4): 739–746.
24.
JiXWuJZhaoY, et al. A new robust control method for active front steering considering the intention of the driver. Proc IMechE, Part D: J Automobile Engineering2015; 229(4): 518–531.
25.
ElmiNOhadiASamadiB. Active front-steering control of a sport utility vehicle using a robust linear quadratic regulator method, with emphasis on the roll dynamics. Proc IMechE, Part D: J Automobile Engineering2013; 227(12): 1636–1649.
26.
MnihVKavukcuogluKSilverD, et al. Human-level control through deep reinforcement learning. Nature2015; 518(7540): 529.
27.
SilverDHubertTSchrittwieserJ, et al. Mastering chess and shogi by self-play with a general reinforcement learning algorithm. arXiv:1712.01815, 2017.
28.
GrigorescuSTrasneaBCociasT, et al. A survey of deep learning techniques for autonomous driving. J Rob Syst2020; 37(3): 362–386.
29.
JuangCFHsuCH. Reinforcement ant optimized fuzzy controller for mobile-robot wall-following control. IEEE Trans Ind Electron2009; 56(10): 3931–3940.
30.
RiedmillerMMontemerloMDahlkampH. Learning to drive a real car in 20 minutes. In: Frontiers in the convergence of bioscience & information technologies, Jeju City, South Korea, 11–13 Octobet 2007, pp.645–650. IEEE.
31.
YangFWangPWangXH. Continuous control in car simulator with deep reinforcement learning. In: Proceedings of the 2018 2nd international conference on computer science and artificial intelligenceShenzhen, China, 8 December 2018, pp.566–570. New York: Association for computing Machinery.
32.
SuttonRSMcallesterDSinghS, et al. Policy gradient methods for reinforcement learning with function approximation. Adv Neural Inf Process Syst1999; 99: 1057–1063.
33.
SilverDLeverGHeessN, et al. Deterministic policy gradient algorithms. In: Proceedings of The 31st international conference on machine learningBeijing, China, 27 January 2014, pp.387–395. PMLR.
34.
LillicrapTPHuntJJPritzelA, et al. Continuous control with deep reinforcement learning. arXiv:1509.02971, 2016.
