In the electric-pneumatic braking system (EPBS), fast and accurate brake pressure regulation is critical to vehicle braking safety and is the basis for active safety functions. However, the lack of signal feedback, limited actuator response accuracy, and extremely strong model stiffness and nonlinearity pose problems for high-precision brake pressure regulation. To solve these problems, this article proposes a Q-learning-based control algorithm to regulate actuator instructions. First, the nonlinearity of the system and complicated actuator operating process is settled by a simplified mathematical model. Then, a decoupled pressure observing method is proposed to solve the observation problem caused by the coupling of mechanical and fluid motion. Finally, this paper proposes the idea of using an optimization method to solve the control problem caused by the response speed of the actuator, a Q-learning algorithm is used to settle the solenoid switching action to minimize response time and steady-state error. Both simulation and practical experiments are conducted to demonstrate the dependability of the model and effectiveness of the control method.
LengBXiongLYuZ, et al. Robust variable structure anti-slip control method of a distributed drive electric vehicle. IEEE Access2020; 8: 162196–162208.
2.
MorrisonGCebonD.Sideslip estimation for articulated heavy vehicles at the limits of adhesion. Veh Syst Dyn2016; 54(11): 1601–1628.
3.
JinXYuZYinG, et al. Improving vehicle handling stability based on combined AFS and DYC system via robust Takagi-Sugeno fuzzy control. IEEE Trans Intell Transp Syst2018; 19(8): 2696–2707.
4.
WuJWangXLiL, et al. Hierarchical control strategy with battery aging consideration for hybrid electric vehicle regenerative braking control. Energy2018; 145: 301–312.
5.
ZhangJZChenXZhangPJ.Integrated control of braking energy regeneration and pneumatic anti-lock braking. Proc IMechE Part D: J Automobile Engineering2010; 224(5): 587–610.
6.
ChenLChenTXuX, et al. Multi-objective coordination control strategy of distributed drive electric vehicle by orientated tire force distribution method. IEEE Access2018; 6: 69559–69574.
7.
LaghroucheSAhmedFSMehmoodA.Pressure and friction observer-based backstepping control for a VGT pneumatic actuator. IEEE Trans Control Syst Technol2014; 22(2): 456–467.
8.
TurksevenMUedaJ.Model-based force control of pneumatic actuators with long transmission lines. IEEE/ASME Trans Mechatron2018; 23(3): 1292–1302.
9.
JiaWYanJ.Pressure drop characteristics and minimum pressure drop velocity for pneumatic conveying of polyacrylamide in a horizontal pipe with bends at both ends. Powder Technol2020; 372: 192–203.
10.
TagamiTMiyazakiTKawaseT, et al. Pressure control of a pneumatic artificial muscle including pneumatic circuit model. IEEE Access2020; 8: 60526–60538.
11.
PandianSRTakemuraFHayakawaY, et al. Pressure observer-controller design for pneumatic cylinder actuators. IEEE/ASME Trans Mechatron2002; 7(4): 490–499.
12.
ChenSYGongSS.Speed tracking control of pneumatic motor servo systems using observation-based adaptive dynamic sliding-mode control. Mech Syst Signal Process2017; 94: 111–128.
13.
WuXLiLWangX, et al. Nonlinear controller design and testing for chatter suppression in an electric-pneumatic braking system with parametric variation. Mech Syst Signal Process2020; 135: 106401.
14.
YuanYWangZYuY, et al. Active disturbance rejection control for a pneumatic motion platform subject to actuator saturation: an extended state observer approach. Automatica2019; 107: 353–361.
15.
ZhaoLZhangBYangH, et al. Optimized linear active disturbance rejection control for pneumatic servo systems via least squares support vector machine. Neurocomputing2017; 242: 178–186.
16.
HanWXiongLYuZ.Interconnected pressure estimation and double closed-loop cascade control for an integrated electrohydraulic brake system. IEEE/ASME Trans Mechatron2020; 25(5): 2460–2471.
17.
GulatiNBarthEJ.A globally stable, load-independent pressure observer for the servo control of pneumatic actuators. IEEE/ASME Trans Mechatron2009; 14(3): 295–306.
18.
GulatiNBarthEJ.Non-linear pressure observer design for pneumatic actuators. In: Proceedings of the 2005 IEEE/ASME international conference on advanced intelligent mechatronicsMonterey, CA, 2005, pp.783–788. New York, NY: IEEE. DOI: 10.1109/AIM.2005. 1511078.
19.
LiuYTKungTTChangKM, et al. Observer-based adaptive sliding mode control for pneumatic servo system. Precis Eng2013; 37(3): 522–530.
20.
AbryFBrunXDi LoretoM, et al. Piston position estimation for an electro-pneumatic actuator at standstill. Control Eng Pract2015; 41: 176–185.
21.
XuCZhaoWLiL, et al. A nash Q-learning based motion decision algorithm with considering interaction to traffic participants. IEEE Trans Vehicular Technol2020; 69(11): 12621–12634.
22.
ZhangWGaiJZhangZ, et al. Double-DQN based path smoothing and tracking control method for robotic vehicle navigation. Comput Electron Agric2019; 166: 104985.
MnihVKavukcuogluKSilverD, et al. Playing Atari with deep reinforcement learning. arXiv:13125602 [cs] 2013; Comment: NIPS Deep Learning Workshop 2013, 1312. 5602.
25.
TaiLLiuM.A robot exploration strategy based on Q-learning network. In: 2016 IEEE international conference on real-time computing and robotics (RCAR)Angkor Wat, 2016, pp.57–62.: New York, NY: IEEE. DOI: 10.1109/RCAR.2016.7784001.
26.
DuanJShiDDiaoR, et al. Deep-reinforcement-learning-based autonomous voltage control for power grid operations. IEEE Trans Power Syst2020; 35(1): 814–817.
27.
van HasseltHGuezASilverD.Deep reinforcement learning with double Q-learning. In: Proceedings of the AAAI conference on artificial intelligence, 2016, vol. 30, no. 1, p.7.
28.
WangZSchaulTHesselM, et al. Dueling network architectures for deep reinforcement learning. In: International conference on machine learning, PMLR, pp.1995–2003.
29.
FanJWangZXieY, et al. A theoretical analysis of deep Q-learning. arXiv:190100137 [cs, math, stat] 2020; Comment: 65 pages, 1901.00137.
30.
XuCZhaoWWangC.An integrated threat assessment algorithm for decision-making of autonomous driving vehicles. IEEE Trans Intell Transp Syst2020; 21(6): 2510–2521.
31.
GongLWangQHuC, et al. Switching control of morphing aircraft based on Q-learning. Chin J Aeronaut2020; 33(2): 672–687.
32.
HanXHeHWuJ, et al. Energy management based on reinforcement learning with double deep Q-learning for a hybrid electric tracked vehicle. Appl Energy2019; 254: 113708.
33.
JouffeL.Fuzzy inference system learning by reinforcement methods. IEEE Trans Syst Man Cybernetics, Part C (Appl Rev)1998; 28(3): 338–355.
34.
HuZWanKGaoX, et al. A dynamic adjusting reward function method for deep reinforcement learning with adjustable parameters. Math Probl Eng2019; 2019: 1–10.
