A longitudinal platoon control method based on Twin Delayed Deep Deterministic Policy Gradient (TD3) and Model Predictive Control (MPC) is proposed to solve the problems of low following efficiency and system instability in longitudinal platoon control. Firstly, Dynamic Bayesian Network (DBN) and Long Short-Term Memory (LSTM) network are introduced to identify the driving behavior of bystanders and derive the MPC objective constraint function containing three indicators of following, comfort and fuel consumption according to the platoon dynamics equation. Secondly, the system prediction model and cost function are introduced into the action-critic network of TD3 to solve the problem of no model training in the traditional TD3 algorithm and to speed up the training speed and accuracy of the network. On this basis, a Bellman equation is proposed to calculate the time-domain difference error and the expected loss function to solve the TD3 network overestimation problem. Finally, the joint simulation platform is built to simulate the platoon driving conditions and compare with the DDPG optimization algorithm and the traditional MPC algorithm respectively. The results show that the improved TD3-MPC algorithm satisfies the constraints and the spacing error is controlled within 0.3 m, and the vehicle speed changes more smoothly in the scenario of speed fluctuation in the front vehicle, and the ride comfort of the platoon is improved, and it has better robustness in the scenario of vehicle cut-in. The experimental results show that when the vehicle speeds are 20, 40, and 60 km/h, compared to MPC, the average spacing errors are reduced by 27.56%, 25.64%, and 28.04%, respectively, and compared to DDPG-MPC, the average spacing errors are reduced by 6.59%, 8.77%, and 7.86%, respectively.
ZhuLTangYYangD.Cellular automata-based modeling and simulation of the mixed traffic flow of vehicle platoon and normal vehicles. Phys A Stat Mech Appl2021; 584: 126368.
2.
ZhenhaiGJunWHongyuH, et al. Multi-argument control mode switching strategy for adaptive cruise control system. Proced Eng2016; 137: 581–589. DOI: 10.1016/j.proeng.2016.01.295.
3.
TsaiCCHsiehSMChenCT.Fuzzy Longitudinal controller design and experimentation for adaptive cruise control and Stop&Go. J Intell Rob Sys2010; 59: 167–189. DOI: 10.1007/s10846-010-9393-z.
4.
PansePSinghASatsangiC.Adaptive Cruise Control using Fuzzy Logic. Int J Dig Appl Contemp Res2015; 3: 7.
5.
SamaniBShamekhiAH. Multi-objective adaptive cruise controller design using nonlinear predictive controller with the objective function with variable weights determined by fuzzy logic controller. Proc Inst Mech Eng D: J Automob Eng2021; 236: 142–154. DOI: 10.1177/09544070211014783.
6.
AmmourMOrjuelaRBassetM. A MPC combined decision making and trajectory planning for autonomous vehicle collision avoidance. IEEE Trans Intell Transp Syst2022; 23: 24805–24817. DOI: 10.1109/TITS.2022.3210276.
7.
HuangJWWZouD. Research on multi-mode switching strategy of adaptive cruise control system based on personalized spacing strategy. Autom Eng2020; 42: 1302–1311. DOI: 10.19562/j.chinasae.qcgc.2020.10.002.
8.
LiuWJDingHFGeMF, et al. Cooperative control for platoon generation of vehicle-to-vehicle networks: a hierarchical nonlinear MPC algorithm. Nonlin Dyn2022; 108: 3561-3578. DOI: 10.1007/s11071-022-07400-y.
9.
ZhuCH.Strategy for vehicle adaptive cruise control considering the reaction headway. J Mech Eng2017; 53: 144–150. DOI: 10.3901/JME.2017.24.144.
10.
GaoHSuHCaiY, et al. Trajectory prediction of cyclist based on dynamic Bayesian network and long short-term memory model at unsignalized intersections. Sci China Inform Sci2021; 64: 172207. DOI: 10.1007/s11432-020-3071-8.
11.
LiPPeiXChenZ, et al. Human-like motion planning of autonomous vehicle based on probabilistic trajectory prediction. Appl Soft Comp2022; 118: 108499. DOI: https://doi.org/10.1016/j.asoc.2022.108499.
12.
SunYChuYXuT, et al. Inverse reinforcement learning based: segmented lane-change trajectory planning with consideration of interactive driving intention. IEEE Transact Vehi Technol2022; 71: 11395–11407. DOI: 10.1109/TVT.2022.3193220.
13.
LiaoYYuGChenP, et al. Modelling personalised car-following behaviour: a memory-based deep reinforcement learning approach. Transport A: Transport Sci2022; 3: 1–29. DOI: 10.1080/23249935.2022.2035846.
14.
JinghuaZHYugongLandKeqiangL.Vehicle cut-in trajectory prediction based on deep learning in a human-machine mixed driving environment. Autom Eng2022; 44: 153–160. DOI: 10.19562/j.chinasae.qcgc.2022.02.001.
15.
YuS-MWuS-NZhaoY-B, et al. Delayed feedback MPC algorithms of vehicle platoons subject to constraints on measurement range and driving behaviors. Asian J Con2018; 20: 2260–2270. DOI: 10.1002/asjc.1761.
16.
LinYMcPheeJAzadNL.Comparison of deep reinforcement learning and model predictive control for adaptive cruise control. IEEE Transact o Intell Veh2021; 6: 221–231. DOI: 10.1109/TIV.2020.3012947.
17.
ZhuMWangYPuZ, et al. Safe, efficient, and comfortable velocity control based on reinforcement learning for autonomous driving. Transport Res Part C Emerg Technol2020; 117: 102662. DOI: 10.1016/j.trc.2020.102662.
18.
HuMXuBQinH, et al. MPC based longitudinal coordinated collision avoidance for multiple connected vehicles. Qinghua Daxue Xuebao/J Tsinghua Univ2017; 57: 1280–1286. DOI: 10.16511/j.cnki.qhdxxb.2017.21.032.
19.
YingfengCLZXiaoqiangSHaiW, et al. An adaptive cruise control scheme based on merging behavior recognition. Autom Eng2021; 43: 1077–1087. DOI: 10.19562/j.chinasae.qcgc.2021.07.015.
20.
GaoHZhuJZhangT, et al. Situational assessment for intelligent vehicles based on stochastic model and gaussian distributions in typical traffic scenarios. IEEE Transact Sys Man Cyber Sys2022; 52: 1426-1436. DOI: 10.1109/TSMC.2020.3019512.
21.
KooijJFPFlohrFPoolEAI, et al. Context-based path prediction for targets with switching dynamics. Int J Comp Vis2019; 127: 239–262. DOI: 10.1007/s11263-018-1104-4.
22.
WangJZhangZLuG.A Bayesian inference based adaptive lane change prediction model. Transport Res Part C Emerg Technol2021; 132: 103363. DOI: https://doi.org/10.1016/j.trc.2021.103363.
23.
ZhuJChenJHuW, et al. Big Learning with Bayesian methods. Nat Sci Rev2017; 4: 627-651. DOI: 10.1093/nsr/nwx044.
24.
MunirUJunzhiZ. Weight changing model predictive controller for adaptive cruise control with multiple objectives. In: 2018 IEEE international conference on mechatronics, robotics and automation (ICMRA)18–21 May 20182018, pp. 62–66.
25.
ZhuM.Strategy for vehicle adaptive cruise control considering the reaction headway. J Mech Eng2017; 53: 144. DOI: 10.3901/JME.2017.24.144.
26.
Ju-huaHHan-pengZMing-chunL.Research on the longitudinal tracking of adaptive cruise control system for vehicles considering of measurement noise. Transact Beijing Inst Technol2020; 40: 254. DOI: 10.15918/j.tbit1001-0645.2018.410.
27.
GuoLGePSunD, et al. Adaptive cruise control based on model predictive control with constraints softening. Appl Sci2020; 10: 1635.
28.
LEI LiliZT. Longitudinal following control of intelligent vehicle fleet based on fuzzy MPC. J Jiangsu Univ (Nat Sci Ed)2022; 43: 394–399. DOI: 10.3969/j.issn.1671-7775.2022.04.004.
29.
Rios-TorresJMalikopoulosAA.Automated and cooperative vehicle merging at highway on-ramps. IEEE Transact Intell Transport Sys2017; 18: 780–789. DOI: 10.1109/TITS.2016.2587582.
30.
BasiriMHGhojoghBAzadNL, et al. Distributed nonlinear model predictive control and metric learning for heterogeneous vehicle platooning with cut-in/cut-out maneuvers. In: 2020 59th IEEE conference on decision and control (CDC)14–18 Dec. 20202020, pp. 2849–2856.
31.
HeDShiYSongX.Weight-free multi-objective predictive cruise control of autonomous vehicles in integrated perturbation analysis and sequential quadratic programming optimization framework. J Dyn Sys Measur, Cont2019; 141: 3270. DOI: 10.1115/1.4043270.
32.
JiaCHeHZhouJ, et al. Learning-based model predictive energy management for fuel cell hybrid electric bus with health-aware control. Appl Ene2024; 355: 122228. DOI: 10.1016/j.apenergy.2023.122228.
33.
HeRLvHZhangS, et al. Lane following method based on improved DDPG algorithm. Sensors2021; 21 2021/07/25. DOI: 10.3390/s21144827.
34.
HanSZhouWLüS, et al. Regularly updated deterministic policy gradient algorithm. Knowl Based Sys2021; 214: 106736. DOI: 10.1016/j.knosys.2020.106736.
35.
Rodriguez-RamosASampedroCBavleH, et al. A deep reinforcement learning strategy for UAV autonomous landing on a moving platform. J Intell Robot2019; 93: 351–366. DOI: 10.1007/s10846-018-0891-8.
36.
JiaCZhouJHeH, et al. Health-conscious deep reinforcement learning energy management for fuel cell buses integrating environmental and look-ahead road information. Energy2024; 290: 130146. DOI: 10.1016/j.energy.2023.130146.
37.
FuCZhangY.Research and application of predictive control method based on deep reinforcement learning for HVAC systems. IEEE Acc2021; 9: 130845–130852. DOI: 10.1109/ACCESS.2021.3114161.
38.
JinWFuQChenJ, et al. A novel building energy consumption prediction method using deep reinforcement learning with consideration of fluctuation points. J Build Eng2022: 105458. DOI: 10.1016/j.jobe.2022.105458.
39.
Zhen-fuCShuo-jieMBoY, et al. Lane changing of autonomous vehicle based on TD3 algorithm in human-machine hybrid driving environment. China J Highway Trans2021; 34: 246–254. DOI: 10.19721/j.cnki.1001-7372.2021.11.020.
40.
JiaCLiKHeH, et al. Health-aware energy management strategy for fuel cell hybrid bus considering air-conditioning control based on TD3 algorithm. Energy2023; 283: 128462. DOI: 10.1016/j.energy.2023.128462.
41.
MaCGaoJChenJ, et al. Development of adaptive cruising strategy of extended range electric vehicle based on driving behavior. Proceed Inst Mech Eng Part D J Automob Eng; 0: 09544070231179853. DOI: 10.1177/09544070231179853.
