A bi-level control framework with temperature rise inference and intelligent decision-making longitudinal control on downgrade hydrodynamic braking process

Abstract

Temperature rise is a salient characteristic of the braking device of heavy-duty vehicles (HDVs) during long downhill braking. The temperature-independent effect on braking performance is an inevitable challenge for the longitudinal controller, which also causes the overheating of mechanical or hydrodynamic braking devices and may bring about brake failures for heavy-duty vehicles. To this end, a universal Bi-level control framework combining a temperature hierarchy system performance prediction method and a deep reinforcement learning (DRL)-based controller is proposed for long downhill braking of heavy-duty vehicles. Firstly, the temperature-independent characteristic is clustered to predict the braking performance under various rotating speeds. Secondly, a data-driven model for temperature rising is built for the long-time braking thermal prediction and estimates the safety remaining braking time using the auxiliary braking. Thirdly, a temperature-hierarchy environmental perceptive control framework with Double Deep Q Network (DDQN) algorithm is exploited to achieve the target speed tracking accuracy. Thermal safety is ensured with the application of fast calculating for the thermal rising, along with the effective estimation of remaining braking performance on endurance braking. The proposed Bi-level longitudinal controller is compared with the average-temperature strategy and the classic PID strategy to validate its superiority in terms of speed-tracking accuracy on robust conditions. The simulation results show that the proposed strategy improves the speed tracking accuracy by 37.31% on constant slope conditions and 68.11% on varying slope conditions compared with the classic PID strategy. Furthermore, a processor-in-the-loop test experiment verifies its real-time application.

Keywords

Bi-level control longitudinal control long-time braking safety deep reinforcement learning double deep Q network

Get full access to this article

View all access options for this article.

References

Xiong

Yang

Leng

, et al. Integrated longitudinal and lateral control for autonomous vehicles with active load transfer strategy at the handling limits. Proc IMechE, Part D: J Automobile Engineering 2021; 235: 961–974.

, et al. Optimization on parallel control of PID and fuzzy in constant-speed braking process with a hydrodynamic retarder. In: 2019 IEEE 8th international conference on fluid power and mechatronics (FPM), 2019, pp.1175–1188. New York: IEEE.

Le Gigan

Vernersson

Lundén

, et al. Disc brakes for heavy vehicles: an experimental study of temperatures and cracks. Proc IMechE, Part D: J Automobile Engineering 2015; 229: 684–707.

Tretsiak

Experimental investigation of the brake system’s efficiency for commercial vehicles equipped with disc brakes. Proc IMechE, Part D: J Automobile Engineering2012; 226: 725–739.

Guo

Yang

, et al. Hydraulic retarder torque control for heavy duty vehicle longitudinal control. Int J Heavy Veh Syst 2019; 26: 854–871.

Lei

Song

Zheng

, et al. Application of fuzzy logic in constant speed control of hydraulic retarder. Adv Mech Eng 2017; 9: 1687814017690956.

Pei

Safe reinforcement learning for autonomous vehicle using Monte Carlo tree search. IEEE Trans Intell Transp Syst 2022; 23: 6766–6773.

Cheng

Guo

H-Q

, et al. Longitudinal collision avoidance and lateral stability adaptive control system based on MPC of autonomous vehicles. IEEE Trans Intell Transp Syst 2020; 21: 2376–2385.

You

Ding

, et al. Trajectory tracking control for WMRs with the time-varying longitudinal slippage based on a new adaptive SMC method. Int J Aerosp Eng 2019; 2019: 1–13.

10.

Zhao

, et al. Towards longitudinal and lateral coupling control of autonomous vehicles using offset free MPC. Control Eng Pract 2022; 121: 105074.

11.

Guo

Pei

, et al. Path tracking control based on tube MPC and time delay motion prediction. IET Intell Transp Syst 2020; 14: 1–12.

12.

, et al. A decision-making strategy for vehicle autonomous braking in emergency via deep reinforcement learning. IEEE Trans Vehicular Technol 2020; 69: 5876–5888.

13.

Guo

Yang

, et al. Heavy-duty vehicle longitudinal automation with hydraulic retarder via H infinity control and off-policy reinforcement learning. Int J Heavy Veh Syst 2020; 82: 97–119.

14.

Hasselt

. Double Q-learning. In: Advances in neural information processing systems 23 (NIPS 2010), 2010, pp.2613–2621. Curran Associates, Inc.

15.

Van Hasselt

Guez

Silver

. Deep reinforcement learning with double Q-learning. In: Proceedings of the thirtieth AAAI conference on artificial intelligence, 2016, pp.2094–2100. AAAI Press.

16.

Zhang

Sun

Yin

, et al. Human-like autonomous vehicle speed control by deep reinforcement learning with double Q-learning. In: 2018 IEEE intelligent vehicles symposium (IV), 2018, pp.1251–1256. New York: IEEE.

17.

Puccetti

Köpf

Rathgeber

, et al. Speed tracking control using online reinforcement learning in a real car. In: 6th international conference on control, automation and robotics, 2020, pp.392-399. New York: IEEE.

18.

Tian

Study on braking characteristics of a novel eddy current-hydraulic hybrid retarder for heavy-duty vehicles. IEEE Trans Energy Convers 2020; 35: 1658–1666.

19.

Liu

, et al. Analysis of unsteady rotor–stator flow with variable viscosity based on experiments and CFD simulations. Numer Heat Transf A Appl 2015; 68: 1351–1368.

20.

Shen

Qiu

, et al. Investigation on the dynamic influence of thermophysical properties of transmission medium on the internal flow field for hydraulic retarder. Int J Heat Mass Transf 2018; 126: 1367–1376.

21.

Ham

Y-G

Kim

J-H

Luo

J-J.

Deep learning for multi-year ENSO forecasts. Nature 2019; 573: 568–572.

22.

Hartigan

Wong

MA.

Algorithm AS 136: a K-means clustering algorithm. Appl Stat 1979; 28: 100–108.

23.

Khan

Ahmed

Roy

Temporal clustering based thermal condition monitoring in building. Sustain Comput Inform Syst 2021; 29: 100441.

24.

Siddiqui

Morales-Menendez

Gupta

, et al. Correlation between temperature and COVID-19 (suspected, confirmed and death) cases based on machine learning analysis. J Pure Appl Microbiol 2020; 14: 1017–1024.

25.

Lin

Peng

Lai

, et al. Short-term power prediction for photovoltaic power plants using a hybrid improved k means-GRA-Elman model based on multivariate meteorological factors and historical power datasets. Energy Convers Manag 2018; 177: 704–717.

26.

Liu

A bi-level control for energy efficiency improvement of a hybrid tracked vehicle. IEEE Trans Ind Inform 2018; 14: 1616–1625.

27.

Yuan

Yang

Research on K-value selection method of K-means clustering algorithm. Multidiscip Sci J 2019; 2: 226–235.

28.

Shutaywi

Kachouie

NN.

Silhouette analysis for performance evaluation in machine learning with applications to clustering. Entropy 2021; 23: 759.

29.

Benesty

Chen

Huang

, et al. Pearson correlation coefficient. In: Cohen

Huang

Chen

, et al (eds) Noise reduction in speech processing. Berlin, Heidelberg: Springer, 2009, pp.1–4.

30.

Watkins

CJCH

. Learning from delayed rewards. PhD Thesis, University of Cambridge, England, 1989.

31.

Russakovsky

Deng

, et al. Imagenet large scale visual recognition challenge. Int J Comput Vis 2015; 115: 211–252.

32.

Anitha

Patil

MM.

RFM model for customer purchase behavior using K-means algorithm. J King Saud Univ Comput Inf Sci 2022; 34: 1785–1792.

33.

Tang

Chen

Yang

, et al. Visual detection and deep reinforcement learning-based car following and energy management for hybrid electric vehicles. IEEE Trans Transp Electrification 2022; 8: 2501–2515.