Parallel parking is the most common form of parking, but it is difficult to park vehicles safely in complex traffic scenes. The paper proposed an automatic parallel parking control strategy based on Q-learning. The convergent Q-table was obtained through offline training to ensure that the automatic parking control system could obtain the optimal parking action during the automatic parking process. Based on the parking kinematics model, the virtual parking scene and the parallel parking control strategy based on fuzzy control were established. The comparative analysis of the two control strategies showed that the parallel automatic parking control strategy based on a Q-learning algorithm could deal with more complex traffic scenes. Finally, the automatic parallel parking hardware-in-the-loop test system was built based on a purely electric drive platform, and the effectiveness of the automatic parking control strategy was further verified through the hardware-in-the-loop test.
KhalidMWangKAslamN, et al. From smart parking towards autonomous valet parking: a survey, challenges and future works. J Netw Comput Appl2021; 175: 102935.
2.
KimYKimBK.Time-optimal trajectory planning based on dynamics for differential-wheeled mobile robots with a geometric corridor. IEEE Trans Ind Electron2017; 64(7): 5502–5512.
3.
YangCLiZCuiR, et al. Neural network-based motion control of an underactuated wheeled inverted pendulum model. IEEE Trans Neural Netw Learn Syst2014; 25(11): 2004–2016.
4.
ChaiRTsourdosASavvarisA, et al. Design and implementation of deep neural network-based control for automatic parking maneuver process. IEEE Trans Neural Netw Learn Syst2022; 33: 1400–1413.
5.
ChoiSBoussardCd’Andréa-NovelB.Easy path planning and robust control for automatic parallel parking. IFAC Proc Volumes2011; 44(1): 656–661.
6.
VorobievaHGlaserSMinoiu-EnacheN, et al. Automatic parallel parking in tiny spots: path planning and control. IEEE Trans Intell Transp Syst2015; 16(1): 396–410.
7.
WangXShiHZhangC.Path planning for intelligent parking system based on improved ant colony optimization. IEEE Access2020; 8: 65267–65273.
8.
LiJChengHGuoH, et al. Survey on artificial intelligence for vehicles. Automot Innov2018; 1: 2–14.
9.
KimHMXuXJungS.A reinforcement learning technique for autonomous back driving control of a vehicle. In: International conference on robot intelligence technology & applications, Daejeon, Korea, 13–15 December 2017, pp.185–193. Berlin: Springer.
10.
WangZShaoQWangC, et al. 2018. Automatic parking trajectory planning based on recurrent neural network. In: 9th International conference on software engineering and service science (ICSESS), Beijing, China, 23–25 November 2018, pp.1–4. New York: IEEE.
11.
YeHJiangHMaS, et al. Linear model predictive control of automatic parking path tracking with soft constraints. Int J Adv Robot Syst2019; 16(3): 1009–1028.
12.
MuCNiZSunC, et al. Air-breathing hypersonic vehicle tracking control based on adaptive dynamic programming. IEEE Trans Neural Netw Learn Syst2017; 28(3): 584–598.
13.
GuiHde RuiterAHJ. Adaptive fault-tolerant spacecraft pose tracking with control allocation. IEEE Trans Control Syst Technol2019; 27(2): 479–494.
14.
WangNSuSFPanX, et al. Yaw-guided trajectory tracking control of an asymmetric underactuated surface vehicle. IEEE Trans Ind Inform2019; 15(6): 3502–3513.
15.
ShiJWuJZhuB, et al. Design of automatic parallel parking system based on multi-point preview theory. SAE technical paper 2018-01-0604, 2018.
16.
MaravallDPatricioMÁde LopeJ.Automatic car parking: a reinforcement learning approach. In: International work-conference on artificial neural networks, Menorca, Spain, 3–6 June 2003, paper no. 2686, pp.214–221. Berlin: Springer.
17.
TongWHussainABoWX, et al. Artificial intelligence for vehicle-to-everything: a survey. IEEE Access2019; 7: 10823–10843.
18.
NowakowskiPSzwarcKBoryczkaU.Combining an artificial intelligence algorithm and a novel vehicle for sustainable e-waste collection. Sci Total Environ2020; 730: 730.
19.
YangLGaoHDuanH.Research on intelligent diagnosis method of electric vehicle charging fault based on artificial intelligence expert system. In: Journal of Physics: conference series, 2021 4th International conference on advanced algorithms and control engineering (ICAACE 2021,1848), Sanya, China, 29–31 January 2021, paper no. 012125, pp.1–6. Bristol: IOP.
20.
GamalOImranMRothH, et al. Assistive parking systems knowledge transfer to end-to-end deep learning for autonomous parking. In: 6th International conference on mechatronics and robotics engineering (ICMRE), Barcelona, Spain, 12–15 February 2020, Paper no. 9065014, pp.216–221. New York: IEEE.
21.
CaneseLCardarilliGCDi NunzioL, et al. Multi-agent reinforcement learning: a review of challenges and applications. Appl Sci2021; 11(11): 4948.
22.
LuoWSunPZhongF, et al. End-to-end active object tracking and its real-world deployment via reinforcement learning. IEEE Trans Pattern Anal Mach Intell2020; 42(6): 1317–1332.
23.
ArwaEOFollyKA.Reinforcement learning techniques for optimal power control in grid-connected microgrids: a comprehensive review. IEEE Access2020; 8: 208992–209007.
24.
SilverDHubertTSchrittwieserJ, et al. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play. Science2018; 362(6419): 1140–1144.
25.
ErickAOFollyKA.Reinforcement learning approaches to power management in grid-tied microgrids: a review. In: 2020 Clemson University power systems conference (PSC) Clemson SC10–13 March 2020, Paper no. 9131138, pp.1–6. New York: IEEE.
26.
JangBKimMHarerimanaG, et al. Q-learning algorithms: a comprehensive classification and applications. IEEE Access2019; 7: 133653–133667.
27.
GuoHZhaoFGuoH, et al. Self-learning energy management for plug-in hybrid electric bus considering expert experience and generalization performance. Int J Energy Res2020; 44: 5659–5674.
28.
LiuCMurpheyYL.Optimal power management based on Q-learning and neuro-dynamic programming for plug-in hybrid electric vehicles. IEEE Trans Neural Netw Learn Syst2020; 31(6): 1942–1954.
29.
DarapaneniNBasuASavlaS, et al. Automated portfolio rebalancing using Q-learning. In: 11th IEEE annual ubiquitous computing, electronics mobile communication conference (UEMCON), New York NY28–31 October 2020, Paper no. 9298035, pp.0596–0602. New York: IEEE.
30.
ChakoleJBKolheMSMahapurushGD, et al. A Q-learning agent for automated trading in equity stock markets. Expert Syst Appl2021; 163: 11763.
31.
LuXXiaoXXiaoL, et al. Reinforcement learning-based microgrid energy trading with a reduced power plant schedule. IEEE Internet Things J2019; 6(6): 10728–10737.
32.
ZhangJChenHSongS, et al. Automatic parking motion planning based on reinforcement learning. J Tongji Univ (Nat Sci)2019; 47(S1): 186–190.
33.
DuZMiaoQZongC.Trajectory planning for automated parking systems using deep reinforcement learning. Int J Autom Technol2020; 21(4): 881–887.
