For the vertical landing phase of the reusable rocket, in order to improve the landing accuracy with consideration of multiple uncertainties, a novel strategy to calculate the ignition height online is proposed based on polynomial guidance law (PGL), particle swarm optimization (PSO), and deep reinforcement learning (DRL). Firstly, a deep neural network (DNN) is designed to describe the relationship between the state of the reusable rocket and the ignition height. To accomplish the guidance task of the vertical landing phase, PGL is modified by introducing the estimated aerodynamic acceleration. Through simulation, the output range of the DNN is estimated by the modified PSO. Then, the reward function is shaped and the parameters of the DNN are trained on a training set of simulation scenarios by the DRL algorithm. Finally, to demonstrate the effectiveness of the proposed strategy, the trained DNN is used to calculate the ignition height of 1500 unlearned simulation scenarios online. The numerical simulation results show that the proposed strategy has higher landing accuracy and lower fuel consumption than the offline strategy of fixed ignition height based on the modified PSO.
HiltonSSabatiniRGardiA, et al.Space traffic management: towards safe and unsegregated space transport operations. Prog Aero Sci2019; 105: 98–125.
2.
ZhangMXuDYueS, et al.Design and dynamic analysis of landing gear system in vertical takeoff and vertical landing reusable launch vehicle. Proc IMechE Part G2019; 233(10): 3700–3713.
3.
Byeong-UnJJaemyungA. Optimal staging of reusable launch vehicles for minimum life cycle cost. Aero Sci Technol2022; 127: 1–13.
4.
JinPChenZLiR, et al.Opportunistic preventive maintenance scheduling for multi-unit reusable rocket engine system based on the variable maintenance task window method. Aero Sci Technol2022; 121: 1–14.
SongZWangC. Powered soft landing guidance method for launchers with non-cluster configured engines. Acta Astronaut2021; 189: 379–390.
7.
SundararajanV.Gaganyaan mission: socio-technical systems design and strategic perspectives of India’s human spaceflight program. In: AIAA Scitech 2021 Forum,Tennessee, USA, 11–15& 19–21 January 2021, 1–12.
8.
SaglianoMHeideckerAMacesJ, et al.Onboard guidance for reusable rockets: aerodynamic descent and powered Landing. In: AIAA SciTech 2021 Forum, Tennessee,USA, 11–15 & 19–21 January 2021, 1–35.
9.
ChengGJingWGaoC. Recovery trajectory planning for the reusable launch vehicle. Aero Sci Technol2021; 117: 1–8.
10.
SzmukMReynoldsTAcikmeseB. Successive convexification for real-time six-degree-of-freedom powered descent guidance with state-triggered constraints. J Guid Control Dynam2020; 43(8): 1399–1413.
11.
IzzoDMartensMPanB. A survey on artificial intelligence trends in spacecraft guidance dynamics and control. Astrodynamics2019; 6(4): 287–299.
12.
IzzoDOzturkE. Real-time guidance for low-thrust transfers using deep neural networks. J Guid Control Dynam2021; 44(2): 315–327.
13.
SilverDHuangAMaddisonC, et al.Mastering the game of Go with deep neural networks and tree search. Nature2016; 529(7587): 484–489.
14.
YangBLiSFengJ, et al.Fast solver for J2-perturbed lambert problem using deep neural network. J Guid Control Dynam2021; 45(5): 1–10.
15.
HeXTanSWuZ, et al.Mission reconstruction for launch vehicles under thrust drop faults based on deep neural networks with asymmetric loss functions. Aero Sci Technol2022; 121: 1–13.
16.
AnkitTRitikaL. Fusion of statistical importance for feature selection in deep neural network-based intrusion detection system. Inf Fusion2023; 90: 353–363.
17.
SuttonRSBartoAG. Reinforcement learning: an introduction. Massachusetts: MIT Press, 2018, pp. 1–6.
18.
ZhengMWuYLiC. Reinforcement learning strategy for spacecraft attitude hyperagile tracking control with uncertainties. Aero Sci Technol2021; 119: 1–15.
19.
AfsarMMCrumpTFarB. Reinforcement learning based recommender systems: a survey. ACM Comput Surv2023; 55(7): 1–38.
20.
ZhaoYMaYHuS, et al.USV formation and path-following control via deep reinforcement learning with random braking. IEEE T Neur Net Lear2021; 32(12): 5468–5478.
21.
MnihVKavukcuogluKSilverD, et al.Human-level control through deep reinforcement learning. Nature2015; 518(7540): 529–533.
22.
DegraveJFeliciFBuchliJ, et al.Magnetic control of tokamak plasmas through deep reinforcement learning. Nature2022; 602(7897): 414–419.
23.
QiuXGaoCJingW. Maneuvering penetration strategies of ballistic missiles based on deep reinforcement learning. Proc IMechE Part G2022; 236(16): 1–11.
24.
HovellKUlrichS. Deep reinforcement learning for spacecraft proximity operations guidance. J Spacecraft Rockets2021; 58(2): 254–264.
25.
LiuDQinZHuaH, et al.Incremental incentive mechanism design for diversified consumers in demand response. Appl Energy2023; 329: 1–10.
26.
EberhartRKennedyJ. A new optimizer using particle swarm theory. In: MHS’95. Proceedings of the sixth international symposium on micro machine and human science. Nagoya, Japan: IEEE, 1995.
27.
MiBChengSLouY, et al.A new many-objective aerodynamic optimization method for symmetrical elliptic airfoils by PSO and direct-manipulation-based parametric mesh deformation. Aero Sci Technol2021; 120: 1–21.
28.
ZhangHYuanMLiangY, et al.A novel particle swarm optimization based on prey-predator relationship. Appl Soft Comput2018; 68: 202–218.
29.
OkhitinaARolduginDTkachevS. Application of the PSO for the construction of a 3-axis stable magnetically actuated satellite angular motion. Acta Astronaut2022; 195: 86–97.
30.
LiuYLiuXCaiG, et al.Detumbling a non-cooperative tumbling target using a low-thrust device. AIAA J2022; 60(5): 2718–2729.
31.
HannachiMElbejiOBenhamedM, et al.Optimal tuning of proportional-integral controller using particle swarm optimization algorithm for control of permanent magnet synchronous generator-based wind turbine with tip speed ratio for maximum power point tracking. Proc IMechE Part G2021; 45(2): 400–412.
32.
YangAKongZSunarsoJ. Design and optimisation of novel hybrid side-stream reactive-extractive distillation for recovery of isopropyl alcohol and ethyl acetate from wastewater. Chem Eng J2023; 451: 1–12.
33.
ZhuSYangHCuiP, et al.Anti-collision zone division based hazard avoidance guidance for asteroid landing with constant thrust. Acta Astronaut2022; 190: 377–387.
34.
LuP. Theory of fractional-polynomial powered descent guidance. J Guid Control Dynam2020; 43(3): 1–21.
35.
LiuXLiSXinM. Comparison of powered descent guidance laws for planetary pin-point landing. Acta Astronaut2021; 187: 101–114.
36.
PaulaNMarquesFSilvaW. Volterra kernels assessment via time-delay neural networks for nonlinear unsteady aerodynamic loading identification. AIAA J2019; 57(4): 1725–1735.
37.
ChengZWangBLiuL, et al.Adaptive polynomial guidance with impact angle constraint under varying velocity. IEEE Access2019; 7: 104210–104217.
38.
ZhouZYaoX. Polynomial guidance law for impact angle control with a seeker look angle limit. Proc IMechE Part G2020; 234(3): 857–870.
39.
GlorotXBengioY. Understanding the difficulty of training deep feedforward neural networks. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics2010; 9: 249–256.
40.
RatnaweeraAHalgamugeSWatsonH. Self-organizing hierarchical particle swarm optimizer with time-varying acceleration coefficients. IEEE Trans Evol Comput2004; 8(3): 240–255.
41.
ShiYEberhartR. A modified particle swarm optimizer. IEEE1998: 69–73.
42.
SilverDLeverGHeessN, et al.Deterministic policy gradient algorithms. In: ICML’14: proceedings of the 31st international conference on machine learning. Beijing, China, 21-26 June 2014.
43.
ButtonLBousquetO. The tradeoffs of large scale learning. In: NIPS’07: proceedings of the neural information processing systems conference, Vancouver British Columbia, Canada, 03-06 December 2007.
44.
KingmaDBaJ. Adam: a method for stochastic optimization. San Diego, USA: The 3rd International Conference for Learning Representations, 2015.
