This paper presents applying reinforcement learning to find the optimal sensor/actuator placement (OSAP) policy and optimal control for the flexible wing. The “co-design” objective is to find the OSAP and its associate controller to render the optimal closed-loop performance. The nonlinear vibration dynamics of the flexible wing are modeled in the linear parameter varying (LPV) approach so that LPV-H∞ controllers can be designed. The co-design problem is formulated into mixed-integer semi-definite programming (MISDP). As a special form of combinatorial optimization, MIDSP solves integer optimization for sensor/actuator selection and convex optimization for controller design. A modified reinforcement learning algorithm is applied to solve this NP-hard optimization problem and obtain a converged solution. In addition, RL is compared with the greedy algorithm and genetic algorithm to demonstrate its strengths and drawbacks in solving high-dimensional MISDP. The solutions obtained by RL and the greedy algorithm are verified and compared in the high-fidelity simulation with the full-order model.
SuWCesnikCE. Nonlinear aeroelasticity of a very flexible Blended-Wing-Body aircraft. J Aircra2010; 47(5): 1539–1553. DOI: 10.2514/1.47317
2.
GuoCLuDZhangM, et al.Active control technology for flexible solar array disturbance suppression. Aeros Sci Tech2020; 106: 106148. DOI: 10.1016/j.ast.2020.106148
3.
OuyangYGuYKouX, et al.Active flutter suppression of wing with morphing flap. Aeros Sci Tech2021; 110: 106457. DOI: 10.1016/j.ast.2020.106457
4.
AzimiMFarzaneh JoubanehE. Dynamic modeling and vibration control of a coupled rigid-flexible high-order structural system: a comparative study. Aeros Sci Tech2020; 102: 105875. DOI: 10.1016/j.ast.2020.105875
5.
GaoSLiuJ. Adaptive neural network vibration control of a flexible aircraft wing system with input signal quantization. Aeros Sci Tech2020; 96: 105593. DOI: 10.1016/j.ast.2019.105593
6.
MaghamiPGJoshiSM. Sensor/actuator placement for flexible space structures. IEEE Trans Aeros Electro Sys1993; 29(2): 345–351.
7.
SkogestadSPostlethwaiteI. Multivariable feedback control: analysis and design, 2. New York: Wiley, 2007.
8.
CaoYBissDPerkinsJ. Assessment of input-output controllability in the presence of control constraints. Compu chem eng1996; 20(4): 337–346.
9.
GeorgesD.The use of observability and controllability gramians or functions for optimal sensor and actuator location in finite-dimensional systems. In: Proceedings of 1995 34th IEEE Conference on Decision and Control, Vol 4, New Orleans,1995. IEEE, pp. 3319–3324.
10.
LeleuSAbou-KandilHBonnassieuxY. Actuators and sensors positioning for active vibration control in flexible systems. In: IFAC Proceedings Volumes 33 (26) (2000) 935 – 940, iFAC Conference on Mechatronic Systems, Darmstadt, Germany, 18-20 September 2000. DOI: 10.1016/S1474-6670(17)39265-0
11.
MüllerPWeberH. Analysis and optimization of certain qualities of controllability and observability for linear dynamical systems. Automatica1972; 8(3): 237–246.
12.
TzoumasVRahimianMAPappasGJ, et al.Minimal actuator placement with bounds on control effort. IEEE Transa Control Net Sys2016; 3(1): 67–78. DOI: 10.1109/TCNS.2015.2444031
13.
GawronskiWK. Dynamics and control of structures: a modal approach. Springer Science and Business Media, 2004.
14.
LiuWHouZDemetriouMA. A computational scheme for the optimal sensor/actuator placement of flexible structures using spatial h2 measures. Mech Sys Signal Proc2006; 20(4): 881–895, DOI: 10.1016/j.ymssp.2005.08.030
15.
GüneyMEşkinatE. Optimal actuator and sensor placement in flexible structures using closed-loop criteria. J Sound Vibra2008; 312(1–2): 210–233.
16.
HanisTHromcikM. Optimal sensors placement and spillover suppression. Mech sys signal proc2012; 28: 367–378.
17.
LiDLiHFritzenC. The connection between effective independence and modal kinetic energy methods for sensor placement. J sound vibra2007; 305(4–5): 945–955.
18.
PadulaSLKincaidRK. Optimization strategies for sensor and actuator placement, 1999.
19.
SinghTDe MauriMDecréW, et al.Feedback control of linear systems with optimal sensor and actuator selection. J Vibra Contr2021; 27(11–12): 1250–1264.
20.
JoshiSBoydS. Sensor selection via convex optimization. IEEE Transac Signal Proc2008; 57(2): 451–462.
21.
JawaidSTSmithSL. Submodularity and greedy algorithms in sensor scheduling for linear dynamical systems. Automatica2015; 61: 282–288.
22.
BruantIGallimardLNikoukarS. Optimization of piezoelectric sensors location and number using a genetic algorithm. Mech Adv Mat Struc2011; 18(7): 469–475.
23.
MazyavkinaNSviridovSIvanovSet al. Reinforcement learning for combinatorial optimization: a survey. Computers and Operations Research2021; 134: 105400.
24.
BengioYLodiAProuvostA. Machine learning for combinatorial optimization: a methodological tour d’horizon. European Journal of Operational Research, 2020.
25.
CappartQGoutierreEBergmanD, et al.Improving optimization bounds using machine learning: Decision diagrams meet deep reinforcement learning. Proc AAAI Conf Artifi Intelli2019; 33: 1443–1451.
26.
KasperKMathelinLAbou-KandilH. A machine learning approach for constrained sensor placement. In: 2015 American Control Conference (ACC),Chicago, 2015, IEEE, pp. 4479–4484.
Al-JibooryAKZhuGGSweiSSM, et al.LPV modeling of a flexible wing aircraft using modal alignment and adaptive gridding methods. Aeros Sci Tech2017; 66: 92–102. DOI: 10.1016/j.ast.2017.03.009
29.
HeTAl-JibooryAKZhuGG, et al.Application of ICC LPV control to a blended-wing-body airplane with guaranteed performance. Aeros Sci Tech2018; 81: 88–98. DOI: 10.1016/j.ast.2018.07.046
30.
HeTZhuGGSweiSS-M, et al.Smooth-switching LPV control for vibration suppression of a flexible airplane wing. Aeros Sci Tech2019; 84: 895–903. DOI: 10.1016/j.ast.2018.11.029
31.
SuWCesnikCE. Strain-based geometrically nonlinear beam formulation for modeling very flexible aircraft. Int J Soli Struct2011; 48(16–17): 2349–2360.
32.
CesnikCEHodgesDH. Vabs: a new concept for composite rotor blade cross-sectional modeling. J Am helicop soc1997; 42(1): 27–38.
33.
AgulhariCMOliveiraRCLFPeresPLD. Robust LMI parser: a computational package to construct LMI conditions for uncertain systems. In: XIX Brazilian Conference on Automation. Brazil: CBA 2012Campina GrandePB, 2012, pp. 2298–2305.
LöfbergJ. Yalmip: a toolbox for modeling and optimization in MATLAB. In: Proceedings of the CACSD Conference. Taipei, Taiwan, 2004, pp. 284–289.
36.
SturmJF. Using SeDuMi 1.02, a MATLAB toolbox for optimization over symmetric cones. Optimiz Meth Soft1999; 11(1): 625–653.
37.
SuttonRSBartoAG. Reinforcement learning: an introduction. MIT press, 2018.
38.
XuXZuoLHuangZ. Reinforcement learning algorithms with function approximation: recent advances and applications. Infor Sci2014; 261: 1–31.
39.
HeTZhuGGSweiSSM, et al.Optimal sensor placement for flexible wings using the greedy algorithm. In: 2020 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 2020, pp. 577–582. DOI: 10.1109/AIM43001.2020.9158923
40.
DasAKempeD. Submodular meets spectral: greedy algorithms for subset selection, sparse approximation and dictionary selection, 2011. arXiv preprint arXiv:1102.3975.
