Abstract
A reinforcement learning (RL) controller with identification of the dynamic parameter of hypersonic morphing flight vehicle (HMFV) is proposed in this paper, successfully realizing the end-to-end control of attack angle in the longitudinal plane. The following improvements are made in this paper: Firstly, the dynamic parameter (rudder efficiency coefficient) of the flight vehicle is added into the state vector, so that the RL controller can understand the control ability of the rudder and generates the optimal control commands in the current state. Secondly, five instead of only one consecutive attack angle deviations are used to jointly generate the state vector, which enables the RL controller to use the model state information of the previous period of time and improve the control stability. Three simulations are set up in this paper. The simulation results show that the RL controller proposed in this paper can achieve stable and high precision attack angle control under large-scale environmental deviations and has strong generalization under different guidance commands.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Liu, J.W., Gao, F., Luo, X.L.: Survey of deep reinforcement learning based on value function and policy gradient. Chin. J. Comput. (6), 1406–1438 (2019)
Zhen, Y., Yuan, J.Q., Chi, Q.X., Hao, M.R.: Research on application of deep reinforcement learning method in aircraft control. Tactical Missile Technol. (4), 112–118 (2020)
Zhang, Y.A., Ma, G.X., Liu, J.M., Sun, Y.M.: Reinforcement learning control modeling and algorithm design for the fixed wing UAV. Flight Dyn. (4), 88–96 (2019)
Li, R.F., Hu, L., Cai, L.: Adaptive tracking control of a hypersonic flight aircraft using neural networks with reinforcement synthesis. Aero Weaponry (6), 3–10 (2018)
Wang, G., Ma, C., Ru, H., Ma, G., Xia, H.: An intelligent control method for non-affine hypersonic vehicle. Flight Control Detect. (4), 59–65 (2021)
Zhang, Z.B., Li, X.H., An, J.P., Man, W.X., Zhang, G.H.: Model-free attitude control of spacecraft based on PID-guide TD3 algorithm. Int. J. Aerosp. Eng. 1–13 (2020)
Shen, Y., Chen, M.: Reinforcement learning based dynamic inverse attitude control of near-space vehicle. In: 2020 39th Chinese Control Conference (CCC), pp. 6972–6977 (2020)
Huang, X., Liu, J.R., Jia, C.H., et al.: Deep deterministic policy gradient algorithm for UAV control. Acta Aeronautica et Astronautica Sinica42(11), 524688 (2021). (in Chinese). https://doi.org/10.7527/S10006-893.2020.24688
Zhang, Z.N., Zhang, R., Nie, W.M., et al.: Adaptive optimal attitude control of reentry vehicles. J. Astronaut. (2), 199–206 (2019)
Nie, Y., Yu, C.M., Bai, W.Y.: Aircraft dynamic parameter identification based on adaptive weight Kalman filter. In: 2021 The China Automation Congress (CAC) (2021)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Nie, Y., Zhang, Y., Bai, W., Cao, Y., Yu, C. (2023). Reinforcement Learning Control for Hypersonic Morphing Flight Vehicle with Identification of Dynamic Parameter. In: Yan, L., Duan, H., Deng, Y. (eds) Advances in Guidance, Navigation and Control. ICGNC 2022. Lecture Notes in Electrical Engineering, vol 845. Springer, Singapore. https://doi.org/10.1007/978-981-19-6613-2_250
Download citation
DOI: https://doi.org/10.1007/978-981-19-6613-2_250
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-6612-5
Online ISBN: 978-981-19-6613-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)