Abstract
With the development of military informatization, military evolution is bound to include a large number of confrontations with artificial intelligence algorithms. In this paper, we discuss the air-to-air combat problem in that both sides have multiple UAVs involved, and propose a UAV Cluster Dominance Decision Algorithm Based on Reinforcement Learning algorithm (UCDD-RL). The combat dominance of each UAV is evaluated according to the live number, closeness, health, and relative positions of both camps in the local UAV cluster. The combat dominance of the UAV is the input of the fuzzy inference system, and its output is the UAV’s combat strategy. The combat strategies include fight, escape, occupy, and change cluster. We perform numerical simulations with the UCDD-RL algorithm in the UAV combat environment. By analyzing the reward, the win rate, and the live number of UAVs in the experiments, we verify that the UCDD-RL algorithm has a positive effect on improving UAV air combat level.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Wang, X., Cao, Y., Liao, W.: Modeling of MAV/UAV collaborative combat network based on UAV autonomous level. In: Wu, M., Niu, Y., Gu, M., Cheng, J. (eds.) ICAUS 2021. LNEE, vol. 861, pp. 2990–2999. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-9492-9_293
Virtanen, K., Karelahti, J., Raivio, T.: Modeling air combat by a moving horizon influence diagram game. J. Guidance Control Dyn. 29(5), 1080–1091 (2006)
Chen, J., Chen, P., Wu, Q., Xu, Y., Qi, N., Fang, T.: A game-theoretic perspective on resource management for large-scale UAV communication networks. Chin. Commun. 18(1), 70–87 (2021). https://doi.org/10.23919/JCC.2021.01.007
Park, H., Lee, B.Y., Tahk, M.J., Yoo, D.W.: Differential game based air combat maneuver generation using scoring function matrix. Int. J. Aeronaut. Space Sci. 17(2), 204–213 (2016)
Shao, J., Xu, Y., Luo, D.: Cooperative combat decision-making research for multi UAVs. Inf. Control 47(03), 347–354 (2018)
Zhao, Z., Rao, Y., Long, H., Sun, X., Liu, Z.: Resource baseline MAPPO for multi-UAV dog fighting. In: Wu, M., Niu, Y., Gu, M., Cheng, J. (eds.) ICAUS 2021. LNEE, vol. 861, pp. 3330–3336. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-9492-9_327
Hung, S.M., Givigi, S.N.: A q-learning approach to flocking with UAVs in a stochastic environment. IEEE Trans. Cybern. 47(1), 186–197 (2016)
Osband, I., Blundell, C., Pritzel, A., Van Roy, B.: Deep exploration via bootstrapped DQN. In: Advances in Neural Information Processing Systems, vol. 29 (2016)
Pan, J., Wang, X., Cheng, Y., Yu, Q.: Multisource transfer double DQN based on actor learning. IEEE Trans. Neural Netw. Learn. Syst. 29(6), 2227–2238 (2018)
Zeng, Y., Xu, X., Jin, S., Zhang, R.: Simultaneous navigation and radio mapping for cellular-connected UAV with deep reinforcement learning. IEEE Trans. Wirel. Commun. 20(7), 4205–4220 (2021)
Li, B., Yang, Z.P., Chen, D.Q., Liang, S.Y., Ma, H.: Maneuvering target tracking of UAV based on mn-ddpg and transfer learning. Def. Technol. 17(2), 457–466 (2021)
Kong, W., Zhou, D., Yang, Z., Zhao, Y., Zhang, K.: UAV autonomous aerial combat maneuver strategy generation with observation error based on state-adversarial deep deterministic policy gradient and inverse reinforcement learning. Electronics 9(7), 1121 (2020)
Lowe, R., Wu, Y.I., Tamar, A., Harb, J., Pieter Abbeel, O., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. In: Advances in Neural Information Processing Systems, vol. 30 (2017)
Hu, J., Wang, L., Hu, T., Guo, C., Wang, Y.: Autonomous maneuver decision making of dual-UAV cooperative air combat based on deep reinforcement learning. Electronics 11(3), 467 (2022)
Jiang, T., Zhuang, D., Xie, H.: Anti-drone policy learning based on self-attention multi-agent deterministic policy gradient. In: Wu, M., Niu, Y., Gu, M., Cheng, J. (eds.) ICAUS 2021. LNEE, vol. 861, pp. 2277–2289. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-9492-9_225
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 Beijing HIWING Sci. and Tech. Info Inst
About this paper
Cite this paper
Li, Y., Gao, Y., Dai, X., Nian, X., Wang, H., Xiong, H. (2023). Reinforcement Learning-Based Cooperative Adversarial Algorithm for UAV Cluster. In: Fu, W., Gu, M., Niu, Y. (eds) Proceedings of 2022 International Conference on Autonomous Unmanned Systems (ICAUS 2022). ICAUS 2022. Lecture Notes in Electrical Engineering, vol 1010. Springer, Singapore. https://doi.org/10.1007/978-981-99-0479-2_102
Download citation
DOI: https://doi.org/10.1007/978-981-99-0479-2_102
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-0478-5
Online ISBN: 978-981-99-0479-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)