EUVIMEDEUVIMED entdecken
Crossref

A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat

Jiajun Chai, Wenzhang Chen, Yuanheng Zhu, Zong-Xin Yao, Dongbin Zhao

IEEE Transactions on Systems, Man, and Cybernetics: Systems · 2023-09 · 53 (9) · 5417-5429

DOI: 10.1109/tsmc.2023.3270444 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023-09. Verfasst von Jiajun Chai und 4 weiteren Autor:innen. Die Arbeit wurde bisher 86-mal zitiert (Crossref) und nennt 48 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Strategic Priority Research Program of Chinese Academy of Sciences.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Unmanned combat air vehicle (UCAV) combat is a challenging scenario with high-dimensional continuous state and action space and highly nonlinear dynamics. In this article, we propose a general hierarchical framework to resolve the within-vision-range (WVR) air-to-air combat problem under six dimensions of degree (6-DOF) dynamics. The core idea is to divide the whole decision-making process into two loops and use reinforcement learning (RL) to solve them separately. The outer loop uses a combat policy to decide the macro command according to the current combat situation. Then the inner loop uses a control policy to answer the macro command by calculating the actual input signals for the aircraft. We design the Markov decision-making process for the control policy and the Markov game between two aircraft. We present a two-stage training mechanism. For the control policy, we design an effective reward function to accurately track various macro behaviors. For the combat policy, we present a fictitious self-play mechanism to improve the combat performance by combating against the historical combat policies. Experiment results show that the control policy can achieve better tracking performance than conventional methods. The fictitious self-play mechanism can learn competitive combat policy, which can achieve high winning rates against conventional methods.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Strategic Priority Research Program of Chinese Academy of Sciences
  • Youth Innovation Promotion Association CAS

Zitieren

APA

Chai, J., Chen, W., Zhu, Y., Yao, Z., & Zhao, D. (2023). A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 53(9), 5417-5429. https://doi.org/10.1109/tsmc.2023.3270444

Vancouver

Chai J, Chen W, Zhu Y, Yao Z, Zhao D. A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat. IEEE Transactions on Systems, Man, and Cybernetics: Systems. 2023;53(9):5417-5429. doi:10.1109/tsmc.2023.3270444

BibTeX
@article{Chai2023,
  author = {Chai, Jiajun and Chen, Wenzhang and Zhu, Yuanheng and Yao, Zong-Xin and Zhao, Dongbin},
  title = {A Hierarchical Deep Reinforcement Learning Framework for 6-DOF UCAV Air-to-Air Combat},
  journal = {IEEE Transactions on Systems, Man, and Cybernetics: Systems},
  year = {2023},
  volume = {53},
  number = {9},
  pages = {5417--5429},
  doi = {10.1109/tsmc.2023.3270444}
}
Kostenlos starten →