Many-Versus-Many AUV Attack-Defense Game in 3-D Scenarios Using Hierarchical Multiagent Reinforcement Learning
IEEE Internet of Things Journal · 01.07.2025 · 12 (13) · 23479-23494
DOI: 10.1109/jiot.2025.3552116 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Internet of Things Journal, erschienen am 01.07.2025. Verfasst von Wenhao Gan und 1 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Shanghai Sailing Program, Young Talent Project of China National Nuclear Corporation.
Abstract (Originalsprache)
This article proposes a deep reinforcement learning (DRL)-based method for many-versus-many attack-defense games involving autonomous underwater vehicles (AUVs) in 3-D space, focusing on training a defense team to counter attackers. The attackers benefit from speed and unpredictability, while defenders leverage numerical superiority. The scenario includes irregular terrain, and AUVs are limited by low-frequency communication. First, a constrained Apollonius model considering AUV 3-D motion characteristics is developed to evaluate the repulsive effect of defenders on attackers. Second, a hybrid 3-D AUV maneuvering framework integrating end-to-velocity and hierarchical approaches is proposed to reduce the complexity of decision-making strategy learning, enabling AUVs to counter multiattacker threats and learn repulsion strategies across subteams. Third, a scalable learning architecture is designed to adapt to different game scales, with an improved update method to enhance advantage and credit estimation efficiency while ensuring convergence. The combination of population expansion-curriculum training and asynchronous parallel training strengthens the generalization of strategies across various environments. Finally, through comparative analysis with mainstream multiagent deep reinforcement learning-based methods, as well as ablation studies on the framework and rewards, our scheme demonstrates superior learning efficiency and generalization ability. Adversarial experiments across different game scales, along with specialized performance tests, indicate that the defense group exhibits strong robustness and adaptive characteristics.
Förderung
- National Natural Science Foundation of China
- Shanghai Sailing Program
- Young Talent Project of China National Nuclear Corporation
- Young Elite Scientists Sponsorship Program by CAST
Zitieren
APA
Gan, W., & Qiao, L. (2025). Many-Versus-Many AUV Attack-Defense Game in 3-D Scenarios Using Hierarchical Multiagent Reinforcement Learning. IEEE Internet of Things Journal, 12(13), 23479-23494. https://doi.org/10.1109/jiot.2025.3552116
Vancouver
Gan W, Qiao L. Many-Versus-Many AUV Attack-Defense Game in 3-D Scenarios Using Hierarchical Multiagent Reinforcement Learning. IEEE Internet of Things Journal. 2025;12(13):23479-23494. doi:10.1109/jiot.2025.3552116
BibTeX
@article{Gan2025,
author = {Gan, Wenhao and Qiao, Lei},
title = {Many-Versus-Many AUV Attack-Defense Game in 3-D Scenarios Using Hierarchical Multiagent Reinforcement Learning},
journal = {IEEE Internet of Things Journal},
year = {2025},
volume = {12},
number = {13},
pages = {23479--23494},
doi = {10.1109/jiot.2025.3552116}
}
EUVIMED