Underwater Target Tracking Based on Hierarchical Software-Defined Multi-AUV Reinforcement Learning: A Multi-AUV Advantage-Attention Actor-Critic Approach
IEEE Transactions on Mobile Computing · 2024-12 · 23 (12) · 13639-13653
DOI: 10.1109/tmc.2024.3437376 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Mobile Computing, 2024-12. Verfasst von Shengchao Zhu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 42-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Open fund of State Key Laboratory of Acoustics, Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
With the rapid development of underwater robots, underwater communication techniques, etc., the Autonomous Underwater Vehicle (AUV) cluster network has emerged as a candidate paradigm to perform underwater civil and military applications, e.g., underwater target tracking. In this paper, we focus on how to utilize networking and multi-agent artificial intelligence technique to improve underwater target tracking. In particular, to improve the flexibility and scalability of the AUV cluster network, we employ Software-Defined Networking (SDN) and Centralized Training with Decentralized Execution (CTDE)-based Multi-Agent Reinforcement Learning (MARL) technologies, to propose a Hierarchical Software-Defined Multiple AUVs Reinforcement Learning (HSD-MARL) framework. For the MARL mechanism in HSD-MARL, we propose an advantage-attention mechanism and present the architecture of Multi-AUV Advantage-Attention Actor-Critic (MA-A3C), to address slow convergence and poor scalability issues on the AUV cluster network of large-scale. Further, to improve the utilization rate of advantage samples especially when the MA-A3C is utilized to perform AUV cluster network-based underwater tracking, we propose an ‘advantage resampling’ method based on experience replay buffer. Evaluation results showcase that our proposed approaches can perform exact underwater target tracking based on AUV cluster network systems and outperform some recent research products in terms of convergence speed, tracking accuracy, etc.
Förderung
- National Natural Science Foundation of China
- Open fund of State Key Laboratory of Acoustics
- Fundamental Research Funds for the Central Universities
Zitieren
APA
Zhu, S., Han, G., Lin, C., & Tao, Q. (2024). Underwater Target Tracking Based on Hierarchical Software-Defined Multi-AUV Reinforcement Learning: A Multi-AUV Advantage-Attention Actor-Critic Approach. IEEE Transactions on Mobile Computing, 23(12), 13639-13653. https://doi.org/10.1109/tmc.2024.3437376
Vancouver
Zhu S, Han G, Lin C, Tao Q. Underwater Target Tracking Based on Hierarchical Software-Defined Multi-AUV Reinforcement Learning: A Multi-AUV Advantage-Attention Actor-Critic Approach. IEEE Transactions on Mobile Computing. 2024;23(12):13639-13653. doi:10.1109/tmc.2024.3437376
BibTeX
@article{Zhu2024,
author = {Zhu, Shengchao and Han, Guangjie and Lin, Chuan and Tao, Qiuzi},
title = {Underwater Target Tracking Based on Hierarchical Software-Defined Multi-AUV Reinforcement Learning: A Multi-AUV Advantage-Attention Actor-Critic Approach},
journal = {IEEE Transactions on Mobile Computing},
year = {2024},
volume = {23},
number = {12},
pages = {13639--13653},
doi = {10.1109/tmc.2024.3437376}
}
EUVIMED