EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SA-MARL: Novel Self-Attention-Based Multi-Agent Reinforcement Learning With Stochastic Gradient Descent

Rabbiya Younas, Hafiz Muhammad Raza Ur Rehman, Ingyu Lee, Byung-Won On, Sungwon Yi, Gyu Sang Choi

IEEE Access · 2025 · 13 · 35674-35687

DOI: 10.1109/access.2025.3544961 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2025. Verfasst von Rabbiya Younas und 5 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Electronics and Telecommunications Research Institute, Korean Government, Research on Multi-Agent Reinforcement Learning Exploration, Communication, and Training Strategy.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the rapidly advancing Reinforcement Learning (RL) field, Multi-Agent Reinforcement Learning (MARL) has emerged as a key player in solving complex real-world challenges. A pivotal development in this realm is the introduction of the mixing network, representing a significant leap forward in the capabilities of multi-agent systems. Drawing inspiration from COMA and VDN methodologies, the mixing network overcomes limitations in extracting combined Q-values from joint state-action interactions. Previous approaches like COMA and VDN faced constraints in fully utilizing the state-provided information during training, limiting their effectiveness. QMIX and QVMinMax addressed this issue by employing neural networks to convert centralized states into weights for a second neural network, akin to hyper- networks. However, these solutions presented challenges such as computational intensity and susceptibility to local minima. To overcome these hurdles, our proposed methodology introduces three key contributions. First, we introduce the state- fusion network, an innovative alternative to traditional mixing, with a self-attention mechanism. Second, to address the local optima problem in MARL algorithms, we leverage the Grey Wolf Optimizer for weight and bias selection, adding a stochastic element for improved optimization. Finally, we comprehensively compare with QMIX, evaluating performance under two optimization methods: Gradient Descent and Stochastic Optimizer. Using the StarCraft II Learning Environment (SC2LE) as our experimental platform, our results demonstrate the superiority of our methodology over QMIX, QVMinMax, and QSOD in absolute performance, particularly when operating under resource constraints. Our proposed methodology contributes to the ongoing evolution of MARL techniques, showcasing advancements in attention mechanisms and optimization strategies for enhanced multi-agent system capabilities.

Förderung

  • Electronics and Telecommunications Research Institute
  • Korean Government
  • Research on Multi-Agent Reinforcement Learning Exploration, Communication, and Training Strategy
  • Air Force Office of Scientific Research
  • Basic Science Research Program through the National Research Foundation of Korea
  • Ministry of Education

Zitieren

APA

Younas, R., Rehman, H. M. R. U., Lee, I., On, B., Yi, S., & Choi, G. S. (2025). SA-MARL: Novel Self-Attention-Based Multi-Agent Reinforcement Learning With Stochastic Gradient Descent. IEEE Access, 13, 35674-35687. https://doi.org/10.1109/access.2025.3544961

Vancouver

Younas R, Rehman HMRU, Lee I, On B, Yi S, Choi GS. SA-MARL: Novel Self-Attention-Based Multi-Agent Reinforcement Learning With Stochastic Gradient Descent. IEEE Access. 2025;13:35674-35687. doi:10.1109/access.2025.3544961

BibTeX
@article{Younas2025,
  author = {Younas, Rabbiya and Rehman, Hafiz Muhammad Raza Ur and Lee, Ingyu and On, Byung-Won and Yi, Sungwon and Choi, Gyu Sang},
  title = {SA-MARL: Novel Self-Attention-Based Multi-Agent Reinforcement Learning With Stochastic Gradient Descent},
  journal = {IEEE Access},
  year = {2025},
  volume = {13},
  pages = {35674--35687},
  doi = {10.1109/access.2025.3544961}
}
Kostenlos starten →