EUVIMEDEUVIMED entdecken
Crossref

Attention With System Entropy for Optimizing Credit Assignment in Cooperative Multi-Agent Reinforcement Learning

Wei Wei, Haibin Li, Shiyuan Zhou, Baifeng Li, Xue Liu

IEEE Transactions on Automation Science and Engineering · 2025 · 22 · 14775-14787

DOI: 10.1109/tase.2025.3563725 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Automation Science and Engineering, 2025. Verfasst von Wei Wei und 4 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 41 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Beijing Natural Science Foundation, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In cooperative multi-agent reinforcement learning (MARL), value function factorization methods have been proposed to address the dimensionality explosion problem encountered in centralized training and decentralized execution (CTDE). The existing value function factorization methods lack a perspective from the global system when addressing credit assignment, failing to measure each agent’s contribution to the current system. A typical limitation is that these factorization approaches primarily rely on the proximity of local individual groups to assign credit. There are challenges such as the sensitivity of credit assignment weights to local information and instability in the learning process. In this study, we focus on redistributing agents’ features during the system evolution and suggest employing an enhanced attention mechanism with system entropy measure to factorize value function. Specifically, this method emphasizes each agent’s representation between their local and global contributions and then redesigns multi-head attention to optimize the credit assignment in value function factorization. To evaluate the effectiveness of our method, we conduct a series of cooperative multi-agent tasks on the StarCraft II platform and compare the results with several state-of-the-art techniques, including Qatten, QMIX, QTRAN, COMA, and VDN. The experimental results demonstrate that our method achieves faster overall convergence speed, higher stability, and robust performance across various scenarios. Note to Practitioners—The fully cooperative multi-agent reinforcement learning (MARL) is extensively applied in practical multi-agent systems, such as coordinating multiple vehicle traffic conditions. Motivated by these practical application scenarios, our study is to enhance MARL’s learning efficiency by introducing an entropy-centered improved multi-head attention mechanism to address credit assignment issues. This method is aimed at measuring the importance weights of each agent within the agent system, thereby further boosting the learning efficiency of MARL. This work holds promise for application in large-scale intelligent agent systems, including multi-robot systems and collaborative multi-vehicle systems, among others.

Förderung

  • National Natural Science Foundation of China
  • Beijing Natural Science Foundation
  • Fundamental Research Funds for the Central Universities

Zitieren

APA

Wei, W., Li, H., Zhou, S., Li, B., & Liu, X. (2025). Attention With System Entropy for Optimizing Credit Assignment in Cooperative Multi-Agent Reinforcement Learning. IEEE Transactions on Automation Science and Engineering, 22, 14775-14787. https://doi.org/10.1109/tase.2025.3563725

Vancouver

Wei W, Li H, Zhou S, Li B, Liu X. Attention With System Entropy for Optimizing Credit Assignment in Cooperative Multi-Agent Reinforcement Learning. IEEE Transactions on Automation Science and Engineering. 2025;22:14775-14787. doi:10.1109/tase.2025.3563725

BibTeX
@article{Wei2025,
  author = {Wei, Wei and Li, Haibin and Zhou, Shiyuan and Li, Baifeng and Liu, Xue},
  title = {Attention With System Entropy for Optimizing Credit Assignment in Cooperative Multi-Agent Reinforcement Learning},
  journal = {IEEE Transactions on Automation Science and Engineering},
  year = {2025},
  volume = {22},
  pages = {14775--14787},
  doi = {10.1109/tase.2025.3563725}
}
Kostenlos starten →