EUVIMEDEUVIMED entdecken
Crossref

CTDS: Centralized Teacher With Decentralized Student for Multiagent Reinforcement Learning

Jian Zhao, Xunhan Hu, Mingyu Yang, Wengang Zhou, Jiangcheng Zhu, Houqiang Li

IEEE Transactions on Games · 2024-03 · 16 (1) · 140-150

DOI: 10.1109/tg.2022.3232390 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Games, 2024-03. Verfasst von Jian Zhao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: National Natural Science Foundation of China, MCC Lab of Information Science and Technology Institution, USTC.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Due to the partial observability and communication constraints in many multiagent reinforcement learning (MARL) tasks, centralized training with decentralized execution (CTDE) has become one of the most widely used MARL paradigms. In CTDE, centralized information is dedicated to learning the allocation of the team reward with a mixing network while the learning of individual Q-values is usually based on local observations. The insufficient utility of global observation will degrade performance in challenging environments. To this end, this work proposes a novel Centralized Teacher with a Decentralized Student (CTDS) framework, which consists of a teacher model and a student model. Specifically, the teacher model allocates the team reward by learning individual Q-values conditioned on global observation while the student model utilizes the partial observations to approximate the Q-values estimated by the teacher model. In this way, CTDS balances the full utilization of global observation during training and the feasibility of decentralized execution for online inference. Our CTDS framework is generic, which is ready to be applied upon existing CTDE methods to boost their performance. We conduct experiments on a challenging set of StarCraft II micromanagement tasks to test the effectiveness of our method and the results show that CTDS outperforms the existing value-based MARL methods.

Förderung

  • National Natural Science Foundation of China
  • MCC Lab of Information Science and Technology Institution, USTC

Zitieren

APA

Zhao, J., Hu, X., Yang, M., Zhou, W., Zhu, J., & Li, H. (2024). CTDS: Centralized Teacher With Decentralized Student for Multiagent Reinforcement Learning. IEEE Transactions on Games, 16(1), 140-150. https://doi.org/10.1109/tg.2022.3232390

Vancouver

Zhao J, Hu X, Yang M, Zhou W, Zhu J, Li H. CTDS: Centralized Teacher With Decentralized Student for Multiagent Reinforcement Learning. IEEE Transactions on Games. 2024;16(1):140-150. doi:10.1109/tg.2022.3232390

BibTeX
@article{Zhao2024,
  author = {Zhao, Jian and Hu, Xunhan and Yang, Mingyu and Zhou, Wengang and Zhu, Jiangcheng and Li, Houqiang},
  title = {CTDS: Centralized Teacher With Decentralized Student for Multiagent Reinforcement Learning},
  journal = {IEEE Transactions on Games},
  year = {2024},
  volume = {16},
  number = {1},
  pages = {140--150},
  doi = {10.1109/tg.2022.3232390}
}
Kostenlos starten →