CTDS: Centralized Teacher With Decentralized Student for Multiagent Reinforcement Learning
IEEE Transactions on Games · 2024-03 · 16 (1) · 140-150
DOI: 10.1109/tg.2022.3232390 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Games, 2024-03. Verfasst von Jian Zhao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: National Natural Science Foundation of China, MCC Lab of Information Science and Technology Institution, USTC.
Abstract (Originalsprache)
Due to the partial observability and communication constraints in many multiagent reinforcement learning (MARL) tasks, centralized training with decentralized execution (CTDE) has become one of the most widely used MARL paradigms. In CTDE, centralized information is dedicated to learning the allocation of the team reward with a mixing network while the learning of individual Q-values is usually based on local observations. The insufficient utility of global observation will degrade performance in challenging environments. To this end, this work proposes a novel Centralized Teacher with a Decentralized Student (CTDS) framework, which consists of a teacher model and a student model. Specifically, the teacher model allocates the team reward by learning individual Q-values conditioned on global observation while the student model utilizes the partial observations to approximate the Q-values estimated by the teacher model. In this way, CTDS balances the full utilization of global observation during training and the feasibility of decentralized execution for online inference. Our CTDS framework is generic, which is ready to be applied upon existing CTDE methods to boost their performance. We conduct experiments on a challenging set of StarCraft II micromanagement tasks to test the effectiveness of our method and the results show that CTDS outperforms the existing value-based MARL methods.
Förderung
- National Natural Science Foundation of China
- MCC Lab of Information Science and Technology Institution, USTC
Zitieren
APA
Zhao, J., Hu, X., Yang, M., Zhou, W., Zhu, J., & Li, H. (2024). CTDS: Centralized Teacher With Decentralized Student for Multiagent Reinforcement Learning. IEEE Transactions on Games, 16(1), 140-150. https://doi.org/10.1109/tg.2022.3232390
Vancouver
Zhao J, Hu X, Yang M, Zhou W, Zhu J, Li H. CTDS: Centralized Teacher With Decentralized Student for Multiagent Reinforcement Learning. IEEE Transactions on Games. 2024;16(1):140-150. doi:10.1109/tg.2022.3232390
BibTeX
@article{Zhao2024,
author = {Zhao, Jian and Hu, Xunhan and Yang, Mingyu and Zhou, Wengang and Zhu, Jiangcheng and Li, Houqiang},
title = {CTDS: Centralized Teacher With Decentralized Student for Multiagent Reinforcement Learning},
journal = {IEEE Transactions on Games},
year = {2024},
volume = {16},
number = {1},
pages = {140--150},
doi = {10.1109/tg.2022.3232390}
}
EUVIMED