A Generic Competitive-Cooperative Actor-Critic Framework for Deep Reinforcement Learning
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-06 · 48 (6) · 7112-7128
DOI: 10.1109/tpami.2026.3664855 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-06. Verfasst von Meng Xu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: Hong Kong Research Grants Council, General Research Fund, Collaborative Research Fund.
Abstract (Originalsprache)
In the field of Deep reinforcement learning (DRL), enhancing exploration capabilities and improving the accuracy of Q-value estimation remain two major challenges. Recently, double-actor DRL methods have emerged as a promising class of DRL approaches, achieving substantial advancements in both exploration and Q-value estimation. However, existing double-actor DRL methods feature actors that operate independently in exploring the environment, lacking mutual learning and collaboration, which leads to suboptimal policies. To address this challenge, this work proposes a generic solution that can be seamlessly integrated into existing double-actor DRL methods by promoting mutual learning among the actors to develop improved policies. Specifically, we calculate the difference in actions output by the actors and minimize this difference as a loss during training to facilitate mutual imitation among the actors. Simultaneously, we also minimize the differences in Q-values output by the various critics as part of the loss, thereby avoiding significant discrepancies in value estimation for the imitated actions. We present two specific implementations of our method and extend these implementations beyond double-actor DRL methods to other DRL approaches to encourage broader adoption. Experimental results demonstrate that our method significantly improves twenty state-of-the-art (SOTA) DRL methods, including SOTA double-actor DRL methods, across eleven tasks, as measured by return and other metrics.
Förderung
- Hong Kong Research Grants Council
- General Research Fund
- Collaborative Research Fund
- National Key Research and Development Program of China
- Key R&D Program of Jiangxi Province
Zitieren
APA
Xu, M., Wen, Z., Chen, X., Zhao, G., Huang, J., & Wang, J. (2026). A Generic Competitive-Cooperative Actor-Critic Framework for Deep Reinforcement Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(6), 7112-7128. https://doi.org/10.1109/tpami.2026.3664855
Vancouver
Xu M, Wen Z, Chen X, Zhao G, Huang J, Wang J. A Generic Competitive-Cooperative Actor-Critic Framework for Deep Reinforcement Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(6):7112-7128. doi:10.1109/tpami.2026.3664855
BibTeX
@article{Xu2026,
author = {Xu, Meng and Wen, Zihao and Chen, Xinhong and Zhao, Guanyi and Huang, Jin and Wang, Jianping},
title = {A Generic Competitive-Cooperative Actor-Critic Framework for Deep Reinforcement Learning},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {6},
pages = {7112--7128},
doi = {10.1109/tpami.2026.3664855}
}
EUVIMED