EUVIMEDEUVIMED entdecken
Crossref

A Generic Competitive-Cooperative Actor-Critic Framework for Deep Reinforcement Learning

Meng Xu, Zihao Wen, Xinhong Chen, Guanyi Zhao, Jin Huang, Jianping Wang

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-06 · 48 (6) · 7112-7128

DOI: 10.1109/tpami.2026.3664855 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-06. Verfasst von Meng Xu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: Hong Kong Research Grants Council, General Research Fund, Collaborative Research Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the field of Deep reinforcement learning (DRL), enhancing exploration capabilities and improving the accuracy of Q-value estimation remain two major challenges. Recently, double-actor DRL methods have emerged as a promising class of DRL approaches, achieving substantial advancements in both exploration and Q-value estimation. However, existing double-actor DRL methods feature actors that operate independently in exploring the environment, lacking mutual learning and collaboration, which leads to suboptimal policies. To address this challenge, this work proposes a generic solution that can be seamlessly integrated into existing double-actor DRL methods by promoting mutual learning among the actors to develop improved policies. Specifically, we calculate the difference in actions output by the actors and minimize this difference as a loss during training to facilitate mutual imitation among the actors. Simultaneously, we also minimize the differences in Q-values output by the various critics as part of the loss, thereby avoiding significant discrepancies in value estimation for the imitated actions. We present two specific implementations of our method and extend these implementations beyond double-actor DRL methods to other DRL approaches to encourage broader adoption. Experimental results demonstrate that our method significantly improves twenty state-of-the-art (SOTA) DRL methods, including SOTA double-actor DRL methods, across eleven tasks, as measured by return and other metrics.

Förderung

  • Hong Kong Research Grants Council
  • General Research Fund
  • Collaborative Research Fund
  • National Key Research and Development Program of China
  • Key R&D Program of Jiangxi Province

Zitieren

APA

Xu, M., Wen, Z., Chen, X., Zhao, G., Huang, J., & Wang, J. (2026). A Generic Competitive-Cooperative Actor-Critic Framework for Deep Reinforcement Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(6), 7112-7128. https://doi.org/10.1109/tpami.2026.3664855

Vancouver

Xu M, Wen Z, Chen X, Zhao G, Huang J, Wang J. A Generic Competitive-Cooperative Actor-Critic Framework for Deep Reinforcement Learning. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(6):7112-7128. doi:10.1109/tpami.2026.3664855

BibTeX
@article{Xu2026,
  author = {Xu, Meng and Wen, Zihao and Chen, Xinhong and Zhao, Guanyi and Huang, Jin and Wang, Jianping},
  title = {A Generic Competitive-Cooperative Actor-Critic Framework for Deep Reinforcement Learning},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {6},
  pages = {7112--7128},
  doi = {10.1109/tpami.2026.3664855}
}
Kostenlos starten →