Model-Based Reinforcement Learning for Offline Zero-Sum Markov Games
Operations Research · 2024-11 · 72 (6) · 2430-2445
DOI: 10.1287/opre.2022.0342 ↗Kurz gefasst
Fachartikel in der Zeitschrift Operations Research, 2024-11. Verfasst von Yuling Yan und 3 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 31 Literaturangaben.
Abstract (Originalsprache)
This paper makes progress toward learning Nash equilibria in two-player, zero-sum Markov games from offline data. Despite a large number of prior works tackling this problem, the state-of-the-art results suffer from the curse of multiple agents in the sense that their sample complexity bounds scale linearly with the total number of joint actions. The current paper proposes a new model-based algorithm, which provably finds an approximate Nash equilibrium with a sample complexity that scales linearly with the total number of individual actions. This work also develops a matching minimax lower bound, demonstrating the minimax optimality of the proposed algorithm for a broad regime of interest. An appealing feature of the result lies in algorithmic simplicity, which reveals the unnecessity of sophisticated variance reduction and sample splitting in achieving sample optimality.
Zitieren
APA
Yan, Y., Li, G., Chen, Y., & Fan, J. (2024). Model-Based Reinforcement Learning for Offline Zero-Sum Markov Games. Operations Research, 72(6), 2430-2445. https://doi.org/10.1287/opre.2022.0342
Vancouver
Yan Y, Li G, Chen Y, Fan J. Model-Based Reinforcement Learning for Offline Zero-Sum Markov Games. Operations Research. 2024;72(6):2430-2445. doi:10.1287/opre.2022.0342
BibTeX
@article{Yan2024,
author = {Yan, Yuling and Li, Gen and Chen, Yuxin and Fan, Jianqing},
title = {Model-Based Reinforcement Learning for Offline Zero-Sum Markov Games},
journal = {Operations Research},
year = {2024},
volume = {72},
number = {6},
pages = {2430--2445},
doi = {10.1287/opre.2022.0342}
}
EUVIMED