EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Heterogeneity, reinforcement learning, and chaos in population games

Jakub Bielawski iD, Thiparat Chotibut iD, Fryderyk Falniowski iD, Michał Misiurewicz iD, Georgios Piliouras iD

Proceedings of the National Academy of Sciences · 2025

DOI: 10.1073/pnas.2319929121 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the National Academy of Sciences, 2025. Verfasst von Jakub Bielawski und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Mathematics, Krakow University of Economics, Kraków 31-510, Poland. Hauptthemen laut MEDLINE: Game Theory, Reinforcement, Psychology, Machine Learning, Nonlinear Dynamics. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Inspired by the challenges at the intersection of Evolutionary Game Theory and Machine Learning, we investigate a class of discrete-time multiagent reinforcement learning (MARL) dynamics in population/nonatomic congestion games, where agents have diverse beliefs and learn at different rates. These congestion games, a well-studied class of potential games, are characterized by individual agents having negligible effects on system performance, strongly aligned incentives, and well-understood advantageous properties of Nash equilibria. Despite the presence of static Nash equilibria, we demonstrate that MARL dynamics with heterogeneous learning rates can deviate from these equilibria, exhibiting instability and even chaotic behavior and resulting in increased social costs. Remarkably, even within these chaotic regimes, we show that the time-averaged macroscopic behavior converges to exact Nash equilibria, thus linking the microscopic dynamic complexity with traditional equilibrium concepts. By employing dynamical systems techniques, we analyze the interaction between individual-level adaptation and population-level outcomes, paving the way for studying heterogeneous learning dynamics in discrete time across more complex game scenarios.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Game TheoryReinforcement, PsychologyMachine LearningNonlinear DynamicsLearningHumans

Stichwörter der Autor:innen

Li-Yorke chaoscongestion gamesevolutionary dynamicsgame theorymultiplicative weights update

Autor:innen und Institutionen

  • Department of Mathematics, Krakow University of Economics, Kraków 31-510, Poland.
  • Chula Intelligent and Complex Systems, Department of Physics, Faculty of Science, Chulalongkorn University, Bangkok 10330, Thailand.
  • Department of Mathematical Sciences, Indiana University-Purdue University Indianapolis, Indianapolis, IN 46202.
  • Google DeepMind, London EC4A 3TW, United Kingdom.

ORCID-Profile: Jakub Bielawski, Thiparat Chotibut, Fryderyk Falniowski, Michał Misiurewicz, Georgios Piliouras

Zitieren

APA

Bielawski, J., Chotibut, T., Falniowski, F., Misiurewicz, M., & Piliouras, G. (2025). Heterogeneity, reinforcement learning, and chaos in population games. Proceedings of the National Academy of Sciences. https://doi.org/10.1073/pnas.2319929121

Vancouver

Bielawski J, Chotibut T, Falniowski F, Misiurewicz M, Piliouras G. Heterogeneity, reinforcement learning, and chaos in population games. Proceedings of the National Academy of Sciences. 2025. doi:10.1073/pnas.2319929121

BibTeX
@article{Bielawski2025,
  author = {Bielawski, Jakub and Chotibut, Thiparat and Falniowski, Fryderyk and Misiurewicz, Michał and Piliouras, Georgios},
  title = {Heterogeneity, reinforcement learning, and chaos in population games},
  journal = {Proceedings of the National Academy of Sciences},
  year = {2025},
  doi = {10.1073/pnas.2319929121}
}
Kostenlos starten →