EUVIMEDEUVIMED entdecken
Crossref

Adaptive network approach to exploration–exploitation trade-off in reinforcement learning

Mohammadamin Moradi iD, Zheng-Meng Zhai iD, Shirin Panahi iD, Ying-Cheng Lai iD

Chaos: An Interdisciplinary Journal of Nonlinear Science · 01.12.2024 · 34 (12)

DOI: 10.1063/5.0221833 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Chaos: An Interdisciplinary Journal of Nonlinear Science, online erschienen am 03.12.2024. Verfasst von Mohammadamin Moradi und 3 weiteren Autor:innen. Beteiligte Einrichtung: School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, Arizona 85287, USA. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 68 Literaturangaben. Gefördert von: Air Force Office of Scientific Research.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A foundational machine-learning architecture is reinforcement learning, where an outstanding problem is achieving an optimal balance between exploration and exploitation. Specifically, exploration enables the agents to discover optimal policies in unknown domains of the environment for gaining potentially large future rewards, while exploitation relies on the already acquired knowledge to maximize the immediate rewards. We articulate an approach to this problem, treating the dynamical process of reinforcement learning as a Markov decision process that can be modeled as a nondeterministic finite automaton and defining a subset of states in the automaton to represent the preference for exploring unknown domains of the environment. Exploration is prioritized by assigning higher transition probabilities to these states. We derive a mathematical framework to systematically balance exploration and exploitation by formulating it as a mixed integer programming (MIP) problem to optimize the agent’s actions and maximize the discovery of novel preferential states. Solving the MIP problem provides a trade-off point between exploiting known states and exploring unexplored regions. We validate the framework computationally with a benchmark system and argue that the articulated automaton is effectively an adaptive network with a time-varying connection matrix, where the states in the automaton are nodes and the transitions among the states represent the edges. The network is adaptive because the transition probabilities evolve over time. The established connection between the adaptive automaton arising from reinforcement learning and the adaptive network opens the door to applying theories of complex dynamical networks to address frontier problems in machine learning and artificial intelligence.

Autor:innen und Institutionen

  • School of Electrical, Computer and Energy Engineering, Arizona State University, Tempe, Arizona 85287, USA.
  • Department of Physics, Arizona State University, Tempe, Arizona 85287, USA.

ORCID-Profile: Mohammadamin Moradi, Zheng-Meng Zhai, Shirin Panahi, Ying-Cheng Lai

Förderung

  • Air Force Office of Scientific Research

Zitieren

APA

Moradi, M., Zhai, Z. M., Panahi, S., & Lai, Y. C. (2024). Adaptive network approach to exploration–exploitation trade-off in reinforcement learning. Chaos: An Interdisciplinary Journal of Nonlinear Science, 34(12). https://doi.org/10.1063/5.0221833

Vancouver

Moradi M, Zhai ZM, Panahi S, Lai YC. Adaptive network approach to exploration–exploitation trade-off in reinforcement learning. Chaos: An Interdisciplinary Journal of Nonlinear Science. 2024;34(12). doi:10.1063/5.0221833

BibTeX
@article{Moradi2024,
  author = {Moradi, Mohammadamin and Zhai, Zheng-Meng and Panahi, Shirin and Lai, Ying-Cheng},
  title = {Adaptive network approach to exploration–exploitation trade-off in reinforcement learning},
  journal = {Chaos: An Interdisciplinary Journal of Nonlinear Science},
  year = {2024},
  volume = {34},
  number = {12},
  doi = {10.1063/5.0221833}
}
Kostenlos starten →