Social satisficing: Multi-agent reinforcement learning with satisficing agents
BioSystems · 2024
DOI: 10.1016/j.biosystems.2024.105276 ↗Kurz gefasst
Fachartikel in der Zeitschrift BioSystems, 2024. Verfasst von Daisuke Uragami und 2 weiteren Autor:innen. Beteiligte Einrichtung: College of Industrial Technology, Nihon University, 1-2-1, Izumi, Narashino, Chiba, 275-8575, Japan. Hauptthemen laut MEDLINE: Reinforcement, Psychology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: JSPS, Japan Society for the Promotion of Science.
Abstract (Originalsprache)
For a reinforcement learning agent to finish trial-and-error in a realistic time duration, it is necessary to limit the scope of exploration during the learning process. However, limiting the exploration scope means limitation in optimality: the agent could fall into a suboptimal solution. This is the nature of local, bottom-up way of learning. An alternative way is to set a goal to be achieved, which is a more global, top-down way. The risk-sensitive satisficing (RS) value function incorporate, as a method of the latter way, the satisficing principle into reinforcement learning and enables agents to quickly converge to exploiting the optimal solution without falling into a suboptimal one, when an appropriate goal (aspiration level) is given. However, how best to determine the aspiration level is still an open problem. This study proposes social satisficing, a framework for multi-agent reinforcement learning which determines the aspiration level through information sharing among multiple agents. In order to verify the effectiveness of this novel method, we conducted simulations in a learning environment with many suboptimal goals (SuboptimaWorld). The results show that the proposed method, which converts the aspiration level at the episodic level into local (state-wise) aspiration levels, possesses a higher learning efficiency than any of the compared methods, and that the novel method has the ability to autonomously adjust exploration scope, while keeping the shared information minimal. This study provides a glimpse into an aspect of human and biological sociality which has been mentioned little in the context of artificial intelligence and machine learning.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- College of Industrial Technology, Nihon University, 1-2-1, Izumi, Narashino, Chiba, 275-8575, Japan.
- School of Science and Engineering, Tokyo Denki University, Hiki, Saitama, 350-0394, Japan.
- School of Science and Engineering, Tokyo Denki University, Hiki, Saitama, 350-0394, Japan. Electronic address: tatsuji.takahashi@gmail.com.
Förderung
- JSPS
- Japan Society for the Promotion of Science
Zitieren
APA
Uragami, D., Sonota, N., & Takahashi, T. (2024). Social satisficing: Multi-agent reinforcement learning with satisficing agents. BioSystems. https://doi.org/10.1016/j.biosystems.2024.105276
Vancouver
Uragami D, Sonota N, Takahashi T. Social satisficing: Multi-agent reinforcement learning with satisficing agents. BioSystems. 2024. doi:10.1016/j.biosystems.2024.105276
BibTeX
@article{Uragami2024,
author = {Uragami, Daisuke and Sonota, Noriaki and Takahashi, Tatsuji},
title = {Social satisficing: Multi-agent reinforcement learning with satisficing agents},
journal = {BioSystems},
year = {2024},
doi = {10.1016/j.biosystems.2024.105276}
}
EUVIMED