EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Social satisficing: Multi-agent reinforcement learning with satisficing agents

Daisuke Uragami, Noriaki Sonota, Tatsuji Takahashi

BioSystems · 2024

DOI: 10.1016/j.biosystems.2024.105276 ↗

Kurz gefasst

Fachartikel in der Zeitschrift BioSystems, 2024. Verfasst von Daisuke Uragami und 2 weiteren Autor:innen. Beteiligte Einrichtung: College of Industrial Technology, Nihon University, 1-2-1, Izumi, Narashino, Chiba, 275-8575, Japan. Hauptthemen laut MEDLINE: Reinforcement, Psychology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: JSPS, Japan Society for the Promotion of Science.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

For a reinforcement learning agent to finish trial-and-error in a realistic time duration, it is necessary to limit the scope of exploration during the learning process. However, limiting the exploration scope means limitation in optimality: the agent could fall into a suboptimal solution. This is the nature of local, bottom-up way of learning. An alternative way is to set a goal to be achieved, which is a more global, top-down way. The risk-sensitive satisficing (RS) value function incorporate, as a method of the latter way, the satisficing principle into reinforcement learning and enables agents to quickly converge to exploiting the optimal solution without falling into a suboptimal one, when an appropriate goal (aspiration level) is given. However, how best to determine the aspiration level is still an open problem. This study proposes social satisficing, a framework for multi-agent reinforcement learning which determines the aspiration level through information sharing among multiple agents. In order to verify the effectiveness of this novel method, we conducted simulations in a learning environment with many suboptimal goals (SuboptimaWorld). The results show that the proposed method, which converts the aspiration level at the episodic level into local (state-wise) aspiration levels, possesses a higher learning efficiency than any of the compared methods, and that the novel method has the ability to autonomously adjust exploration scope, while keeping the shared information minimal. This study provides a glimpse into an aspect of human and biological sociality which has been mentioned little in the context of artificial intelligence and machine learning.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Reinforcement, PsychologyHumansComputer SimulationAlgorithmsLearningArtificial Intelligence

Stichwörter der Autor:innen

Aspiration levelDistributed reinforcement learningExploration-exploitation dilemmaSocial learningSuboptimaWorld

Autor:innen und Institutionen

  • College of Industrial Technology, Nihon University, 1-2-1, Izumi, Narashino, Chiba, 275-8575, Japan.
  • School of Science and Engineering, Tokyo Denki University, Hiki, Saitama, 350-0394, Japan.
  • School of Science and Engineering, Tokyo Denki University, Hiki, Saitama, 350-0394, Japan. Electronic address: tatsuji.takahashi@gmail.com.

Förderung

  • JSPS
  • Japan Society for the Promotion of Science

Zitieren

APA

Uragami, D., Sonota, N., & Takahashi, T. (2024). Social satisficing: Multi-agent reinforcement learning with satisficing agents. BioSystems. https://doi.org/10.1016/j.biosystems.2024.105276

Vancouver

Uragami D, Sonota N, Takahashi T. Social satisficing: Multi-agent reinforcement learning with satisficing agents. BioSystems. 2024. doi:10.1016/j.biosystems.2024.105276

BibTeX
@article{Uragami2024,
  author = {Uragami, Daisuke and Sonota, Noriaki and Takahashi, Tatsuji},
  title = {Social satisficing: Multi-agent reinforcement learning with satisficing agents},
  journal = {BioSystems},
  year = {2024},
  doi = {10.1016/j.biosystems.2024.105276}
}
Kostenlos starten →