EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Emergence of belief-like representations through reinforcement learning

Jay A Hennig iD, Sandra A Romero Pinto, Takahiro Yamaguchi, Scott W Linderman, Naoshige Uchida, Samuel J Gershman

PLOS Computational Biology · 2023

DOI: 10.1371/journal.pcbi.1011067 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, 2023. Verfasst von Jay A Hennig und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America. Hauptthemen laut MEDLINE: Learning, Reinforcement, Psychology. Die Arbeit wurde bisher 44-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NINDS NIH HHS, National Institutes of Health, Air Force Research Laboratory.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

To behave adaptively, animals must learn to predict future reward, or value. To do this, animals are thought to learn reward predictions using reinforcement learning. However, in contrast to classical models, animals must learn to estimate value using only incomplete state information. Previous work suggests that animals estimate value in partially observable tasks by first forming “beliefs”—optimal Bayesian estimates of the hidden states in the task. Although this is one way to solve the problem of partial observability, it is not the only way, nor is it the most computationally scalable solution in complex, real-world environments. Here we show that a recurrent neural network (RNN) can learn to estimate value directly from observations, generating reward prediction errors that resemble those observed experimentally, without any explicit objective of estimating beliefs. We integrate statistical, functional, and dynamical systems perspectives on beliefs to show that the RNN’s learned representation encodes belief information, but only when the RNN’s capacity is sufficiently large. These results illustrate how animals can estimate value in tasks without explicitly estimating beliefs, yielding a representation useful for systems with limited capacity.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LearningReinforcement, PsychologyAnimalsBayes TheoremRewardNeural Networks, Computer

Autor:innen und Institutionen

  • Department of Psychology, Harvard University, Cambridge, Massachusetts, United States of America.
  • Center for Brain Science, Harvard University, Cambridge, Massachusetts, United States of America.
  • Department of Molecular and Cellular Biology, Harvard University, Cambridge, Massachusetts, United States of America.
  • Program in Speech and Hearing Bioscience and Technology, Harvard Medical School, Boston, Massachusetts, USA.
  • Future Research Department, Toyota Research Institute of North America, Toyota Motor North America, Ann Arbor, Michigan, United States of America.
  • Wu Tsai Neurosciences Institute, Stanford University, Stanford, California, United States of America.

ORCID-Profile: Jay A Hennig

Förderung

  • NINDS NIH HHS
  • National Institutes of Health
  • Air Force Research Laboratory

Zitieren

APA

Hennig, J. A., Romero Pinto, S. A., Yamaguchi, T., Linderman, S. W., Uchida, N., & Gershman, S. J. (2023). Emergence of belief-like representations through reinforcement learning. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1011067

Vancouver

Hennig JA, Romero Pinto SA, Yamaguchi T, Linderman SW, Uchida N, Gershman SJ. Emergence of belief-like representations through reinforcement learning. PLOS Computational Biology. 2023. doi:10.1371/journal.pcbi.1011067

BibTeX
@article{Hennig2023,
  author = {Hennig, Jay A and Romero Pinto, Sandra A and Yamaguchi, Takahiro and Linderman, Scott W and Uchida, Naoshige and Gershman, Samuel J},
  title = {Emergence of belief-like representations through reinforcement learning},
  journal = {PLOS Computational Biology},
  year = {2023},
  doi = {10.1371/journal.pcbi.1011067}
}
Kostenlos starten →