EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A probabilistic successor representation for context-dependent learning.

Jesse P Geerts iD, Samuel J Gershman, Neil Burgess iD, Kimberly L Stachenfeld

Psychological Review · 2024

DOI: 10.1037/rev0000414 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Psychological Review, 2024. Verfasst von Jesse P Geerts und 3 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Cognitive Neuroscience, University College London. Hauptthemen laut MEDLINE: Reinforcement, Psychology, Learning. Die Arbeit wurde bisher 14-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wellcome Trust, Gatsby Charitable Foundation, Air Force Office of Scientific Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Two of the main impediments to learning complex tasks are that relationships between different stimuli, including rewards, can be uncertain and context-dependent. Reinforcement learning (RL) provides a framework for learning, by predicting total future reward directly (model-free RL), or via predictions of future states (model-based RL). Within this framework, "successor representation" (SR) predicts total future occupancy of all states. A recent theoretical proposal suggests that the hippocampus encodes the SR in order to facilitate prediction of future reward. However, this proposal does not take into account how learning should adapt under uncertainty and switches of context. Here, we introduce a theory of learning SRs using prediction errors which includes optimally balancing uncertainty in new observations versus existing knowledge. We then generalize that approach to a multicontext setting, allowing the model to learn and maintain multiple task-specific SRs and infer which one to use at any moment based on the accuracy of its predictions. Thus, the context used for predictions can be determined by both the contents of the states themselves and the distribution of transitions between them. This probabilistic SR model captures animal behavior in tasks which require contextual memory and generalization, and unifies previous SR theory with hippocampal-dependent contextual decision-making. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Reinforcement, PsychologyLearningAnimalsHumansRewardUncertaintyGeneralization, Psychological

Autor:innen und Institutionen

  • Institute of Cognitive Neuroscience, University College London.
  • Department of Psychology, Harvard University.
  • DeepMind.

ORCID-Profile: Jesse P Geerts, Neil Burgess

Förderung

  • Wellcome Trust
  • Gatsby Charitable Foundation
  • Air Force Office of Scientific Research
  • European Research Council

Zitieren

APA

Geerts, J. P., Gershman, S. J., Burgess, N., & Stachenfeld, K. L. (2024). A probabilistic successor representation for context-dependent learning. Psychological Review. https://doi.org/10.1037/rev0000414

Vancouver

Geerts JP, Gershman SJ, Burgess N, Stachenfeld KL. A probabilistic successor representation for context-dependent learning. Psychological Review. 2024. doi:10.1037/rev0000414

BibTeX
@article{Geerts2024,
  author = {Geerts, Jesse P and Gershman, Samuel J and Burgess, Neil and Stachenfeld, Kimberly L},
  title = {A probabilistic successor representation for context-dependent learning},
  journal = {Psychological Review},
  year = {2024},
  doi = {10.1037/rev0000414}
}
Kostenlos starten →