A probabilistic successor representation for context-dependent learning.
Psychological Review · 2024
DOI: 10.1037/rev0000414 ↗Kurz gefasst
Fachartikel in der Zeitschrift Psychological Review, 2024. Verfasst von Jesse P Geerts und 3 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Cognitive Neuroscience, University College London. Hauptthemen laut MEDLINE: Reinforcement, Psychology, Learning. Die Arbeit wurde bisher 14-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wellcome Trust, Gatsby Charitable Foundation, Air Force Office of Scientific Research.
Abstract (Originalsprache)
Two of the main impediments to learning complex tasks are that relationships between different stimuli, including rewards, can be uncertain and context-dependent. Reinforcement learning (RL) provides a framework for learning, by predicting total future reward directly (model-free RL), or via predictions of future states (model-based RL). Within this framework, "successor representation" (SR) predicts total future occupancy of all states. A recent theoretical proposal suggests that the hippocampus encodes the SR in order to facilitate prediction of future reward. However, this proposal does not take into account how learning should adapt under uncertainty and switches of context. Here, we introduce a theory of learning SRs using prediction errors which includes optimally balancing uncertainty in new observations versus existing knowledge. We then generalize that approach to a multicontext setting, allowing the model to learn and maintain multiple task-specific SRs and infer which one to use at any moment based on the accuracy of its predictions. Thus, the context used for predictions can be determined by both the contents of the states themselves and the distribution of transitions between them. This probabilistic SR model captures animal behavior in tasks which require contextual memory and generalization, and unifies previous SR theory with hippocampal-dependent contextual decision-making. (PsycInfo Database Record (c) 2024 APA, all rights reserved).
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Institute of Cognitive Neuroscience, University College London.
- Department of Psychology, Harvard University.
- DeepMind.
ORCID-Profile: Jesse P Geerts, Neil Burgess
Förderung
- Wellcome Trust
- Gatsby Charitable Foundation
- Air Force Office of Scientific Research
- European Research Council
Zitieren
APA
Geerts, J. P., Gershman, S. J., Burgess, N., & Stachenfeld, K. L. (2024). A probabilistic successor representation for context-dependent learning. Psychological Review. https://doi.org/10.1037/rev0000414
Vancouver
Geerts JP, Gershman SJ, Burgess N, Stachenfeld KL. A probabilistic successor representation for context-dependent learning. Psychological Review. 2024. doi:10.1037/rev0000414
BibTeX
@article{Geerts2024,
author = {Geerts, Jesse P and Gershman, Samuel J and Burgess, Neil and Stachenfeld, Kimberly L},
title = {A probabilistic successor representation for context-dependent learning},
journal = {Psychological Review},
year = {2024},
doi = {10.1037/rev0000414}
}
EUVIMED