EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Surprise-minimization as a solution to the structural credit assignment problem

Franz Wurm iD, Benjamin Ernst, Marco Steinhauser

PLOS Computational Biology · 2024

DOI: 10.1371/journal.pcbi.1012175 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, 2024. Verfasst von Franz Wurm und 2 weiteren Autor:innen. Beteiligte Einrichtung: Catholic University of Eichstätt-Ingolstadt, Eichstätt, Germany. Hauptthemen laut MEDLINE: Choice Behavior. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 73 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The structural credit assignment problem arises when the causal structure between actions and subsequent outcomes is hidden from direct observation. To solve this problem and enable goal-directed behavior, an agent has to infer structure and form a representation thereof. In the scope of this study, we investigate a possible solution in the human brain. We recorded behavioral and electrophysiological data from human participants in a novel variant of the bandit task, where multiple actions lead to multiple outcomes. Crucially, the mapping between actions and outcomes was hidden and not instructed to the participants. Human choice behavior revealed clear hallmarks of credit assignment and learning. Moreover, a computational model which formalizes action selection as the competition between multiple representations of the hidden structure was fit to account for participants data. Starting in a state of uncertainty about the correct representation, the central mechanism of this model is the arbitration of action control towards the representation which minimizes surprise about outcomes. Crucially, single-trial latent-variable analysis reveals that the neural patterns clearly support central quantitative predictions of this surprise minimization model. The results suggest that neural activity is not only related to reinforcement learning under correct as well as incorrect task representations but also reflects central mechanisms of credit assignment and behavioral arbitration.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Choice BehaviorHumansMaleFemaleAdultBrainModels, NeurologicalComputational BiologyYoung AdultLearningReinforcement, PsychologyComputer SimulationElectroencephalography

Autor:innen und Institutionen

  • Catholic University of Eichstätt-Ingolstadt, Eichstätt, Germany.
  • Leiden University, Leiden, the Netherlands.
  • Leiden Institute for Brain and Cognition, Leiden University, Leiden, the Netherlands.

ORCID-Profile: Franz Wurm

Zitieren

APA

Wurm, F., Ernst, B., & Steinhauser, M. (2024). Surprise-minimization as a solution to the structural credit assignment problem. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1012175

Vancouver

Wurm F, Ernst B, Steinhauser M. Surprise-minimization as a solution to the structural credit assignment problem. PLOS Computational Biology. 2024. doi:10.1371/journal.pcbi.1012175

BibTeX
@article{Wurm2024,
  author = {Wurm, Franz and Ernst, Benjamin and Steinhauser, Marco},
  title = {Surprise-minimization as a solution to the structural credit assignment problem},
  journal = {PLOS Computational Biology},
  year = {2024},
  doi = {10.1371/journal.pcbi.1012175}
}
Kostenlos starten →