EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex

Tim Vriens iD, Eliana Vassena iD, Giovanni Pezzulo, Gianluca Baldassarre, Massimo Silvetti iD

PLOS Computational Biology · 2025

DOI: 10.1371/journal.pcbi.1013025 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, 2025. Verfasst von Tim Vriens und 4 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Cognitive Sciences and Technologies, CNR, Rome, Italy. Hauptthemen laut MEDLINE: Gyrus Cinguli, Reinforcement, Psychology, Learning. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 70 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Consiglio Nazionale delle Ricerche (CNR, Italy), Netherlands Organisation for Scientific Research (NWO), European Research Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The role of the dorsal anterior cingulate cortex (dACC) in cognition is a frequently studied yet highly debated topic in neuroscience. Most authors agree that the dACC is involved in either cognitive control (e.g., voluntary inhibition of automatic responses) or monitoring (e.g., comparing expectations with outcomes, detecting errors, tracking surprise). A consensus on which theoretical perspective best explains dACC contribution to behaviour is still lacking, as two distinct sets of studies report dACC activation in tasks requiring surprise tracking for performance monitoring and cognitive control without involving surprise monitoring, respectively. This creates a theoretical impasse, as no single current account can reconcile these findings. Here we propose a novel hypothesis on dACC function that integrates both the monitoring and the cognitive control perspectives in a unifying, meta-Reinforcement Learning framework, in which cognitive control is optimized by meta-learning based on tracking Bayesian surprise. We tested the quantitative predictions from our theory in three different functional neuroimaging experiments at the basis of the current theory crisis. We show that the meta-Reinforcement Learning perspective successfully captures all the neuroimaging results by predicting both cognitive control and monitoring functions, proposing a solution to the theory crisis about dACC function within an integrative framework. In sum, our results suggest that dACC function can be framed as a meta-learning optimisation of cognitive control, providing an integrative perspective on its roles in cognitive control, surprise tracking, and performance monitoring.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Gyrus CinguliReinforcement, PsychologyLearningHumansCognitionBayes TheoremMaleMagnetic Resonance ImagingAdultFemaleComputational BiologyYoung AdultModels, NeurologicalBrain Mapping

Autor:innen und Institutionen

  • Institute of Cognitive Sciences and Technologies, CNR, Rome, Italy.
  • Behavioural Science Institute, Radboud University, Nijmegen, The Netherlands.
  • Donders Institute for Brain, Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands.

ORCID-Profile: Tim Vriens, Eliana Vassena, Massimo Silvetti

Förderung

  • Consiglio Nazionale delle Ricerche (CNR, Italy)
  • Netherlands Organisation for Scientific Research (NWO)
  • European Research Council
  • Italian National Recovery and Resilience Plan (NRRP)
  • European Union – NextGenerationEU
  • Ministero dell’Istruzione, dell’Università e della Ricerca
  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek
  • Italian National Recovery and Resilience Plan /European Union – NextGenerationEU

Zitieren

APA

Vriens, T., Vassena, E., Pezzulo, G., Baldassarre, G., & Silvetti, M. (2025). Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1013025

Vancouver

Vriens T, Vassena E, Pezzulo G, Baldassarre G, Silvetti M. Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex. PLOS Computational Biology. 2025. doi:10.1371/journal.pcbi.1013025

BibTeX
@article{Vriens2025,
  author = {Vriens, Tim and Vassena, Eliana and Pezzulo, Giovanni and Baldassarre, Gianluca and Silvetti, Massimo},
  title = {Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex},
  journal = {PLOS Computational Biology},
  year = {2025},
  doi = {10.1371/journal.pcbi.1013025}
}
Kostenlos starten →