EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Recurrent neural networks that learn multi-step visual routines with reinforcement learning

Sami Mollard iD, Catherine Wacongne, Sander M Bohte, Pieter R Roelfsema iD

PLOS Computational Biology · 29.04.2024 · 20 (4) · e1012030

DOI: 10.1371/journal.pcbi.1012030 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, online erschienen am 29.04.2024. Verfasst von Sami Mollard und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Vision & Cognition, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands. Hauptthemen laut MEDLINE: Visual Cortex, Neural Networks, Computer, Reinforcement, Psychology, Models, Neurological. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Union's Horizon 2020 Framework Programme for Research and Innovation, Horizon Europe, Horizon 2020 Framework Programme.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Many cognitive problems can be decomposed into series of subproblems that are solved sequentially by the brain. When subproblems are solved, relevant intermediate results need to be stored by neurons and propagated to the next subproblem, until the overarching goal has been completed. We will here consider visual tasks, which can be decomposed into sequences of elemental visual operations. Experimental evidence suggests that intermediate results of the elemental operations are stored in working memory as an enhancement of neural activity in the visual cortex. The focus of enhanced activity is then available for subsequent operations to act upon. The main question at stake is how the elemental operations and their sequencing can emerge in neural networks that are trained with only rewards, in a reinforcement learning setting. We here propose a new recurrent neural network architecture that can learn composite visual tasks that require the application of successive elemental operations. Specifically, we selected three tasks for which electrophysiological recordings of monkeys’ visual cortex are available. To train the networks, we used RELEARNN, a biologically plausible four-factor Hebbian learning rule, which is local both in time and space. We report that networks learn elemental operations, such as contour grouping and visual search, and execute sequences of operations, solely based on the characteristics of the visual stimuli and the reward structure of a task. After training was completed, the activity of the units of the neural network elicited by behaviorally relevant image items was stronger than that elicited by irrelevant ones, just as has been observed in the visual cortex of monkeys solving the same tasks. Relevant information that needed to be exchanged between subroutines was maintained as a focus of enhanced activity and passed on to the subsequent subroutines. Our results demonstrate how a biologically plausible learning rule can train a recurrent neural network on multistep visual tasks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Visual CortexNeural Networks, ComputerReinforcement, PsychologyModels, NeurologicalAnimalsComputational BiologyMemory, Short-TermNeuronsLearningVisual PerceptionMacaca mulatta

Autor:innen und Institutionen

  • Department of Vision & Cognition, Netherlands Institute for Neuroscience, Amsterdam, The Netherlands.
  • AnotherBrain, Paris, France.
  • Machine Learning Group, Centrum Wiskunde & Informatica, Amsterdam, The Netherlands.
  • Swammerdam Institute for Life Sciences, University of Amsterdam, Amsterdam, The Netherlands.
  • Laboratory of Visual Brain Therapy, Sorbonne Université, Institut National de la Santé et de la Recherche Médicale, Centre National de la Recherche Scientifique, Institut de la Vision, Paris, France.
  • Department of Integrative Neurophysiology, Center for Neurogenomics and Cognitive Research, VU University, Amsterdam, The Netherlands.

ORCID-Profile: Sami Mollard, Pieter R Roelfsema

Förderung

  • European Union's Horizon 2020 Framework Programme for Research and Innovation
  • Horizon Europe
  • Horizon 2020 Framework Programme
  • European Research Council
  • Nederlandse Organisatie voor Wetenschappelijk Onderzoek
  • Agence Nationale de la Recherche

Zitieren

APA

Mollard, S., Wacongne, C., Bohte, S. M., & Roelfsema, P. R. (2024). Recurrent neural networks that learn multi-step visual routines with reinforcement learning. PLOS Computational Biology, 20(4), e1012030. https://doi.org/10.1371/journal.pcbi.1012030

Vancouver

Mollard S, Wacongne C, Bohte SM, Roelfsema PR. Recurrent neural networks that learn multi-step visual routines with reinforcement learning. PLOS Computational Biology. 2024;20(4):e1012030. doi:10.1371/journal.pcbi.1012030

BibTeX
@article{Mollard2024,
  author = {Mollard, Sami and Wacongne, Catherine and Bohte, Sander M and Roelfsema, Pieter R},
  title = {Recurrent neural networks that learn multi-step visual routines with reinforcement learning},
  journal = {PLOS Computational Biology},
  year = {2024},
  volume = {20},
  number = {4},
  pages = {e1012030},
  doi = {10.1371/journal.pcbi.1012030}
}
Kostenlos starten →