Trained recurrent neural networks develop phase-locked limit cycles in a working memory task
PLOS Computational Biology · 05.02.2024 · 20 (2) · e1011852
DOI: 10.1371/journal.pcbi.1011852 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Computational Biology, online erschienen am 05.02.2024. Verfasst von Matthijs Pals und 2 weiteren Autor:innen. Beteiligte Einrichtung: Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen, Tübingen, Germany. Hauptthemen laut MEDLINE: Memory, Short-Term, Brain. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deutsche Forschungsgemeinschaft (DFG, German Research Foundation), German Federal Ministry of Education and Research (BMBF), ISRAEL SCIENCE FOUNDATION.
Abstract (Originalsprache)
Neural oscillations are ubiquitously observed in many brain areas. One proposed functional role of these oscillations is that they serve as an internal clock, or ‘frame of reference’. Information can be encoded by the timing of neural activity relative to the phase of such oscillations. In line with this hypothesis, there have been multiple empirical observations of such phase codes in the brain. Here we ask: What kind of neural dynamics support phase coding of information with neural oscillations? We tackled this question by analyzing recurrent neural networks (RNNs) that were trained on a working memory task. The networks were given access to an external reference oscillation and tasked to produce an oscillation, such that the phase difference between the reference and output oscillation maintains the identity of transient stimuli. We found that networks converged to stable oscillatory dynamics. Reverse engineering these networks revealed that each phase-coded memory corresponds to a separate limit cycle attractor. We characterized how the stability of the attractor dynamics depends on both reference oscillation amplitude and frequency, properties that can be experimentally observed. To understand the connectivity structures that underlie these dynamics, we showed that trained networks can be described as two phase-coupled oscillators. Using this insight, we condensed our trained networks to a reduced model consisting of two functional modules: One that generates an oscillation and one that implements a coupling function between the internal oscillation and external reference. In summary, by reverse engineering the dynamics and connectivity of trained RNNs, we propose a mechanism by which neural networks can harness reference oscillations for working memory. Specifically, we propose that a phase-coding network generates autonomous oscillations which it couples to an external reference oscillation in a multi-stable fashion.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Machine Learning in Science, Excellence Cluster Machine Learning, University of Tübingen, Tübingen, Germany.
- Tübingen AI Center, University of Tübingen, Tübingen, Germany.
- Department Empirical Inference, Max Planck Institute for Intelligent Systems, Tübingen, Germany.
- Rappaport Faculty of Medicine Technion, Israel Institute of Technology, Haifa, Israel.
- Network Biology Research Laboratory, Israel Institute of Technology, Haifa, Israel.
ORCID-Profile: Matthijs Pals, Omri Barak
Förderung
- Deutsche Forschungsgemeinschaft (DFG, German Research Foundation)
- German Federal Ministry of Education and Research (BMBF)
- ISRAEL SCIENCE FOUNDATION
- Bundesministerium für Bildung und Forschung
- Human Frontier Science Program
Zitieren
APA
Pals, M., Macke, J. H., & Barak, O. (2024). Trained recurrent neural networks develop phase-locked limit cycles in a working memory task. PLOS Computational Biology, 20(2), e1011852. https://doi.org/10.1371/journal.pcbi.1011852
Vancouver
Pals M, Macke JH, Barak O. Trained recurrent neural networks develop phase-locked limit cycles in a working memory task. PLOS Computational Biology. 2024;20(2):e1011852. doi:10.1371/journal.pcbi.1011852
BibTeX
@article{Pals2024,
author = {Pals, Matthijs and Macke, Jakob H and Barak, Omri},
title = {Trained recurrent neural networks develop phase-locked limit cycles in a working memory task},
journal = {PLOS Computational Biology},
year = {2024},
volume = {20},
number = {2},
pages = {e1011852},
doi = {10.1371/journal.pcbi.1011852}
}
EUVIMED