Heterogeneity in Neuronal Dynamics Is Learned by Gradient Descent for Temporal Processing Tasks
Neural Computation · 18.03.2023 · 35 (4) · 555-592
DOI: 10.1162/neco_a_01571 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neural Computation, online erschienen am 18.03.2023. Verfasst von Chloe N Winston und 3 weiteren Autor:innen. Beteiligte Einrichtung: Departments of Neuroscience and Computer Science, University of Washington, Seattle, WA 98195, U.S.A. Hauptthemen laut MEDLINE: Time Perception. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIBIB NIH HHS, NIDA NIH HHS.
Abstract (Originalsprache)
Individual neurons in the brain have complex intrinsic dynamics that are highly diverse. We hypothesize that the complex dynamics produced by networks of complex and heterogeneous neurons may contribute to the brain's ability to process and respond to temporally complex data. To study the role of complex and heterogeneous neuronal dynamics in network computation, we develop a rate-based neuronal model, the generalized-leaky-integrate-and-fire-rate (GLIFR) model, which is a rate equivalent of the generalized-leaky-integrate-and-fire model. The GLIFR model has multiple dynamical mechanisms, which add to the complexity of its activity while maintaining differentiability. We focus on the role of after-spike currents, currents induced or modulated by neuronal spikes, in producing rich temporal dynamics. We use machine learning techniques to learn both synaptic weights and parameters underlying intrinsic dynamics to solve temporal tasks. The GLIFR model allows the use of standard gradient descent techniques rather than surrogate gradient descent, which has been used in spiking neural networks. After establishing the ability to optimize parameters using gradient descent in single neurons, we ask how networks of GLIFR neurons learn and perform on temporally challenging tasks, such as sequential MNIST. We find that these networks learn diverse parameters, which gives rise to diversity in neuronal dynamics, as demonstrated by clustering of neuronal parameters. GLIFR networks have mixed performance when compared to vanilla recurrent neural networks, with higher performance in pixel-by-pixel MNIST but lower in line-by-line MNIST. However, they appear to be more robust to random silencing. We find that the ability to learn heterogeneity and the presence of after-spike currents contribute to these gains in performance. Our work demonstrates both the computational robustness of neuronal complexity and diversity in networks and a feasible method of training such models using exact gradients.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Departments of Neuroscience and Computer Science, University of Washington, Seattle, WA 98195, U.S.A.
- University of Washington Computational Neuroscience Center, Seattle, WA 98195, U.S.A. wincnw@gmail.com.
- Allen Institute for Brain Science, Seattle, WA 98109, U.S.A. dana.mastrovito@alleninstitute.org.
- University of Washington Computational Neuroscience Center, Seattle, WA 98195, U.S.A.
- Allen Institute for Brain Science, Seattle, WA 98109, U.S.A.
- Department of Applied Mathematics, University of Washington, Seattle, WA 98195, U.S.A. etsb@uw.edu.
Förderung
- NIBIB NIH HHS
- NIDA NIH HHS
Zitieren
APA
Winston, C. N., Mastrovito, D., Shea-Brown, E., & Mihalas, S. (2023). Heterogeneity in Neuronal Dynamics Is Learned by Gradient Descent for Temporal Processing Tasks. Neural Computation, 35(4), 555-592. https://doi.org/10.1162/neco_a_01571
Vancouver
Winston CN, Mastrovito D, Shea-Brown E, Mihalas S. Heterogeneity in Neuronal Dynamics Is Learned by Gradient Descent for Temporal Processing Tasks. Neural Computation. 2023;35(4):555-592. doi:10.1162/neco_a_01571
BibTeX
@article{Winston2023,
author = {Winston, Chloe N and Mastrovito, Dana and Shea-Brown, Eric and Mihalas, Stefan},
title = {Heterogeneity in Neuronal Dynamics Is Learned by Gradient Descent for Temporal Processing Tasks},
journal = {Neural Computation},
year = {2023},
volume = {35},
number = {4},
pages = {555--592},
doi = {10.1162/neco_a_01571}
}
EUVIMED