Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning
PLOS Computational Biology · 2025
DOI: 10.1371/journal.pcbi.1012899 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Computational Biology, 2025. Verfasst von Huu Hoang und 6 weiteren Autor:innen. Beteiligte Einrichtung: Neural Information Analysis Laboratories, Advanced Telecommunications Research Institute International, Kyoto, Japan. Hauptthemen laut MEDLINE: Reward, Reinforcement, Psychology, Supervised Machine Learning, Cerebellum. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 84 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Scientific Research in Innovative Areas, Grants-in-Aid for Transformative Research Areas, Japan Agency for Medical Research and Development (AMED).
Abstract (Originalsprache)
Although the cerebellum is typically associated with supervised learning algorithms, it also exhibits extensive involvement in reward processing. In this study, we investigated the cerebellum’s role in executing reinforcement learning algorithms, with a particular emphasis on essential reward-prediction errors. We employed the Q-learning model to accurately reproduce the licking responses of mice in a Go/No-go auditory-discrimination task. This method enabled the calculation of reinforcement learning variables, such as reward, predicted reward, and reward-prediction errors in each learning trial. Through tensor component analysis of two-photon Ca 2+ imaging data from more than 6,000 Purkinje cells, we found that climbing fiber inputs of the two distinct components, which were specifically activated during Go and No-go cues in the learning process, showed an inverse relationship with predictive reward-prediction errors. Assuming bidirectional parallel-fiber Purkinje-cell synaptic plasticity, we constructed a cerebellar neural-network model with 5,000 spiking neurons of granule cells, Purkinje cells, cerebellar nuclei neurons, and inferior olive neurons. The network model qualitatively reproduced distinct changes in licking behaviors, climbing-fiber firing rates, and their synchronization during discrimination learning separately for Go/No-go conditions. We found that Purkinje cells in the two components could develop specific motor commands for their respective auditory cues, guided by the predictive reward-prediction errors from their climbing fiber inputs. These results indicate a possible role of context-specific actors in modular reinforcement learning, integrating with cerebellar supervised learning capabilities.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Neural Information Analysis Laboratories, Advanced Telecommunications Research Institute International, Kyoto, Japan.
- Laboratory for Multi-scale Biological Psychiatry, RIKEN Center for Brain Science, Saitama, Japan.
- Department of Physiology, The University of Tokyo, Tokyo, Japan.
- Department of Neurophysiology, The University of Tokyo, Tokyo, Japan.
- International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo, Tokyo, Japan.
- Department of Neurophysiology, University of Yamanashi, Yamanashi, Japan.
ORCID-Profile: Huu Hoang, Shinichiro Tsutsumi
Förderung
- Scientific Research in Innovative Areas
- Grants-in-Aid for Transformative Research Areas
- Japan Agency for Medical Research and Development (AMED)
- Innovative Science and Technology Initiative for Security
- Acquisition, Technology & Logistics Agency (ATLA)
- Japan Society for the Promotion of Science (JSPS)
- Exploratory Research for Advanced Technology
Zitieren
APA
Hoang, H., Tsutsumi, S., Matsuzaki, M., Kano, M., Toyama, K., Kitamura, K., & Kawato, M. (2025). Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1012899
Vancouver
Hoang H, Tsutsumi S, Matsuzaki M, Kano M, Toyama K, Kitamura K, et al. Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning. PLOS Computational Biology. 2025. doi:10.1371/journal.pcbi.1012899
BibTeX
@article{Hoang2025,
author = {Hoang, Huu and Tsutsumi, Shinichiro and Matsuzaki, Masanori and Kano, Masanobu and Toyama, Keisuke and Kitamura, Kazuo and Kawato, Mitsuo},
title = {Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning},
journal = {PLOS Computational Biology},
year = {2025},
doi = {10.1371/journal.pcbi.1012899}
}
EUVIMED