EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning

Huu Hoang iD, Shinichiro Tsutsumi iD, Masanori Matsuzaki, Masanobu Kano, Keisuke Toyama, Kazuo Kitamura, Mitsuo Kawato

PLOS Computational Biology · 2025

DOI: 10.1371/journal.pcbi.1012899 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS Computational Biology, 2025. Verfasst von Huu Hoang und 6 weiteren Autor:innen. Beteiligte Einrichtung: Neural Information Analysis Laboratories, Advanced Telecommunications Research Institute International, Kyoto, Japan. Hauptthemen laut MEDLINE: Reward, Reinforcement, Psychology, Supervised Machine Learning, Cerebellum. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 84 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Scientific Research in Innovative Areas, Grants-in-Aid for Transformative Research Areas, Japan Agency for Medical Research and Development (AMED).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Although the cerebellum is typically associated with supervised learning algorithms, it also exhibits extensive involvement in reward processing. In this study, we investigated the cerebellum’s role in executing reinforcement learning algorithms, with a particular emphasis on essential reward-prediction errors. We employed the Q-learning model to accurately reproduce the licking responses of mice in a Go/No-go auditory-discrimination task. This method enabled the calculation of reinforcement learning variables, such as reward, predicted reward, and reward-prediction errors in each learning trial. Through tensor component analysis of two-photon Ca 2+ imaging data from more than 6,000 Purkinje cells, we found that climbing fiber inputs of the two distinct components, which were specifically activated during Go and No-go cues in the learning process, showed an inverse relationship with predictive reward-prediction errors. Assuming bidirectional parallel-fiber Purkinje-cell synaptic plasticity, we constructed a cerebellar neural-network model with 5,000 spiking neurons of granule cells, Purkinje cells, cerebellar nuclei neurons, and inferior olive neurons. The network model qualitatively reproduced distinct changes in licking behaviors, climbing-fiber firing rates, and their synchronization during discrimination learning separately for Go/No-go conditions. We found that Purkinje cells in the two components could develop specific motor commands for their respective auditory cues, guided by the predictive reward-prediction errors from their climbing fiber inputs. These results indicate a possible role of context-specific actors in modular reinforcement learning, integrating with cerebellar supervised learning capabilities.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

RewardReinforcement, PsychologySupervised Machine LearningCerebellumModels, NeurologicalAnimalsMicePurkinje CellsAlgorithmsMaleComputational BiologyMice, Inbred C57BL

Autor:innen und Institutionen

  • Neural Information Analysis Laboratories, Advanced Telecommunications Research Institute International, Kyoto, Japan.
  • Laboratory for Multi-scale Biological Psychiatry, RIKEN Center for Brain Science, Saitama, Japan.
  • Department of Physiology, The University of Tokyo, Tokyo, Japan.
  • Department of Neurophysiology, The University of Tokyo, Tokyo, Japan.
  • International Research Center for Neurointelligence (WPI-IRCN), The University of Tokyo, Tokyo, Japan.
  • Department of Neurophysiology, University of Yamanashi, Yamanashi, Japan.

ORCID-Profile: Huu Hoang, Shinichiro Tsutsumi

Förderung

  • Scientific Research in Innovative Areas
  • Grants-in-Aid for Transformative Research Areas
  • Japan Agency for Medical Research and Development (AMED)
  • Innovative Science and Technology Initiative for Security
  • Acquisition, Technology & Logistics Agency (ATLA)
  • Japan Society for the Promotion of Science (JSPS)
  • Exploratory Research for Advanced Technology

Zitieren

APA

Hoang, H., Tsutsumi, S., Matsuzaki, M., Kano, M., Toyama, K., Kitamura, K., & Kawato, M. (2025). Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1012899

Vancouver

Hoang H, Tsutsumi S, Matsuzaki M, Kano M, Toyama K, Kitamura K, et al. Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning. PLOS Computational Biology. 2025. doi:10.1371/journal.pcbi.1012899

BibTeX
@article{Hoang2025,
  author = {Hoang, Huu and Tsutsumi, Shinichiro and Matsuzaki, Masanori and Kano, Masanobu and Toyama, Keisuke and Kitamura, Kazuo and Kawato, Mitsuo},
  title = {Predictive reward-prediction errors of climbing fiber inputs integrate modular reinforcement learning with supervised learning},
  journal = {PLOS Computational Biology},
  year = {2025},
  doi = {10.1371/journal.pcbi.1012899}
}
Kostenlos starten →