An introduction to reinforcement learning for neuroscience
Neurons, Behavior, Data analysis, and Theory · 2024
DOI: 10.51628/001c.127771 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neurons, Behavior, Data analysis, and Theory, 2024. Verfasst von Kristopher T. Jensen. Die Arbeit wurde bisher 8-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Reinforcement learning has a rich history in neuroscience, from early work on dopamine as a reward prediction error signal for temporal difference learning (Schultz et al., 1997) to recent work suggesting that dopamine could implement a form of ‘distributional reinforcement learning’ popularized in deep learning (Dabney et al., 2020). Throughout this literature, there has been a tight link between theoretical advances in reinforcement learning and neuroscientific experiments and findings. As a result, the theories describing our experimental data have become increasingly complex and difficult to navigate. In this review, we cover the basic theory underlying classical work in reinforcement learning and build up to an introductory overview of methods in modern deep reinforcement learning that have found applications in systems neuroscience. We start with an overview of the reinforcement learning problem and classical temporal difference algorithms, followed by a discussion of ‘model-free’ and ‘model-based’ reinforcement learning together with methods such as DYNA and successor representations that fall in between these two extremes. Throughout these sections, we highlight the close parallels between such machine learning methods and related work in both experimental and theoretical neuroscience. We then provide an introduction to deep reinforcement learning with examples of how these methods have been used to model different learning phenomena in systems neuroscience, such as meta-reinforcement learning (Wang et al., 2018) and distributional reinforcement learning (Dabney et al., 2020). Code that implements the methods discussed in this work and generates the figures is also provided.
Zitieren
APA
Jensen, K. T. (2024). An introduction to reinforcement learning for neuroscience. Neurons, Behavior, Data analysis, and Theory. https://doi.org/10.51628/001c.127771
Vancouver
Jensen KT. An introduction to reinforcement learning for neuroscience. Neurons, Behavior, Data analysis, and Theory. 2024. doi:10.51628/001c.127771
BibTeX
@article{Jensen2024,
author = {Jensen, Kristopher T.},
title = {An introduction to reinforcement learning for neuroscience},
journal = {Neurons, Behavior, Data analysis, and Theory},
year = {2024},
doi = {10.51628/001c.127771}
}
EUVIMED