Learning with sparse reward in a gap junction network inspired by the insect mushroom body
PLOS Computational Biology · 2024
DOI: 10.1371/journal.pcbi.1012086 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS Computational Biology, 2024. Verfasst von Tianqi Wei und 2 weiteren Autor:innen. Beteiligte Einrichtung: Institute of Perception, Action, and Behaviour, School of Informatics, University of Edinburgh, Edinburgh, United Kingdom. Hauptthemen laut MEDLINE: Mushroom Bodies, Reward, Algorithms, Gap Junctions. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 73 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Huawei Technologies Co., Ltd, Huawei Technologies.
Abstract (Originalsprache)
Animals can learn in real-life scenarios where rewards are often only available when a goal is achieved. This ‘distal’ or ‘sparse’ reward problem remains a challenge for conventional reinforcement learning algorithms. Here we investigate an algorithm for learning in such scenarios, inspired by the possibility that axo-axonal gap junction connections, observed in neural circuits with parallel fibres such as the insect mushroom body, could form a resistive network. In such a network, an active node represents the task state, connections between nodes represent state transitions and their connection to actions, and current flow to a target state can guide decision making. Building on evidence that gap junction weights are adaptive, we propose that experience of a task can modulate the connections to form a graph encoding the task structure. We demonstrate that the approach can be used for efficient reinforcement learning under sparse rewards, and discuss whether it is plausible as an account of the insect mushroom body.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Institute of Perception, Action, and Behaviour, School of Informatics, University of Edinburgh, Edinburgh, United Kingdom.
- School of Artificial Intelligence, Sun Yat-sen University, Zhuhai, Guangdong, China.
- Huawei Technologies Co., Ltd., Shenzhen, Guangdong, China.
ORCID-Profile: Barbara Webb
Förderung
- Huawei Technologies Co., Ltd
- Huawei Technologies
Zitieren
APA
Wei, T., Guo, Q., & Webb, B. (2024). Learning with sparse reward in a gap junction network inspired by the insect mushroom body. PLOS Computational Biology. https://doi.org/10.1371/journal.pcbi.1012086
Vancouver
Wei T, Guo Q, Webb B. Learning with sparse reward in a gap junction network inspired by the insect mushroom body. PLOS Computational Biology. 2024. doi:10.1371/journal.pcbi.1012086
BibTeX
@article{Wei2024,
author = {Wei, Tianqi and Guo, Qinghai and Webb, Barbara},
title = {Learning with sparse reward in a gap junction network inspired by the insect mushroom body},
journal = {PLOS Computational Biology},
year = {2024},
doi = {10.1371/journal.pcbi.1012086}
}
EUVIMED