Relating Human Error–Based Learning to Modern Deep RL Algorithms
Neural Computation · 2024
DOI: 10.1162/neco_a_01721 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neural Computation, 2024. Verfasst von Michele Garibbo und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Engineering Mathematics, Faculty of Engineering, University of Bristol, Bristol BS8 1QU, U.K. michele.garibbo@bristol.ac.uk. Hauptthemen laut MEDLINE: Deep Learning, Algorithms, Reinforcement, Psychology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 57 Literaturangaben.
Abstract (Originalsprache)
In human error–based learning, the size and direction of a scalar error (i.e., the “directed error”) are used to update future actions. Modern deep reinforcement learning (RL) methods perform a similar operation but in terms of scalar rewards. Despite this similarity, the relationship between action updates of deep RL and human error–based learning has yet to be investigated. Here, we systematically compare the three major families of deep RL algorithms to human error–based learning. We show that all three deep RL approaches are qualitatively different from human error–based learning, as assessed by a mirror-reversal perturbation experiment. To bridge this gap, we developed an alternative deep RL algorithm inspired by human error–based learning, model-based deterministic policy gradients (MB-DPG). We showed that MB-DPG captures human error–based learning under mirror-reversal and rotational perturbations and that MB-DPG learns faster than canonical model-free algorithms on complex arm-based reaching tasks, while being more robust to (forward) model misspecification than model-based RL.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Engineering Mathematics, Faculty of Engineering, University of Bristol, Bristol BS8 1QU, U.K. michele.garibbo@bristol.ac.uk.
- School of Psychological Science, University of Bristol, Bristol BS8 1QU, U.K. c.ludwig@bristol.ac.uk.
- Department of Engineering Mathematics, Faculty of Engineering, University of Bristol, Bristol BS8 1QU, U.K. n.lepora@bristol.ac.uk.
- Department of Computer Science, Faculty of Engineering, University of Bristol, Bristol BS8 1QU, U.K. laurence.aitchison@bristol.ac.uk.
Zitieren
APA
Garibbo, M., Ludwig, C. J. H., Lepora, N. F., & Aitchison, L. (2024). Relating Human Error–Based Learning to Modern Deep RL Algorithms. Neural Computation. https://doi.org/10.1162/neco_a_01721
Vancouver
Garibbo M, Ludwig CJH, Lepora NF, Aitchison L. Relating Human Error–Based Learning to Modern Deep RL Algorithms. Neural Computation. 2024. doi:10.1162/neco_a_01721
BibTeX
@article{Garibbo2024,
author = {Garibbo, Michele and Ludwig, Casimir J H and Lepora, Nathan F and Aitchison, Laurence},
title = {Relating Human Error–Based Learning to Modern Deep RL Algorithms},
journal = {Neural Computation},
year = {2024},
doi = {10.1162/neco_a_01721}
}
EUVIMED