EUVIMEDEUVIMED entdecken
Crossref

Relating Human Error–Based Learning to Modern Deep RL Algorithms

Michele Garibbo, Casimir J H Ludwig, Nathan F Lepora, Laurence Aitchison

Neural Computation · 2024

DOI: 10.1162/neco_a_01721 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neural Computation, 2024. Verfasst von Michele Garibbo und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Engineering Mathematics, Faculty of Engineering, University of Bristol, Bristol BS8 1QU, U.K. michele.garibbo@bristol.ac.uk. Hauptthemen laut MEDLINE: Deep Learning, Algorithms, Reinforcement, Psychology. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 57 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In human error–based learning, the size and direction of a scalar error (i.e., the “directed error”) are used to update future actions. Modern deep reinforcement learning (RL) methods perform a similar operation but in terms of scalar rewards. Despite this similarity, the relationship between action updates of deep RL and human error–based learning has yet to be investigated. Here, we systematically compare the three major families of deep RL algorithms to human error–based learning. We show that all three deep RL approaches are qualitatively different from human error–based learning, as assessed by a mirror-reversal perturbation experiment. To bridge this gap, we developed an alternative deep RL algorithm inspired by human error–based learning, model-based deterministic policy gradients (MB-DPG). We showed that MB-DPG captures human error–based learning under mirror-reversal and rotational perturbations and that MB-DPG learns faster than canonical model-free algorithms on complex arm-based reaching tasks, while being more robust to (forward) model misspecification than model-based RL.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Deep LearningAlgorithmsReinforcement, PsychologyHumansLearningNeural Networks, Computer

Autor:innen und Institutionen

  • Department of Engineering Mathematics, Faculty of Engineering, University of Bristol, Bristol BS8 1QU, U.K. michele.garibbo@bristol.ac.uk.
  • School of Psychological Science, University of Bristol, Bristol BS8 1QU, U.K. c.ludwig@bristol.ac.uk.
  • Department of Engineering Mathematics, Faculty of Engineering, University of Bristol, Bristol BS8 1QU, U.K. n.lepora@bristol.ac.uk.
  • Department of Computer Science, Faculty of Engineering, University of Bristol, Bristol BS8 1QU, U.K. laurence.aitchison@bristol.ac.uk.

Zitieren

APA

Garibbo, M., Ludwig, C. J. H., Lepora, N. F., & Aitchison, L. (2024). Relating Human Error–Based Learning to Modern Deep RL Algorithms. Neural Computation. https://doi.org/10.1162/neco_a_01721

Vancouver

Garibbo M, Ludwig CJH, Lepora NF, Aitchison L. Relating Human Error–Based Learning to Modern Deep RL Algorithms. Neural Computation. 2024. doi:10.1162/neco_a_01721

BibTeX
@article{Garibbo2024,
  author = {Garibbo, Michele and Ludwig, Casimir J H and Lepora, Nathan F and Aitchison, Laurence},
  title = {Relating Human Error–Based Learning to Modern Deep RL Algorithms},
  journal = {Neural Computation},
  year = {2024},
  doi = {10.1162/neco_a_01721}
}
Kostenlos starten →