EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples

Li Meng, Anis Yazidi, Morten Goodwin, Paal Engelstad

Proceedings of the Northern Lights Deep Learning Workshop · 2022

DOI: 10.7557/18.6237 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the Northern Lights Deep Learning Workshop, 2022. Verfasst von Li Meng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this article, we propose a novel algorithm for deep reinforcement learning named Expert Q-learning. Expert Q-learning is inspired by Dueling Q-learning and aims to incorporate semi-supervised learning into reinforcement learning through splitting Q-values into state values and action advantages. We require that an offline expert assesses the value of a state in a coarse manner using three discrete values. An expert network is designed in addition to the Q-network, which updates each time following the regular offline minibatch update whenever the expert example buffer is not empty. Using the board game Othello, we compare our algorithm with the baseline Q-learning algorithm, which is a combination of Double Q-learning and Dueling Q-learning. Our results show that Expert Q-learning is indeed useful and more resistant to the overestimation bias. The baseline Q-learning algorithm exhibits unstable and suboptimal behavior in non-deterministic settings, whereas Expert Q-learning demonstrates more robust performance with higher scores, illustrating that our algorithm is indeed suitable to integrate state values from expert examples into Q-learning.

Zitieren

APA

Meng, L., Yazidi, A., Goodwin, M., & Engelstad, P. (2022). Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples. Proceedings of the Northern Lights Deep Learning Workshop. https://doi.org/10.7557/18.6237

Vancouver

Meng L, Yazidi A, Goodwin M, Engelstad P. Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples. Proceedings of the Northern Lights Deep Learning Workshop. 2022. doi:10.7557/18.6237

BibTeX
@article{Meng2022,
  author = {Meng, Li and Yazidi, Anis and Goodwin, Morten and Engelstad, Paal},
  title = {Expert Q-learning: Deep Reinforcement Learning with Coarse State Values from Offline Expert Examples},
  journal = {Proceedings of the Northern Lights Deep Learning Workshop},
  year = {2022},
  doi = {10.7557/18.6237}
}
Kostenlos starten →