A Survey of Zero-shot Generalisation in Deep Reinforcement Learning
Journal of Artificial Intelligence Research · 09.01.2023 · 76 · 201-264
DOI: 10.1613/jair.1.14174 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Artificial Intelligence Research, online erschienen am 09.01.2023. Verfasst von Robert Kirk und 3 weiteren Autor:innen. Die Arbeit wurde bisher 194-mal zitiert (Crossref).
Abstract (Originalsprache)
The study of zero-shot generalisation (ZSG) in deep Reinforcement Learning (RL) aims to produce RL algorithms whose policies generalise well to novel unseen situations at deployment time, avoiding overfitting to their training environments. Tackling this is vital if we are to deploy reinforcement learning algorithms in real world scenarios, where the environment will be diverse, dynamic and unpredictable. This survey is an overview of this nascent field. We rely on a unifying formalism and terminology for discussing different ZSG problems, building upon previous works. We go on to categorise existing benchmarks for ZSG, as well as current methods for tackling these problems. Finally, we provide a critical discussion of the current state of the field, including recommendations for future work. Among other conclusions, we argue that taking a purely procedural content generation approach to benchmark design is not conducive to progress in ZSG, we suggest fast online adaptation and tackling RL-specific problems as some areas for future work on methods for ZSG, and we recommend building benchmarks in underexplored problem settings such as offline RL ZSG and reward-function variation.
Zitieren
APA
Kirk, R., Zhang, A., Grefenstette, E., & Rocktäschel, T. (2023). A Survey of Zero-shot Generalisation in Deep Reinforcement Learning. Journal of Artificial Intelligence Research, 76, 201-264. https://doi.org/10.1613/jair.1.14174
Vancouver
Kirk R, Zhang A, Grefenstette E, Rocktäschel T. A Survey of Zero-shot Generalisation in Deep Reinforcement Learning. Journal of Artificial Intelligence Research. 2023;76:201-264. doi:10.1613/jair.1.14174
BibTeX
@article{Kirk2023,
author = {Kirk, Robert and Zhang, Amy and Grefenstette, Edward and Rocktäschel, Tim},
title = {A Survey of Zero-shot Generalisation in Deep Reinforcement Learning},
journal = {Journal of Artificial Intelligence Research},
year = {2023},
volume = {76},
pages = {201--264},
doi = {10.1613/jair.1.14174}
}
EUVIMED