Weighted Policy Constraints for Offline Reinforcement Learning
Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (8) · 9435-9443
DOI: 10.1609/aaai.v37i8.26130 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Zhiyong Peng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref).
Abstract (Originalsprache)
Offline reinforcement learning (RL) aims to learn policy from the passively collected offline dataset. Applying existing RL methods on the static dataset straightforwardly will raise distribution shift, causing these unconstrained RL methods to fail. To cope with the distribution shift problem, a common practice in offline RL is to constrain the policy explicitly or implicitly close to behavioral policy. However, the available dataset usually contains sub-optimal or inferior actions, constraining the policy near all these actions will make the policy inevitably learn inferior behaviors, limiting the performance of the algorithm. Based on this observation, we propose a weighted policy constraints (wPC) method that only constrains the learned policy to desirable behaviors, making room for policy improvement on other parts. Our algorithm outperforms existing state-of-the-art offline RL algorithms on the D4RL offline gym datasets. Moreover, the proposed algorithm is simple to implement with few hyper-parameters, making the proposed wPC algorithm a robust offline RL method with low computational complexity.
Zitieren
APA
Peng, Z., Han, C., Liu, Y., & Zhou, Z. (2023). Weighted Policy Constraints for Offline Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37(8), 9435-9443. https://doi.org/10.1609/aaai.v37i8.26130
Vancouver
Peng Z, Han C, Liu Y, Zhou Z. Weighted Policy Constraints for Offline Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(8):9435-9443. doi:10.1609/aaai.v37i8.26130
BibTeX
@article{Peng2023,
author = {Peng, Zhiyong and Han, Changlin and Liu, Yadong and Zhou, Zongtan},
title = {Weighted Policy Constraints for Offline Reinforcement Learning},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2023},
volume = {37},
number = {8},
pages = {9435--9443},
doi = {10.1609/aaai.v37i8.26130}
}
EUVIMED