EUVIMEDEUVIMED entdecken
Crossref

An Implicit Trust Region Approach to Behavior Regularized Offline Reinforcement Learning

Zhe Zhang, Xiaoyang Tan

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (15) · 16944-16952

DOI: 10.1609/aaai.v38i15.29637 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Zhe Zhang und 1 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We revisit behavior regularization, a popular approach to mitigate the extrapolation error in offline reinforcement learning (RL), showing that current behavior regularization may suffer from unstable learning and hinder policy improvement. Motivated by this, a novel reward shaping-based behavior regularization method is proposed, where the log-probability ratio between the learned policy and the behavior policy is monitored during learning. We show that this is equivalent to an implicit but computationally lightweight trust region mechanism, which is beneficial to mitigate the influence of estimation errors of the value function, leading to more stable performance improvement. Empirical results on the popular D4RL benchmark verify the effectiveness of the presented method with promising performance compared with some state-of-the-art offline RL algorithms.

Zitieren

APA

Zhang, Z., & Tan, X. (2024). An Implicit Trust Region Approach to Behavior Regularized Offline Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15), 16944-16952. https://doi.org/10.1609/aaai.v38i15.29637

Vancouver

Zhang Z, Tan X. An Implicit Trust Region Approach to Behavior Regularized Offline Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(15):16944-16952. doi:10.1609/aaai.v38i15.29637

BibTeX
@article{Zhang2024,
  author = {Zhang, Zhe and Tan, Xiaoyang},
  title = {An Implicit Trust Region Approach to Behavior Regularized Offline Reinforcement Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {15},
  pages = {16944--16952},
  doi = {10.1609/aaai.v38i15.29637}
}
Kostenlos starten →