EUVIMEDEUVIMED entdecken
Crossref

Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning

Jinxin Liu, Ziqi Zhang, Zhenyu Wei, Zifeng Zhuang, Yachen Kang, Sibo Gai, Donglin Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (12) · 13945-13953

DOI: 10.1609/aaai.v38i12.29302 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Jinxin Liu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Offline reinforcement learning (RL) aims to learn a policy using only pre-collected and fixed data. Although avoiding the time-consuming online interactions in RL, it poses challenges for out-of-distribution (OOD) state actions and often suffers from data inefficiency for training. Despite many efforts being devoted to addressing OOD state actions, the latter (data inefficiency) receives little attention in offline RL. To address this, this paper proposes the cross-domain offline RL, which assumes offline data incorporate additional source-domain data from varying transition dynamics (environments), and expects it to contribute to the offline data efficiency. To do so, we identify a new challenge of OOD transition dynamics, beyond the common OOD state actions issue, when utilizing cross-domain offline data. Then, we propose our method BOSA, which employs two support-constrained objectives to address the above OOD issues. Through extensive experiments in the cross-domain offline RL setting, we demonstrate BOSA can greatly improve offline data efficiency: using only 10% of the target data, BOSA could achieve 74.4% of the SOTA offline RL performance that uses 100% of the target data. Additionally, we also show BOSA can be effortlessly plugged into model-based offline RL and noising data augmentation techniques (used for generating source-domain data), which naturally avoids the potential dynamics mismatch between target-domain data and newly generated source-domain data.

Zitieren

APA

Liu, J., Zhang, Z., Wei, Z., Zhuang, Z., Kang, Y., Gai, S., & Wang, D. (2024). Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(12), 13945-13953. https://doi.org/10.1609/aaai.v38i12.29302

Vancouver

Liu J, Zhang Z, Wei Z, Zhuang Z, Kang Y, Gai S, et al. Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(12):13945-13953. doi:10.1609/aaai.v38i12.29302

BibTeX
@article{Liu2024,
  author = {Liu, Jinxin and Zhang, Ziqi and Wei, Zhenyu and Zhuang, Zifeng and Kang, Yachen and Gai, Sibo and Wang, Donglin},
  title = {Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {12},
  pages = {13945--13953},
  doi = {10.1609/aaai.v38i12.29302}
}
Kostenlos starten →