EUVIMEDEUVIMED entdecken
Crossref

Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning

Suzan Ece Ada, Erhan Oztop, Emre Ugur

IEEE Robotics and Automation Letters · 2024-04 · 9 (4) · 3116-3123

DOI: 10.1109/lra.2024.3363530 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Robotics and Automation Letters, 2024-04. Verfasst von Suzan Ece Ada und 2 weiteren Autor:innen. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 43 Literaturangaben. Gefördert von: JSPS KAKENHI, New Energy and Industrial Technology Development Organization, Türkiye Bilimsel ve Teknolojik Araştırma Kurumu.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Offline Reinforcement Learning (RL) methods leverage previous experiences to learn better policies than the behavior policy used for data collection. However, they face challenges handling distribution shifts due to the lack of online interaction during training. To this end, we propose a novel method named State Reconstruction for Diffusion Policies (SRDP) that incorporates state reconstruction feature learning in the recent class of diffusion policies to address the problem of out-of-distribution (OOD) generalization. Our method promotes learning of generalizable state representation to alleviate the distribution shift caused by OOD states. To illustrate the OOD generalization and faster convergence of SRDP, we design a novel 2D Multimodal Contextual Bandit environment and realize it on a 6-DoF real-world UR10 robot, as well as in simulation, and compare its performance with prior algorithms. In particular, we show the importance of the proposed state reconstruction via ablation studies. In addition, we assess the performance of our model on standard continuous control benchmarks (D4RL), namely the navigation of an 8-DoF ant and forward locomotion of half-cheetah, hopper, and walker2d, achieving state-of-the-art results. Finally, we demonstrate that our method can achieve 167% improvement over the competing baseline on a sparse continuous control navigation task where various regions of the state space are removed from the offline RL dataset, including the region encapsulating the goal.

Förderung

  • JSPS KAKENHI
  • New Energy and Industrial Technology Development Organization
  • Türkiye Bilimsel ve Teknolojik Araştırma Kurumu
  • European Commission
  • INVERSE Project

Zitieren

APA

Ada, S. E., Oztop, E., & Ugur, E. (2024). Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning. IEEE Robotics and Automation Letters, 9(4), 3116-3123. https://doi.org/10.1109/lra.2024.3363530

Vancouver

Ada SE, Oztop E, Ugur E. Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning. IEEE Robotics and Automation Letters. 2024;9(4):3116-3123. doi:10.1109/lra.2024.3363530

BibTeX
@article{Ada2024,
  author = {Ada, Suzan Ece and Oztop, Erhan and Ugur, Emre},
  title = {Diffusion Policies for Out-of-Distribution Generalization in Offline Reinforcement Learning},
  journal = {IEEE Robotics and Automation Letters},
  year = {2024},
  volume = {9},
  number = {4},
  pages = {3116--3123},
  doi = {10.1109/lra.2024.3363530}
}
Kostenlos starten →