EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Unifying Unsupervised and Offline RL for Fast Adaptation Using World Models

Daniel Khapun, Dan Rosenbaum

IEEE Robotics and Automation Letters · 2026-05 · 11 (5) · 5693-5700

DOI: 10.1109/lra.2026.3673814 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Robotics and Automation Letters, 2026-05. Verfasst von Daniel Khapun und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep reinforcement learning has proven an effective method to solve many intricate tasks, yet it still struggles with data efficiency and generalization to novel scenarios, as required in settings such as robotics. Recent approaches to deal with this include (1) unsupervised pretraining of the agent in an environment without reward signals, and (2) training the agent using offline data coming from various possible sources. In this letter we propose to consider both of these approaches together, resulting in a setting where different types of data streams are available and fast online adaptation to new tasks is required. Towards this goal we consider the Unsupervised Reinforcement Learning Benchmark and show that unsupervised training’s primary value lies in its use as a source of exploration trajectories, beyond its role in pretraining a policy. Following this observation we develop a method based on a world-model as a generative model of offline exploration data and model predictive control (MPC) planning. We show that this approach outperforms previous methods and demonstrates task adaptation which is 10 times faster than previously shown. We then propose a setup that includes access to both unsupervised exploratory data and offline expert demonstrations when testing the agents’ online performance on adaptation to novel tasks in the environment.

Zitieren

APA

Khapun, D., & Rosenbaum, D. (2026). Unifying Unsupervised and Offline RL for Fast Adaptation Using World Models. IEEE Robotics and Automation Letters, 11(5), 5693-5700. https://doi.org/10.1109/lra.2026.3673814

Vancouver

Khapun D, Rosenbaum D. Unifying Unsupervised and Offline RL for Fast Adaptation Using World Models. IEEE Robotics and Automation Letters. 2026;11(5):5693-5700. doi:10.1109/lra.2026.3673814

BibTeX
@article{Khapun2026,
  author = {Khapun, Daniel and Rosenbaum, Dan},
  title = {Unifying Unsupervised and Offline RL for Fast Adaptation Using World Models},
  journal = {IEEE Robotics and Automation Letters},
  year = {2026},
  volume = {11},
  number = {5},
  pages = {5693--5700},
  doi = {10.1109/lra.2026.3673814}
}
Kostenlos starten →