EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Imagine to Ensure Safety in Hierarchical Reinforcement Learning

Gregory Gorbov, Artem Latyshev, Aleksandr Panov

Machine Learning and Knowledge Extraction · 13.08.2026 · 8 (8) · 243

DOI: 10.3390/make8080243 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 13.08.2026. Verfasst von Gregory Gorbov und 2 weiteren Autor:innen. Die Arbeit nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Science and Higher Education of the Russian Federation.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This work investigates the safe exploration problem in reinforcement learning, where an agent must maximize cumulative performance while simultaneously satisfying safety constraints. This challenge becomes particularly difficult in long-horizon tasks because safe reinforcement learning methods often become overly conservative, restricting exploration and preventing agents from reaching distant goals, while estimation errors accumulate over extended horizons. We propose Imagine To Ensure Safety in Hierarchical Reinforcement Learning (ITES), which combines a learnable world model with high-level and low-level policies to promote safety at both hierarchical levels. The novelty of ITES lies in jointly ensuring safe subgoal generation and safe subgoal execution within a hierarchical framework. The high-level policy generates intermediate subgoals that guide exploration toward safe regions, while the low-level policy uses imagined rollouts in the learned world model to reduce unsafe behavior during subgoal execution. We evaluate ITES on the long-horizon SafeAntMaze C-shape, SafeAntMaze W-shape, and SafePusher benchmarks, as well as on short-horizon Safety Gym tasks, using task performance and episodic cost under prescribed safety budgets. The results show that ITES achieves substantially higher task success and constraint-compliant success rates on the long-horizon benchmarks while maintaining mean episodic costs below the prescribed budgets.

Förderung

  • Science and Higher Education of the Russian Federation

Zitieren

APA

Gorbov, G., Latyshev, A., & Panov, A. (2026). Imagine to Ensure Safety in Hierarchical Reinforcement Learning. Machine Learning and Knowledge Extraction, 8(8), 243. https://doi.org/10.3390/make8080243

Vancouver

Gorbov G, Latyshev A, Panov A. Imagine to Ensure Safety in Hierarchical Reinforcement Learning. Machine Learning and Knowledge Extraction. 2026;8(8):243. doi:10.3390/make8080243

BibTeX
@article{Gorbov2026,
  author = {Gorbov, Gregory and Latyshev, Artem and Panov, Aleksandr},
  title = {Imagine to Ensure Safety in Hierarchical Reinforcement Learning},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {8},
  pages = {243},
  doi = {10.3390/make8080243}
}
Kostenlos starten →