EUVIMEDΑνακαλύψτε το EUVIMED
Open access · CC BY 4.0

Imagine to Ensure Safety in Hierarchical Reinforcement Learning

Gregory Gorbov, Artem Latyshev, Aleksandr Panov

Machine Learning and Knowledge Extraction · 2026-08-13 · 8 (8) · 243

DOI: 10.3390/make8080243 ↗

At a glance

Journal article in Machine Learning and Knowledge Extraction, published online on 2026-08-13. Written by Gregory Gorbov and 2 co-authors. 36 references. The full text is freely available (open access, CC BY 4.0). Funded by: Science and Higher Education of the Russian Federation.

ΠερίληψηΒιβλιογραφικά στοιχείαCiteRelated studies

Abstract (original language)

This work investigates the safe exploration problem in reinforcement learning, where an agent must maximize cumulative performance while simultaneously satisfying safety constraints. This challenge becomes particularly difficult in long-horizon tasks because safe reinforcement learning methods often become overly conservative, restricting exploration and preventing agents from reaching distant goals, while estimation errors accumulate over extended horizons. We propose Imagine To Ensure Safety in Hierarchical Reinforcement Learning (ITES), which combines a learnable world model with high-level and low-level policies to promote safety at both hierarchical levels. The novelty of ITES lies in jointly ensuring safe subgoal generation and safe subgoal execution within a hierarchical framework. The high-level policy generates intermediate subgoals that guide exploration toward safe regions, while the low-level policy uses imagined rollouts in the learned world model to reduce unsafe behavior during subgoal execution. We evaluate ITES on the long-horizon SafeAntMaze C-shape, SafeAntMaze W-shape, and SafePusher benchmarks, as well as on short-horizon Safety Gym tasks, using task performance and episodic cost under prescribed safety budgets. The results show that ITES achieves substantially higher task success and constraint-compliant success rates on the long-horizon benchmarks while maintaining mean episodic costs below the prescribed budgets.

Funding

  • Science and Higher Education of the Russian Federation

Cite

APA

Gorbov, G., Latyshev, A., & Panov, A. (2026). Imagine to Ensure Safety in Hierarchical Reinforcement Learning. Machine Learning and Knowledge Extraction, 8(8), 243. https://doi.org/10.3390/make8080243

Vancouver

Gorbov G, Latyshev A, Panov A. Imagine to Ensure Safety in Hierarchical Reinforcement Learning. Machine Learning and Knowledge Extraction. 2026;8(8):243. doi:10.3390/make8080243

BibTeX
@article{Gorbov2026,
  author = {Gorbov, Gregory and Latyshev, Artem and Panov, Aleksandr},
  title = {Imagine to Ensure Safety in Hierarchical Reinforcement Learning},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {8},
  pages = {243},
  doi = {10.3390/make8080243}
}
Start for free →