Multi-Length Skills with Priors for Reinforcement Learning
ACM Transactions on Probabilistic Machine Learning · 2025
DOI: 10.1145/3728647 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Probabilistic Machine Learning, 2025. Verfasst von Carlos I. Jerez und 1 weiteren Autor:innen. Die Arbeit nennt 56 Literaturangaben.
Abstract (Originalsprache)
Leveraging prior experience has been effective in accelerating reinforcement learning (RL). One approach for leveraging prior experience consists of learning skills, which are temporally extended sequences of actions, and priors over those skills, which are then used to learn new tasks. Existing methods either learn single-length skills or learn skills with every possible length, e.g., options. Instead, we propose using a more select set of skill lengths, where the agent can learn repetitive, monotonous sequences of actions as “long” skills, and unique, highly complex sequences of actions as “short” skills. To implement that intuition, we introduce multi-length skills with priors (MLSP) for RL, a novel approach that leverages prior experience and presents a new RL formulation that considers multi-length skills. We, first, introduce hierarchical multi-length skill learning (HIMES), which is a deep latent variable model that learns multi-length skills by hierarchical composing skills to form longer skills, then we provide a notion of optimal length based on reconstruction error, from which a length prior is learned, and lastly a prior over skills is learned. Finally, MLSP integrates these models to guide downstream learning, where now two policies are learned: The length policy that selects which skill length to use, and then the skill policy that selects what skill of a particular skill length to use. We validate our approach on complex robotic manipulation tasks and show that MLSP significantly accelerates downstream task learning with respect to competitive benchmarks. Code and videos of the trained agents are available at https://github.com/cijerezg/MLSP .
Zitieren
APA
Jerez, C. I., & Zhang, J. (2025). Multi-Length Skills with Priors for Reinforcement Learning. ACM Transactions on Probabilistic Machine Learning. https://doi.org/10.1145/3728647
Vancouver
Jerez CI, Zhang J. Multi-Length Skills with Priors for Reinforcement Learning. ACM Transactions on Probabilistic Machine Learning. 2025. doi:10.1145/3728647
BibTeX
@article{Jerez2025,
author = {Jerez, Carlos I. and Zhang, Jun},
title = {Multi-Length Skills with Priors for Reinforcement Learning},
journal = {ACM Transactions on Probabilistic Machine Learning},
year = {2025},
doi = {10.1145/3728647}
}
EUVIMED