Expert-Trajectory-Based Features for Apprenticeship Learning via Inverse Reinforcement Learning for Robotic Manipulation
Applied Sciences · 29.11.2024 · 14 (23) · 11131
DOI: 10.3390/app142311131 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 29.11.2024. Verfasst von Francisco J. Naranjo-Campos und 2 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: ISCIII, Proyectos Sinérgicos de I+D la Comunidad de Madrid, EU structural funds.
Abstract (Originalsprache)
This paper explores the application of Inverse Reinforcement Learning (IRL) in robotics, focusing on inferring reward functions from expert demonstrations of robot arm manipulation tasks. By leveraging IRL, we aim to develop efficient and adaptable techniques for learning robust solutions to complex tasks in continuous state spaces. Our approach combines Apprenticeship Learning via IRL with Proximal Policy Optimization (PPO), expert-trajectory-based features, and the application of a reverse discount. The feature space is constructed by sampling expert trajectories to capture essential task characteristics, enhancing learning efficiency and generalizability by concentrating on critical states. To prevent the vanishing of feature expectations in goal states, we introduce a reverse discounting application to prioritize feature expectations in final states. We validate our methodology through experiments in a simple GridWorld environment, demonstrating that reverse discounting enhances the alignment of the agent’s features with those of the expert. Additionally, we explore how the parameters of the proposed feature definition influence performance. Further experiments on robotic manipulation tasks using the TIAGo robot compare our approach with state-of-the-art methods, confirming its effectiveness and adaptability in complex continuous state spaces across diverse manipulation tasks.
Förderung
- ISCIII
- Proyectos Sinérgicos de I+D la Comunidad de Madrid
- EU structural funds
- IROPER: Robótica inteligente para necesidades personales
- “NextGenerationEU/PRTR” of European Union
Zitieren
APA
Naranjo-Campos, F. J., Victores, J. G., & Balaguer, C. (2024). Expert-Trajectory-Based Features for Apprenticeship Learning via Inverse Reinforcement Learning for Robotic Manipulation. Applied Sciences, 14(23), 11131. https://doi.org/10.3390/app142311131
Vancouver
Naranjo-Campos FJ, Victores JG, Balaguer C. Expert-Trajectory-Based Features for Apprenticeship Learning via Inverse Reinforcement Learning for Robotic Manipulation. Applied Sciences. 2024;14(23):11131. doi:10.3390/app142311131
BibTeX
@article{NaranjoCampos2024,
author = {Naranjo-Campos, Francisco J. and Victores, Juan G. and Balaguer, Carlos},
title = {Expert-Trajectory-Based Features for Apprenticeship Learning via Inverse Reinforcement Learning for Robotic Manipulation},
journal = {Applied Sciences},
year = {2024},
volume = {14},
number = {23},
pages = {11131},
doi = {10.3390/app142311131}
}
EUVIMED