EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ILeSiA: Interactive Learning of Robot Situational Awareness From Camera Input

Petr Vanc, Giovanni Franzese, Jan Kristof Behrens, Cosimo Della Santina, Karla Stepanova, Jens Kober, Robert Babuska

IEEE Robotics and Automation Letters · 2025-10 · 10 (10) · 10490-10497

DOI: 10.1109/lra.2025.3601037 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Robotics and Automation Letters, 2025-10. Verfasst von Petr Vanc und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: CTU Student Grant Agency, European Union through Project Robotics and Advanced Industrial Production, Grantová Agentura České Republiky.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Learning from demonstration is a promising approach for teaching robots new skills. However, a central challenge in the execution of acquired skills is the ability to recognize faults and prevent failures. This is essential because demonstrations typically cover only a limited set of scenarios and often only the successful ones. During task execution, unforeseen situations may arise, such as changes in the robot's environment or interaction with human operators. To recognize such situations, this letter focuses on teaching the robot situational awareness by using a camera input and labeling frames as safe or risky. We train a Gaussian Process (GP) regression model fed by a low-dimensional latent space representation of the input images. The model outputs a continuous risk score ranging from zero to one, quantifying the degree of risk at each timestep. This allows for pausing task execution in unsafe situations and directly adding new training data, labeled by the human user. Our experiments on a robotic manipulator show that the proposed method can reliably detect both known and novel faults using only a single example for each new fault. In contrast, a standard multi-layer perceptron (MLP) performs well only on faults it has encountered during training. Our method enables the next generation of cobots to be rapidly deployed with easy-to-set-up, vision-based risk assessment, proactively safeguarding humans and detecting misaligned parts or missing objects before failures occur.

Förderung

  • CTU Student Grant Agency
  • European Union through Project Robotics and Advanced Industrial Production
  • Grantová Agentura České Republiky
  • National Growth Fund program NXTGEN Hightech

Zitieren

APA

Vanc, P., Franzese, G., Behrens, J. K., Santina, C. D., Stepanova, K., Kober, J., & Babuska, R. (2025). ILeSiA: Interactive Learning of Robot Situational Awareness From Camera Input. IEEE Robotics and Automation Letters, 10(10), 10490-10497. https://doi.org/10.1109/lra.2025.3601037

Vancouver

Vanc P, Franzese G, Behrens JK, Santina CD, Stepanova K, Kober J, et al. ILeSiA: Interactive Learning of Robot Situational Awareness From Camera Input. IEEE Robotics and Automation Letters. 2025;10(10):10490-10497. doi:10.1109/lra.2025.3601037

BibTeX
@article{Vanc2025,
  author = {Vanc, Petr and Franzese, Giovanni and Behrens, Jan Kristof and Santina, Cosimo Della and Stepanova, Karla and Kober, Jens and Babuska, Robert},
  title = {ILeSiA: Interactive Learning of Robot Situational Awareness From Camera Input},
  journal = {IEEE Robotics and Automation Letters},
  year = {2025},
  volume = {10},
  number = {10},
  pages = {10490--10497},
  doi = {10.1109/lra.2025.3601037}
}
Kostenlos starten →