EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Integrating Egocentric and Robotic Vision for Object Identification Using Siamese Networks and Superquadric Estimations in Partial Occlusion Scenarios

Elisabeth Menendez iD, Santiago Martínez iD, Fernando Díaz-de-María iD, Carlos Balaguer iD

Biomimetics · 08.02.2024 · 9 (2) · 100

DOI: 10.3390/biomimetics9020100 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Biomimetics, online erschienen am 08.02.2024. Verfasst von Elisabeth Menendez und 3 weiteren Autor:innen. Beteiligte Einrichtung: System Engineering and Automation Department, University Carlos III, Av de la Universidad, 30, 28911 Madrid, Spain. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Proyectos Sinérgicos de I+D la Comunidad de Madrid, COMPANION-CM, Inteligencia artificial y modelos cognitivos para la interacción simétrica humano-robot en el ámbito de la robótica asistencial.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper introduces a novel method that enables robots to identify objects based on user gaze, tracked via eye-tracking glasses. This is achieved without prior knowledge of the objects’ categories or their locations and without external markers. The method integrates a two-part system: a category-agnostic object shape and pose estimator using superquadrics and Siamese networks. The superquadrics-based component estimates the shapes and poses of all objects, while the Siamese network matches the object targeted by the user’s gaze with the robot’s viewpoint. Both components are effectively designed to function in scenarios with partial occlusions. A key feature of the system is the user’s ability to move freely around the scenario, allowing dynamic object selection via gaze from any position. The system is capable of handling significant viewpoint differences between the user and the robot and adapts easily to new objects. In tests under partial occlusion conditions, the Siamese networks demonstrated an 85.2% accuracy in aligning the user-selected object with the robot’s viewpoint. This gaze-based Human–Robot Interaction approach demonstrates its practicality and adaptability in real-world scenarios.

Schlagwörter

Stichwörter der Autor:innen

Siamese networkgazehuman–robot interactionimage matchingpose estimationprimitive shapessuperquadrics

Autor:innen und Institutionen

  • System Engineering and Automation Department, University Carlos III, Av de la Universidad, 30, 28911 Madrid, Spain.
  • Signal Theory and Communications Department, University Carlos III, Av de la Universidad, 30, 28911 Madrid, Spain.

ORCID-Profile: Elisabeth Menendez, Santiago Martínez, Fernando Díaz-de-María, Carlos Balaguer

Förderung

  • Proyectos Sinérgicos de I+D la Comunidad de Madrid
  • COMPANION-CM, Inteligencia artificial y modelos cognitivos para la interacción simétrica humano-robot en el ámbito de la robótica asistencial

Zitieren

APA

Menendez, E., Martínez, S., Díaz-de-María, F., & Balaguer, C. (2024). Integrating Egocentric and Robotic Vision for Object Identification Using Siamese Networks and Superquadric Estimations in Partial Occlusion Scenarios. Biomimetics, 9(2), 100. https://doi.org/10.3390/biomimetics9020100

Vancouver

Menendez E, Martínez S, Díaz-de-María F, Balaguer C. Integrating Egocentric and Robotic Vision for Object Identification Using Siamese Networks and Superquadric Estimations in Partial Occlusion Scenarios. Biomimetics. 2024;9(2):100. doi:10.3390/biomimetics9020100

BibTeX
@article{Menendez2024,
  author = {Menendez, Elisabeth and Martínez, Santiago and Díaz-de-María, Fernando and Balaguer, Carlos},
  title = {Integrating Egocentric and Robotic Vision for Object Identification Using Siamese Networks and Superquadric Estimations in Partial Occlusion Scenarios},
  journal = {Biomimetics},
  year = {2024},
  volume = {9},
  number = {2},
  pages = {100},
  doi = {10.3390/biomimetics9020100}
}
Kostenlos starten →