EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Towards Recognition of Human Actions in Collaborative Tasks with Robots: Extending Action Recognition with Tool Recognition Methods

Lukas Büsch iD, Julian Koch iD, Daniel Schoepflin iD, Michelle Schulze iD, Thorsten Schüppstuhl iD

Sensors · 2023

DOI: 10.3390/s23125718 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2023. Verfasst von Lukas Büsch und 4 weiteren Autor:innen. Beteiligte Einrichtung: Hamburg University of Technology, Institute of Aircraft Production Technology, Denickestraße 17, 21073 Hamburg, Germany. Hauptthemen laut MEDLINE: Robotics. Die Arbeit wurde bisher 20-mal zitiert (Crossref) und nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Federal Ministry of Education and Research, Hamburg University of Technology, German Federal Ministry of Education and Research (BMBF).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper presents a novel method for online tool recognition in manual assembly processes. The goal was to develop and implement a method that can be integrated with existing Human Action Recognition (HAR) methods in collaborative tasks. We examined the state-of-the-art for progress detection in manual assembly via HAR-based methods, as well as visual tool-recognition approaches. A novel online tool-recognition pipeline for handheld tools is introduced, utilizing a two-stage approach. First, a Region Of Interest (ROI) was extracted by determining the wrist position using skeletal data. Afterward, this ROI was cropped, and the tool located within this ROI was classified. This pipeline enabled several algorithms for object recognition and demonstrated the generalizability of our approach. An extensive training dataset for tool-recognition purposes is presented, which was evaluated with two image-classification approaches. An offline pipeline evaluation was performed with twelve tool classes. Additionally, various online tests were conducted covering different aspects of this vision application, such as two assembly scenarios, unknown instances of known classes, as well as challenging backgrounds. The introduced pipeline was competitive with other approaches regarding prediction accuracy, robustness, diversity, extendability/flexibility, and online capability.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

RoboticsHumansHuman ActivitiesVisual Perception

Stichwörter der Autor:innen

assemblyassembly progress detectionassembly step recognitionhuman action recognitionhuman–robot collaborationindustrial object recognitiontool detectiontool recognition

Autor:innen und Institutionen

  • Hamburg University of Technology, Institute of Aircraft Production Technology, Denickestraße 17, 21073 Hamburg, Germany.

ORCID-Profile: Lukas Büsch, Julian Koch, Daniel Schoepflin, Michelle Schulze, Thorsten Schüppstuhl

Förderung

  • Federal Ministry of Education and Research
  • Hamburg University of Technology
  • German Federal Ministry of Education and Research (BMBF)
  • Funding Programme Open Access Publishing of Hamburg University of Technology (TUHH)

Zitieren

APA

Büsch, L., Koch, J., Schoepflin, D., Schulze, M., & Schüppstuhl, T. (2023). Towards Recognition of Human Actions in Collaborative Tasks with Robots: Extending Action Recognition with Tool Recognition Methods. Sensors. https://doi.org/10.3390/s23125718

Vancouver

Büsch L, Koch J, Schoepflin D, Schulze M, Schüppstuhl T. Towards Recognition of Human Actions in Collaborative Tasks with Robots: Extending Action Recognition with Tool Recognition Methods. Sensors. 2023. doi:10.3390/s23125718

BibTeX
@article{Bsch2023,
  author = {Büsch, Lukas and Koch, Julian and Schoepflin, Daniel and Schulze, Michelle and Schüppstuhl, Thorsten},
  title = {Towards Recognition of Human Actions in Collaborative Tasks with Robots: Extending Action Recognition with Tool Recognition Methods},
  journal = {Sensors},
  year = {2023},
  doi = {10.3390/s23125718}
}
Kostenlos starten →