EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Self-supervised representation learning for robust fine-grained human hand action recognition in industrial assembly lines

Fabian Sturm, Martin Trat, Rahul Sathiyababu, Harshitha Allipilli, Benjamin Menz, Elke Hergenroether, Melanie Siegel

Machine Vision and Applications · 2025-01 · 36 (1)

DOI: 10.1007/s00138-024-01638-9 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machine Vision and Applications, online erschienen am 13.12.2024. Verfasst von Fabian Sturm und 6 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Technische Hochschule Nürnberg.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Humans are still indispensable on industrial assembly lines, but in the event of an error, they need support from intelligent systems. In addition to the objects to be observed, it is equally important to understand the fine-grained hand movements of a human to be able to track the entire process. However, these deep-learning-based hand action recognition methods are very label intensive, which cannot be offered by all industrial companies due to the associated costs. This work therefore presents a self-supervised learning approach for industrial assembly processes that allows a spatio-temporal transformer architecture to be pre-trained on a variety of information from real-world video footage of daily life. Subsequently, this deep learning model is adapted to the industrial assembly task at hand using only a few labels. Well-known real-world datasets best suited for representation learning of such hand actions in a regression tasks are outlined and to what extent they optimize the subsequent supervised trained classification task. This subsequent fine-tuning is supplemented by concept drift detection, which makes the resulting productively employed models more robust against concept drift and future changing assembly movements.

Förderung

  • Technische Hochschule Nürnberg

Zitieren

APA

Sturm, F., Trat, M., Sathiyababu, R., Allipilli, H., Menz, B., Hergenroether, E., & Siegel, M. (2025). Self-supervised representation learning for robust fine-grained human hand action recognition in industrial assembly lines. Machine Vision and Applications, 36(1). https://doi.org/10.1007/s00138-024-01638-9

Vancouver

Sturm F, Trat M, Sathiyababu R, Allipilli H, Menz B, Hergenroether E, et al. Self-supervised representation learning for robust fine-grained human hand action recognition in industrial assembly lines. Machine Vision and Applications. 2025;36(1). doi:10.1007/s00138-024-01638-9

BibTeX
@article{Sturm2025,
  author = {Sturm, Fabian and Trat, Martin and Sathiyababu, Rahul and Allipilli, Harshitha and Menz, Benjamin and Hergenroether, Elke and Siegel, Melanie},
  title = {Self-supervised representation learning for robust fine-grained human hand action recognition in industrial assembly lines},
  journal = {Machine Vision and Applications},
  year = {2025},
  volume = {36},
  number = {1},
  doi = {10.1007/s00138-024-01638-9}
}
Kostenlos starten →