Sign Language Motion Generation from Sign Characteristics
Sensors · 2023
DOI: 10.3390/s23239365 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2023. Verfasst von Manuel Gil-Martín und 4 weiteren Autor:innen. Beteiligte Einrichtung: Grupo de Tecnología del Habla y Aprendizaje Automático (T.H.A.U. Group), Information Processing and Telecommunications Center, Escuela Técnica Superi…. Hauptthemen laut MEDLINE: Algorithms, Sign Language. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Amazon through the IPTC-Amazon collaboration.
Abstract (Originalsprache)
This paper proposes, analyzes, and evaluates a deep learning architecture based on transformers for generating sign language motion from sign phonemes (represented using HamNoSys: a notation system developed at the University of Hamburg). The sign phonemes provide information about sign characteristics like hand configuration, localization, or movements. The use of sign phonemes is crucial for generating sign motion with a high level of details (including finger extensions and flexions). The transformer-based approach also includes a stop detection module for predicting the end of the generation process. Both aspects, motion generation and stop detection, are evaluated in detail. For motion generation, the dynamic time warping distance is used to compute the similarity between two landmarks sequences (ground truth and generated). The stop detection module is evaluated considering detection accuracy and ROC (receiver operating characteristic) curves. The paper proposes and evaluates several strategies to obtain the system configuration with the best performance. These strategies include different padding strategies, interpolation approaches, and data augmentation techniques. The best configuration of a fully automatic system obtains an average DTW distance per frame of 0.1057 and an area under the ROC curve (AUC) higher than 0.94.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Grupo de Tecnología del Habla y Aprendizaje Automático (T.H.A.U. Group), Information Processing and Telecommunications Center, Escuela Técnica Superior de Ingenieros de Telecomunicación, Universidad Politécnica de Madri…
- Alexa AI, Aleja Grunwaldzka 472, 80-309 Gdańsk, Poland.
- Alexa AI, C. de Ramírez de Prado 5, 28045 Madrid, Spain.
ORCID-Profile: Manuel Gil-Martín, Andrzej Pomirski, Rubén San-Segundo
Förderung
- Amazon through the IPTC-Amazon collaboration
Zitieren
APA
Gil-Martín, M., Villa-Monedero, M., Pomirski, A., Sáez-Trigueros, D., & San-Segundo, R. (2023). Sign Language Motion Generation from Sign Characteristics. Sensors. https://doi.org/10.3390/s23239365
Vancouver
Gil-Martín M, Villa-Monedero M, Pomirski A, Sáez-Trigueros D, San-Segundo R. Sign Language Motion Generation from Sign Characteristics. Sensors. 2023. doi:10.3390/s23239365
BibTeX
@article{GilMartn2023,
author = {Gil-Martín, Manuel and Villa-Monedero, María and Pomirski, Andrzej and Sáez-Trigueros, Daniel and San-Segundo, Rubén},
title = {Sign Language Motion Generation from Sign Characteristics},
journal = {Sensors},
year = {2023},
doi = {10.3390/s23239365}
}
EUVIMED