EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Modelling Sign Language with Encoder-Only Transformers and Human Pose Estimation Keypoint Data

Luke T. Woods, Zeeshan A. Rana

Mathematics · 01.05.2023 · 11 (9) · 2129

DOI: 10.3390/math11092129 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 01.05.2023. Verfasst von Luke T. Woods und 1 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 104 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Leidos Industrial Engineers Limited.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present a study on modelling American Sign Language (ASL) with encoder-only transformers and human pose estimation keypoint data. Using an enhanced version of the publicly available Word-level ASL (WLASL) dataset, and a novel normalisation technique based on signer body size, we show the impact model architecture has on accurately classifying sets of 10, 50, 100, and 300 isolated, dynamic signs using two-dimensional keypoint coordinates only. We demonstrate the importance of running and reporting results from repeated experiments to describe and evaluate model performance. We include descriptions of the algorithms used to normalise the data and generate the train, validation, and test data splits. We report top-1, top-5, and top-10 accuracy results, evaluated with two separate model checkpoint metrics based on validation accuracy and loss. We find models with fewer than 100k learnable parameters can achieve high accuracy on reduced vocabulary datasets, paving the way for lightweight consumer hardware to perform tasks that are traditionally resource-intensive, requiring expensive, high-end equipment. We achieve top-1, top-5, and top-10 accuracies of 97%, 100%, and 100%, respectively, on a vocabulary size of 10 signs; 87%, 97%, and 98% on 50 signs; 83%, 96%, and 97% on 100 signs; and 71%, 90%, and 94% on 300 signs, thereby setting a new benchmark for this task.

Förderung

  • Leidos Industrial Engineers Limited

Zitieren

APA

Woods, L. T., & Rana, Z. A. (2023). Modelling Sign Language with Encoder-Only Transformers and Human Pose Estimation Keypoint Data. Mathematics, 11(9), 2129. https://doi.org/10.3390/math11092129

Vancouver

Woods LT, Rana ZA. Modelling Sign Language with Encoder-Only Transformers and Human Pose Estimation Keypoint Data. Mathematics. 2023;11(9):2129. doi:10.3390/math11092129

BibTeX
@article{Woods2023,
  author = {Woods, Luke T. and Rana, Zeeshan A.},
  title = {Modelling Sign Language with Encoder-Only Transformers and Human Pose Estimation Keypoint Data},
  journal = {Mathematics},
  year = {2023},
  volume = {11},
  number = {9},
  pages = {2129},
  doi = {10.3390/math11092129}
}
Kostenlos starten →