EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

SignFormer-GCN: Continuous sign language translation using spatio-temporal graph convolutional networks

Safaeid Hossain Arib, Rabeya Akter, Sejuti Rahman iD, Shafin Rahman iD

PLOS ONE · 2025

DOI: 10.1371/journal.pone.0316298 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS ONE, 2025. Verfasst von Safaeid Hossain Arib und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh. Hauptthemen laut MEDLINE: Sign Language, Neural Networks, Computer, Translating. Die Arbeit wurde bisher 16-mal zitiert (Crossref) und nennt 75 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science and Technology, Government of the People’s Republic of Bangladesh, Conference Travel and Research Grant NSU.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sign language is a complex visual language system that uses hand gestures, facial expressions, and body movements to convey meaning. It is the primary means of communication for millions of deaf and hard-of-hearing individuals worldwide. Tracking physical actions, such as hand movements and arm orientation, alongside expressive actions, including facial expressions, mouth movements, eye movements, eyebrow gestures, head movements, and body postures, using only RGB features can be limiting due to discrepancies in backgrounds and signers across different datasets. Despite this limitation, most Sign Language Translation (SLT) research relies solely on RGB features. We used keypoint features, and RGB features to capture better the pose and configuration of body parts involved in sign language actions and complement the RGB features. Similarly, most works on SLT research have used transformers, which are good at capturing broader, high-level context and focusing on the most relevant video frames. Still, the inherent graph structure associated with sign language is neglected and fails to capture low-level details. To solve this, we used a joint encoding technique using a transformer and STGCN architecture to capture the context of sign language expressions and spatial and temporal dependencies on skeleton graphs. Our method, SignFormer-GCN, achieves competitive performance in RWTH-PHOENIX-2014T, How2Sign, and BornilDB v1.0 datasets experimentally, showcasing its effectiveness in enhancing translation accuracy through different sign languages. The code is available at the following link: https://github.com/rabeya-akter/SignLanguageTranslation .

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Sign LanguageNeural Networks, ComputerTranslatingHumansGestures

Autor:innen und Institutionen

  • Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh.
  • Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh.

ORCID-Profile: Sejuti Rahman, Shafin Rahman

Förderung

  • Ministry of Science and Technology, Government of the People’s Republic of Bangladesh
  • Conference Travel and Research Grant NSU

Zitieren

APA

Arib, S. H., Akter, R., Rahman, S., & Rahman, S. (2025). SignFormer-GCN: Continuous sign language translation using spatio-temporal graph convolutional networks. PLOS ONE. https://doi.org/10.1371/journal.pone.0316298

Vancouver

Arib SH, Akter R, Rahman S, Rahman S. SignFormer-GCN: Continuous sign language translation using spatio-temporal graph convolutional networks. PLOS ONE. 2025. doi:10.1371/journal.pone.0316298

BibTeX
@article{Arib2025,
  author = {Arib, Safaeid Hossain and Akter, Rabeya and Rahman, Sejuti and Rahman, Shafin},
  title = {SignFormer-GCN: Continuous sign language translation using spatio-temporal graph convolutional networks},
  journal = {PLOS ONE},
  year = {2025},
  doi = {10.1371/journal.pone.0316298}
}
Kostenlos starten →