EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access · CC BY 4.0

A comparative analysis of video vision transformers on word-level sign language datasets

Jubayer Ahmed Bhuiyan Shawon iD, Md Kamrul Hasan, Hasan Mahmud

PLOS One · 2026

DOI: 10.1371/journal.pone.0341909 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift PLOS One, 2026. Verfasst von Jubayer Ahmed Bhuiyan Shawon und 2 weiteren Autor:innen. Beteiligte Einrichtung: Systems and Software Lab (SSL), Department of CSE, Islamic University of Technology (IUT), OIC, Board Bazar, Gazipur, Bangladesh. Hauptthemen laut MEDLINE: Sign Language, Video Recording, Image Processing, Computer-Assisted. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 72 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Sign Language Recognition (SLR) involves the automatic identification and classification of sign gestures from images or video, converting them into text or speech to improve accessibility for the hard-of-hearing community. In Bangladesh, Bangla Sign Language (BdSL) serves as the primary mode of communication for many individuals with hearing loss. This study fine-tunes state-of-the-art video transformer architectures VideoMAE, ViViT, and TimeSformer on BdSLW60, a small-scale BdSL dataset with 60 frequent signs. We standardized the videos to 30 FPS, resulting in 9,307 user trial clips. To evaluate scalability and robustness, the models were also fine-tuned on BdSLW401, a large-scale dataset with 401 sign classes. Additionally, we benchmark performance against public datasets, including LSA64 and WLASL. Data augmentation techniques such as random cropping, horizontal flipping, and short-side scaling were applied to improve model robustness. To ensure balanced evaluation across folds during model selection, we employed 10-fold stratified cross-validation on the training set of the BdSLW60 dataset, while signer-independent evaluation was carried out using held-out test data from unseen users U4 and U8. Results show that video transformer models significantly outperform traditional machine learning and deep learning approaches. Performance is influenced by factors such as dataset size, signer appearance, frame distribution, frame rate, and model architecture. Among the models, the VideoMAE variant (MCG-NJU/videomae-base-finetuned-kinetics) achieved the highest accuracies 96.9% on the frame rate corrected BdSLW60 dataset and 81.04% on the front-facing signs of BdSLW401 demonstrating strong potential for scalable and accurate BdSL recognition.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Sign LanguageVideo RecordingImage Processing, Computer-AssistedHumansBangladeshGestures

Autor:innen und Institutionen

  • Systems and Software Lab (SSL), Department of CSE, Islamic University of Technology (IUT), OIC, Board Bazar, Gazipur, Bangladesh.

ORCID-Profile: Jubayer Ahmed Bhuiyan Shawon

Zitieren

APA

Shawon, J. A. B., Hasan, M. K., & Mahmud, H. (2026). A comparative analysis of video vision transformers on word-level sign language datasets. PLOS One. https://doi.org/10.1371/journal.pone.0341909

Vancouver

Shawon JAB, Hasan MK, Mahmud H. A comparative analysis of video vision transformers on word-level sign language datasets. PLOS One. 2026. doi:10.1371/journal.pone.0341909

BibTeX
@article{Shawon2026,
  author = {Shawon, Jubayer Ahmed Bhuiyan and Hasan, Md Kamrul and Mahmud, Hasan},
  title = {A comparative analysis of video vision transformers on word-level sign language datasets},
  journal = {PLOS One},
  year = {2026},
  doi = {10.1371/journal.pone.0341909}
}
Kostenlos starten →