EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CLIP-ArASL: A Lightweight Multimodal Model for Arabic Sign Language Recognition

Naif Alasmari

Applied Sciences · 07.03.2026 · 16 (5) · 2573

DOI: 10.3390/app16052573 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 07.03.2026. Verfasst von Naif Alasmari. Die Arbeit nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deanship of Research and Graduate Studies at King Khalid University, KSA.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Arabic sign language (ArASL) is the primary communication medium for Deaf and hard-of-hearing people across Arabic-speaking communities. Most current ArASL recognition systems are based solely on visual features and do not incorporate linguistic or semantic information that could improve generalization and semantic grounding. This paper introduces CLIP-ArASL, a lightweight CLIP-style multimodal approach for static ArASL letter recognition that aligns visual hand gestures with bilingual textual descriptions. The approach integrates an EfficientNet-B0 image encoder with a MiniLM text encoder to learn a shared embedding space using a hybrid objective that combines contrastive and cross-entropy losses. This design supports supervised classification on seen classes and zero-shot prediction on unseen classes using textual class representations. The proposed approach is evaluated on two public datasets, ArASL2018 and ArASL21L. Under supervised evaluation, recognition accuracies of 99.25±0.14% and 91.51±1.29% are achieved, respectively. Zero-shot performance is assessed by withholding 20% of gesture classes during training and predicting them using only their textual descriptions. In this setting, accuracies of 55.2±12.15% on ArASL2018 and 37.6±9.07% on ArASL21L are obtained. These results show that multimodal vision–language alignment supports semantic transfer and enables recognition of unseen classes.

Förderung

  • Deanship of Research and Graduate Studies at King Khalid University, KSA

Zitieren

APA

Alasmari, N. (2026). CLIP-ArASL: A Lightweight Multimodal Model for Arabic Sign Language Recognition. Applied Sciences, 16(5), 2573. https://doi.org/10.3390/app16052573

Vancouver

Alasmari N. CLIP-ArASL: A Lightweight Multimodal Model for Arabic Sign Language Recognition. Applied Sciences. 2026;16(5):2573. doi:10.3390/app16052573

BibTeX
@article{Alasmari2026,
  author = {Alasmari, Naif},
  title = {CLIP-ArASL: A Lightweight Multimodal Model for Arabic Sign Language Recognition},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {5},
  pages = {2573},
  doi = {10.3390/app16052573}
}
Kostenlos starten →