CLIP-ArASL: A Lightweight Multimodal Model for Arabic Sign Language Recognition
Applied Sciences · 07.03.2026 · 16 (5) · 2573
DOI: 10.3390/app16052573 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 07.03.2026. Verfasst von Naif Alasmari. Die Arbeit nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Deanship of Research and Graduate Studies at King Khalid University, KSA.
Abstract (Originalsprache)
Arabic sign language (ArASL) is the primary communication medium for Deaf and hard-of-hearing people across Arabic-speaking communities. Most current ArASL recognition systems are based solely on visual features and do not incorporate linguistic or semantic information that could improve generalization and semantic grounding. This paper introduces CLIP-ArASL, a lightweight CLIP-style multimodal approach for static ArASL letter recognition that aligns visual hand gestures with bilingual textual descriptions. The approach integrates an EfficientNet-B0 image encoder with a MiniLM text encoder to learn a shared embedding space using a hybrid objective that combines contrastive and cross-entropy losses. This design supports supervised classification on seen classes and zero-shot prediction on unseen classes using textual class representations. The proposed approach is evaluated on two public datasets, ArASL2018 and ArASL21L. Under supervised evaluation, recognition accuracies of 99.25±0.14% and 91.51±1.29% are achieved, respectively. Zero-shot performance is assessed by withholding 20% of gesture classes during training and predicting them using only their textual descriptions. In this setting, accuracies of 55.2±12.15% on ArASL2018 and 37.6±9.07% on ArASL21L are obtained. These results show that multimodal vision–language alignment supports semantic transfer and enables recognition of unseen classes.
Förderung
- Deanship of Research and Graduate Studies at King Khalid University, KSA
Zitieren
APA
Alasmari, N. (2026). CLIP-ArASL: A Lightweight Multimodal Model for Arabic Sign Language Recognition. Applied Sciences, 16(5), 2573. https://doi.org/10.3390/app16052573
Vancouver
Alasmari N. CLIP-ArASL: A Lightweight Multimodal Model for Arabic Sign Language Recognition. Applied Sciences. 2026;16(5):2573. doi:10.3390/app16052573
BibTeX
@article{Alasmari2026,
author = {Alasmari, Naif},
title = {CLIP-ArASL: A Lightweight Multimodal Model for Arabic Sign Language Recognition},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {5},
pages = {2573},
doi = {10.3390/app16052573}
}
EUVIMED