EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Hybrid Deep Learning Models for Arabic Sign Language Recognition in Healthcare Applications

Ibtihel Mansour, Mohamed Hamroun, Sonia Lajmi, Ryma Abassi, Damien Sauveron

Big Data and Cognitive Computing · 08.11.2025 · 9 (11) · 281

DOI: 10.3390/bdcc9110281 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 08.11.2025. Verfasst von Ibtihel Mansour und 4 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 76 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: XLIM, UMR CNRS 7252, University of Limoges.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deaf and hearing-impaired individuals rely on sign language, a visual communication system using hand shapes, facial expressions, and body gestures. Sign languages vary by region. For example, Arabic Sign Language (ArSL) is notably different from American Sign Language (ASL). This project focuses on creating an Arabic Sign Language Recognition (ArSLR) System tailored for healthcare, aiming to bridge communication gaps resulting from a lack of sign-proficient professionals and limited region-specific technological solutions. Our research addresses limitations in sign language recognition systems by introducing a novel framework centered on ResNet50ViT, a hybrid architecture that synergistically combines ResNet50’s robust local feature extraction with the global contextual modeling of Vision Transformers (ViT). We also explored a tailored Vision Transformer variant (SignViT) for Arabic Sign Language as a comparative model. Our main contribution is the ResNet50ViT model, which significantly outperforms existing approaches, specifically targeting the challenges of capturing sequential hand movements, which traditional CNN-based methods struggle with. We utilized an extensive dataset incorporating both static (36 signs) and dynamic (92 signs) medical signs. Through targeted preprocessing techniques and optimization strategies, we achieved significant performance improvements over conventional approaches. In our experiments, the proposed ResNet50-ViT achieved a remarkable 99.86% accuracy on the ArSL dataset, setting a new state-of-the-art, demonstrating the effectiveness of integrating ResNet50’s hierarchical local feature extraction with Vision Transformer’s global contextual modeling. For comparison, a fine-tuned Vision Transformer (SignViT) attained 98.03% accuracy, confirming the strength of transformer-based approaches but underscoring the clear performance gain enabled by our hybrid architecture. We expect that RAFID will help deaf patients communicate better with healthcare providers without needing human interpreters.

Förderung

  • XLIM, UMR CNRS 7252, University of Limoges

Zitieren

APA

Mansour, I., Hamroun, M., Lajmi, S., Abassi, R., & Sauveron, D. (2025). Hybrid Deep Learning Models for Arabic Sign Language Recognition in Healthcare Applications. Big Data and Cognitive Computing, 9(11), 281. https://doi.org/10.3390/bdcc9110281

Vancouver

Mansour I, Hamroun M, Lajmi S, Abassi R, Sauveron D. Hybrid Deep Learning Models for Arabic Sign Language Recognition in Healthcare Applications. Big Data and Cognitive Computing. 2025;9(11):281. doi:10.3390/bdcc9110281

BibTeX
@article{Mansour2025,
  author = {Mansour, Ibtihel and Hamroun, Mohamed and Lajmi, Sonia and Abassi, Ryma and Sauveron, Damien},
  title = {Hybrid Deep Learning Models for Arabic Sign Language Recognition in Healthcare Applications},
  journal = {Big Data and Cognitive Computing},
  year = {2025},
  volume = {9},
  number = {11},
  pages = {281},
  doi = {10.3390/bdcc9110281}
}
Kostenlos starten →