Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking
Sensors · 28.03.2025 · 25 (7) · 2138
DOI: 10.3390/s25072138 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 28.03.2025. Verfasst von Bader Alsharif und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical Engineering and Computer Science, Florida Atlantic University, 777 Glades Road, Boca Raton, FL 33431, USA. Hauptthemen laut MEDLINE: Sign Language, Deep Learning. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Communication barriers pose significant challenges for the Deaf and Hard-of-Hearing (DHH) community, limiting their access to essential services, social interactions, and professional opportunities. To bridge this gap, assistive technologies leveraging artificial intelligence (AI) and deep learning have gained prominence. This study presents a real-time American Sign Language (ASL) interpretation system that integrates deep learning with keypoint tracking to enhance accessibility and foster inclusivity. By combining the YOLOv11 model for gesture recognition with MediaPipe for precise hand tracking, the system achieves high accuracy in identifying ASL alphabet letters in real time. The proposed approach addresses challenges such as gesture ambiguity, environmental variations, and computational efficiency. Additionally, this system enables users to spell out names and locations, further improving its practical applications. Experimental results demonstrate that the model attains a mean Average Precision (mAP@0.5) of 98.2%, with an inference speed optimized for real-world deployment. This research underscores the critical role of AI-driven assistive technologies in empowering the DHH community by enabling seamless communication and interaction.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Electrical Engineering and Computer Science, Florida Atlantic University, 777 Glades Road, Boca Raton, FL 33431, USA.
- Department of Computer Science and Engineering, College of Telecommunication and Information, Technical and Vocational Training Corporation, Riyadh 12464, Saudi Arabia.
- College of Computer Science and Engineering, Taibah University, Yanbu 46421, Saudi Arabia.
ORCID-Profile: Bader Alsharif, Easa Alalwany, Imad Mahgoub, Mohammad Ilyas
Zitieren
APA
Alsharif, B., Alalwany, E., Ibrahim, A., Mahgoub, I., & Ilyas, M. (2025). Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking. Sensors, 25(7), 2138. https://doi.org/10.3390/s25072138
Vancouver
Alsharif B, Alalwany E, Ibrahim A, Mahgoub I, Ilyas M. Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking. Sensors. 2025;25(7):2138. doi:10.3390/s25072138
BibTeX
@article{Alsharif2025,
author = {Alsharif, Bader and Alalwany, Easa and Ibrahim, Ali and Mahgoub, Imad and Ilyas, Mohammad},
title = {Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking},
journal = {Sensors},
year = {2025},
volume = {25},
number = {7},
pages = {2138},
doi = {10.3390/s25072138}
}
EUVIMED