EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking

Bader Alsharif iD, Easa Alalwany iD, Ali Ibrahim, Imad Mahgoub iD, Mohammad Ilyas iD

Sensors · 28.03.2025 · 25 (7) · 2138

DOI: 10.3390/s25072138 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 28.03.2025. Verfasst von Bader Alsharif und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical Engineering and Computer Science, Florida Atlantic University, 777 Glades Road, Boca Raton, FL 33431, USA. Hauptthemen laut MEDLINE: Sign Language, Deep Learning. Die Arbeit wurde bisher 39-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Communication barriers pose significant challenges for the Deaf and Hard-of-Hearing (DHH) community, limiting their access to essential services, social interactions, and professional opportunities. To bridge this gap, assistive technologies leveraging artificial intelligence (AI) and deep learning have gained prominence. This study presents a real-time American Sign Language (ASL) interpretation system that integrates deep learning with keypoint tracking to enhance accessibility and foster inclusivity. By combining the YOLOv11 model for gesture recognition with MediaPipe for precise hand tracking, the system achieves high accuracy in identifying ASL alphabet letters in real time. The proposed approach addresses challenges such as gesture ambiguity, environmental variations, and computational efficiency. Additionally, this system enables users to spell out names and locations, further improving its practical applications. Experimental results demonstrate that the model attains a mean Average Precision (mAP@0.5) of 98.2%, with an inference speed optimized for real-world deployment. This research underscores the critical role of AI-driven assistive technologies in empowering the DHH community by enabling seamless communication and interaction.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Sign LanguageDeep LearningHumansGesturesArtificial Intelligence

Stichwörter der Autor:innen

AI for accessibilityMediaPipeYOLO11assistive technologydeep learninghuman–computer interactionreal-Time ASL recognitiontransfer learning

Autor:innen und Institutionen

  • Department of Electrical Engineering and Computer Science, Florida Atlantic University, 777 Glades Road, Boca Raton, FL 33431, USA.
  • Department of Computer Science and Engineering, College of Telecommunication and Information, Technical and Vocational Training Corporation, Riyadh 12464, Saudi Arabia.
  • College of Computer Science and Engineering, Taibah University, Yanbu 46421, Saudi Arabia.

ORCID-Profile: Bader Alsharif, Easa Alalwany, Imad Mahgoub, Mohammad Ilyas

Zitieren

APA

Alsharif, B., Alalwany, E., Ibrahim, A., Mahgoub, I., & Ilyas, M. (2025). Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking. Sensors, 25(7), 2138. https://doi.org/10.3390/s25072138

Vancouver

Alsharif B, Alalwany E, Ibrahim A, Mahgoub I, Ilyas M. Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking. Sensors. 2025;25(7):2138. doi:10.3390/s25072138

BibTeX
@article{Alsharif2025,
  author = {Alsharif, Bader and Alalwany, Easa and Ibrahim, Ali and Mahgoub, Imad and Ilyas, Mohammad},
  title = {Real-Time American Sign Language Interpretation Using Deep Learning and Keypoint Tracking},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {7},
  pages = {2138},
  doi = {10.3390/s25072138}
}
Kostenlos starten →