A Comprehensive Approach to Indian Sign Language Recognition: Leveraging LSTM and MediaPipe Holistic for Dynamic and Static Hand Gesture Recognition
EAI Endorsed Transactions on AI and Robotics · 19.05.2025 · 4
DOI: 10.4108/airo.8693 ↗Kurz gefasst
Fachartikel in der Zeitschrift EAI Endorsed Transactions on AI and Robotics, online erschienen am 19.05.2025. Verfasst von Prachi Rawat und 3 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref). Der Volltext ist frei zugänglich (Open Access, CC BY-NC-SA 4.0).
Abstract (Originalsprache)
Recognizing Indian Sign Language (ISL) gestures effectively is crucial for improving communication accessibility for deaf community. This study introduces an innovative approach that integrates a Sequential Long Short-Term Memory (LSTM) model with MediaPipe Holistic for accurate and real-time gesture recognition. This work outlines a straightforward approach to recognizing Indian Sign Language (ISL) gestures effectively. The process is divided into three steps: Extracting features from data, Cleaning, Labelling and identifying gestures using MediaPipe Holistic. The system tracks landmarks on the face, hands, and body across video frames, capturing essential details such as temporal and spatial features for interpreting gestures. First, the data is cleaned and labeled by removing unclear fuzzy images and null entries. Then after, the processed data is passed into a Sequential LSTM model, which has two LSTM layers and a dense output layer. In the proposed approach, model’s performance is improved by integrating techniques such as early stopping and categorical cross-entropy. The model is trained and tested using a customized ISL dataset that included 11 distinct gestures, and it achieved a high accuracy rate of 96.97%. The framework emphasizes the model's robustness across diverse lighting conditions and real-world scenarios, ensuring its applicability in sectors such as healthcare, education, and public service. By enhancing communication for ISL users, it effectively addresses existing gaps and improves accessibility in these domains.
Zitieren
APA
Rawat, P., Kumar, P., Tamta, V. K., & Kumar, A. (2025). A Comprehensive Approach to Indian Sign Language Recognition: Leveraging LSTM and MediaPipe Holistic for Dynamic and Static Hand Gesture Recognition. EAI Endorsed Transactions on AI and Robotics, 4. https://doi.org/10.4108/airo.8693
Vancouver
Rawat P, Kumar P, Tamta VK, Kumar A. A Comprehensive Approach to Indian Sign Language Recognition: Leveraging LSTM and MediaPipe Holistic for Dynamic and Static Hand Gesture Recognition. EAI Endorsed Transactions on AI and Robotics. 2025;4. doi:10.4108/airo.8693
BibTeX
@article{Rawat2025,
author = {Rawat, Prachi and Kumar, Papendra and Tamta, Vivek Kumar and Kumar, Anuj},
title = {A Comprehensive Approach to Indian Sign Language Recognition: Leveraging LSTM and MediaPipe Holistic for Dynamic and Static Hand Gesture Recognition},
journal = {EAI Endorsed Transactions on AI and Robotics},
year = {2025},
volume = {4},
doi = {10.4108/airo.8693}
}
EUVIMED