AI-Powered Multimodal Sign Language Recognition, Translation and Assistive Communication System
INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · 27.04.2026 · 10 (04) · 1-9
DOI: 10.55041/ijsrem61508 ↗Kurz gefasst
Fachartikel in der Zeitschrift INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT, online erschienen am 27.04.2026. Verfasst von Abhisha KJ und 4 weiteren Autor:innen.
Abstract (Originalsprache)
Abstract- Sign language is an essential communication medium for individuals with hearing and speech impairments, yet communication barriers still exist due to the limited availability of intelligent translation tools. This paper proposes an AI-powered multimodal sign language recognition, translation, and assistive communication system designed to improve accessibility and interaction between sign language users and the general community. The system utilizes MediaPipe to detect and extract 21 hand landmarks from sign gestures captured through a standard webcam. These landmarks are converted into structured JSON feature vectors and used to train a 1D Convolutional Neural Network (1D-CNN) model for accurate recognition of predefined as well as custom user-defined signs. The proposed system supports real-time and offline sign detection, sentence formation. multi-language translation, and text-to-speech output for enhanced communication. In addition, the platform enables multimodal interaction through voice, text, image, and video inputs. For video inputs, audio is extracted and converted to text, translated if necessary, and synchronized subtitles are embedded into the video. The system also includes a motivational support module that allows trained users to provide assistance and guidance. The proposed approach offers an inclusive, scalable, and hardware-independent solution for assistive communication. Key Words: Sign Language Recognition, Deep Learning, MediaPipe, Multimodal Interaction,1D Convolutional Neural Network
Zitieren
APA
KJ, A., KS, A., M, A., M, A., & CS, S. (2026). AI-Powered Multimodal Sign Language Recognition, Translation and Assistive Communication System. INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT, 10(04), 1-9. https://doi.org/10.55041/ijsrem61508
Vancouver
KJ A, KS A, M A, M A, CS S. AI-Powered Multimodal Sign Language Recognition, Translation and Assistive Communication System. INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT. 2026;10(04):1-9. doi:10.55041/ijsrem61508
BibTeX
@article{KJ2026,
author = {KJ, Abhisha and KS, Akash and M, Arathi and M, Ashik and CS, Swathy},
title = {AI-Powered Multimodal Sign Language Recognition, Translation and Assistive Communication System},
journal = {INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT},
year = {2026},
volume = {10},
number = {04},
pages = {1--9},
doi = {10.55041/ijsrem61508}
}
EUVIMED