A Full-Stack Multimodal Assistive Communication System
International Journal of Scientific Research in Science and Technology · 22.04.2026 · 13 (2) · 871-876
DOI: 10.32628/ijsrst2613366 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Scientific Research in Science and Technology, online erschienen am 22.04.2026. Verfasst von Shivangi Jindal und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
MANUSCRIPT is a full-stack web-based assistive communication platform designed to bridge the communication gap between sign-language / visually impaired users and the hearing/ seeing population. The system enables real-time conversion of hand gestures and spoken language into readable text using computer vision and speech recognition technologies. It integrates MediaPipe Hands for gesture detection and the Web Speech API for speech-to-text conversion, supported by a Flask backend for secure data storage, authentication, and scalability. The primary problem addressed is the lack of accessible, scalable, and affordable communication tools for individuals with hearing, visual or speech impairments. Existing solutions are either limited to single modalities or rely heavily on server-based systems, leading to latency and privacy concerns. The objective of this project is to develop a hybrid system that combines real-time client-side processing with backend support to enhance usability, scalability, and data persistence. The system achieves an average response time of 0.17s for gesture recognition and 0.8s for speech-to-text conversion, enabling near real-time communication. The expected outcome is a robust, user-friendly platform capable of facilitating seamless communication using gestures and speech, with potential applications in education, healthcare, and everyday interactions.
Zitieren
APA
Jindal, S., & Kori, K. (2026). A Full-Stack Multimodal Assistive Communication System. International Journal of Scientific Research in Science and Technology, 13(2), 871-876. https://doi.org/10.32628/ijsrst2613366
Vancouver
Jindal S, Kori K. A Full-Stack Multimodal Assistive Communication System. International Journal of Scientific Research in Science and Technology. 2026;13(2):871-876. doi:10.32628/ijsrst2613366
BibTeX
@article{Jindal2026,
author = {Jindal, Shivangi and Kori, Kajal},
title = {A Full-Stack Multimodal Assistive Communication System},
journal = {International Journal of Scientific Research in Science and Technology},
year = {2026},
volume = {13},
number = {2},
pages = {871--876},
doi = {10.32628/ijsrst2613366}
}
EUVIMED