EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Full-Stack Multimodal Assistive Communication System

Shivangi Jindal, Kajal Kori

International Journal of Scientific Research in Science and Technology · 22.04.2026 · 13 (2) · 871-876

DOI: 10.32628/ijsrst2613366 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Scientific Research in Science and Technology, online erschienen am 22.04.2026. Verfasst von Shivangi Jindal und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

MANUSCRIPT is a full-stack web-based assistive communication platform designed to bridge the communication gap between sign-language / visually impaired users and the hearing/ seeing population. The system enables real-time conversion of hand gestures and spoken language into readable text using computer vision and speech recognition technologies. It integrates MediaPipe Hands for gesture detection and the Web Speech API for speech-to-text conversion, supported by a Flask backend for secure data storage, authentication, and scalability. The primary problem addressed is the lack of accessible, scalable, and affordable communication tools for individuals with hearing, visual or speech impairments. Existing solutions are either limited to single modalities or rely heavily on server-based systems, leading to latency and privacy concerns. The objective of this project is to develop a hybrid system that combines real-time client-side processing with backend support to enhance usability, scalability, and data persistence. The system achieves an average response time of 0.17s for gesture recognition and 0.8s for speech-to-text conversion, enabling near real-time communication. The expected outcome is a robust, user-friendly platform capable of facilitating seamless communication using gestures and speech, with potential applications in education, healthcare, and everyday interactions.

Zitieren

APA

Jindal, S., & Kori, K. (2026). A Full-Stack Multimodal Assistive Communication System. International Journal of Scientific Research in Science and Technology, 13(2), 871-876. https://doi.org/10.32628/ijsrst2613366

Vancouver

Jindal S, Kori K. A Full-Stack Multimodal Assistive Communication System. International Journal of Scientific Research in Science and Technology. 2026;13(2):871-876. doi:10.32628/ijsrst2613366

BibTeX
@article{Jindal2026,
  author = {Jindal, Shivangi and Kori, Kajal},
  title = {A Full-Stack Multimodal Assistive Communication System},
  journal = {International Journal of Scientific Research in Science and Technology},
  year = {2026},
  volume = {13},
  number = {2},
  pages = {871--876},
  doi = {10.32628/ijsrst2613366}
}
Kostenlos starten →