Neurolens: A Multimodal Real-Time Stress Detection System using Computer Vision and Speech Emotion Recognition
INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT · 31.03.2026 · 10 (03) · 1-9
DOI: 10.55041/ijsrem58725 ↗Kurz gefasst
Fachartikel in der Zeitschrift INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT, online erschienen am 31.03.2026. Verfasst von Chaitali Mhatre und 4 weiteren Autor:innen.
Abstract (Originalsprache)
- Chronic psychological stress impairs cognitive performance, academic outcomes, and long-term well-being, yet most automated detection systems rely on a single sensing modality, limiting their robustness under real-world conditions. Unimodal approaches—whether vision-based, physiological, or acoustic—are individually vulnerable to noise, occlusion, and signal artifacts, motivating the need for integrated multimodal frameworks. This paper presents Neurolens, a real-time multimodal stress detection system that concurrently processes facial video through a fine-tuned You Only Look Once version 8 (YOLOv8) model trained on a publicly available facial emotion dataset, wearable physiological signals—including electrodermal activity (EDA), blood volume pulse (BVP), and skin temperature— through a hybrid convolutional neural network–long short- term memory (CNN-LSTM) architecture trained on the WESAD wearable stress and affect detection dataset, and speech audio through a Wav2Vec2 transformer-based speech encoder. A weighted late-fusion module integrates per- modality stress scores into a unified Stress Index rendered on an interactive real-time dashboard with adaptive push notifications and ambient brightness control. System demonstrations confirm correct identification of stress- indicative facial states such as anger and elevated physiological arousal from CSV-uploaded sensor data, alongside neutral baseline detection with appropriately reduced stress index values. These results establish Neurolens as a scalable, non-invasive, and reproducible framework for continuous passive stress monitoring in academic, clinical, and professional environments. Keywords—multimodal fusion; Wav2Vec2; CNN-LSTM; facial emotion recognition; speech emotion recognition; wearable sensors.
Zitieren
APA
Mhatre, C., Pal, S., Rajbhar, A., Patra, P., & Rathod, Y. (2026). Neurolens: A Multimodal Real-Time Stress Detection System using Computer Vision and Speech Emotion Recognition. INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT, 10(03), 1-9. https://doi.org/10.55041/ijsrem58725
Vancouver
Mhatre C, Pal S, Rajbhar A, Patra P, Rathod Y. Neurolens: A Multimodal Real-Time Stress Detection System using Computer Vision and Speech Emotion Recognition. INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT. 2026;10(03):1-9. doi:10.55041/ijsrem58725
BibTeX
@article{Mhatre2026,
author = {Mhatre, Chaitali and Pal, Shivam and Rajbhar, Aryan and Patra, Pritesh and Rathod, Yuvraj},
title = {Neurolens: A Multimodal Real-Time Stress Detection System using Computer Vision and Speech Emotion Recognition},
journal = {INTERNATIONAL JOURNAL OF SCIENTIFIC RESEARCH IN ENGINEERING AND MANAGEMENT},
year = {2026},
volume = {10},
number = {03},
pages = {1--9},
doi = {10.55041/ijsrem58725}
}
EUVIMED