A multi-modal deep learning approach for stress detection using physiological signals: integrating time and frequency domain features
Frontiers in Physiology · 01.04.2025 · 16
DOI: 10.3389/fphys.2025.1584299 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Physiology, online erschienen am 01.04.2025. Verfasst von Jun-Zhi Xiang und 4 weiteren Autor:innen. Beteiligte Einrichtung: Emergency Department, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, China. Die Arbeit wurde bisher 85-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wenzhou Municipal Science and Technology Bureau.
Abstract (Originalsprache)
Objective
This study aims to develop a multimodal deep learning-based stress detection method (MMFD-SD) using intermittently collected physiological signals from wearable devices, including accelerometer data, electrodermal activity (EDA), heart rate (HR), and skin temperature. Given the unique demands and high-intensity work environment of the nursing profession, stress measurement in nurses serves as a representative case, reflecting stress levels in other high-pressure occupations.
Methods
We propose a multimodal deep learning framework that integrates time-domain and frequency-domain features for stress detection. To enhance model robustness and generalization, data augmentation techniques such as sliding window and jittering are applied. Feature extraction includes statistical features derived from raw time-domain signals and frequency-domain features obtained via Fast Fourier Transform (FFT). A customized deep learning architecture employs convolutional neural networks (CNNs) to process time-domain and frequency-domain features separately, followed by fully connected layers for final classification. To address class imbalance, the Synthetic Minority Over-sampling Technique (SMOTE) is utilized. The model is trained and evaluated on a multimodal physiological signal dataset with stress level labels.
Results
Experimental results demonstrate that the MMFD-SD method achieves outstanding performance in stress detection, with an accuracy of 91.00% and an F1-score of 0.91. Compared to traditional machine learning classifiers such as logistic regression, random forest, and XGBoost, the proposed method significantly improves both accuracy and robustness. Ablation studies reveal that the integration of time-domain and frequency-domain features plays a crucial role in enhancing model performance. Additionally, sensitivity analysis confirms the model’s stability and adaptability across different hyperparameter settings.
Conclusion
The proposed MMFD-SD model provides an accurate and robust stress detection approach by integrating time-domain and frequency-domain features. Designed for occupational environments with intermittent data collection, it effectively addresses real-world stress monitoring challenges. Future research can explore the fusion of additional modalities, real-time stress detection, and improvements in model generalization to enhance its practical applicability.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Emergency Department, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, China.
- School of Artificial Intelligence, Zhejiang College of Security Technology, Wenzhou, Zhejiang, China.
- School of Software Technology, Zhejiang University, Ningbo, Zhejiang, China.
- Graduate School, Angeles University Foundation, Angeles, Philippines.
Förderung
- Wenzhou Municipal Science and Technology Bureau
Zitieren
APA
Xiang, J. Z., Wang, Q. Y., Fang, Z. B., Esquivel, J. A., & Su, Z. X. (2025). A multi-modal deep learning approach for stress detection using physiological signals: integrating time and frequency domain features. Frontiers in Physiology, 16. https://doi.org/10.3389/fphys.2025.1584299
Vancouver
Xiang JZ, Wang QY, Fang ZB, Esquivel JA, Su ZX. A multi-modal deep learning approach for stress detection using physiological signals: integrating time and frequency domain features. Frontiers in Physiology. 2025;16. doi:10.3389/fphys.2025.1584299
BibTeX
@article{Xiang2025,
author = {Xiang, Jun-Zhi and Wang, Qin-Yong and Fang, Zhi-Bin and Esquivel, James A and Su, Zhi-Xian},
title = {A multi-modal deep learning approach for stress detection using physiological signals: integrating time and frequency domain features},
journal = {Frontiers in Physiology},
year = {2025},
volume = {16},
doi = {10.3389/fphys.2025.1584299}
}
EUVIMED