EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A multi-modal deep learning approach for stress detection using physiological signals: integrating time and frequency domain features

Jun-Zhi Xiang, Qin-Yong Wang, Zhi-Bin Fang, James A Esquivel, Zhi-Xian Su

Frontiers in Physiology · 01.04.2025 · 16

DOI: 10.3389/fphys.2025.1584299 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Physiology, online erschienen am 01.04.2025. Verfasst von Jun-Zhi Xiang und 4 weiteren Autor:innen. Beteiligte Einrichtung: Emergency Department, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, China. Die Arbeit wurde bisher 85-mal zitiert (Crossref) und nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Wenzhou Municipal Science and Technology Bureau.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

This study aims to develop a multimodal deep learning-based stress detection method (MMFD-SD) using intermittently collected physiological signals from wearable devices, including accelerometer data, electrodermal activity (EDA), heart rate (HR), and skin temperature. Given the unique demands and high-intensity work environment of the nursing profession, stress measurement in nurses serves as a representative case, reflecting stress levels in other high-pressure occupations.

Methods

We propose a multimodal deep learning framework that integrates time-domain and frequency-domain features for stress detection. To enhance model robustness and generalization, data augmentation techniques such as sliding window and jittering are applied. Feature extraction includes statistical features derived from raw time-domain signals and frequency-domain features obtained via Fast Fourier Transform (FFT). A customized deep learning architecture employs convolutional neural networks (CNNs) to process time-domain and frequency-domain features separately, followed by fully connected layers for final classification. To address class imbalance, the Synthetic Minority Over-sampling Technique (SMOTE) is utilized. The model is trained and evaluated on a multimodal physiological signal dataset with stress level labels.

Results

Experimental results demonstrate that the MMFD-SD method achieves outstanding performance in stress detection, with an accuracy of 91.00% and an F1-score of 0.91. Compared to traditional machine learning classifiers such as logistic regression, random forest, and XGBoost, the proposed method significantly improves both accuracy and robustness. Ablation studies reveal that the integration of time-domain and frequency-domain features plays a crucial role in enhancing model performance. Additionally, sensitivity analysis confirms the model’s stability and adaptability across different hyperparameter settings.

Conclusion

The proposed MMFD-SD model provides an accurate and robust stress detection approach by integrating time-domain and frequency-domain features. Designed for occupational environments with intermittent data collection, it effectively addresses real-world stress monitoring challenges. Future research can explore the fusion of additional modalities, real-time stress detection, and improvements in model generalization to enhance its practical applicability.

Schlagwörter

Stichwörter der Autor:innen

fast fourier transformmulti-modal deep learningstress detectiontime and frequency domain featureswearable devices

Autor:innen und Institutionen

  • Emergency Department, The First Affiliated Hospital of Wenzhou Medical University, Wenzhou, Zhejiang, China.
  • School of Artificial Intelligence, Zhejiang College of Security Technology, Wenzhou, Zhejiang, China.
  • School of Software Technology, Zhejiang University, Ningbo, Zhejiang, China.
  • Graduate School, Angeles University Foundation, Angeles, Philippines.

Förderung

  • Wenzhou Municipal Science and Technology Bureau

Zitieren

APA

Xiang, J. Z., Wang, Q. Y., Fang, Z. B., Esquivel, J. A., & Su, Z. X. (2025). A multi-modal deep learning approach for stress detection using physiological signals: integrating time and frequency domain features. Frontiers in Physiology, 16. https://doi.org/10.3389/fphys.2025.1584299

Vancouver

Xiang JZ, Wang QY, Fang ZB, Esquivel JA, Su ZX. A multi-modal deep learning approach for stress detection using physiological signals: integrating time and frequency domain features. Frontiers in Physiology. 2025;16. doi:10.3389/fphys.2025.1584299

BibTeX
@article{Xiang2025,
  author = {Xiang, Jun-Zhi and Wang, Qin-Yong and Fang, Zhi-Bin and Esquivel, James A and Su, Zhi-Xian},
  title = {A multi-modal deep learning approach for stress detection using physiological signals: integrating time and frequency domain features},
  journal = {Frontiers in Physiology},
  year = {2025},
  volume = {16},
  doi = {10.3389/fphys.2025.1584299}
}
Kostenlos starten →