EUVIMEDEUVIMED entdecken
Crossref

Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition

Yih-Liang Shen, Pei-Chin Hsieh, Tai-Shih Chi

IEEE Transactions on Affective Computing · 2025-07 · 16 (3) · 1693-1704

DOI: 10.1109/taffc.2025.3531638 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025-07. Verfasst von Yih-Liang Shen und 2 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: Ministry of Science and Technology, Taiwan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Deep learning based speech emotion recognition (SER) models have shown impressive results in controlled environments, but their performance significantly degrades in noisy conditions. This paper proposes a robust two-stream SER model by combining spectro-temporal modulation features with conventional acoustic features. Experiments were conducted on German (EMODB) and English (RAVDESS) datasets using the clean-train-noisy-test paradigm. The results demonstrate that spectro-temporal modulation features offer superior robustness in noisy conditions compared with conventional acoustic features such as MFCCs and time-frequency features from Mel-spectrograms. Additionally, we analyze weights of modulation features and demonstrate the model emphasizes contours of formants and harmonics, which are crucial features for speech perception in noise, for robust SER. Incorporating the stream of spectro-temporal modulations not only enhances the robustness of the model but also provides deeper insights into the task of SER in noise.

Förderung

  • Ministry of Science and Technology, Taiwan

Zitieren

APA

Shen, Y., Hsieh, P., & Chi, T. (2025). Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition. IEEE Transactions on Affective Computing, 16(3), 1693-1704. https://doi.org/10.1109/taffc.2025.3531638

Vancouver

Shen Y, Hsieh P, Chi T. Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition. IEEE Transactions on Affective Computing. 2025;16(3):1693-1704. doi:10.1109/taffc.2025.3531638

BibTeX
@article{Shen2025,
  author = {Shen, Yih-Liang and Hsieh, Pei-Chin and Chi, Tai-Shih},
  title = {Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition},
  journal = {IEEE Transactions on Affective Computing},
  year = {2025},
  volume = {16},
  number = {3},
  pages = {1693--1704},
  doi = {10.1109/taffc.2025.3531638}
}
Kostenlos starten →