Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition
IEEE Transactions on Affective Computing · 2025-07 · 16 (3) · 1693-1704
DOI: 10.1109/taffc.2025.3531638 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025-07. Verfasst von Yih-Liang Shen und 2 weiteren Autor:innen. Die Arbeit wurde bisher 8-mal zitiert (Crossref) und nennt 50 Literaturangaben. Gefördert von: Ministry of Science and Technology, Taiwan.
Abstract (Originalsprache)
Deep learning based speech emotion recognition (SER) models have shown impressive results in controlled environments, but their performance significantly degrades in noisy conditions. This paper proposes a robust two-stream SER model by combining spectro-temporal modulation features with conventional acoustic features. Experiments were conducted on German (EMODB) and English (RAVDESS) datasets using the clean-train-noisy-test paradigm. The results demonstrate that spectro-temporal modulation features offer superior robustness in noisy conditions compared with conventional acoustic features such as MFCCs and time-frequency features from Mel-spectrograms. Additionally, we analyze weights of modulation features and demonstrate the model emphasizes contours of formants and harmonics, which are crucial features for speech perception in noise, for robust SER. Incorporating the stream of spectro-temporal modulations not only enhances the robustness of the model but also provides deeper insights into the task of SER in noise.
Förderung
- Ministry of Science and Technology, Taiwan
Zitieren
APA
Shen, Y., Hsieh, P., & Chi, T. (2025). Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition. IEEE Transactions on Affective Computing, 16(3), 1693-1704. https://doi.org/10.1109/taffc.2025.3531638
Vancouver
Shen Y, Hsieh P, Chi T. Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition. IEEE Transactions on Affective Computing. 2025;16(3):1693-1704. doi:10.1109/taffc.2025.3531638
BibTeX
@article{Shen2025,
author = {Shen, Yih-Liang and Hsieh, Pei-Chin and Chi, Tai-Shih},
title = {Spectro-Temporal Modulations Incorporated Two-Stream Robust Speech Emotion Recognition},
journal = {IEEE Transactions on Affective Computing},
year = {2025},
volume = {16},
number = {3},
pages = {1693--1704},
doi = {10.1109/taffc.2025.3531638}
}
EUVIMED