EUVIMEDEUVIMED entdecken
Crossref

SeeNet: A Soft Emotion Expert and Data Augmentation Method to Enhance Speech Emotion Recognition

Qifei Li, Yingming Gao, Yuhua Wen, Ziping Zhao, Ya Li, Bjorn W. Schuller

IEEE Transactions on Affective Computing · 2025-07 · 16 (3) · 2142-2156

DOI: 10.1109/taffc.2025.3555406 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025-07. Verfasst von Qifei Li und 5 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 86 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Key Project of the National Language Commission, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Speech emotion recognition (SER) systems are designed to enable machines to recognize emotional states in human speech during human-computer interactions, enhancing the interactive experience. While considerable progress has been achieved in this field recently, SER systems still encounter challenges related to performance and robustness, primarily stemming from the limited labeled data. To this end, we propose a novel multitask learning framework to learn a distinctive and robust emotional representation by our “Soft Emotion Expert Network (SeeNet)”. SeeNet consists of three components: a pretrained model, an auxiliary task soft emotion expert (SEE) module and an energy-based mixup (EBM) data augmentation module. The pretrained model and EBM module are employed to mitigate the challenges arising from limited labeled data, thereby enhancing the model performance and bolstering robustness. The SEE module as an auxiliary task is designed to assist the main task of SER by enhancing the distinction between samples exhibiting high similarity across categories. This aims to further improve the performance and robustness of the system. Comprehensive experiments on three different settings and multiple datasets are conducted to evaluate the performance and robustness of our proposed method. The experimental results demonstrate that SeeNet surpasses the state-of-the-art (SOTA) methods in both performance and robustness.

Förderung

  • National Natural Science Foundation of China
  • Key Project of the National Language Commission
  • Fundamental Research Funds for the Central Universities
  • BUPT Excellent Ph.D. Students Foundation
  • National Social Science Fund of China

Zitieren

APA

Li, Q., Gao, Y., Wen, Y., Zhao, Z., Li, Y., & Schuller, B. W. (2025). SeeNet: A Soft Emotion Expert and Data Augmentation Method to Enhance Speech Emotion Recognition. IEEE Transactions on Affective Computing, 16(3), 2142-2156. https://doi.org/10.1109/taffc.2025.3555406

Vancouver

Li Q, Gao Y, Wen Y, Zhao Z, Li Y, Schuller BW. SeeNet: A Soft Emotion Expert and Data Augmentation Method to Enhance Speech Emotion Recognition. IEEE Transactions on Affective Computing. 2025;16(3):2142-2156. doi:10.1109/taffc.2025.3555406

BibTeX
@article{Li2025,
  author = {Li, Qifei and Gao, Yingming and Wen, Yuhua and Zhao, Ziping and Li, Ya and Schuller, Bjorn W.},
  title = {SeeNet: A Soft Emotion Expert and Data Augmentation Method to Enhance Speech Emotion Recognition},
  journal = {IEEE Transactions on Affective Computing},
  year = {2025},
  volume = {16},
  number = {3},
  pages = {2142--2156},
  doi = {10.1109/taffc.2025.3555406}
}
Kostenlos starten →