EUVIMEDEUVIMED entdecken
Crossref

MCRVT: Multi-Hierarchical Cross-Reconstruction Networks With Versatile Transformer for Speech Emotion Recognition

Xin-Heng Li, Zhen-Tao Liu, Yu-Jie Zou, Jinhua She, Kaoru Hirota

IEEE Transactions on Affective Computing · 2025-07 · 16 (3) · 2189-2199

DOI: 10.1109/taffc.2025.3557153 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025-07. Verfasst von Xin-Heng Li und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 35 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Hubei Provincial Natural Science Foundation of China, Wuhan Municipal Natural Science Foundation Exploratory Program Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Two challenges for Speech Emotion Recognition (SER) are efficiently capturing features to explore speech-emotion correlations and reducing the SER model redundancy while increasing performance. To address these challenges, we propose Multi-Hierarchical Cross-Reconstruction Networks with Versatile Transformer (MCRVT), including a Spectrogram-Based SER Model with Versatile Attention (SSER-VA), a pre-trained model (WavLM), and Multi-Hierarchical Feature Fusion with Cross Attention (MHFF-CA). Specifically, SSER-VA takes the log-Mel spectrogram as the input, and we propose a versatile attention module to reduce redundancy and increase efficiency. MHFF-CA is employed to refine intermediate features from WavLM and SSER-VA to improve competitiveness in various tasks, where Contrastive Reconstruction Networks (CRN) and Cross Attention-based Feature Fusion (CAF) are utilized to reconstruct and fuse features, respectively. Effectiveness experiments on IEMOCAP and MELD datasets illustrate MCRVT's superior performance compared to SOTA methods. Exploratory experiments on D-vlog, BioCAS2024, and IEMOCAP (speaker-independent setting) datasets demonstrate the competitiveness of our method in various speech classification tasks.

Förderung

  • National Natural Science Foundation of China
  • Hubei Provincial Natural Science Foundation of China
  • Wuhan Municipal Natural Science Foundation Exploratory Program Project
  • Wuhan Science and Technology Project
  • Fundamental Research Funds for National University, China University of Geosciences, Wuhan

Zitieren

APA

Li, X., Liu, Z., Zou, Y., She, J., & Hirota, K. (2025). MCRVT: Multi-Hierarchical Cross-Reconstruction Networks With Versatile Transformer for Speech Emotion Recognition. IEEE Transactions on Affective Computing, 16(3), 2189-2199. https://doi.org/10.1109/taffc.2025.3557153

Vancouver

Li X, Liu Z, Zou Y, She J, Hirota K. MCRVT: Multi-Hierarchical Cross-Reconstruction Networks With Versatile Transformer for Speech Emotion Recognition. IEEE Transactions on Affective Computing. 2025;16(3):2189-2199. doi:10.1109/taffc.2025.3557153

BibTeX
@article{Li2025,
  author = {Li, Xin-Heng and Liu, Zhen-Tao and Zou, Yu-Jie and She, Jinhua and Hirota, Kaoru},
  title = {MCRVT: Multi-Hierarchical Cross-Reconstruction Networks With Versatile Transformer for Speech Emotion Recognition},
  journal = {IEEE Transactions on Affective Computing},
  year = {2025},
  volume = {16},
  number = {3},
  pages = {2189--2199},
  doi = {10.1109/taffc.2025.3557153}
}
Kostenlos starten →