MCRVT: Multi-Hierarchical Cross-Reconstruction Networks With Versatile Transformer for Speech Emotion Recognition
IEEE Transactions on Affective Computing · 2025-07 · 16 (3) · 2189-2199
DOI: 10.1109/taffc.2025.3557153 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025-07. Verfasst von Xin-Heng Li und 4 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 35 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Hubei Provincial Natural Science Foundation of China, Wuhan Municipal Natural Science Foundation Exploratory Program Project.
Abstract (Originalsprache)
Two challenges for Speech Emotion Recognition (SER) are efficiently capturing features to explore speech-emotion correlations and reducing the SER model redundancy while increasing performance. To address these challenges, we propose Multi-Hierarchical Cross-Reconstruction Networks with Versatile Transformer (MCRVT), including a Spectrogram-Based SER Model with Versatile Attention (SSER-VA), a pre-trained model (WavLM), and Multi-Hierarchical Feature Fusion with Cross Attention (MHFF-CA). Specifically, SSER-VA takes the log-Mel spectrogram as the input, and we propose a versatile attention module to reduce redundancy and increase efficiency. MHFF-CA is employed to refine intermediate features from WavLM and SSER-VA to improve competitiveness in various tasks, where Contrastive Reconstruction Networks (CRN) and Cross Attention-based Feature Fusion (CAF) are utilized to reconstruct and fuse features, respectively. Effectiveness experiments on IEMOCAP and MELD datasets illustrate MCRVT's superior performance compared to SOTA methods. Exploratory experiments on D-vlog, BioCAS2024, and IEMOCAP (speaker-independent setting) datasets demonstrate the competitiveness of our method in various speech classification tasks.
Förderung
- National Natural Science Foundation of China
- Hubei Provincial Natural Science Foundation of China
- Wuhan Municipal Natural Science Foundation Exploratory Program Project
- Wuhan Science and Technology Project
- Fundamental Research Funds for National University, China University of Geosciences, Wuhan
Zitieren
APA
Li, X., Liu, Z., Zou, Y., She, J., & Hirota, K. (2025). MCRVT: Multi-Hierarchical Cross-Reconstruction Networks With Versatile Transformer for Speech Emotion Recognition. IEEE Transactions on Affective Computing, 16(3), 2189-2199. https://doi.org/10.1109/taffc.2025.3557153
Vancouver
Li X, Liu Z, Zou Y, She J, Hirota K. MCRVT: Multi-Hierarchical Cross-Reconstruction Networks With Versatile Transformer for Speech Emotion Recognition. IEEE Transactions on Affective Computing. 2025;16(3):2189-2199. doi:10.1109/taffc.2025.3557153
BibTeX
@article{Li2025,
author = {Li, Xin-Heng and Liu, Zhen-Tao and Zou, Yu-Jie and She, Jinhua and Hirota, Kaoru},
title = {MCRVT: Multi-Hierarchical Cross-Reconstruction Networks With Versatile Transformer for Speech Emotion Recognition},
journal = {IEEE Transactions on Affective Computing},
year = {2025},
volume = {16},
number = {3},
pages = {2189--2199},
doi = {10.1109/taffc.2025.3557153}
}
EUVIMED