EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MAGTF-Net: Dynamic Speech Emotion Recognition with Multi-Scale Graph Attention and LLD Feature Fusion

Shiyin Zhu iD, Yinggang Xie iD, Zhiliang Wang

Sensors · 04.12.2025 · 25 (23) · 7378

DOI: 10.3390/s25237378 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 04.12.2025. Verfasst von Shiyin Zhu und 2 weiteren Autor:innen. Beteiligte Einrichtung: Key Laboratory of Information and Communication Systems, Ministry of Information Industry, Beijing Information Science and Technology University, Bei…. Hauptthemen laut MEDLINE: Emotions, Speech. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Beijing Natural Science Foundation, Gold-Bridge Funds for Beijing.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this paper, we propose a novel speech emotion recognition model, named MAGTF-Net (Multi-scale Attention Graph Transformer Fusion Network), which addresses the challenges faced by traditional hand-crafted feature-based approaches in modeling complex emotional nuances and dynamic contextual dependencies. Although existing state-of-the-art methods have achieved improvements in recognition performance, they often fail to simultaneously capture both local acoustic features and global temporal structures, and they lack adaptability to variable-length speech utterances, thereby limiting their accuracy and robustness in recognizing complex emotional expressions. To tackle these challenges, we design a log-Mel spectrogram feature extraction branch that combines a Multi-scale Attention Graph (MAG) structure with a Transformer encoder, where the Transformer module adaptively performs dynamic modeling of speech sequences with varying lengths. In addition, a low-level descriptor (LLD) feature branch is introduced, where a multilayer perceptron (MLP) is employed for complementary feature modeling. The two feature branches are fused and subsequently classified through a fully connected layer, further enhancing the expressive capability of emotional representations. Moreover, a label-smoothing-enhanced cross-entropy loss function is adopted to improve the model’s recognition performance on difficult-to-classify emotional categories. Experiments conducted on the IEMOCAP dataset demonstrate that MAGTF-Net achieves weighted accuracy (WA) and unweighted accuracy (UA) scores of 69.15% and 70.86%, respectively, outperforming several baseline models. Further ablation studies validate the significant contributions of each module in the Mel-spectrogram branch and the LLD feature branch to the overall performance improvement. The proposed method effectively integrates local, global, and multi-source feature information, significantly enhancing the recognition of complex emotional expressions and providing new theoretical and practical insights for the field of speech emotion recognition.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

EmotionsSpeechHumansAlgorithmsNeural Networks, Computer

Stichwörter der Autor:innen

feature fusiongraph neural networksmultimodal learningspeech emotion recognitiontransformer encoder

Autor:innen und Institutionen

  • Key Laboratory of Information and Communication Systems, Ministry of Information Industry, Beijing Information Science and Technology University, Beijing 100192, China.
  • School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing 100083, China.

ORCID-Profile: Shiyin Zhu, Yinggang Xie

Förderung

  • Beijing Natural Science Foundation
  • Gold-Bridge Funds for Beijing

Zitieren

APA

Zhu, S., Xie, Y., & Wang, Z. (2025). MAGTF-Net: Dynamic Speech Emotion Recognition with Multi-Scale Graph Attention and LLD Feature Fusion. Sensors, 25(23), 7378. https://doi.org/10.3390/s25237378

Vancouver

Zhu S, Xie Y, Wang Z. MAGTF-Net: Dynamic Speech Emotion Recognition with Multi-Scale Graph Attention and LLD Feature Fusion. Sensors. 2025;25(23):7378. doi:10.3390/s25237378

BibTeX
@article{Zhu2025,
  author = {Zhu, Shiyin and Xie, Yinggang and Wang, Zhiliang},
  title = {MAGTF-Net: Dynamic Speech Emotion Recognition with Multi-Scale Graph Attention and LLD Feature Fusion},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {23},
  pages = {7378},
  doi = {10.3390/s25237378}
}
Kostenlos starten →