Enhancing Emotion Recognition in Speech Based on Self-Supervised Learning: Cross-Attention Fusion of Acoustic and Semantic Features
IEEE Access · 2025 · 13 · 56283-56295
DOI: 10.1109/access.2025.3554454 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2025. Verfasst von Bashar M. Deeb und 2 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Center for Trusted Artificial Intelligence, Ivannikov Institute for System Programming, Russian Academy of Sciences.
Abstract (Originalsprache)
Speech Emotion Recognition has gained considerable attention in speech processing and machine learning due to its potential applications in human-computer interaction, mental health monitoring, and customer service. However, state-of-the-art models for speech emotion recognition use many parameters, which leads to computational complexity. In this paper, we introduce a novel deep-learning model to enhance the accuracy of emotional content detection in speech signals while maintaining a lightweight architecture compared to state-of-the-art models. The proposed model incorporates a feature encoder that significantly improves the emotional representation of acoustic features and a cross-attention mechanism to fuse acoustic features, such as Spectrograms, with semantic features extracted from the pre-trained self-supervised learning framework, enriching the emotional representation of speech. An extensive experimental study demonstrates that the proposed model achieves a weighted accuracy of 74.6% on the IEMOCAP dataset, competitive with the state-of-the-art baselines. In addition, our proposed model achieves a latency of 24 milliseconds on moderate devices while containing up to three times fewer parameters.
Förderung
- Research Center for Trusted Artificial Intelligence, Ivannikov Institute for System Programming, Russian Academy of Sciences
Zitieren
APA
Deeb, B. M., Savchenko, A. V., & Makarov, I. (2025). Enhancing Emotion Recognition in Speech Based on Self-Supervised Learning: Cross-Attention Fusion of Acoustic and Semantic Features. IEEE Access, 13, 56283-56295. https://doi.org/10.1109/access.2025.3554454
Vancouver
Deeb BM, Savchenko AV, Makarov I. Enhancing Emotion Recognition in Speech Based on Self-Supervised Learning: Cross-Attention Fusion of Acoustic and Semantic Features. IEEE Access. 2025;13:56283-56295. doi:10.1109/access.2025.3554454
BibTeX
@article{Deeb2025,
author = {Deeb, Bashar M. and Savchenko, Andrey V. and Makarov, Ilya},
title = {Enhancing Emotion Recognition in Speech Based on Self-Supervised Learning: Cross-Attention Fusion of Acoustic and Semantic Features},
journal = {IEEE Access},
year = {2025},
volume = {13},
pages = {56283--56295},
doi = {10.1109/access.2025.3554454}
}
EUVIMED