Individual-Aware Attention Modulation for Unseen Speaker Emotion Recognition
IEEE Transactions on Affective Computing · 2025-04 · 16 (2) · 1205-1218
DOI: 10.1109/taffc.2024.3498937 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025-04. Verfasst von Yuanbo Fang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 56 Literaturangaben. Gefördert von: Natural Science Foundation of Guangdong Province, Key-Area Research and Development Program of Guangdong Province, Guangdong Provincial Key Laboratory of Human Digital Twin.
Abstract (Originalsprache)
In practical human-computer interaction (HCI) applications, robust speech emotion recognition (SER) for unseen speakers is crucial. Prior research has primarily focused on extracting common representations to enhance the generalization of cross-individual SER. However, most methods ignore the positive effects of individual characteristics. Actually, each speaker can be regarded as an independent individual domain. Personalized SER can be improved if the emotional expressions of individual speech characteristics are effectively utilized. To address the challenges in recognizing emotions for unseen speakers, this paper proposes a novel individual-aware attention modulation (IAM) model. Specifically, the IAM uses meta-learning techniques to extract modulation parameters for obtaining individual-related emotion expressions from individual characteristics. The base model is then modulated to facilitate the transfer of the common emotion representation space to an individual-specific emotion representation space. This transformation is achieved by applying attention modulation within the transformer-based model developed in this paper. In addition, we employ a meta-learning-based method to optimize model parameters, enhancing the adaptability of the model to unseen speakers, and a control factor is introduced to regulate the degree of individual modulation, thus enhancing the robustness of the modulation process. Experimental results demonstrate that the proposed model achieves significantly improved cross-individual SER performance.
Förderung
- Natural Science Foundation of Guangdong Province
- Key-Area Research and Development Program of Guangdong Province
- Guangdong Provincial Key Laboratory of Human Digital Twin
- Cognitive Radio and Information Processing Key Laboratory Authorized by China's Ministry of Education Foundation
Zitieren
APA
Fang, Y., Xing, X., Chu, Z., Du, Y., & Xu, X. (2025). Individual-Aware Attention Modulation for Unseen Speaker Emotion Recognition. IEEE Transactions on Affective Computing, 16(2), 1205-1218. https://doi.org/10.1109/taffc.2024.3498937
Vancouver
Fang Y, Xing X, Chu Z, Du Y, Xu X. Individual-Aware Attention Modulation for Unseen Speaker Emotion Recognition. IEEE Transactions on Affective Computing. 2025;16(2):1205-1218. doi:10.1109/taffc.2024.3498937
BibTeX
@article{Fang2025,
author = {Fang, Yuanbo and Xing, Xiaofen and Chu, Zhaojie and Du, Yifeng and Xu, Xiangmin},
title = {Individual-Aware Attention Modulation for Unseen Speaker Emotion Recognition},
journal = {IEEE Transactions on Affective Computing},
year = {2025},
volume = {16},
number = {2},
pages = {1205--1218},
doi = {10.1109/taffc.2024.3498937}
}
EUVIMED