Cross-Attention Audio–Visual Fusion Based on Multi-Scale Vision Transformers for Emotion Recognition
Journal of Advanced Computational Intelligence and Intelligent Informatics · 20.05.2026 · 30 (3) · 749-760
DOI: 10.20965/jaciii.2026.p0749 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Advanced Computational Intelligence and Intelligent Informatics, online erschienen am 20.05.2026. Verfasst von Chengao Bao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 43 Literaturangaben.
Abstract (Originalsprache)
A multimodal emotion recognition method based on a multi-scale vision Transformer and cross attention mechanism (MSFCA) is proposed. The proposed method integrates a multi-scale vision Transformer and a cross-attention mechanism to fully exploit the complementary information between facial expressions and speech modalities, thereby enabling more effective feature extraction and fusion. By incorporating L1 regularization, the sparsity and robustness of the fused features are enhanced, thereby improving the accuracy and generalization capability of multi-modal emotion recognition. Experimental results on the eNTERFACE’05 and RAVDESS multi-modal emotion recognition datasets demonstrate that the proposed MSFCA method achieves recognition accuracies of 86.16% and 87.14%, respectively, which satisfy the requirements for reliable multi-modal emotion recognition in practical applications.
Zitieren
APA
Bao, C., Chen, L., Li, M., Wu, M., Pedrycz, W., & Hirota, K. (2026). Cross-Attention Audio–Visual Fusion Based on Multi-Scale Vision Transformers for Emotion Recognition. Journal of Advanced Computational Intelligence and Intelligent Informatics, 30(3), 749-760. https://doi.org/10.20965/jaciii.2026.p0749
Vancouver
Bao C, Chen L, Li M, Wu M, Pedrycz W, Hirota K. Cross-Attention Audio–Visual Fusion Based on Multi-Scale Vision Transformers for Emotion Recognition. Journal of Advanced Computational Intelligence and Intelligent Informatics. 2026;30(3):749-760. doi:10.20965/jaciii.2026.p0749
BibTeX
@article{Bao2026,
author = {Bao, Chengao and Chen, Luefeng and Li, Min and Wu, Min and Pedrycz, Witold and Hirota, Kaoru},
title = {Cross-Attention Audio–Visual Fusion Based on Multi-Scale Vision Transformers for Emotion Recognition},
journal = {Journal of Advanced Computational Intelligence and Intelligent Informatics},
year = {2026},
volume = {30},
number = {3},
pages = {749--760},
doi = {10.20965/jaciii.2026.p0749}
}
EUVIMED