EUVIMEDEUVIMED entdecken
Crossref

Cross-Attention Audio–Visual Fusion Based on Multi-Scale Vision Transformers for Emotion Recognition

Chengao Bao, Luefeng Chen, Min Li, Min Wu, Witold Pedrycz, Kaoru Hirota

Journal of Advanced Computational Intelligence and Intelligent Informatics · 20.05.2026 · 30 (3) · 749-760

DOI: 10.20965/jaciii.2026.p0749 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Advanced Computational Intelligence and Intelligent Informatics, online erschienen am 20.05.2026. Verfasst von Chengao Bao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 43 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A multimodal emotion recognition method based on a multi-scale vision Transformer and cross attention mechanism (MSFCA) is proposed. The proposed method integrates a multi-scale vision Transformer and a cross-attention mechanism to fully exploit the complementary information between facial expressions and speech modalities, thereby enabling more effective feature extraction and fusion. By incorporating L1 regularization, the sparsity and robustness of the fused features are enhanced, thereby improving the accuracy and generalization capability of multi-modal emotion recognition. Experimental results on the eNTERFACE’05 and RAVDESS multi-modal emotion recognition datasets demonstrate that the proposed MSFCA method achieves recognition accuracies of 86.16% and 87.14%, respectively, which satisfy the requirements for reliable multi-modal emotion recognition in practical applications.

Zitieren

APA

Bao, C., Chen, L., Li, M., Wu, M., Pedrycz, W., & Hirota, K. (2026). Cross-Attention Audio–Visual Fusion Based on Multi-Scale Vision Transformers for Emotion Recognition. Journal of Advanced Computational Intelligence and Intelligent Informatics, 30(3), 749-760. https://doi.org/10.20965/jaciii.2026.p0749

Vancouver

Bao C, Chen L, Li M, Wu M, Pedrycz W, Hirota K. Cross-Attention Audio–Visual Fusion Based on Multi-Scale Vision Transformers for Emotion Recognition. Journal of Advanced Computational Intelligence and Intelligent Informatics. 2026;30(3):749-760. doi:10.20965/jaciii.2026.p0749

BibTeX
@article{Bao2026,
  author = {Bao, Chengao and Chen, Luefeng and Li, Min and Wu, Min and Pedrycz, Witold and Hirota, Kaoru},
  title = {Cross-Attention Audio–Visual Fusion Based on Multi-Scale Vision Transformers for Emotion Recognition},
  journal = {Journal of Advanced Computational Intelligence and Intelligent Informatics},
  year = {2026},
  volume = {30},
  number = {3},
  pages = {749--760},
  doi = {10.20965/jaciii.2026.p0749}
}
Kostenlos starten →