EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Cross-Modal Attention Fusion: A Deep Learning and Affective Computing Model for Emotion Recognition

Himanshu Kumar, Martin Aruldoss, Martin Wynn

Multimodal Technologies and Interaction · 24.11.2025 · 9 (12) · 116

DOI: 10.3390/mti9120116 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Multimodal Technologies and Interaction, online erschienen am 24.11.2025. Verfasst von Himanshu Kumar und 2 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 88 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Artificial emotional intelligence is a sub-domain of human–computer interaction research that aims to develop deep learning models capable of detecting and interpreting human emotional states through various modalities. A major challenge in this domain is identifying meaningful correlations between heterogeneous modalities—for example, between audio and visual data—due to their distinct temporal and spatial properties. Traditional fusion techniques used in multimodal learning to combine data from different sources often fail to adequately capture meaningful and less computational cross-modal interactions, and struggle to adapt to varying modality reliability. Following a review of the relevant literature, this study adopts an experimental research method to develop and evaluate a mathematical cross-modal fusion model, thereby addressing a gap in the extant research literature. The framework uses the Tucker tensor decomposition to analyse the multi-dimensional array of data into a set of matrices to support the integration of temporal features from audio and spatiotemporal features from visual modalities. A cross-attention mechanism is incorporated to enhance cross-modal interaction, enabling each modality to attend to the relevant information from the other. The efficacy of the model is rigorously evaluated on three publicly available datasets and the results conclusively demonstrate that the proposed fusion technique outperforms conventional fusion methods and several more recent approaches. The findings break new ground in this field of study and will be of interest to researchers and developers in artificial emotional intelligence.

Zitieren

APA

Kumar, H., Aruldoss, M., & Wynn, M. (2025). Cross-Modal Attention Fusion: A Deep Learning and Affective Computing Model for Emotion Recognition. Multimodal Technologies and Interaction, 9(12), 116. https://doi.org/10.3390/mti9120116

Vancouver

Kumar H, Aruldoss M, Wynn M. Cross-Modal Attention Fusion: A Deep Learning and Affective Computing Model for Emotion Recognition. Multimodal Technologies and Interaction. 2025;9(12):116. doi:10.3390/mti9120116

BibTeX
@article{Kumar2025,
  author = {Kumar, Himanshu and Aruldoss, Martin and Wynn, Martin},
  title = {Cross-Modal Attention Fusion: A Deep Learning and Affective Computing Model for Emotion Recognition},
  journal = {Multimodal Technologies and Interaction},
  year = {2025},
  volume = {9},
  number = {12},
  pages = {116},
  doi = {10.3390/mti9120116}
}
Kostenlos starten →