EUVIMEDEUVIMED entdecken
Crossref

Multimodal Discriminative Network for Emotion Recognition Across Individuals

Minxu Liu, Donghai Guan, Chuhang Zheng, Qi Zhu

IEEE Transactions on Cognitive and Developmental Systems · 2025-10 · 17 (5) · 1323-1335

DOI: 10.1109/tcds.2025.3552124 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Cognitive and Developmental Systems, 2025-10. Verfasst von Minxu Liu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 52 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Jiangsu Province, Jiangsu Province 100 Foreign Experts Introduction Plan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal emotion recognition is gaining significant attention for ability to fuse complementary information from diverse physiological and behavioral signals, which benefits the understanding of emotional disorders. However, challenges arise in multimodal fusion due to uncertainties inherent in different modalities, such as complex signal coupling and modality heterogeneity. Furthermore, the feature distribution drift in intersubject emotion recognition hinders the generalization ability of the method and significantly degrades performance on new individuals. To address the above issues, we propose a cross-subject multimodal emotion robust recognition framework that effectively extracts subject-independent intrinsic emotional identification information from heterogeneous multimodal emotion data. First, we develop a multichannel network with self-attention and cross-attention mechanisms to capture modality-specific and complementary features among different modalities, respectively. Second, we incorporate contrastive loss into the multichannel attention network to enhance feature extraction across different channels, thereby facilitating the disentanglement of emotion-specific information. Moreover, a self-expression learning-based network layer is devised to enhance feature discriminability and subject alignment. It aligns samples in a discriminative space using block diagonal matrices and maps multiple individuals to a shared subspace using a block off-diagonal matrix. Finally, attention is used to merge multichannel features, and multilayer perceptron is employed for classification. Experimental results on multimodal emotion datasets confirm that our proposed approach surpasses the current state-of-the-art in terms of emotion recognition accuracy, with particularly significant gains observed in the challenging cross-subject multimodal recognition scenarios.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation of Jiangsu Province
  • Jiangsu Province 100 Foreign Experts Introduction Plan

Zitieren

APA

Liu, M., Guan, D., Zheng, C., & Zhu, Q. (2025). Multimodal Discriminative Network for Emotion Recognition Across Individuals. IEEE Transactions on Cognitive and Developmental Systems, 17(5), 1323-1335. https://doi.org/10.1109/tcds.2025.3552124

Vancouver

Liu M, Guan D, Zheng C, Zhu Q. Multimodal Discriminative Network for Emotion Recognition Across Individuals. IEEE Transactions on Cognitive and Developmental Systems. 2025;17(5):1323-1335. doi:10.1109/tcds.2025.3552124

BibTeX
@article{Liu2025,
  author = {Liu, Minxu and Guan, Donghai and Zheng, Chuhang and Zhu, Qi},
  title = {Multimodal Discriminative Network for Emotion Recognition Across Individuals},
  journal = {IEEE Transactions on Cognitive and Developmental Systems},
  year = {2025},
  volume = {17},
  number = {5},
  pages = {1323--1335},
  doi = {10.1109/tcds.2025.3552124}
}
Kostenlos starten →