EUVIMEDEUVIMED entdecken
Crossref

Cross-Modal Knowledge Distillation for Enhanced Unimodal Emotion Recognition

Ziyu Jia, Yucheng Liu, Haichao Wang, Tianzi Jiang

IEEE Transactions on Affective Computing · 2025 · 1-13

DOI: 10.1109/taffc.2025.3583594 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025. Verfasst von Ziyu Jia und 3 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal physiological signal-based emotion recognition technology represents an emerging branch in the field of affective computing. By combining various physiological signals, its performance significantly surpasses traditional unimodal methods. Notably, the joint application of electroencephalography (EEG) and galvanic skin response (GSR) show high effectiveness in emotion recognition tasks. However, the difficulty and high cost of EEG signal acquisition limit its widespread use in practical applications. To address this challenge, we propose a Cross-Modal Emotion Knowledge Distillation (CMEKD) framework. This framework not only extracts the heterogeneity and interactivity between GSR and EEG signals but also conveys a comprehensive fusion of multimodal features into an unimodal GSR model through knowledge distillation techniques, thereby enhancing unimodal model performance. We employ cosine similarity-based knowledge representation to reduce the gap between the multimodal model and the unimodal model. Additionally, an adaptive feedback mechanism is introduced to dynamically adjust the distillation process based on the performance of the unimodal model, further improving the classification performance of emotion recognition. Experiment results demonstrate that this framework achieves state-of-the-art performance on two public datasets. By reducing reliance on multimodal data in the application of the emotion recognition technique, we significantly improve the practicality and feasibility of emotion recognition technology. The code for this paper will be made publicly available on GitHub:https://anonymous.4open.science/r/CMEKD-2440/.

Zitieren

APA

Jia, Z., Liu, Y., Wang, H., & Jiang, T. (2025). Cross-Modal Knowledge Distillation for Enhanced Unimodal Emotion Recognition. IEEE Transactions on Affective Computing, 1-13. https://doi.org/10.1109/taffc.2025.3583594

Vancouver

Jia Z, Liu Y, Wang H, Jiang T. Cross-Modal Knowledge Distillation for Enhanced Unimodal Emotion Recognition. IEEE Transactions on Affective Computing. 2025:1-13. doi:10.1109/taffc.2025.3583594

BibTeX
@article{Jia2025,
  author = {Jia, Ziyu and Liu, Yucheng and Wang, Haichao and Jiang, Tianzi},
  title = {Cross-Modal Knowledge Distillation for Enhanced Unimodal Emotion Recognition},
  journal = {IEEE Transactions on Affective Computing},
  year = {2025},
  pages = {1--13},
  doi = {10.1109/taffc.2025.3583594}
}
Kostenlos starten →