Multimodal Knowledge Distillation for Emotion Recognition
Brain Sciences · 30.06.2025 · 15 (7) · 707
DOI: 10.3390/brainsci15070707 ↗Kurz gefasst
Fachartikel in der Zeitschrift Brain Sciences, online erschienen am 30.06.2025. Verfasst von Zhenxuan Zhang und 1 weiteren Autor:innen. Beteiligte Einrichtung: State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Multimodal emotion recognition has emerged as a prominent field in affective computing, offering superior performance compared to single-modality methods. Among various physiological signals, EEG signals and EOG data are highly valued for their complementary strengths in emotion recognition. However, the practical application of EEG-based approaches is often hindered by high costs and operational complexity, making EOG a more feasible alternative in real-world scenarios. To address this limitation, this study introduces a novel framework for multimodal knowledge distillation, designed to improve the practicality of emotion decoding while maintaining high accuracy, with the framework including a multimodal fusion module to extract and integrate interactive and heterogeneous features, and a unimodal student model structurally aligned with the multimodal teacher model for better knowledge alignment. The framework combines EEG and EOG signals into a unified model and distills the fused multimodal features into a simplified EOG-only model. To facilitate efficient knowledge transfer, the approach incorporates a dynamic feedback mechanism that adjusts the guidance provided by the multimodal model to the unimodal model during the distillation process based on performance metrics. The proposed method was comprehensively evaluated on two datasets based on EEG and EOG signals. The accuracy of the valence and arousal of the proposed model in the DEAP dataset are 70.38% and 60.41%, respectively. The accuracy of valence and arousal in the BJTU-Emotion dataset are 61.31% and 60.31%, respectively. The proposed method achieves state-of-the-art classification performance compared to the baseline method, with statistically significant improvements confirmed by paired t-tests (p < 0.05), and the framework effectively transfers knowledge from multimodal models to unimodal EOG models, enhancing the practicality of emotion recognition while maintaining high accuracy, thus expanding the applicability of emotion recognition in real-world scenarios.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China.
- School of Cyberspace Security, Beijing Jiaotong University, Beijing 100044, China.
ORCID-Profile: Guanyu Lu
Zitieren
APA
Zhang, Z., & Lu, G. (2025). Multimodal Knowledge Distillation for Emotion Recognition. Brain Sciences, 15(7), 707. https://doi.org/10.3390/brainsci15070707
Vancouver
Zhang Z, Lu G. Multimodal Knowledge Distillation for Emotion Recognition. Brain Sciences. 2025;15(7):707. doi:10.3390/brainsci15070707
BibTeX
@article{Zhang2025,
author = {Zhang, Zhenxuan and Lu, Guanyu},
title = {Multimodal Knowledge Distillation for Emotion Recognition},
journal = {Brain Sciences},
year = {2025},
volume = {15},
number = {7},
pages = {707},
doi = {10.3390/brainsci15070707}
}
EUVIMED