EUVIMEDEUVIMED entdecken
Crossref

MS²FL: Modality-shared and Modality-specific Feature Learning for Multimodal Emotion Recognition

Xinhui Li, Hao Chen, Minchao Wu, Qingwei Song, Fan Li, Jinyang Huang, Feng-Qi Cui, Zhao Lv

ACM Transactions on Autonomous and Adaptive Systems · 19.08.2026

DOI: 10.1145/3836771 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Autonomous and Adaptive Systems, online erschienen am 19.08.2026. Verfasst von Xinhui Li und 7 weiteren Autor:innen. Die Arbeit nennt 59 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal emotion recognition based on complementary physiological signals such as electroencephalogram (EEG) and eye movements can effectively reflect human emotional states, demonstrating significant potential in fields such as rehabilitation monitoring and driving safety. However, existing multimodal emotion recognition approaches often focus solely on either feature fusion or feature alignment strategies, lacking a unified framework capturing both complementary and modality-specific representations. To address this limitation, we propose a modality-shared and modality-specific feature learning framework (MS \({}^{2}\) FL) for multimodal emotion recognition. Specifically, a dual-path encoder is employed first to extract the spatiotemporal frequency features of EEG signals and dynamic response features of eye movement signals, respectively. Then, a specificity-preserving feature distribution alignment mechanism is introduced to alleviate distribution discrepancies between modalities while reinforcing modality-specific modeling of distinctive features. Finally, a feature distribution enhancement fusion strategy is utilized to effectively integrate the shared features across modalities, thus improving the representational capacity of the fused features. Experimental results demonstrate that MS \({}^{2}\) FL achieves recognition accuracies of 96.99% and 89.83% on the SEED and SEED-V datasets, respectively, significantly improving emotion recognition performance. This study provides a concise and effective solution for emotion perception neurotechnology and offers a practical solution for emotion perception health monitoring and rehabilitation.

Zitieren

APA

Li, X., Chen, H., Wu, M., Song, Q., Li, F., Huang, J., Cui, F., & Lv, Z. (2026). MS²FL: Modality-shared and Modality-specific Feature Learning for Multimodal Emotion Recognition. ACM Transactions on Autonomous and Adaptive Systems. https://doi.org/10.1145/3836771

Vancouver

Li X, Chen H, Wu M, Song Q, Li F, Huang J, et al. MS²FL: Modality-shared and Modality-specific Feature Learning for Multimodal Emotion Recognition. ACM Transactions on Autonomous and Adaptive Systems. 2026. doi:10.1145/3836771

BibTeX
@article{Li2026,
  author = {Li, Xinhui and Chen, Hao and Wu, Minchao and Song, Qingwei and Li, Fan and Huang, Jinyang and Cui, Feng-Qi and Lv, Zhao},
  title = {MS²FL: Modality-shared and Modality-specific Feature Learning for Multimodal Emotion Recognition},
  journal = {ACM Transactions on Autonomous and Adaptive Systems},
  year = {2026},
  doi = {10.1145/3836771}
}
Kostenlos starten →