EUVIMEDEUVIMED entdecken
Crossref

Phy-FusionNet: A Memory-Augmented Transformer for Multimodal Emotion Recognition With Periodicity and Contextual Attention

Tianyi Wu, Erick Purwanto, Yongrun Huang, Yang Su

IEEE Transactions on Affective Computing · 2026-01 · 17 (1) · 235-246

DOI: 10.1109/taffc.2025.3609046 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2026-01. Verfasst von Tianyi Wu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 58 Literaturangaben. Gefördert von: Xi’an Jiaotong-Liverpool University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate emotion recognition from physiological signals is critical for applications in healthcare, autonomous systems, and human-computer interaction. However, prevailing methods often fail to model long-term dependencies and overlook periodic patterns inherent in physiological data. To address these challenges, we propose Phy-FusionNet, a novel memory-augmented transformer architecture for multimodal emotion recognition. Phy-FusionNet introduces a Memory Stream Module with FIFO-queue and decay-based updates to preserve long-term contextual information. It further integrates Fourier-based positional encoding and frequency-aware attention, enabling robust detection of periodic emotional cues. An Adaptive Temporal Attention Module enhances computational efficiency and enables dynamic relevance in temporal feature extraction. For cross-modal fusion, we employ a transformer-based Multimodal Binding Learning framework that balances modality-specific and shared features. Extensive experiments on five public datasets—WESAD, CL-Drive, PPB-Emo, PhyMER, and EEG-VUI—demonstrate that Phy-FusionNet outperforms state-of-the-art models, achieving up to 16.3% improvement in accuracy and superior robustness across diverse emotional states and noisy environments. Notably, the model maintains low performance variance across emotion classes, with F1-Score differences under 2.5%, indicating stable recognition even for subtle or overlapping emotions. Our results underscore the importance of integrating memory, frequency, and adaptive attention for effective affective computing. The code will be publicly available on GitHub.

Förderung

  • Xi’an Jiaotong-Liverpool University

Zitieren

APA

Wu, T., Purwanto, E., Huang, Y., & Su, Y. (2026). Phy-FusionNet: A Memory-Augmented Transformer for Multimodal Emotion Recognition With Periodicity and Contextual Attention. IEEE Transactions on Affective Computing, 17(1), 235-246. https://doi.org/10.1109/taffc.2025.3609046

Vancouver

Wu T, Purwanto E, Huang Y, Su Y. Phy-FusionNet: A Memory-Augmented Transformer for Multimodal Emotion Recognition With Periodicity and Contextual Attention. IEEE Transactions on Affective Computing. 2026;17(1):235-246. doi:10.1109/taffc.2025.3609046

BibTeX
@article{Wu2026,
  author = {Wu, Tianyi and Purwanto, Erick and Huang, Yongrun and Su, Yang},
  title = {Phy-FusionNet: A Memory-Augmented Transformer for Multimodal Emotion Recognition With Periodicity and Contextual Attention},
  journal = {IEEE Transactions on Affective Computing},
  year = {2026},
  volume = {17},
  number = {1},
  pages = {235--246},
  doi = {10.1109/taffc.2025.3609046}
}
Kostenlos starten →