EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A novel signal channel attention network for multi-modal emotion recognition

Ziang Du, Xia Ye, Pujie Zhao

Frontiers in Neurorobotics · 11.09.2024 · 18

DOI: 10.3389/fnbot.2024.1442080 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Neurorobotics, online erschienen am 11.09.2024. Verfasst von Ziang Du und 2 weiteren Autor:innen. Beteiligte Einrichtung: Xi'an Research Institute of High-Tech, Xi'an, Shaanxi, China. Die Arbeit wurde bisher 13-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Physiological signal recognition is crucial in emotion recognition, and recent advancements in multi-modal fusion have enabled the integration of various physiological signals for improved recognition tasks. However, current models for emotion recognition with hyper complex multi-modal signals face limitations due to fusion methods and insufficient attention mechanisms, preventing further enhancement in classification performance. To address these challenges, we propose a new model framework named Signal Channel Attention Network (SCA-Net), which comprises three main components: an encoder, an attention fusion module, and a decoder. In the attention fusion module, we developed five types of attention mechanisms inspired by existing research and performed comparative experiments using the public dataset MAHNOB-HCI. All of these experiments demonstrate the effectiveness of the attention module we addressed for our baseline model in improving both accuracy and F1 score metrics. We also conducted ablation experiments within the most effective attention fusion module to verify the benefits of multi-modal fusion. Additionally, we adjusted the training process for different attention fusion modules by employing varying early stopping parameters to prevent model overfitting.

Schlagwörter

Stichwörter der Autor:innen

attention fusion moduleemotion recognitionhypercomplex neural networksmulti-modal fusionphysiological signals

Autor:innen und Institutionen

  • Xi'an Research Institute of High-Tech, Xi'an, Shaanxi, China.

Zitieren

APA

Du, Z., Ye, X., & Zhao, P. (2024). A novel signal channel attention network for multi-modal emotion recognition. Frontiers in Neurorobotics, 18. https://doi.org/10.3389/fnbot.2024.1442080

Vancouver

Du Z, Ye X, Zhao P. A novel signal channel attention network for multi-modal emotion recognition. Frontiers in Neurorobotics. 2024;18. doi:10.3389/fnbot.2024.1442080

BibTeX
@article{Du2024,
  author = {Du, Ziang and Ye, Xia and Zhao, Pujie},
  title = {A novel signal channel attention network for multi-modal emotion recognition},
  journal = {Frontiers in Neurorobotics},
  year = {2024},
  volume = {18},
  doi = {10.3389/fnbot.2024.1442080}
}
Kostenlos starten →