EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Dynamic hypernode injection and graph attention fusion for multimodal sentiment analysis of behavioral signals

Wenfei Cao, Jian Xu iD, Yinghao Li, Huizi Yan, Zhuowei Hu, Wenxu Chen

PLOS One · 2026

DOI: 10.1371/journal.pone.0359425 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2026. Verfasst von Wenfei Cao und 5 weiteren Autor:innen. Beteiligte Einrichtung: College of Intelligent Systems Science and Engineering, Hubei Minzu University, Enshi, China. Hauptthemen laut MEDLINE: Language, Attention. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Language use, vocal behavior, and facial activity provide complementary indicators of affective state, but multimodal models can be affected by noisy temporal observations and insufficient global guidance before cross-modal interaction. We developed a framework combining bidirectional gated recurrent unit encoders, attention pooling, dynamic hypernode injection, and graph attention fusion. Textual, acoustic, and visual sequences were mapped into a shared latent space and compressed into modality-level representations. A sample-dependent hypernode and a learnable static prior were then injected through gated residual connections before graph propagation. The model was evaluated on CMU-MOSI and CMU-MOSEI using five random seeds and validation-MAE checkpoint selection. On CMU-MOSI, the model obtained an MAE of 0.7290 ± 0.0099 , a correlation of 0.7880 ± 0.0076 , and an Acc-7 of 0.4561 ± 0.0160 . On CMU-MOSEI, the corresponding results were 0.5541 ± 0.0053 , 0.7551 ± 0.0058 , and 0.5269 ± 0.0064 . A five-run comparison of pre-GAT, post-GAT, and no injection showed that pre-GAT injection provided the strongest overall regression-oriented balance, although post-GAT was slightly higher on MOSEI Acc-7. Gate, graph-attention, representation, and balanced case diagnostics showed that the injected prior was sample dependent and modality selective. These findings support pre-propagation hypernode injection under complete, word-aligned tri-modal inputs, without establishing universal superiority or robustness to missing and corrupted modalities.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

LanguageAttentionHumansAlgorithms

Autor:innen und Institutionen

  • College of Intelligent Systems Science and Engineering, Hubei Minzu University, Enshi, China.

ORCID-Profile: Jian Xu

Zitieren

APA

Cao, W., Xu, J., Li, Y., Yan, H., Hu, Z., & Chen, W. (2026). Dynamic hypernode injection and graph attention fusion for multimodal sentiment analysis of behavioral signals. PLOS One. https://doi.org/10.1371/journal.pone.0359425

Vancouver

Cao W, Xu J, Li Y, Yan H, Hu Z, Chen W. Dynamic hypernode injection and graph attention fusion for multimodal sentiment analysis of behavioral signals. PLOS One. 2026. doi:10.1371/journal.pone.0359425

BibTeX
@article{Cao2026,
  author = {Cao, Wenfei and Xu, Jian and Li, Yinghao and Yan, Huizi and Hu, Zhuowei and Chen, Wenxu},
  title = {Dynamic hypernode injection and graph attention fusion for multimodal sentiment analysis of behavioral signals},
  journal = {PLOS One},
  year = {2026},
  doi = {10.1371/journal.pone.0359425}
}
Kostenlos starten →