EUVIMEDEUVIMED entdecken
Crossref

Second-Order Visual Attention Prediction in Multiagent Videos via Emotion-Aware Belief Inference

Chenglizhao Chen, Xinyu Liu, Mengke Song, Guotao Wang, Shanchen Pang, Hong Qin, Jufeng Yang, Hui Yu

IEEE Transactions on Image Processing · 2026

DOI: 10.1109/tip.2026.3736736 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Image Processing, 2026. Verfasst von Chenglizhao Chen und 7 weiteren Autor:innen. Hauptthemen laut MEDLINE: Attention, Emotions, Video Recording, Image Processing, Computer-Assisted. Die Arbeit nennt 57 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Taishan Scholars Program for Young Experts of Shandong Province, Excellent Young Scientists Fund of Shandong Provincial Natural Science Foundation.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In human social cognition, understanding others' attention relies not only on explicit cues such as gaze and posture but also on observers' inferences of implicit factors, including emotions, task goals, and social roles. This belief-based ability, known as second-order attention modeling, is a key component of theory of mind (ToM), enabling individuals to infer others' attentional states during multiagent interactions. Existing vision systems mainly focus on first-order attention modeling, such as gaze estimation and gaze following, which predict physical gaze locations. Although effective at the perceptual level, these methods cannot capture belief formation or account for modulatory factors such as emotions, limiting their applicability to higher-level social cognition. To address this limitation, we propose a belief-driven framework for second-order attention modeling, formulating observer inference under uncertainty as conditional probability estimation and integrating emotional cues, task semantics, and social roles. The framework extends attention modeling from gaze perception to emotion-modulated belief reasoning, enabling prediction of observer-reported attentional beliefs from visual, visually inferred affective, and social-context cues when direct gaze evidence is limited or ambiguous. We further introduce a large-scale dataset with diverse agents, tasks, and emotional contexts, whose observer-reported labels are validated through test-retest analysis, together with an emotion-conditioned evaluation metric for social-cognitive assessment. Experiments demonstrate that the proposed framework outperforms existing methods and produces predictions consistent with observer reports, showing effectiveness for emotion-aware second-order attention modeling.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AttentionEmotionsVideo RecordingImage Processing, Computer-AssistedTheory of MindHumansFixation, OcularCues

Förderung

  • National Natural Science Foundation of China
  • Taishan Scholars Program for Young Experts of Shandong Province
  • Excellent Young Scientists Fund of Shandong Provincial Natural Science Foundation

Zitieren

APA

Chen, C., Liu, X., Song, M., Wang, G., Pang, S., Qin, H., Yang, J., & Yu, H. (2026). Second-Order Visual Attention Prediction in Multiagent Videos via Emotion-Aware Belief Inference. IEEE Transactions on Image Processing. https://doi.org/10.1109/tip.2026.3736736

Vancouver

Chen C, Liu X, Song M, Wang G, Pang S, Qin H, et al. Second-Order Visual Attention Prediction in Multiagent Videos via Emotion-Aware Belief Inference. IEEE Transactions on Image Processing. 2026. doi:10.1109/tip.2026.3736736

BibTeX
@article{Chen2026,
  author = {Chen, Chenglizhao and Liu, Xinyu and Song, Mengke and Wang, Guotao and Pang, Shanchen and Qin, Hong and Yang, Jufeng and Yu, Hui},
  title = {Second-Order Visual Attention Prediction in Multiagent Videos via Emotion-Aware Belief Inference},
  journal = {IEEE Transactions on Image Processing},
  year = {2026},
  doi = {10.1109/tip.2026.3736736}
}
Kostenlos starten →