EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Component-Guided 3D Convolutional Neural Network with Multi-Stream Temporal Attention for Robust Video-Based Expression Recognition

Pushpa Rajput Narayana Singh

Journal of Intelligent Decision Making and Information Science · 09.09.2026 · 3 (18s) · 3121-3136

DOI: 10.17762/jidmis.v3.6074 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Intelligent Decision Making and Information Science, online erschienen am 09.09.2026. Verfasst von Pushpa Rajput Narayana Singh. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video-based facial expression recognition (FER) remains a challenging task because it requires the effective integration of spatial appearance, temporal dynamics, and localized facial movements associated with both macro- and micro-expressions. Conventional 3D convolutional neural networks (3D-CNNs) and CNN-LSTM architectures typically assign similar importance to all frames in a sequence and process facial features globally. Consequently, they may fail to adequately capture informative temporal segments, such as the expression apex, and subtle movements of localized facial muscles. To address these limitations, this paper proposes a Temporal Attention-Augmented 3D-CNN, a multi-stream architecture designed to enhance spatiotemporal feature representation. The proposed model integrates a residual 3D-CNN backbone for appearance learning, a temporal self-attention (TSA) gating mechanism that adaptively emphasizes informative feature volumes along the temporal dimension, a parallel motion-aware stream that extracts short-term motion information from frame-difference volumes, and a facial component modeling module. The latter employs landmark-guided region-of-interest pooling to extract features from the eyes, eyebrows, mouth, and nose, followed by cross-region self-attention to capture interactions among facial components. To ensure stable optimization on a relatively small benchmark dataset, an epoch-wise warmup strategy is employed to progressively activate the motion and facial component streams, complemented by inverse-frequency class weighting and stream-level regularization.

Zitieren

APA

Singh, P. R. N. (2026). A Component-Guided 3D Convolutional Neural Network with Multi-Stream Temporal Attention for Robust Video-Based Expression Recognition. Journal of Intelligent Decision Making and Information Science, 3(18s), 3121-3136. https://doi.org/10.17762/jidmis.v3.6074

Vancouver

Singh PRN. A Component-Guided 3D Convolutional Neural Network with Multi-Stream Temporal Attention for Robust Video-Based Expression Recognition. Journal of Intelligent Decision Making and Information Science. 2026;3(18s):3121-3136. doi:10.17762/jidmis.v3.6074

BibTeX
@article{Singh2026,
  author = {Singh, Pushpa Rajput Narayana},
  title = {A Component-Guided 3D Convolutional Neural Network with Multi-Stream Temporal Attention for Robust Video-Based Expression Recognition},
  journal = {Journal of Intelligent Decision Making and Information Science},
  year = {2026},
  volume = {3},
  number = {18s},
  pages = {3121--3136},
  doi = {10.17762/jidmis.v3.6074}
}
Kostenlos starten →