Static for Dynamic: Towards a Deeper Understanding of Dynamic Facial Expressions Using Static Expression Data
IEEE Transactions on Affective Computing · 2026-01 · 17 (1) · 438-451
DOI: 10.1109/taffc.2025.3623135 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2026-01. Verfasst von Yin Chen und 6 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 61 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
Dynamic facial expression recognition (DFER) infers emotions from the temporal evolution of expressions, unlike static facial expression recognition (SFER), which relies solely on a single snapshot. This temporal analysis provides richer information and promises greater recognition capability. However, current DFER methods often exhibit unsatisfied performance largely due to fewer training samples compared to SFER. Given the inherent correlation between static and dynamic expressions, we hypothesize that leveraging the abundant SFER data can enhance DFER. To this end, we propose Static-for-Dynamic (S4D), a unified dual-modal learning framework that integrates SFER data as a complementary resource for DFER. Specifically, S4D employs dual-modal self-supervised pre-training on facial images and videos using a shared Vision Transformer (ViT) encoder-decoder architecture, yielding improved spatiotemporal representations. The pre-trained encoder is then fine-tuned on static and dynamic expression datasets in a multi-task learning setup to facilitate emotional information interaction. Unfortunately, vanilla multi-task learning in our study results in negative transfer. To address this, we propose an innovative Mixture of Adapter Experts (MoAE) module that facilitates task-specific knowledge acquisition while effectively extracting shared knowledge from both static and dynamic expression data. Extensive experiments demonstrate that S4D achieves a deeper understanding of DFER, setting new state-of-the-art performance on FERV39K, MAFW, and DFEW benchmarks, with weighted average recall (WAR) of 53.65%, 58.44%, and 76.68%, respectively. Additionally, a systematic correlation analysis between SFER and DFER tasks is presented, which further elucidates the potential benefits of leveraging SFER.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities
- High-Performance Computing Platform of Hefei University of Technology
Zitieren
APA
Chen, Y., Li, J., Zhang, Y., Hu, Z., Shan, S., Wang, M., & Hong, R. (2026). Static for Dynamic: Towards a Deeper Understanding of Dynamic Facial Expressions Using Static Expression Data. IEEE Transactions on Affective Computing, 17(1), 438-451. https://doi.org/10.1109/taffc.2025.3623135
Vancouver
Chen Y, Li J, Zhang Y, Hu Z, Shan S, Wang M, et al. Static for Dynamic: Towards a Deeper Understanding of Dynamic Facial Expressions Using Static Expression Data. IEEE Transactions on Affective Computing. 2026;17(1):438-451. doi:10.1109/taffc.2025.3623135
BibTeX
@article{Chen2026,
author = {Chen, Yin and Li, Jia and Zhang, Yu and Hu, Zhenzhen and Shan, Shiguang and Wang, Meng and Hong, Richang},
title = {Static for Dynamic: Towards a Deeper Understanding of Dynamic Facial Expressions Using Static Expression Data},
journal = {IEEE Transactions on Affective Computing},
year = {2026},
volume = {17},
number = {1},
pages = {438--451},
doi = {10.1109/taffc.2025.3623135}
}
EUVIMED