MFDAN: Multi-Level Flow-Driven Attention Network for Micro-Expression Recognition
IEEE Transactions on Circuits and Systems for Video Technology · 2024-12 · 34 (12) · 12823-12836
DOI: 10.1109/tcsvt.2024.3437481 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-12. Verfasst von Wenhao Cai und 6 weiteren Autor:innen. Die Arbeit wurde bisher 42-mal zitiert (Crossref) und nennt 65 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Beijing Natural Science Foundation, Natural Science Foundation of Shandong Province.
Abstract (Originalsprache)
Facial expressions are an essential part of human emotional communication, and micro-expressions (MEs), as transient and imperceptible non-verbal signals, can potentially reveal real human emotions. However, subtle motion variations, limited and unbalanced samples make micro-expression recognition (MER) challenging. In this paper, we design a novel dual-branch learning framework of multi-level flow-driven attention for micro-expression recognition (MFDAN), which innovatively integrates optical flow prior to guide the attention learning in the image encoding branch, enabling the model to focus on the most discriminative facial regions for subtle motion patterns. Firstly, we extract optical flow information by an optical flow encoding module. Then, in the image coding module, we construct a Transformer structure containing an optical flow-driven attention mechanism, which can effectively locate the interest region of micro-expressions in the image according to the position information of optical flow to capture more sensitive and fine-grained micro-expressions. By interoperating prior knowledge with data learning, and introducing the Dropkey operation and Focal Loss, our method can handle subtle micro-expression features on small imbalanced datasets. Through extensive experiments on three independent datasets and a composite database, including SMIC-HS, SAMM, and CASME II, robust leave-one-subject-out (LOSO) evaluation results show that our method outperforms state-of-the-art methods especially on the composite database.
Förderung
- National Natural Science Foundation of China
- Beijing Natural Science Foundation
- Natural Science Foundation of Shandong Province
- Young Elite Scientists Sponsorship Program by China Association for Science and Technology
- Shanghai Sailing Program
- Beijing Hospitals Authority Clinical Medicine Development of Special Funding
Zitieren
APA
Cai, W., Zhao, J., Yi, R., Yu, M., Duan, F., Pan, Z., & Liu, Y. (2024). MFDAN: Multi-Level Flow-Driven Attention Network for Micro-Expression Recognition. IEEE Transactions on Circuits and Systems for Video Technology, 34(12), 12823-12836. https://doi.org/10.1109/tcsvt.2024.3437481
Vancouver
Cai W, Zhao J, Yi R, Yu M, Duan F, Pan Z, et al. MFDAN: Multi-Level Flow-Driven Attention Network for Micro-Expression Recognition. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(12):12823-12836. doi:10.1109/tcsvt.2024.3437481
BibTeX
@article{Cai2024,
author = {Cai, Wenhao and Zhao, Junli and Yi, Ran and Yu, Minjing and Duan, Fuqing and Pan, Zhenkuan and Liu, Yong-Jin},
title = {MFDAN: Multi-Level Flow-Driven Attention Network for Micro-Expression Recognition},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2024},
volume = {34},
number = {12},
pages = {12823--12836},
doi = {10.1109/tcsvt.2024.3437481}
}
EUVIMED