EUVIMEDEUVIMED entdecken
Crossref

MER-CLIP: AU-Guided Vision-Language Alignment for Micro-Expression Recognition

Shifeng Liu, Xinglong Mao, Sirui Zhao, Peiming Li, Tong Xu, Enhong Chen

IEEE Transactions on Affective Computing · 2025-10 · 16 (4) · 3028-3042

DOI: 10.1109/taffc.2025.3584918 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Affective Computing, 2025-10. Verfasst von Shifeng Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 65 Literaturangaben. Gefördert von: National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As a critical psychological stress response, micro-expressions (MEs) are fleeting and subtle facial movements revealing genuine emotions. Automatic ME recognition (MER) holds valuable applications in fields such as criminal investigation and psychological diagnosis. The Facial Action Coding System (FACS) encodes expressions by identifying activations of specific facial action units (AUs), serving as a key reference for ME analysis. However, current MER methods typically limit AU utilization to defining regions of interest (ROIs) or relying on specific prior knowledge, often resulting in limited performance and poor generalization. To address this, we integrate the CLIP model's powerful cross-modal semantic alignment capability into MER and propose a novel approach namely MER-CLIP. Specifically, we convert AU labels into detailed textual descriptions of facial muscle movements, guiding fine-grained spatiotemporal ME learning by aligning visual dynamics and textual AU-based representations. Additionally, we introduce an Emotion Inference Module to capture the nuanced relationships between ME patterns and emotions with higher-level semantic understanding. To mitigate overfitting caused by the scarcity of ME data, we put forward LocalStaticFaceMix, an effective data augmentation strategy blending facial images to enhance facial diversity while preserving critical ME features. Finally, comprehensive experiments on four benchmark ME datasets confirm the superiority of MER-CLIP. Notably, UF1 scores on CAS(ME)$^{3}$reach 0.7832, 0.6544, and 0.4997 for 3-, 4-, and 7-class classification tasks, significantly outperforming previous methods.

Förderung

  • National Natural Science Foundation of China

Zitieren

APA

Liu, S., Mao, X., Zhao, S., Li, P., Xu, T., & Chen, E. (2025). MER-CLIP: AU-Guided Vision-Language Alignment for Micro-Expression Recognition. IEEE Transactions on Affective Computing, 16(4), 3028-3042. https://doi.org/10.1109/taffc.2025.3584918

Vancouver

Liu S, Mao X, Zhao S, Li P, Xu T, Chen E. MER-CLIP: AU-Guided Vision-Language Alignment for Micro-Expression Recognition. IEEE Transactions on Affective Computing. 2025;16(4):3028-3042. doi:10.1109/taffc.2025.3584918

BibTeX
@article{Liu2025,
  author = {Liu, Shifeng and Mao, Xinglong and Zhao, Sirui and Li, Peiming and Xu, Tong and Chen, Enhong},
  title = {MER-CLIP: AU-Guided Vision-Language Alignment for Micro-Expression Recognition},
  journal = {IEEE Transactions on Affective Computing},
  year = {2025},
  volume = {16},
  number = {4},
  pages = {3028--3042},
  doi = {10.1109/taffc.2025.3584918}
}
Kostenlos starten →