Prompt-Controlled Multi-Modal Tuning for Person-Specific Few-Shot Micro-Expression Recognition
Sensors · 2026
DOI: 10.3390/s26196083 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Ruiqi Wang und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Artificial Intelligence, The University of Science and Technology Beijing, Beijing 100080, China. Hauptthemen laut MEDLINE: Pattern Recognition, Automated. Die Arbeit nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, the Major Basic Research Project of Hunan Province.
Abstract (Originalsprache)
Micro-expression recognition (MER) is a challenging step in various multimedia applications, such as media understanding and human–computer interaction, as it can reveal genuine human emotions. However, traditional MER often overlooks person-specific facial nuances, limiting generalization and personalized adaptability in practical applications. In this paper, we propose a novel person-specific few-shot MER benchmark that uses only an image/video of the target person as a reference to extend the boundaries of generalization and personalized adaptability in MER. A Prompt-Controlled Multi-Modal Tuning (PCMMT) framework is presented to tackle this benchmark. We introduce a prompt bottleneck mechanism in PCMMT that leverages text prompts and the CLIP multi-modal embedding space to bridge the query and reference visual inputs. Our prompt bottleneck also controls the flow of information across modalities to extract person-specific, subtle micro-expression motions. Moreover, adapter groups are designed at the layer level, with multi-modal tokens fine-tuned separately to refine person-specific cues and enhance generalization. The experimental results show that the proposed person-specific few-shot setting achieves better personalized generalization than traditional MER settings, and our PCMMT outperforms previous state-of-the-art models across various MER benchmarks.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Artificial Intelligence, The University of Science and Technology Beijing, Beijing 100080, China.
- College of Electrical and Information Engineering, Hunan University, Changsha 410082, China.
- The Key Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University, Nanjing 211100, China.
- School of Artificial Intelligence and Robotics, Hunan University, Changsha 410082, China.
- China Mobile Communications Group Hunan, Changsha 410082, China.
ORCID-Profile: Jiateng Liu, Tianxiang Cao
Förderung
- National Natural Science Foundation of China
- the Major Basic Research Project of Hunan Province
Zitieren
APA
Wang, R., Li, K., Liu, J., Cao, T., Yu, Y., Shi, H., & Wang, Y. (2026). Prompt-Controlled Multi-Modal Tuning for Person-Specific Few-Shot Micro-Expression Recognition. Sensors. https://doi.org/10.3390/s26196083
Vancouver
Wang R, Li K, Liu J, Cao T, Yu Y, Shi H, et al. Prompt-Controlled Multi-Modal Tuning for Person-Specific Few-Shot Micro-Expression Recognition. Sensors. 2026. doi:10.3390/s26196083
BibTeX
@article{Wang2026,
author = {Wang, Ruiqi and Li, Kerong and Liu, Jiateng and Cao, Tianxiang and Yu, Yingtian and Shi, Hengcan and Wang, Yaonan},
title = {Prompt-Controlled Multi-Modal Tuning for Person-Specific Few-Shot Micro-Expression Recognition},
journal = {Sensors},
year = {2026},
doi = {10.3390/s26196083}
}
EUVIMED