EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Prompt-Controlled Multi-Modal Tuning for Person-Specific Few-Shot Micro-Expression Recognition

Ruiqi Wang, Kerong Li, Jiateng Liu iD, Tianxiang Cao iD, Yingtian Yu, Hengcan Shi, Yaonan Wang

Sensors · 2026

DOI: 10.3390/s26196083 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2026. Verfasst von Ruiqi Wang und 6 weiteren Autor:innen. Beteiligte Einrichtung: School of Artificial Intelligence, The University of Science and Technology Beijing, Beijing 100080, China. Hauptthemen laut MEDLINE: Pattern Recognition, Automated. Die Arbeit nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, the Major Basic Research Project of Hunan Province.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Micro-expression recognition (MER) is a challenging step in various multimedia applications, such as media understanding and human–computer interaction, as it can reveal genuine human emotions. However, traditional MER often overlooks person-specific facial nuances, limiting generalization and personalized adaptability in practical applications. In this paper, we propose a novel person-specific few-shot MER benchmark that uses only an image/video of the target person as a reference to extend the boundaries of generalization and personalized adaptability in MER. A Prompt-Controlled Multi-Modal Tuning (PCMMT) framework is presented to tackle this benchmark. We introduce a prompt bottleneck mechanism in PCMMT that leverages text prompts and the CLIP multi-modal embedding space to bridge the query and reference visual inputs. Our prompt bottleneck also controls the flow of information across modalities to extract person-specific, subtle micro-expression motions. Moreover, adapter groups are designed at the layer level, with multi-modal tokens fine-tuned separately to refine person-specific cues and enhance generalization. The experimental results show that the proposed person-specific few-shot setting achieves better personalized generalization than traditional MER settings, and our PCMMT outperforms previous state-of-the-art models across various MER benchmarks.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Pattern Recognition, AutomatedHumansAlgorithmsFacial ExpressionEmotions

Stichwörter der Autor:innen

CLIPadapter groupsfew-shotmicro-expression recognition

Autor:innen und Institutionen

  • School of Artificial Intelligence, The University of Science and Technology Beijing, Beijing 100080, China.
  • College of Electrical and Information Engineering, Hunan University, Changsha 410082, China.
  • The Key Laboratory of Child Development and Learning Science of Ministry of Education, Southeast University, Nanjing 211100, China.
  • School of Artificial Intelligence and Robotics, Hunan University, Changsha 410082, China.
  • China Mobile Communications Group Hunan, Changsha 410082, China.

ORCID-Profile: Jiateng Liu, Tianxiang Cao

Förderung

  • National Natural Science Foundation of China
  • the Major Basic Research Project of Hunan Province

Zitieren

APA

Wang, R., Li, K., Liu, J., Cao, T., Yu, Y., Shi, H., & Wang, Y. (2026). Prompt-Controlled Multi-Modal Tuning for Person-Specific Few-Shot Micro-Expression Recognition. Sensors. https://doi.org/10.3390/s26196083

Vancouver

Wang R, Li K, Liu J, Cao T, Yu Y, Shi H, et al. Prompt-Controlled Multi-Modal Tuning for Person-Specific Few-Shot Micro-Expression Recognition. Sensors. 2026. doi:10.3390/s26196083

BibTeX
@article{Wang2026,
  author = {Wang, Ruiqi and Li, Kerong and Liu, Jiateng and Cao, Tianxiang and Yu, Yingtian and Shi, Hengcan and Wang, Yaonan},
  title = {Prompt-Controlled Multi-Modal Tuning for Person-Specific Few-Shot Micro-Expression Recognition},
  journal = {Sensors},
  year = {2026},
  doi = {10.3390/s26196083}
}
Kostenlos starten →