Large Language Model-Assisted Distillation–Fusion Framework for Visual Emotion Recognition
Algorithms · 11.08.2026 · 19 (8) · 669
DOI: 10.3390/a19080669 ↗Kurz gefasst
Fachartikel in der Zeitschrift Algorithms, online erschienen am 11.08.2026. Verfasst von Yujun Ma und 5 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Young and Middle-aged Scientific and Technological Innovation Team Plan in Higher Education Institutions in Hubei Province, China.
Abstract (Originalsprache)
Visual emotion recognition plays a critical role in human–computer interaction and mental health applications. Although existing Vision–Language Models (VLMs) alleviate the limitations of conventional vision models in high-level semantic understanding, they still face three main challenges: limited emotional semantic understanding, insufficient visual emotional perception capability, and high computational costs when deploying both models simultaneously. To address these issues, a large language model-assisted distillation–fusion framework (VERLADF) is proposed, which introduces emotion instruction data generated by GPT to fine-tune a VLM, thereby enhancing its emotional semantic understanding capability. Furthermore, we transfer the visual emotion discrimination knowledge of a conventional vision model into the VLM using a distillation module while keeping the VLM frozen during training, which reduces the computational costs. Following that, we design a fusion and prediction module that adaptively fuses predictions from the instruction-tuned VLM and the distillation module for final emotion recognition. The experimental results on the Abstract, ArtPhoto, Emotion6, and FI datasets demonstrate that VERLADF achieves recognition accuracies of 36.71%, 52.38%, 74.73%, and 79.69%, respectively, significantly outperforming many methods in the literature and demonstrating the effectiveness of the proposed framework.
Förderung
- National Natural Science Foundation of China
- Young and Middle-aged Scientific and Technological Innovation Team Plan in Higher Education Institutions in Hubei Province, China
Zitieren
APA
Ma, Y., Zeng, Y., Chen, Z., Ye, Z., Zhou, W., & Xiang, C. (2026). Large Language Model-Assisted Distillation–Fusion Framework for Visual Emotion Recognition. Algorithms, 19(8), 669. https://doi.org/10.3390/a19080669
Vancouver
Ma Y, Zeng Y, Chen Z, Ye Z, Zhou W, Xiang C. Large Language Model-Assisted Distillation–Fusion Framework for Visual Emotion Recognition. Algorithms. 2026;19(8):669. doi:10.3390/a19080669
BibTeX
@article{Ma2026,
author = {Ma, Yujun and Zeng, Yunjie and Chen, Zhiyuan and Ye, Zhiwei and Zhou, Wen and Xiang, Chunli},
title = {Large Language Model-Assisted Distillation–Fusion Framework for Visual Emotion Recognition},
journal = {Algorithms},
year = {2026},
volume = {19},
number = {8},
pages = {669},
doi = {10.3390/a19080669}
}
EUVIMED