EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Large Language Model-Assisted Distillation–Fusion Framework for Visual Emotion Recognition

Yujun Ma, Yunjie Zeng, Zhiyuan Chen, Zhiwei Ye, Wen Zhou, Chunli Xiang

Algorithms · 11.08.2026 · 19 (8) · 669

DOI: 10.3390/a19080669 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Algorithms, online erschienen am 11.08.2026. Verfasst von Yujun Ma und 5 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Young and Middle-aged Scientific and Technological Innovation Team Plan in Higher Education Institutions in Hubei Province, China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual emotion recognition plays a critical role in human–computer interaction and mental health applications. Although existing Vision–Language Models (VLMs) alleviate the limitations of conventional vision models in high-level semantic understanding, they still face three main challenges: limited emotional semantic understanding, insufficient visual emotional perception capability, and high computational costs when deploying both models simultaneously. To address these issues, a large language model-assisted distillation–fusion framework (VERLADF) is proposed, which introduces emotion instruction data generated by GPT to fine-tune a VLM, thereby enhancing its emotional semantic understanding capability. Furthermore, we transfer the visual emotion discrimination knowledge of a conventional vision model into the VLM using a distillation module while keeping the VLM frozen during training, which reduces the computational costs. Following that, we design a fusion and prediction module that adaptively fuses predictions from the instruction-tuned VLM and the distillation module for final emotion recognition. The experimental results on the Abstract, ArtPhoto, Emotion6, and FI datasets demonstrate that VERLADF achieves recognition accuracies of 36.71%, 52.38%, 74.73%, and 79.69%, respectively, significantly outperforming many methods in the literature and demonstrating the effectiveness of the proposed framework.

Förderung

  • National Natural Science Foundation of China
  • Young and Middle-aged Scientific and Technological Innovation Team Plan in Higher Education Institutions in Hubei Province, China

Zitieren

APA

Ma, Y., Zeng, Y., Chen, Z., Ye, Z., Zhou, W., & Xiang, C. (2026). Large Language Model-Assisted Distillation–Fusion Framework for Visual Emotion Recognition. Algorithms, 19(8), 669. https://doi.org/10.3390/a19080669

Vancouver

Ma Y, Zeng Y, Chen Z, Ye Z, Zhou W, Xiang C. Large Language Model-Assisted Distillation–Fusion Framework for Visual Emotion Recognition. Algorithms. 2026;19(8):669. doi:10.3390/a19080669

BibTeX
@article{Ma2026,
  author = {Ma, Yujun and Zeng, Yunjie and Chen, Zhiyuan and Ye, Zhiwei and Zhou, Wen and Xiang, Chunli},
  title = {Large Language Model-Assisted Distillation–Fusion Framework for Visual Emotion Recognition},
  journal = {Algorithms},
  year = {2026},
  volume = {19},
  number = {8},
  pages = {669},
  doi = {10.3390/a19080669}
}
Kostenlos starten →