EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Dual-Distillation Vision-Language Model for Multimodal Emotion Recognition in Conversation with Quantized Edge Deployment

DeogHwa Kim, Yu il Lee, Da Hyun Yoon, Byeong Jun Kim, Deok-Hwan Kim

Applied Sciences · 23.03.2026 · 16 (6) · 3103

DOI: 10.3390/app16063103 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 23.03.2026. Verfasst von DeogHwa Kim und 4 weiteren Autor:innen. Die Arbeit nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation of Korea, Inha University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal Emotion Recognition in Conversation (ERC) has attracted attention as a key technology in human–computer interaction, mental healthcare, and intelligent services. However, deploying ERC in real-world settings remains challenging due to reliability gaps across modalities, instability in visual representations, and the high computational cost of large pretrained models. In particular, on resource-constrained edge devices, it is difficult to reduce model size and inference latency while preserving accuracy. To address these challenges, we jointly propose a knowledge-distillation-based multimodal ERC model, called DDVLM, with an edge-optimized Weight-Only Quantization (WOQ) pipeline for efficient edge deployment. DDVLM assigns the textual modality as the teacher and the visual modality as the student, transferring emotion-distribution knowledge to improve non-verbal representations and stabilize multimodal learning. In addition, Exponential Moving Average (EMA)-based self-distillation enhances the consistency and generalization capability of text features. Meanwhile, the proposed WOQ pipeline quantizes linear-layer weights to INT8 while preserving precision-sensitive operations in mixed precision, thereby minimizing accuracy loss and reducing model size, memory usage, and inference latency. Experiments on the MELD dataset demonstrated that the proposed approach achieves state-of-the-art performance while also enabling real-time inference on edge devices such as NVIDIA Jetson. Overall, this work presents a practical ERC framework that jointly considers accuracy and deployability.

Förderung

  • National Research Foundation of Korea
  • Inha University

Zitieren

APA

Kim, D., Lee, Y. I., Yoon, D. H., Kim, B. J., & Kim, D. (2026). Dual-Distillation Vision-Language Model for Multimodal Emotion Recognition in Conversation with Quantized Edge Deployment. Applied Sciences, 16(6), 3103. https://doi.org/10.3390/app16063103

Vancouver

Kim D, Lee YI, Yoon DH, Kim BJ, Kim D. Dual-Distillation Vision-Language Model for Multimodal Emotion Recognition in Conversation with Quantized Edge Deployment. Applied Sciences. 2026;16(6):3103. doi:10.3390/app16063103

BibTeX
@article{Kim2026,
  author = {Kim, DeogHwa and Lee, Yu il and Yoon, Da Hyun and Kim, Byeong Jun and Kim, Deok-Hwan},
  title = {Dual-Distillation Vision-Language Model for Multimodal Emotion Recognition in Conversation with Quantized Edge Deployment},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {6},
  pages = {3103},
  doi = {10.3390/app16063103}
}
Kostenlos starten →