Dual-Distillation Vision-Language Model for Multimodal Emotion Recognition in Conversation with Quantized Edge Deployment
Applied Sciences · 23.03.2026 · 16 (6) · 3103
DOI: 10.3390/app16063103 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 23.03.2026. Verfasst von DeogHwa Kim und 4 weiteren Autor:innen. Die Arbeit nennt 33 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation of Korea, Inha University.
Abstract (Originalsprache)
Multimodal Emotion Recognition in Conversation (ERC) has attracted attention as a key technology in human–computer interaction, mental healthcare, and intelligent services. However, deploying ERC in real-world settings remains challenging due to reliability gaps across modalities, instability in visual representations, and the high computational cost of large pretrained models. In particular, on resource-constrained edge devices, it is difficult to reduce model size and inference latency while preserving accuracy. To address these challenges, we jointly propose a knowledge-distillation-based multimodal ERC model, called DDVLM, with an edge-optimized Weight-Only Quantization (WOQ) pipeline for efficient edge deployment. DDVLM assigns the textual modality as the teacher and the visual modality as the student, transferring emotion-distribution knowledge to improve non-verbal representations and stabilize multimodal learning. In addition, Exponential Moving Average (EMA)-based self-distillation enhances the consistency and generalization capability of text features. Meanwhile, the proposed WOQ pipeline quantizes linear-layer weights to INT8 while preserving precision-sensitive operations in mixed precision, thereby minimizing accuracy loss and reducing model size, memory usage, and inference latency. Experiments on the MELD dataset demonstrated that the proposed approach achieves state-of-the-art performance while also enabling real-time inference on edge devices such as NVIDIA Jetson. Overall, this work presents a practical ERC framework that jointly considers accuracy and deployability.
Förderung
- National Research Foundation of Korea
- Inha University
Zitieren
APA
Kim, D., Lee, Y. I., Yoon, D. H., Kim, B. J., & Kim, D. (2026). Dual-Distillation Vision-Language Model for Multimodal Emotion Recognition in Conversation with Quantized Edge Deployment. Applied Sciences, 16(6), 3103. https://doi.org/10.3390/app16063103
Vancouver
Kim D, Lee YI, Yoon DH, Kim BJ, Kim D. Dual-Distillation Vision-Language Model for Multimodal Emotion Recognition in Conversation with Quantized Edge Deployment. Applied Sciences. 2026;16(6):3103. doi:10.3390/app16063103
BibTeX
@article{Kim2026,
author = {Kim, DeogHwa and Lee, Yu il and Yoon, Da Hyun and Kim, Byeong Jun and Kim, Deok-Hwan},
title = {Dual-Distillation Vision-Language Model for Multimodal Emotion Recognition in Conversation with Quantized Edge Deployment},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {6},
pages = {3103},
doi = {10.3390/app16063103}
}
EUVIMED