Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (5) · 3858-3866
DOI: 10.1609/aaai.v40i5.37387 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Xiang Fang und 6 weiteren Autor:innen.
Abstract (Originalsprache)
Video-Language Models (VLMs) have demonstrated impressive multi-modal reasoning capabilities across diverse computer vision applications. However, these VLMs are task-specific and assume that both video and language inputs are complete. However, real-world VLM applications might face challenges due to deactivated sensors (e.g., cameras are unavailable due to data privacy), yielding modality-incomplete data and leading to inconsistency between training and testing data. While straightforward incomplete input can boast training generalization-ability and lead to training failure, its potential risks to VLMs regarding safety and trustworthiness have been largely neglected. To this end, we make the first attempt to propose a unified incomplete video-language model to process the incomplete multi-modal inputs. Extensive experimental results show that our method can serve as a plug-and-play module for previous works to improve their performance in various multi-modal tasks.
Zitieren
APA
Fang, X., Fang, W., Wang, C., Tang, K., Liu, D., Wang, S., & Ji, W. (2026). Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs. Proceedings of the AAAI Conference on Artificial Intelligence, 40(5), 3858-3866. https://doi.org/10.1609/aaai.v40i5.37387
Vancouver
Fang X, Fang W, Wang C, Tang K, Liu D, Wang S, et al. Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(5):3858-3866. doi:10.1609/aaai.v40i5.37387
BibTeX
@article{Fang2026,
author = {Fang, Xiang and Fang, Wanlong and Wang, Changshuo and Tang, Keke and Liu, Daizong and Wang, Siyi and Ji, Wei},
title = {Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {5},
pages = {3858--3866},
doi = {10.1609/aaai.v40i5.37387}
}
EUVIMED