Inverse Optimal Transport for Efficient Adaptation of Vision-Language Models
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25018-25026
DOI: 10.1609/aaai.v40i30.39690 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Shupeng Qiu und 1 weiteren Autor:innen.
Abstract (Originalsprache)
Vision–language models (VLMs) such as CLIP have unlocked powerful zero-shot transfer, yet efficient adaptation to downstream tasks remains challenging. Existing methods often depend on graph structures and dataset-specific tuning, making them sensitive to modality gaps and computationally costly at scale. In this paper, we propose IOTA (Inverse Optimal Transport Adaptation), a lightweight algorithm that reformulates VLMs inference from the perspective of inverse optimal transport (IOT), providing a unified view of training and inference. Under the IOT framework, IOTA enhances zero-shot alignment via a theory-guided unbalanced OT strategy and refines textual prototypes using OT-based pseudo-labels with a marginal-aware adaptive threshold, enabling reliable supervision without gradient updates. The framework naturally extends to few-shot scenarios through a label-guided masking mechanism. By decoupling image–text interactions from other inter-modal dependencies, IOTA avoids task-specific tuning and expensive affinity construction. Extensive experiments on standard benchmarks show that IOTA consistently improves zero-shot and few-shot performance while reducing memory and computation overhead, validating both its theoretical insight and plug-and-play practicality.
Zitieren
APA
Qiu, S., & Ren, C. (2026). Inverse Optimal Transport for Efficient Adaptation of Vision-Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25018-25026. https://doi.org/10.1609/aaai.v40i30.39690
Vancouver
Qiu S, Ren C. Inverse Optimal Transport for Efficient Adaptation of Vision-Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25018-25026. doi:10.1609/aaai.v40i30.39690
BibTeX
@article{Qiu2026,
author = {Qiu, Shupeng and Ren, Chuan-Xian},
title = {Inverse Optimal Transport for Efficient Adaptation of Vision-Language Models},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {30},
pages = {25018--25026},
doi = {10.1609/aaai.v40i30.39690}
}
EUVIMED