EUVIMEDEUVIMED entdecken
Crossref

Inverse Optimal Transport for Efficient Adaptation of Vision-Language Models

Shupeng Qiu, Chuan-Xian Ren

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (30) · 25018-25026

DOI: 10.1609/aaai.v40i30.39690 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Shupeng Qiu und 1 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Vision–language models (VLMs) such as CLIP have unlocked powerful zero-shot transfer, yet efficient adaptation to downstream tasks remains challenging. Existing methods often depend on graph structures and dataset-specific tuning, making them sensitive to modality gaps and computationally costly at scale. In this paper, we propose IOTA (Inverse Optimal Transport Adaptation), a lightweight algorithm that reformulates VLMs inference from the perspective of inverse optimal transport (IOT), providing a unified view of training and inference. Under the IOT framework, IOTA enhances zero-shot alignment via a theory-guided unbalanced OT strategy and refines textual prototypes using OT-based pseudo-labels with a marginal-aware adaptive threshold, enabling reliable supervision without gradient updates. The framework naturally extends to few-shot scenarios through a label-guided masking mechanism. By decoupling image–text interactions from other inter-modal dependencies, IOTA avoids task-specific tuning and expensive affinity construction. Extensive experiments on standard benchmarks show that IOTA consistently improves zero-shot and few-shot performance while reducing memory and computation overhead, validating both its theoretical insight and plug-and-play practicality.

Zitieren

APA

Qiu, S., & Ren, C. (2026). Inverse Optimal Transport for Efficient Adaptation of Vision-Language Models. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30), 25018-25026. https://doi.org/10.1609/aaai.v40i30.39690

Vancouver

Qiu S, Ren C. Inverse Optimal Transport for Efficient Adaptation of Vision-Language Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(30):25018-25026. doi:10.1609/aaai.v40i30.39690

BibTeX
@article{Qiu2026,
  author = {Qiu, Shupeng and Ren, Chuan-Xian},
  title = {Inverse Optimal Transport for Efficient Adaptation of Vision-Language Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {30},
  pages = {25018--25026},
  doi = {10.1609/aaai.v40i30.39690}
}
Kostenlos starten →