EUVIMEDEUVIMED entdecken
Crossref

VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering

Chun-Mei Feng, Yang Bai, Tao Luo, Zhen Li, Salman Khan, Wangmeng Zuo, Rick Siow Mong Goh, Yong Liu

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (3) · 2942-2950

DOI: 10.1609/aaai.v39i3.32301 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Chun-Mei Feng und 7 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Albeit progress has been made in Composed Image Retrieval (CIR), we empirically find that a certain percentage of failure retrieval results are not consistent with their relative captions. To address this issue, this work provides a Visual Question Answering (VQA) perspective to boost the performance of CIR. The resulting VQA4CIR is a post-processing approach and can be directly plugged into existing CIR methods. Given the top-C retrieved images by a CIR method, VQA4CIR aims to decrease the adverse effect of the failure retrieval results being inconsistent with the relative caption. To find the retrieved images inconsistent with the relative caption, we resort to the "QA generation → VQA" self-verification pipeline. For QA generation, we suggest fine-tuning LLM (e.g., LLaMA) to generate several pairs of questions and answers from each relative caption. We then fine-tune LVLM (e.g., LLaVA) to obtain the VQA model. By feeding the retrieved image and question to the VQA model, one can find the images inconsistent with relative caption when the answer by VQA is inconsistent with the answer in the QA pair. Consequently, the CIR performance can be boosted by modifying the ranks of inconsistently retrieved images. Experimental results show that our proposed method outperforms state-of-the-art CIR methods on the CIRR and Fashion-IQ datasets.

Zitieren

APA

Feng, C., Bai, Y., Luo, T., Li, Z., Khan, S., Zuo, W., Goh, R. S. M., & Liu, Y. (2025). VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering. Proceedings of the AAAI Conference on Artificial Intelligence, 39(3), 2942-2950. https://doi.org/10.1609/aaai.v39i3.32301

Vancouver

Feng C, Bai Y, Luo T, Li Z, Khan S, Zuo W, et al. VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(3):2942-2950. doi:10.1609/aaai.v39i3.32301

BibTeX
@article{Feng2025,
  author = {Feng, Chun-Mei and Bai, Yang and Luo, Tao and Li, Zhen and Khan, Salman and Zuo, Wangmeng and Goh, Rick Siow Mong and Liu, Yong},
  title = {VQA4CIR: Boosting Composed Image Retrieval with Visual Question Answering},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {3},
  pages = {2942--2950},
  doi = {10.1609/aaai.v39i3.32301}
}
Kostenlos starten →