EUVIMEDEUVIMED entdecken
Crossref

VIXEN: Visual Text Comparison Network for Image Difference Captioning

Alexander Black, Jing Shi, Yifei Fan, Tu Bui, John Collomosse

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (2) · 846-854

DOI: 10.1609/aaai.v38i2.27843 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Alexander Black und 4 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We present VIXEN - a technique that succinctly summarizes in text the visual differences between a pair of images in order to highlight any content manipulation present. Our proposed network linearly maps image features in a pairwise manner, constructing a soft prompt for a pretrained large language model. We address the challenge of low volume of training data and lack of manipulation variety in existing image difference captioning (IDC) datasets by training on synthetically manipulated images from the recent InstructPix2Pix dataset generated via prompt-to-prompt editing framework. We augment this dataset with change summaries produced via GPT-3. We show that VIXEN produces state-of-the-art, comprehensible difference captions for diverse image contents and edit types, offering a potential mitigation against misinformation disseminated via manipulated image content. Code and data are available at http://github.com/alexblck/vixen

Zitieren

APA

Black, A., Shi, J., Fan, Y., Bui, T., & Collomosse, J. (2024). VIXEN: Visual Text Comparison Network for Image Difference Captioning. Proceedings of the AAAI Conference on Artificial Intelligence, 38(2), 846-854. https://doi.org/10.1609/aaai.v38i2.27843

Vancouver

Black A, Shi J, Fan Y, Bui T, Collomosse J. VIXEN: Visual Text Comparison Network for Image Difference Captioning. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(2):846-854. doi:10.1609/aaai.v38i2.27843

BibTeX
@article{Black2024,
  author = {Black, Alexander and Shi, Jing and Fan, Yifei and Bui, Tu and Collomosse, John},
  title = {VIXEN: Visual Text Comparison Network for Image Difference Captioning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {2},
  pages = {846--854},
  doi = {10.1609/aaai.v38i2.27843}
}
Kostenlos starten →