EUVIMEDEUVIMED entdecken
Crossref

Negative Entity Suppression for Zero-Shot Captioning with Synthetic Images

Zimao Lu, Hui Xu, Bing Liu, Ke Wang

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (9) · 7636-7643

DOI: 10.1609/aaai.v40i9.37705 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zimao Lu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text-only training provides an attractive approach to address data scarcity challenges in zero-shot image captioning (ZIC), avoiding the expense of collecting paired image-text annotations. However, although these approaches perform well within training domains, they suffer from poor cross-domain generalization, often producing hallucinated content when encountering novel visual environments. Retrieval-based methods attempt to mitigate this limitation by leveraging external knowledge, but they can paradoxically exacerbate hallucination when retrieved captions contain entities irrelevant to the inputs. We introduce the concept of negative entities—objects that appear in generated caption but are absent from the input—and propose Negative Entity Suppression (NES) to tackle this challenge. NES seamlessly integrates three stages: (1) it employs synthetic images to ensure consistent image-to-text retrieval across both training and inference; (2) it filters negative entities from retrieved content to enhance accuracy; and (3) it applies attention-level suppression using identified negative entities to further minimize the impact of hallucination-prone features. Evaluation across multiple benchmarks demonstrates that NES maintains competitive in-domain performance while improving cross-domain transfer and reducing hallucination rates, achieving new state-of-the-art results in ZIC.

Zitieren

APA

Lu, Z., Xu, H., Liu, B., & Wang, K. (2026). Negative Entity Suppression for Zero-Shot Captioning with Synthetic Images. Proceedings of the AAAI Conference on Artificial Intelligence, 40(9), 7636-7643. https://doi.org/10.1609/aaai.v40i9.37705

Vancouver

Lu Z, Xu H, Liu B, Wang K. Negative Entity Suppression for Zero-Shot Captioning with Synthetic Images. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(9):7636-7643. doi:10.1609/aaai.v40i9.37705

BibTeX
@article{Lu2026,
  author = {Lu, Zimao and Xu, Hui and Liu, Bing and Wang, Ke},
  title = {Negative Entity Suppression for Zero-Shot Captioning with Synthetic Images},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {9},
  pages = {7636--7643},
  doi = {10.1609/aaai.v40i9.37705}
}
Kostenlos starten →