Negative Entity Suppression for Zero-Shot Captioning with Synthetic Images
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (9) · 7636-7643
DOI: 10.1609/aaai.v40i9.37705 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zimao Lu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
Text-only training provides an attractive approach to address data scarcity challenges in zero-shot image captioning (ZIC), avoiding the expense of collecting paired image-text annotations. However, although these approaches perform well within training domains, they suffer from poor cross-domain generalization, often producing hallucinated content when encountering novel visual environments. Retrieval-based methods attempt to mitigate this limitation by leveraging external knowledge, but they can paradoxically exacerbate hallucination when retrieved captions contain entities irrelevant to the inputs. We introduce the concept of negative entities—objects that appear in generated caption but are absent from the input—and propose Negative Entity Suppression (NES) to tackle this challenge. NES seamlessly integrates three stages: (1) it employs synthetic images to ensure consistent image-to-text retrieval across both training and inference; (2) it filters negative entities from retrieved content to enhance accuracy; and (3) it applies attention-level suppression using identified negative entities to further minimize the impact of hallucination-prone features. Evaluation across multiple benchmarks demonstrates that NES maintains competitive in-domain performance while improving cross-domain transfer and reducing hallucination rates, achieving new state-of-the-art results in ZIC.
Zitieren
APA
Lu, Z., Xu, H., Liu, B., & Wang, K. (2026). Negative Entity Suppression for Zero-Shot Captioning with Synthetic Images. Proceedings of the AAAI Conference on Artificial Intelligence, 40(9), 7636-7643. https://doi.org/10.1609/aaai.v40i9.37705
Vancouver
Lu Z, Xu H, Liu B, Wang K. Negative Entity Suppression for Zero-Shot Captioning with Synthetic Images. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(9):7636-7643. doi:10.1609/aaai.v40i9.37705
BibTeX
@article{Lu2026,
author = {Lu, Zimao and Xu, Hui and Liu, Bing and Wang, Ke},
title = {Negative Entity Suppression for Zero-Shot Captioning with Synthetic Images},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {9},
pages = {7636--7643},
doi = {10.1609/aaai.v40i9.37705}
}
EUVIMED