EUVIMEDEUVIMED entdecken
Crossref

LLM-Free Image Captioning Evaluation in Reference-Flexible Settings

Shinnosuke Hirano, Yuiga Wada, Kazuki Matsuda, Seitaro Otsuki, Komei Sugiura

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (6) · 4708-4716

DOI: 10.1609/aaai.v40i6.42472 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Shinnosuke Hirano und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We focus on the automatic evaluation of image captions in both reference-based and reference-free settings. Existing metrics based on large language models (LLMs) favor their own generations; therefore, the neutrality is in question. Most LLM-free metrics do not suffer from such an issue, whereas they do not always demonstrate high performance. To address these issues, we propose Pearl, an LLM-free supervised metric for image captioning, which is applicable to both reference-based and reference-free settings. We introduce a novel mechanism that learns the representations of image--caption and caption--caption similarities. Furthermore, we construct a human-annotated dataset for image captioning metrics that comprises approximately 333k human judgments collected from 2,360 annotators across over 75k images. Pearl outperformed other existing LLM-free metrics on the Composite, Flickr8K-Expert, Flickr8K-CF, Nebula, and FOIL datasets in both reference-based and reference-free settings.

Zitieren

APA

Hirano, S., Wada, Y., Matsuda, K., Otsuki, S., & Sugiura, K. (2026). LLM-Free Image Captioning Evaluation in Reference-Flexible Settings. Proceedings of the AAAI Conference on Artificial Intelligence, 40(6), 4708-4716. https://doi.org/10.1609/aaai.v40i6.42472

Vancouver

Hirano S, Wada Y, Matsuda K, Otsuki S, Sugiura K. LLM-Free Image Captioning Evaluation in Reference-Flexible Settings. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(6):4708-4716. doi:10.1609/aaai.v40i6.42472

BibTeX
@article{Hirano2026,
  author = {Hirano, Shinnosuke and Wada, Yuiga and Matsuda, Kazuki and Otsuki, Seitaro and Sugiura, Komei},
  title = {LLM-Free Image Captioning Evaluation in Reference-Flexible Settings},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {6},
  pages = {4708--4716},
  doi = {10.1609/aaai.v40i6.42472}
}
Kostenlos starten →