Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends, and Metrics Analysis
Transactions of the Association for Computational Linguistics · 13.11.2025 · 13 · 1597-1644
DOI: 10.1162/tacl.a.52 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 02.12.2025. Verfasst von Uri Berger und 3 weiteren Autor:innen. Die Arbeit wurde bisher 5-mal zitiert (Crossref) und nennt 375 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
The task of image captioning has recently been gaining popularity, and with it the complex task of evaluating the quality of image captioning models. In this work, we present the first survey and taxonomy of over 70 different image captioning metrics and their usage in hundreds of papers, specifically designed to help users select the most suitable metric for their needs. We find that despite the diversity of proposed metrics, the vast majority of studies rely on only five popular metrics, which we show to be weakly correlated with human ratings. We hypothesize that combining a diverse set of metrics can enhance correlation with human ratings. As an initial step, we demonstrate that a linear regression-based ensemble method, which we call EnsembEval, trained on one human ratings dataset, achieves improved correlation across five additional datasets, showing there is a lot of room for improvement by leveraging a diverse set of metrics.1
Zitieren
APA
Berger, U., Stanovsky, G., Abend, O., & Frermann, L. (2025). Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends, and Metrics Analysis. Transactions of the Association for Computational Linguistics, 13, 1597-1644. https://doi.org/10.1162/tacl.a.52
Vancouver
Berger U, Stanovsky G, Abend O, Frermann L. Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends, and Metrics Analysis. Transactions of the Association for Computational Linguistics. 2025;13:1597-1644. doi:10.1162/tacl.a.52
BibTeX
@article{Berger2025,
author = {Berger, Uri and Stanovsky, Gabriel and Abend, Omri and Frermann, Lea},
title = {Surveying the Landscape of Image Captioning Evaluation: A Comprehensive Taxonomy, Trends, and Metrics Analysis},
journal = {Transactions of the Association for Computational Linguistics},
year = {2025},
volume = {13},
pages = {1597--1644},
doi = {10.1162/tacl.a.52}
}
EUVIMED