Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking
Transactions of the Association for Computational Linguistics · 15.04.2026 · 14 · 530-561
DOI: 10.1162/tacl.a.647 ↗Kurz gefasst
Fachartikel in der Zeitschrift Transactions of the Association for Computational Linguistics, online erschienen am 15.04.2026. Verfasst von Mubashara Akhtar und 2 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Current automated fact-checking (AFC) approaches typically evaluate evidence either implicitly via the predicted verdicts or through exact matches with predefined closed knowledge sources, such as Wikipedia. However, these methods are limited due to their reliance on evaluation metrics originally designed for other purposes and constraints from closed knowledge sources. In this work, we introduce Ev2R which combines the strengths of reference-based evaluation and verdict-level proxy scoring. Ev2R jointly assesses how well the evidence aligns with the gold references and how reliably it supports the verdict, addressing the shortcomings of prior methods. We evaluate Ev2R against three types of evidence evaluation approaches: reference-based, proxy-reference, and reference-less baselines. Assessments against human ratings and adversarial tests demonstrate that Ev2R consistently outperforms existing scoring approaches in accuracy and robustness. It achieves stronger correlation with human judgments and greater robustness to adversarial perturbations, establishing it as a reliable metric for evidence evaluation in AFC.1
Zitieren
APA
Akhtar, M., Schlichtkrull, M., & Vlachos, A. (2026). Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking. Transactions of the Association for Computational Linguistics, 14, 530-561. https://doi.org/10.1162/tacl.a.647
Vancouver
Akhtar M, Schlichtkrull M, Vlachos A. Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking. Transactions of the Association for Computational Linguistics. 2026;14:530-561. doi:10.1162/tacl.a.647
BibTeX
@article{Akhtar2026,
author = {Akhtar, Mubashara and Schlichtkrull, Michael and Vlachos, Andreas},
title = {Ev2R: Evaluating Evidence Retrieval in Automated Fact-Checking},
journal = {Transactions of the Association for Computational Linguistics},
year = {2026},
volume = {14},
pages = {530--561},
doi = {10.1162/tacl.a.647}
}
EUVIMED