Narrative-Aware Scientific Claim Verification Agent with Evidence Ranking for ClimateCheck
Journal of Technology Informatics and Engineering · 30.04.2026 · 5 (1) · 327-340
DOI: 10.51903/jtie.v5i1.549 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Technology Informatics and Engineering, online erschienen am 30.04.2026. Verfasst von Wenhao Su und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
Climate misinformation often combines a factual proposition with a recognizable narrative, such as denying observed warming, rejecting human causation, minimizing impacts, attacking mitigation, or casting doubt on climate science. This paper presents a lightweight narrative-aware scientific claim verification agent for the official ClimateCheck setting. The revised evaluation uses the official annotated ClimateCheck training data, the official publications corpus of 394,269 abstracts, and a claim-level validation split of the annotated data. The public ClimateCheck test file is treated as a blind claim list because its public fields do not contain verification or narrative labels. The system combines hashed BM25, TF-IDF retrieval, latent semantic analysis, narrative-family probabilities, and a logistic-regression verifier. Full-corpus retrieval shows that BM25 remains the strongest first-stage retriever, with Recall@10 = 0.466, while the narrative-aware hybrid obtains Recall@10 = 0.444. In the judged candidate reranking setting, the narrative-aware ranker obtains the highest Candidate Recall@1 = 0.789 and MAP = 0.848, compared with 0.759 and 0.843 for TF-IDF. End-to-end verification remains difficult: the BM25 top-1 pipeline reaches Macro-F1 = 0.408, while the narrative-aware pipeline reaches Macro-F1 = 0.355. Claim-level narrative evaluation no longer produces a perfect score; single-label top-family Macro-F1 is 0.422, and fine-grained multi-label CARDS-code Macro-F1 is 0.098. These results show that narrative information is useful for reranking already plausible evidence candidates, but it does not replace strong lexical retrieval and does not by itself solve claim verification.
Zitieren
APA
Su, W., Chen, S., & Qian, E. (2026). Narrative-Aware Scientific Claim Verification Agent with Evidence Ranking for ClimateCheck. Journal of Technology Informatics and Engineering, 5(1), 327-340. https://doi.org/10.51903/jtie.v5i1.549
Vancouver
Su W, Chen S, Qian E. Narrative-Aware Scientific Claim Verification Agent with Evidence Ranking for ClimateCheck. Journal of Technology Informatics and Engineering. 2026;5(1):327-340. doi:10.51903/jtie.v5i1.549
BibTeX
@article{Su2026,
author = {Su, Wenhao and Chen, Siyu and Qian, Ethan},
title = {Narrative-Aware Scientific Claim Verification Agent with Evidence Ranking for ClimateCheck},
journal = {Journal of Technology Informatics and Engineering},
year = {2026},
volume = {5},
number = {1},
pages = {327--340},
doi = {10.51903/jtie.v5i1.549}
}
EUVIMED