Evaluating Faithfulness in Agentic RAG Systems for e-Governance Applications Using LLM-Based Judging Frameworks
Big Data and Cognitive Computing · 03.12.2025 · 9 (12) · 309
DOI: 10.3390/bdcc9120309 ↗Kurz gefasst
Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 03.12.2025. Verfasst von George Papageorgiou und 4 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
As Large Language Models (LLMs) are core components in Retrieval-Augmented Generation (RAG) systems for knowledge-intensive tasks, concerns regarding hallucinations, redundancy, and unverifiable outputs have intensified, particularly in high-stakes domains, such as e-government. This study proposes a modular, multi-pipeline framework for statement-level faithfulness evaluation for characterizing hallucination and redundancy across both simple and agentic RAG pipelines. Using GPT-4.1, Claude Sonnet-4.0, and Gemini 2.5 Pro as LLM-based judges, this study examines how tool-specific attribution within agentic multi-tool architectures influences the interpretability and traceability of the generated content. By using a modular agentic RAG framework combining symbolic (GraphRAG), semantic (embedding), and real-time (web) retrieval, we benchmark hallucination and redundancy patterns, using state-of-the-art LLM judges. The study examines RAG and agent-based pipelines that attribute outputs to distinct tools, in contrast to traditional single-source RAG systems that rely on aggregated retrieval. Using e-government data sourced from the European Commission’s Press Corner, our evaluation framework assesses not only the frequency, but also the source-aware detectability of hallucinated content. The findings provide actionable insights into how source granularity and retrieval orchestration impact faithfulness evaluation across different pipeline architectures, while also suggesting new directions for explainability-aware RAG design. The study contributes a reproducible, modular framework for automated faithfulness assessment, with implications for transparency, governance compliance, and trustworthy AI deployment.
Zitieren
APA
Papageorgiou, G., Sarlis, V., Maragoudakis, M., Magnisalis, I., & Tjortjis, C. (2025). Evaluating Faithfulness in Agentic RAG Systems for e-Governance Applications Using LLM-Based Judging Frameworks. Big Data and Cognitive Computing, 9(12), 309. https://doi.org/10.3390/bdcc9120309
Vancouver
Papageorgiou G, Sarlis V, Maragoudakis M, Magnisalis I, Tjortjis C. Evaluating Faithfulness in Agentic RAG Systems for e-Governance Applications Using LLM-Based Judging Frameworks. Big Data and Cognitive Computing. 2025;9(12):309. doi:10.3390/bdcc9120309
BibTeX
@article{Papageorgiou2025,
author = {Papageorgiou, George and Sarlis, Vangelis and Maragoudakis, Manolis and Magnisalis, Ioannis and Tjortjis, Christos},
title = {Evaluating Faithfulness in Agentic RAG Systems for e-Governance Applications Using LLM-Based Judging Frameworks},
journal = {Big Data and Cognitive Computing},
year = {2025},
volume = {9},
number = {12},
pages = {309},
doi = {10.3390/bdcc9120309}
}
EUVIMED