EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating Faithfulness in Agentic RAG Systems for e-Governance Applications Using LLM-Based Judging Frameworks

George Papageorgiou, Vangelis Sarlis, Manolis Maragoudakis, Ioannis Magnisalis, Christos Tjortjis

Big Data and Cognitive Computing · 03.12.2025 · 9 (12) · 309

DOI: 10.3390/bdcc9120309 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 03.12.2025. Verfasst von George Papageorgiou und 4 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 63 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

As Large Language Models (LLMs) are core components in Retrieval-Augmented Generation (RAG) systems for knowledge-intensive tasks, concerns regarding hallucinations, redundancy, and unverifiable outputs have intensified, particularly in high-stakes domains, such as e-government. This study proposes a modular, multi-pipeline framework for statement-level faithfulness evaluation for characterizing hallucination and redundancy across both simple and agentic RAG pipelines. Using GPT-4.1, Claude Sonnet-4.0, and Gemini 2.5 Pro as LLM-based judges, this study examines how tool-specific attribution within agentic multi-tool architectures influences the interpretability and traceability of the generated content. By using a modular agentic RAG framework combining symbolic (GraphRAG), semantic (embedding), and real-time (web) retrieval, we benchmark hallucination and redundancy patterns, using state-of-the-art LLM judges. The study examines RAG and agent-based pipelines that attribute outputs to distinct tools, in contrast to traditional single-source RAG systems that rely on aggregated retrieval. Using e-government data sourced from the European Commission’s Press Corner, our evaluation framework assesses not only the frequency, but also the source-aware detectability of hallucinated content. The findings provide actionable insights into how source granularity and retrieval orchestration impact faithfulness evaluation across different pipeline architectures, while also suggesting new directions for explainability-aware RAG design. The study contributes a reproducible, modular framework for automated faithfulness assessment, with implications for transparency, governance compliance, and trustworthy AI deployment.

Zitieren

APA

Papageorgiou, G., Sarlis, V., Maragoudakis, M., Magnisalis, I., & Tjortjis, C. (2025). Evaluating Faithfulness in Agentic RAG Systems for e-Governance Applications Using LLM-Based Judging Frameworks. Big Data and Cognitive Computing, 9(12), 309. https://doi.org/10.3390/bdcc9120309

Vancouver

Papageorgiou G, Sarlis V, Maragoudakis M, Magnisalis I, Tjortjis C. Evaluating Faithfulness in Agentic RAG Systems for e-Governance Applications Using LLM-Based Judging Frameworks. Big Data and Cognitive Computing. 2025;9(12):309. doi:10.3390/bdcc9120309

BibTeX
@article{Papageorgiou2025,
  author = {Papageorgiou, George and Sarlis, Vangelis and Maragoudakis, Manolis and Magnisalis, Ioannis and Tjortjis, Christos},
  title = {Evaluating Faithfulness in Agentic RAG Systems for e-Governance Applications Using LLM-Based Judging Frameworks},
  journal = {Big Data and Cognitive Computing},
  year = {2025},
  volume = {9},
  number = {12},
  pages = {309},
  doi = {10.3390/bdcc9120309}
}
Kostenlos starten →