EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Evidence-Chain Reliable RAG: Hallucination Detection, Source Attribution, and Deterministic Provenance Explanations

Jiaying Jin

Journal of Technology Informatics and Engineering · 25.08.2025 · 4 (2) · 520-533

DOI: 10.51903/jtie.v4i2.535 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Technology Informatics and Engineering, online erschienen am 25.08.2025. Verfasst von Jiaying Jin. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Retrieval-augmented generation (RAG) reduces unsupported generation by grounding answers in source content, but retrieval alone does not guarantee that every output claim is attributable to evidence. This paper presents Evidence-Chain Reliable RAG, an empirical hallucination-detection and provenance framework that scores whether generated response sentences are supported by the corresponding RAG source record. The evaluation uses the complete RAGTruth JSONL data available for this study: 2,965 source records, 17,790 assistant responses, and 14,289 exact-offset hallucination spans across Data2Text, QA, and summarization. The experiment converts word-level spans into response-level, sentence-level, and character-span targets; extracts lexical, BM25, TF-IDF, unsupported-number, unsupported-entity, refusal, and Evidence-Chain Score features; and compares seven methods. On the official held-out test split of 2,700 responses, RandomForest achieved the best case-level F1 of 0.626 and PR-AUC of 0.553. The proposed ECS-Span model achieved case-level F1 of 0.614, ROC-AUC of 0.742, and PR-AUC of 0.536 while also producing deterministic provenance explanations. At sentence level, RandomForest again achieved the highest F1 of 0.321; the proposed method obtained F1 of 0.312, ROC-AUC of 0.777, and PR-AUC of 0.245. Exact character-span localization remained difficult, with character-level F1 of 0.197 because sentence-level predictions often include supported text around shorter hallucinated spans. The findings indicate that evidence-chain features are useful for interpretable RAG auditing, but precise span extraction requires token-level sequence labeling or a comparable fine-grained model.

Zitieren

APA

Jin, J. (2025). Evidence-Chain Reliable RAG: Hallucination Detection, Source Attribution, and Deterministic Provenance Explanations. Journal of Technology Informatics and Engineering, 4(2), 520-533. https://doi.org/10.51903/jtie.v4i2.535

Vancouver

Jin J. Evidence-Chain Reliable RAG: Hallucination Detection, Source Attribution, and Deterministic Provenance Explanations. Journal of Technology Informatics and Engineering. 2025;4(2):520-533. doi:10.51903/jtie.v4i2.535

BibTeX
@article{Jin2025,
  author = {Jin, Jiaying},
  title = {Evidence-Chain Reliable RAG: Hallucination Detection, Source Attribution, and Deterministic Provenance Explanations},
  journal = {Journal of Technology Informatics and Engineering},
  year = {2025},
  volume = {4},
  number = {2},
  pages = {520--533},
  doi = {10.51903/jtie.v4i2.535}
}
Kostenlos starten →