EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Reproducible Evidence-Centric Evaluation of Multi-Hop Retrieval-Augmented QA on MuSiQue

Harry Wilson, Leo Carter

Artificial Intelligence and Machine Learning Review · 07.07.2025 · 6 (3) · 18-33

DOI: 10.69987/aimlr.2025.60302 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Artificial Intelligence and Machine Learning Review, online erschienen am 07.07.2025. Verfasst von Harry Wilson und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Retrieval-augmented generation (RAG) is a natural fit for multi-hop question answering (QA) because it can explicitly retrieve and aggregate evidence across passages. However, the tight coupling between retrieval, evidence selection, and answer prediction complicates evaluation: a high answer score can mask missing evidence, and strong evidence recall can still fail if the reader is distractor-sensitive. This paper presents a fully reproducible, evidence-centric experimental study of multi-hop RAG-style pipelines on MuSiQue-Ans v1.0, a benchmark designed to require multi-hop reasoning. Using the official MuSiQue-Ans development split (2,417 questions; 20 candidate passages per question; 2–4 annotated supporting passages), we measure (i) answer Exact Match (EM) and token-level F1, (ii) retrieval Hit@k and evidence Recall@k, and (iii) answer containment in retrieved contexts. We implement lexical retrievers (BM25 and TF-IDF) and a deterministic lexical reader (LexR) that extracts candidate answer spans from the most query-overlapping sentences. On MuSiQue dev with k=10, BM25 achieves 95.95% Hit@10 and 69.11% evidence Recall@10, while producing 1.78% EM and 3.82% F1. Oracle retrieval that returns only supporting passages raises EM/F1 to 3.68%/8.75%, quantifying a large reader bottleneck even under perfect evidence. Detailed ablations, curves, runtime measurements, and an error taxonomy show that distractor passages degrade the reader as k increases and that retrieval misses explain 36.04% of BM25 failures at k=10. All results reported in this manuscript are empirically measured from the dataset and generated by a fixed-parameter, seeded evaluation pipeline.

Zitieren

APA

Wilson, H., & Carter, L. (2025). Reproducible Evidence-Centric Evaluation of Multi-Hop Retrieval-Augmented QA on MuSiQue. Artificial Intelligence and Machine Learning Review, 6(3), 18-33. https://doi.org/10.69987/aimlr.2025.60302

Vancouver

Wilson H, Carter L. Reproducible Evidence-Centric Evaluation of Multi-Hop Retrieval-Augmented QA on MuSiQue. Artificial Intelligence and Machine Learning Review. 2025;6(3):18-33. doi:10.69987/aimlr.2025.60302

BibTeX
@article{Wilson2025,
  author = {Wilson, Harry and Carter, Leo},
  title = {Reproducible Evidence-Centric Evaluation of Multi-Hop Retrieval-Augmented QA on MuSiQue},
  journal = {Artificial Intelligence and Machine Learning Review},
  year = {2025},
  volume = {6},
  number = {3},
  pages = {18--33},
  doi = {10.69987/aimlr.2025.60302}
}
Kostenlos starten →