EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

QA-RAG: Exploring LLM Reliance on External Knowledge

Aigerim Mansurova, Aiganym Mansurova, Aliya Nugumanova

Big Data and Cognitive Computing · 09.09.2024 · 8 (9) · 115

DOI: 10.3390/bdcc8090115 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 09.09.2024. Verfasst von Aigerim Mansurova und 2 weiteren Autor:innen. Die Arbeit wurde bisher 62-mal zitiert (Crossref) und nennt 39 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Ministry of Science and Higher Education of the Republic of Kazakhstan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) can store factual knowledge within their parameters and have achieved superior results in question-answering tasks. However, challenges persist in providing provenance for their decisions and keeping their knowledge up to date. Some approaches aim to address these challenges by combining external knowledge with parametric memory. In contrast, our proposed QA-RAG solution relies solely on the data stored within an external knowledge base, specifically a dense vector index database. In this paper, we compare RAG configurations using two LLMs—Llama 2b and 13b—systematically examining their performance in three key RAG capabilities: noise robustness, knowledge gap detection, and external truth integration. The evaluation reveals that while our approach achieves an accuracy of 83.3%, showcasing its effectiveness across all baselines, the model still struggles significantly in terms of external truth integration. These findings suggest that considerable work is still required to fully leverage RAG in question-answering tasks.

Förderung

  • Ministry of Science and Higher Education of the Republic of Kazakhstan

Zitieren

APA

Mansurova, A., Mansurova, A., & Nugumanova, A. (2024). QA-RAG: Exploring LLM Reliance on External Knowledge. Big Data and Cognitive Computing, 8(9), 115. https://doi.org/10.3390/bdcc8090115

Vancouver

Mansurova A, Mansurova A, Nugumanova A. QA-RAG: Exploring LLM Reliance on External Knowledge. Big Data and Cognitive Computing. 2024;8(9):115. doi:10.3390/bdcc8090115

BibTeX
@article{Mansurova2024,
  author = {Mansurova, Aigerim and Mansurova, Aiganym and Nugumanova, Aliya},
  title = {QA-RAG: Exploring LLM Reliance on External Knowledge},
  journal = {Big Data and Cognitive Computing},
  year = {2024},
  volume = {8},
  number = {9},
  pages = {115},
  doi = {10.3390/bdcc8090115}
}
Kostenlos starten →