EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Retrieval-Augmented Generation vs. Baseline LLMs: A Multi-Metric Evaluation for Knowledge-Intensive Content

Aparna Vinayan Kozhipuram, Samar Shailendra, Rajan Kadel

Information · 04.09.2025 · 16 (9) · 766

DOI: 10.3390/info16090766 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information, online erschienen am 04.09.2025. Verfasst von Aparna Vinayan Kozhipuram und 2 weiteren Autor:innen. Die Arbeit wurde bisher 18-mal zitiert (Crossref) und nennt 54 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

(1)

Background

The development of Generative Artificial Intelligence (GenAI) is transforming knowledge-intensive domains such as Education. However, Large Language Models (LLMs), which serve as the foundational components for GenAI tools, are trained on static datasets, often producing misleading, factually incorrect, or outdated responses. Our study explores the performance gains of Retrieval-Augmented LLMs over baseline LLMs while also identifying the trade-off opportunity between smaller-parameter LLMs augmented with user-specific data to larger parameter LLMs. (2)

Methods

We experimented with four different LLMs, each with a different number of parameters, to generate outputs. These outputs were then evaluated across seven lexical and semantic metrics to identify performance trends in Retrieval-Augmented Generation (RAG)-Augmented LLMs and analyze the impact of parameter size on LLM performance. (3) Results and Discussions: We have synthesized 968 different combinations to identify this trend with the help of different LLM sizes/parameters: TinyLlama 1.1B, Mistral 7B, Llama 3.1 8B, and Llama 1 13 B. These studies were grouped into two themes: RAG-Augmented LLM percentage improvements to baseline LLMs and compelling trade-off possibilities of RAG-Augmented smaller-parameter LLMs to larger-parameter LLMs. Our experiments show that RAG-Augmented LLMs demonstrate high lexical and semantic scores relative to baseline LLMs. This offers RAG-Augmented LLMs as a compelling trade-off for reducing the number of parameters in LLMs and lowering overall resource demands. (4)

Conclusions

The findings outline that by leveraging RAG-Augmented LLMs, smaller-parameter LLMs can perform better or equivalently to large-parameter LLMs, particularly demonstrating strong lexical improvements. They reduce the risks of hallucination and keep the output more contextualized, making them a better choice for knowledge-intensive content in academic and research sectors.

Zitieren

APA

Kozhipuram, A. V., Shailendra, S., & Kadel, R. (2025). Retrieval-Augmented Generation vs. Baseline LLMs: A Multi-Metric Evaluation for Knowledge-Intensive Content. Information, 16(9), 766. https://doi.org/10.3390/info16090766

Vancouver

Kozhipuram AV, Shailendra S, Kadel R. Retrieval-Augmented Generation vs. Baseline LLMs: A Multi-Metric Evaluation for Knowledge-Intensive Content. Information. 2025;16(9):766. doi:10.3390/info16090766

BibTeX
@article{Kozhipuram2025,
  author = {Kozhipuram, Aparna Vinayan and Shailendra, Samar and Kadel, Rajan},
  title = {Retrieval-Augmented Generation vs. Baseline LLMs: A Multi-Metric Evaluation for Knowledge-Intensive Content},
  journal = {Information},
  year = {2025},
  volume = {16},
  number = {9},
  pages = {766},
  doi = {10.3390/info16090766}
}
Kostenlos starten →