EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Context-Aware Retrieval Augmented Generation Using Similarity Validation to Handle Context Inconsistencies in Large Language Models

Enrico Collini, Felix Indra Kurniadi, Paolo Nesi, Gianni Pantaleo

IEEE Access · 2025 · 13 · 170065-170080

DOI: 10.1109/access.2025.3614553 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2025. Verfasst von Enrico Collini und 3 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 65 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Ministry University of Research (MUR), project CAI4DSA of FAIR PE1.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) have transformed natural language processing by offering human-like responses. However, issues such as incorrect information (hallucinations) and errors in specific subject areas remain, especially in Retrieval Augmented Generation (RAG) systems. This study introduces a Context-Aware Retrieval Augmented Generation (CA-RAG), which simplifies the process by removing the need to separately find relevant chunks of a document. Instead, after dividing the document into chunks, both the question and chunks are directly given to the LLMs to produce answers. The method then focuses on improving the answers through additional post-processing, aiming to reduce errors and make the answers more relevant to the question. To evaluate the effectiveness of CA-RAG, two scenarios have been designed. Scenario 1 involved experiments using widely adopted and recognized benchmark datasets, such as TriviaQA, Natural Questions, AmbigQA and Stanford Question Answering Dataset (SQuAD). In this context, the proposed CA-RAG method, combined with similarity measure (either cosine similarity or dot product) between generated answers and chunks, achieved the highest F1-score in TriviaQA and AmbigQA. Scenario 2 tested CA-RAG robustness using a custom dataset comprising domain-specific and unstructured documents. Results from automated and manual evaluations revealed that CA-RAG with post-processing consistently outperformed traditional RAG. These findings highlight the critical role of post-processing techniques and similarity measures in improving the accuracy and relevance of generated answers. CA-RAG demonstrates strong potential as a reliable and versatile solution for Retrieval Augmented Generation tasks across diverse datasets and domains.

Förderung

  • Ministry University of Research (MUR), project CAI4DSA of FAIR PE1

Zitieren

APA

Collini, E., Kurniadi, F. I., Nesi, P., & Pantaleo, G. (2025). Context-Aware Retrieval Augmented Generation Using Similarity Validation to Handle Context Inconsistencies in Large Language Models. IEEE Access, 13, 170065-170080. https://doi.org/10.1109/access.2025.3614553

Vancouver

Collini E, Kurniadi FI, Nesi P, Pantaleo G. Context-Aware Retrieval Augmented Generation Using Similarity Validation to Handle Context Inconsistencies in Large Language Models. IEEE Access. 2025;13:170065-170080. doi:10.1109/access.2025.3614553

BibTeX
@article{Collini2025,
  author = {Collini, Enrico and Kurniadi, Felix Indra and Nesi, Paolo and Pantaleo, Gianni},
  title = {Context-Aware Retrieval Augmented Generation Using Similarity Validation to Handle Context Inconsistencies in Large Language Models},
  journal = {IEEE Access},
  year = {2025},
  volume = {13},
  pages = {170065--170080},
  doi = {10.1109/access.2025.3614553}
}
Kostenlos starten →