EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Specializing LLMs to Low-Documented Domains with RAG: An Analysis Across Models and Retrieval Depths

Jacopo Mereu, Alessandro Carcangiu, Valentino Artizzu, Federico Maria Cau, Lucio Davide Spano

SN Computer Science · 28.03.2026 · 7 (4)

DOI: 10.1007/s42979-026-04844-6 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SN Computer Science, online erschienen am 28.03.2026. Verfasst von Jacopo Mereu und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Università degli Studi di Cagliari.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large Language Models (LLMs) are increasingly used to support technical tasks such as software development. However, they often struggle in low-documented or fast-evolving domains, where missing training data leads to inaccurate or incomplete responses. This paper presents a reproducible pipeline based on Retrieval-Augmented Generation to specialize LLMs for such domains by integrating curated external knowledge. We detail a systematic process to build a high-quality Q&A dataset from public instructional sources and developer forums and apply it to the Unity XR Interaction Toolkit (XRIv2) as a case study. We construct a domain-specific benchmark of 101 question-answer pairs based on real learning resources and evaluate five open and proprietary LLMs (GPT-3.5-Turbo, GPT-4o Mini, LLaMA2, LLaMA3, and Mistral) under varying retrieval settings. Results show that standard automatic metrics (e.g., METEOR) struggle to detect quality differences, while LLM-as-a-Judge evaluations reveal significant model-specific improvements as more documents are retrieved. Our findings offer practical guidance for tuning retrieval strategies and highlight the potential for generalizing this approach to other technical domains requiring targeted LLM specialization.

Förderung

  • Università degli Studi di Cagliari

Zitieren

APA

Mereu, J., Carcangiu, A., Artizzu, V., Cau, F. M., & Spano, L. D. (2026). Specializing LLMs to Low-Documented Domains with RAG: An Analysis Across Models and Retrieval Depths. SN Computer Science, 7(4). https://doi.org/10.1007/s42979-026-04844-6

Vancouver

Mereu J, Carcangiu A, Artizzu V, Cau FM, Spano LD. Specializing LLMs to Low-Documented Domains with RAG: An Analysis Across Models and Retrieval Depths. SN Computer Science. 2026;7(4). doi:10.1007/s42979-026-04844-6

BibTeX
@article{Mereu2026,
  author = {Mereu, Jacopo and Carcangiu, Alessandro and Artizzu, Valentino and Cau, Federico Maria and Spano, Lucio Davide},
  title = {Specializing LLMs to Low-Documented Domains with RAG: An Analysis Across Models and Retrieval Depths},
  journal = {SN Computer Science},
  year = {2026},
  volume = {7},
  number = {4},
  doi = {10.1007/s42979-026-04844-6}
}
Kostenlos starten →