EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

An LLaMA 3.1-Based Chatbot with Retrieval-Augmented Generation (RAG) for Academic Services at UPN “Veteran” Yogyakarta

Farel Prayanto, Rifki Perwira

Digital System and Computing · 17.06.2026 · 2 (1) · 19-26

DOI: 10.58920/dsc0201633 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Digital System and Computing, online erschienen am 17.06.2026. Verfasst von Farel Prayanto und 1 weiteren Autor:innen. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

While universities heavily rely on digital information systems, static websites and manual administrative communication often limit accessibility and responsiveness for students seeking academic information. To address this, this study developed and evaluated an academic chatbot using the LLaMA 3.1 large language model integrated with a Retrieval-Augmented Generation (RAG) framework for Informatics students at Universitas Pembangunan Nasional “Veteran” Yogyakarta. Employing a Rapid Application Development approach, 263 institutional document chunks were processed to construct a knowledge base for a hybrid retrieval pipeline that combines BM25 lexical search and semantic vector similarity. The proposed system was comprehensively benchmarked against standalone lexical-only and semantic-only baselines using both RAG-specific and natural language generation (NLG) metrics. Experimental results demonstrated that the hybrid strategy achieved the highest answer faithfulness (0.712) and context recall (0.895), representing a 29.5% and 32.8% improvement in faithfulness over the respective standalone baselines, thereby ensuring superior factual consistency. Furthermore, the hybrid system recorded a Token F1 Score of 0.499, a BLEU score of 0.233, and a faster average response time of 7.64 seconds due to parallel query execution and context-size optimization. Finally, exploratory user evaluation yielded high satisfaction with an overall score of 4.46 out of 5.00, confirming its viability for real-world academic assistance.

Zitieren

APA

Prayanto, F., & Perwira, R. (2026). An LLaMA 3.1-Based Chatbot with Retrieval-Augmented Generation (RAG) for Academic Services at UPN “Veteran” Yogyakarta. Digital System and Computing, 2(1), 19-26. https://doi.org/10.58920/dsc0201633

Vancouver

Prayanto F, Perwira R. An LLaMA 3.1-Based Chatbot with Retrieval-Augmented Generation (RAG) for Academic Services at UPN “Veteran” Yogyakarta. Digital System and Computing. 2026;2(1):19-26. doi:10.58920/dsc0201633

BibTeX
@article{Prayanto2026,
  author = {Prayanto, Farel and Perwira, Rifki},
  title = {An LLaMA 3.1-Based Chatbot with Retrieval-Augmented Generation (RAG) for Academic Services at UPN “Veteran” Yogyakarta},
  journal = {Digital System and Computing},
  year = {2026},
  volume = {2},
  number = {1},
  pages = {19--26},
  doi = {10.58920/dsc0201633}
}
Kostenlos starten →