Development of a Retrieval-Augmented Generation Chatbot for Academic Regulation Information Services
bit-Tech · 10.08.2026 · 9 (1) · 318-329
DOI: 10.32877/bt.v9i1.3863 ↗Kurz gefasst
Fachartikel in der Zeitschrift bit-Tech, online erschienen am 10.08.2026. Verfasst von Aisma Nurlaili und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
Academic information services in higher education institutions still face challenges because important information is often distributed through static, lengthy, and difficult-to-navigate PDF regulatory documents. This condition makes it difficult for students to obtain specific academic information efficiently and may increase repetitive inquiries directed to academic administrative staff. This study aims to develop and evaluate a Retrieval-Augmented Generation (RAG)-based chatbot for academic regulation information services using a locally deployed Large Language Model (LLM). The main contribution of this study lies in the implementation of a local RAG architecture for hierarchical academic regulation documents by combining hybrid chunking, hybrid retrieval, reranking, and evaluation using retrieval metrics and RAGAS. The proposed method includes document preprocessing, document segmentation with variations in chunk size and chunk overlap, embedding generation, storage in ChromaDB, hybrid retrieval combining semantic and lexical search, and integration of Gemma 3:4B through Ollama. System evaluation was conducted through six experimental scenarios with different retrieval configurations. Retrieval performance was measured using Precision, Recall, and Mean Reciprocal Rank (MRR), while response quality was evaluated using Faithfulness, Answer Relevancy, Context Precision, and Context Recall. The best configuration was achieved with Top-K 10, chunk size 1024, and chunk overlap 400, producing scores of 0.7950, 0.7868, 0.8093, and 0.8958, respectively. These results indicate promising feasibility for supporting document-based academic regulation information services, although broader validation is still required due to the limited evaluation dataset and restricted document scope.
Zitieren
APA
Nurlaili, A., Afandi, M. I., & Fitri, A. S. (2026). Development of a Retrieval-Augmented Generation Chatbot for Academic Regulation Information Services. bit-Tech, 9(1), 318-329. https://doi.org/10.32877/bt.v9i1.3863
Vancouver
Nurlaili A, Afandi MI, Fitri AS. Development of a Retrieval-Augmented Generation Chatbot for Academic Regulation Information Services. bit-Tech. 2026;9(1):318-329. doi:10.32877/bt.v9i1.3863
BibTeX
@article{Nurlaili2026,
author = {Nurlaili, Aisma and Afandi, Mohamad Irwan and Fitri, Anindo Saka},
title = {Development of a Retrieval-Augmented Generation Chatbot for Academic Regulation Information Services},
journal = {bit-Tech},
year = {2026},
volume = {9},
number = {1},
pages = {318--329},
doi = {10.32877/bt.v9i1.3863}
}
EUVIMED