A RAG-Based Academic Information Chatbot Using Lightweight LLaMA and Indo-Sencence-BERT
Indonesian Journal of Artificial Intelligence and Data Mining · 10.12.2025 · 8 (3) · 687-696
DOI: 10.24014/ijaidm.v8i3.38150 ↗Kurz gefasst
Fachartikel in der Zeitschrift Indonesian Journal of Artificial Intelligence and Data Mining, online erschienen am 10.12.2025. Verfasst von Muhamad Saman und 3 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben.
Abstract (Originalsprache)
In the current digital era, Institut Teknologi Kalimantan (ITK) encounters challenges in delivering academic information that is fast, accurate, and easily accessible to students, lecturers, and academic staff. Access to important information such as administrative procedures, report writing guidelines, and academic policies remains largely reliant on manual systems and static handbooks. To address this issue, this study investigates a chatbot system utilizing the Retrieval-Augmented Generation (RAG) framework through LLaMA model. The chatbot combines semantic retrieval and natural language generation to provide relevant and accurate answers based on existing academic documents. Evaluation was conducted on two lightweight LlaMA models: 1.5 and 3B parameters. Furthermore, different embedding vector also evaluated along with Indo-Sentence-BERT as well as the chunking size. The most optimal configuration was achieved using LLaMA 3B as the generative model and Indo-Sentence-BERT as the retriever, with a chunk size of 200 tokens and an overlap of 10 tokens. This setup achieved a RAGAS score of approximately 0.9, a competitive MRR of 0.5, and response latency under 1 second. Although LLaMA 1B recorded a higher MRR (0.6), its low RAGAS score made it less favorable. Overall, the LLaMA 3B and Indo-Sentence-BERT configuration is recommended to enhance the efficiency of academic information retrieval at ITK.
Zitieren
APA
Saman, M., Alfarisy, G. A. F., Amelia, R., & Fadhliana, N. R. (2025). A RAG-Based Academic Information Chatbot Using Lightweight LLaMA and Indo-Sencence-BERT. Indonesian Journal of Artificial Intelligence and Data Mining, 8(3), 687-696. https://doi.org/10.24014/ijaidm.v8i3.38150
Vancouver
Saman M, Alfarisy GAF, Amelia R, Fadhliana NR. A RAG-Based Academic Information Chatbot Using Lightweight LLaMA and Indo-Sencence-BERT. Indonesian Journal of Artificial Intelligence and Data Mining. 2025;8(3):687-696. doi:10.24014/ijaidm.v8i3.38150
BibTeX
@article{Saman2025,
author = {Saman, Muhamad and Alfarisy, Gusti Ahmad Fanshuri and Amelia, Rizky and Fadhliana, Nisa Rizqiza},
title = {A RAG-Based Academic Information Chatbot Using Lightweight LLaMA and Indo-Sencence-BERT},
journal = {Indonesian Journal of Artificial Intelligence and Data Mining},
year = {2025},
volume = {8},
number = {3},
pages = {687--696},
doi = {10.24014/ijaidm.v8i3.38150}
}
EUVIMED