EUVIMEDEUVIMED entdecken
Crossref

A RAG-Based Academic Information Chatbot Using Lightweight LLaMA and Indo-Sencence-BERT

Muhamad Saman, Gusti Ahmad Fanshuri Alfarisy, Rizky Amelia, Nisa Rizqiza Fadhliana

Indonesian Journal of Artificial Intelligence and Data Mining · 10.12.2025 · 8 (3) · 687-696

DOI: 10.24014/ijaidm.v8i3.38150 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Indonesian Journal of Artificial Intelligence and Data Mining, online erschienen am 10.12.2025. Verfasst von Muhamad Saman und 3 weiteren Autor:innen. Die Arbeit nennt 25 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In the current digital era, Institut Teknologi Kalimantan (ITK) encounters challenges in delivering academic information that is fast, accurate, and easily accessible to students, lecturers, and academic staff. Access to important information such as administrative procedures, report writing guidelines, and academic policies remains largely reliant on manual systems and static handbooks. To address this issue, this study investigates a chatbot system utilizing the Retrieval-Augmented Generation (RAG) framework through LLaMA model. The chatbot combines semantic retrieval and natural language generation to provide relevant and accurate answers based on existing academic documents. Evaluation was conducted on two lightweight LlaMA models: 1.5 and 3B parameters. Furthermore, different embedding vector also evaluated along with Indo-Sentence-BERT as well as the chunking size. The most optimal configuration was achieved using LLaMA 3B as the generative model and Indo-Sentence-BERT as the retriever, with a chunk size of 200 tokens and an overlap of 10 tokens. This setup achieved a RAGAS score of approximately 0.9, a competitive MRR of 0.5, and response latency under 1 second. Although LLaMA 1B recorded a higher MRR (0.6), its low RAGAS score made it less favorable. Overall, the LLaMA 3B and Indo-Sentence-BERT configuration is recommended to enhance the efficiency of academic information retrieval at ITK.

Zitieren

APA

Saman, M., Alfarisy, G. A. F., Amelia, R., & Fadhliana, N. R. (2025). A RAG-Based Academic Information Chatbot Using Lightweight LLaMA and Indo-Sencence-BERT. Indonesian Journal of Artificial Intelligence and Data Mining, 8(3), 687-696. https://doi.org/10.24014/ijaidm.v8i3.38150

Vancouver

Saman M, Alfarisy GAF, Amelia R, Fadhliana NR. A RAG-Based Academic Information Chatbot Using Lightweight LLaMA and Indo-Sencence-BERT. Indonesian Journal of Artificial Intelligence and Data Mining. 2025;8(3):687-696. doi:10.24014/ijaidm.v8i3.38150

BibTeX
@article{Saman2025,
  author = {Saman, Muhamad and Alfarisy, Gusti Ahmad Fanshuri and Amelia, Rizky and Fadhliana, Nisa Rizqiza},
  title = {A RAG-Based Academic Information Chatbot Using Lightweight LLaMA and Indo-Sencence-BERT},
  journal = {Indonesian Journal of Artificial Intelligence and Data Mining},
  year = {2025},
  volume = {8},
  number = {3},
  pages = {687--696},
  doi = {10.24014/ijaidm.v8i3.38150}
}
Kostenlos starten →