DEVELOPMENT OF A RAG-BASED SCHOLARLY RESEARCH CHATBOT WITH FINE-TUNED DENSE RETRIEVAL
Nigerian Journal of Technology · 22.07.2026 · 45 (2) · 401-410
DOI: 10.67358/njt.2026.5975 ↗Kurz gefasst
Fachartikel in der Zeitschrift Nigerian Journal of Technology, online erschienen am 22.07.2026. Verfasst von Abiodun Oguntimilehin und 1 weiteren Autor:innen. Die Arbeit nennt 18 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Foundation models, particularly Large Language Models (LLMs), show promise for chatbots for scholarly research chatbots but suffer from “hallucinations” and limited domain knowledge. This work addresses these reliability issues by developing a Retrieval-Augmented Generation (RAG) chatbot designed explicitly for scholarly research interactions within the Natural Language Processing (NLP) domain. The system utilizes a domain-specific corpus from the Association of Computational Linguistics (ACL) Anthology. The corpus was processed into text chunks and indexed using a high-dimensional vector database to facilitate efficient information retrieval. The generation component comprises an instruct-tuned LLM; Mixtral 8x7b, which synthesizes the retrieved context into coherent, evidence-based responses while maintaining the original semantic integrity of the scholarly source. Retrieval is performed using semantic similarity search. To improve dense retrieval precision, the BAAI/bge-large Sentence Transformer model was fine-tuned on a synthetic dataset generated from the corpus. Results show that fine-tuning and integrating the BAAI/bge-large model significantly improves the chatbot's ability to retrieve relevant information. Compared to the base model, retrieval accuracy of the fine-tuned model increases by up to 15% across different metrics: Accuracy@k, Precision@k, Recall@k, where k represents the count of retrieved chunks considered for evaluation, and Mean Reciprocal Rank (MRR@10). Notably, accuracy reaches an impressive 97%, demonstrating a significant boost in retrieving the most relevant scholarly information for user queries. These findings underscore the efficacy of fine-tuned Retrieval-Augmented Generation (RAG) systems in developing reliable, grounded chatbots for academic environments.
Zitieren
APA
Oguntimilehin, A., & BALOGUN, O. (2026). DEVELOPMENT OF A RAG-BASED SCHOLARLY RESEARCH CHATBOT WITH FINE-TUNED DENSE RETRIEVAL. Nigerian Journal of Technology, 45(2), 401-410. https://doi.org/10.67358/njt.2026.5975
Vancouver
Oguntimilehin A, BALOGUN O. DEVELOPMENT OF A RAG-BASED SCHOLARLY RESEARCH CHATBOT WITH FINE-TUNED DENSE RETRIEVAL. Nigerian Journal of Technology. 2026;45(2):401-410. doi:10.67358/njt.2026.5975
BibTeX
@article{Oguntimilehin2026,
author = {Oguntimilehin, Abiodun and BALOGUN, OLAMIDE},
title = {DEVELOPMENT OF A RAG-BASED SCHOLARLY RESEARCH CHATBOT WITH FINE-TUNED DENSE RETRIEVAL},
journal = {Nigerian Journal of Technology},
year = {2026},
volume = {45},
number = {2},
pages = {401--410},
doi = {10.67358/njt.2026.5975}
}
EUVIMED