Design and Performance Evaluation of LLM-Based RAG Pipelines for Chatbot Services in International Student Admissions
Electronics · 02.08.2025 · 14 (15) · 3095
DOI: 10.3390/electronics14153095 ↗Kurz gefasst
Fachartikel in der Zeitschrift Electronics, online erschienen am 02.08.2025. Verfasst von Maksuda Khasanova Zafar kizi und 1 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: MSIT (Ministry of Science, ICT), Republic of Korea.
Abstract (Originalsprache)
Recent advancements in large language models (LLMs) have significantly enhanced the effectiveness of Retrieval-Augmented Generation (RAG) systems. This study focuses on the development and evaluation of a domain-specific AI chatbot designed to support international student admissions by leveraging LLM-based RAG pipelines. We implement and compare multiple pipeline configurations, combining retrieval methods (e.g., Dense, MMR, Hybrid), chunking strategies (e.g., Semantic, Recursive), and both open-source and commercial LLMs. Dual evaluation datasets of LLM-generated and human-tagged QA sets are used to measure answer relevancy, faithfulness, context precision, and recall, alongside heuristic NLP metrics. Furthermore, latency analysis across different RAG stages is conducted to assess deployment feasibility in real-world educational environments. Results show that well-optimized open-source RAG pipelines can offer comparable performance to GPT-4o while maintaining scalability and cost-efficiency. These findings suggest that the proposed chatbot system can provide a practical and technically sound solution for international student services in resource-constrained academic institutions.
Förderung
- MSIT (Ministry of Science, ICT), Republic of Korea
Zitieren
APA
kizi, M. K. Z., & Suh, Y. (2025). Design and Performance Evaluation of LLM-Based RAG Pipelines for Chatbot Services in International Student Admissions. Electronics, 14(15), 3095. https://doi.org/10.3390/electronics14153095
Vancouver
kizi MKZ, Suh Y. Design and Performance Evaluation of LLM-Based RAG Pipelines for Chatbot Services in International Student Admissions. Electronics. 2025;14(15):3095. doi:10.3390/electronics14153095
BibTeX
@article{kizi2025,
author = {kizi, Maksuda Khasanova Zafar and Suh, Youngjung},
title = {Design and Performance Evaluation of LLM-Based RAG Pipelines for Chatbot Services in International Student Admissions},
journal = {Electronics},
year = {2025},
volume = {14},
number = {15},
pages = {3095},
doi = {10.3390/electronics14153095}
}
EUVIMED