EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Design and Performance Evaluation of LLM-Based RAG Pipelines for Chatbot Services in International Student Admissions

Maksuda Khasanova Zafar kizi, Youngjung Suh

Electronics · 02.08.2025 · 14 (15) · 3095

DOI: 10.3390/electronics14153095 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 02.08.2025. Verfasst von Maksuda Khasanova Zafar kizi und 1 weiteren Autor:innen. Die Arbeit wurde bisher 15-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: MSIT (Ministry of Science, ICT), Republic of Korea.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent advancements in large language models (LLMs) have significantly enhanced the effectiveness of Retrieval-Augmented Generation (RAG) systems. This study focuses on the development and evaluation of a domain-specific AI chatbot designed to support international student admissions by leveraging LLM-based RAG pipelines. We implement and compare multiple pipeline configurations, combining retrieval methods (e.g., Dense, MMR, Hybrid), chunking strategies (e.g., Semantic, Recursive), and both open-source and commercial LLMs. Dual evaluation datasets of LLM-generated and human-tagged QA sets are used to measure answer relevancy, faithfulness, context precision, and recall, alongside heuristic NLP metrics. Furthermore, latency analysis across different RAG stages is conducted to assess deployment feasibility in real-world educational environments. Results show that well-optimized open-source RAG pipelines can offer comparable performance to GPT-4o while maintaining scalability and cost-efficiency. These findings suggest that the proposed chatbot system can provide a practical and technically sound solution for international student services in resource-constrained academic institutions.

Förderung

  • MSIT (Ministry of Science, ICT), Republic of Korea

Zitieren

APA

kizi, M. K. Z., & Suh, Y. (2025). Design and Performance Evaluation of LLM-Based RAG Pipelines for Chatbot Services in International Student Admissions. Electronics, 14(15), 3095. https://doi.org/10.3390/electronics14153095

Vancouver

kizi MKZ, Suh Y. Design and Performance Evaluation of LLM-Based RAG Pipelines for Chatbot Services in International Student Admissions. Electronics. 2025;14(15):3095. doi:10.3390/electronics14153095

BibTeX
@article{kizi2025,
  author = {kizi, Maksuda Khasanova Zafar and Suh, Youngjung},
  title = {Design and Performance Evaluation of LLM-Based RAG Pipelines for Chatbot Services in International Student Admissions},
  journal = {Electronics},
  year = {2025},
  volume = {14},
  number = {15},
  pages = {3095},
  doi = {10.3390/electronics14153095}
}
Kostenlos starten →