EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Systematic Analysis of Retrieval-Augmented Generation-Based LLMs for Medical Chatbot Applications

Arunabh Bora, Heriberto Cuayáhuitl

Machine Learning and Knowledge Extraction · 18.10.2024 · 6 (4) · 2355-2374

DOI: 10.3390/make6040116 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 18.10.2024. Verfasst von Arunabh Bora und 1 weiteren Autor:innen. Die Arbeit wurde bisher 67-mal zitiert (Crossref) und nennt 47 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Artificial Intelligence (AI) has the potential to revolutionise the medical and healthcare sectors. AI and related technologies could significantly address some supply-and-demand challenges in the healthcare system, such as medical AI assistants, chatbots and robots. This paper focuses on tailoring LLMs to medical data utilising a Retrieval-Augmented Generation (RAG) database to evaluate their performance in a computationally resource-constrained environment. Existing studies primarily focus on fine-tuning LLMs on medical data, but this paper combines RAG and fine-tuned models and compares them against base models using RAG or only fine-tuning. Open-source LLMs (Flan-T5-Large, LLaMA-2-7B, and Mistral-7B) are fine-tuned using the medical datasets Meadow-MedQA and MedMCQA. Experiments are reported for response generation and multiple-choice question answering. The latter uses two distinct methodologies: Type A, as standard question answering via direct choice selection; and Type B, as language generation and probability confidence score generation of choices available. Results in the medical domain revealed that Fine-tuning and RAG are crucial for improved performance, and that methodology Type A outperforms Type B.

Zitieren

APA

Bora, A., & Cuayáhuitl, H. (2024). Systematic Analysis of Retrieval-Augmented Generation-Based LLMs for Medical Chatbot Applications. Machine Learning and Knowledge Extraction, 6(4), 2355-2374. https://doi.org/10.3390/make6040116

Vancouver

Bora A, Cuayáhuitl H. Systematic Analysis of Retrieval-Augmented Generation-Based LLMs for Medical Chatbot Applications. Machine Learning and Knowledge Extraction. 2024;6(4):2355-2374. doi:10.3390/make6040116

BibTeX
@article{Bora2024,
  author = {Bora, Arunabh and Cuayáhuitl, Heriberto},
  title = {Systematic Analysis of Retrieval-Augmented Generation-Based LLMs for Medical Chatbot Applications},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2024},
  volume = {6},
  number = {4},
  pages = {2355--2374},
  doi = {10.3390/make6040116}
}
Kostenlos starten →