Improving medical reasoning through retrieval and self-reflection with retrieval-augmented large language models
Bioinformatics · 28.06.2024 · 40 (Supplement_1) · i119-i129
DOI: 10.1093/bioinformatics/btae238 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 28.06.2024. Verfasst von Minbyul Jeong und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, Korea University, Seoul 02841, Republic of Korea. Hauptthemen laut MEDLINE: Information Storage and Retrieval. Die Arbeit wurde bisher 137-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Research Foundation of Korea, Ministry of Health & Welfare, Republic of Korea, Ministry of Science and ICT.
Abstract (Originalsprache)
Summary Recent proprietary large language models (LLMs), such as GPT-4, have achieved a milestone in tackling diverse challenges in the biomedical domain, ranging from multiple-choice questions to long-form generations. To address challenges that still cannot be handled with the encoded knowledge of LLMs, various retrieval-augmented generation (RAG) methods have been developed by searching documents from the knowledge corpus and appending them unconditionally or selectively to the input of LLMs for generation. However, when applying existing methods to different domain-specific problems, poor generalization becomes apparent, leading to fetching incorrect documents or making inaccurate judgments. In this paper, we introduce Self-BioRAG, a framework reliable for biomedical text that specializes in generating explanations, retrieving domain-specific documents, and self-reflecting generated responses. We utilize 84k filtered biomedical instruction sets to train Self-BioRAG that can assess its generated explanations with customized reflective tokens. Our work proves that domain-specific components, such as a retriever, domain-related document corpus, and instruction sets are necessary for adhering to domain-related instructions. Using three major medical question-answering benchmark datasets, experimental results of Self-BioRAG demonstrate significant performance gains by achieving a 7.2% absolute improvement on average over the state-of-the-art open-foundation model with a parameter size of 7B or less. Similarly, Self-BioRAG outperforms RAG by 8% Rouge-1 score in generating more proficient answers on two long-form question-answering benchmarks on average. Overall, we analyze that Self-BioRAG finds the clues in the question, retrieves relevant documents if needed, and understands how to answer with information from retrieved documents and encoded knowledge as a medical expert does. We release our data and code for training our framework components and model weights (7B and 13B) to enhance capabilities in biomedical and clinical domains. Availability and implementation Self-BioRAG is available at https://github.com/dmis-lab/self-biorag.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Computer Science, Korea University, Seoul 02841, Republic of Korea.
- Department of Software Convergence, School of Computing, Kyung Hee University, Republic of Korea.
- AIGEN Sciences, Seoul 04778, Republic of Korea.
ORCID-Profile: Minbyul Jeong, Jaewoo Kang
Förderung
- National Research Foundation of Korea
- Ministry of Health & Welfare, Republic of Korea
- Ministry of Science and ICT
- Kyung Hee University
- Institute of Information & Communications Technology Planning & Evaluation
- MSIT
Zitieren
APA
Jeong, M., Sohn, J., Sung, M., & Kang, J. (2024). Improving medical reasoning through retrieval and self-reflection with retrieval-augmented large language models. Bioinformatics, 40(Supplement_1), i119-i129. https://doi.org/10.1093/bioinformatics/btae238
Vancouver
Jeong M, Sohn J, Sung M, Kang J. Improving medical reasoning through retrieval and self-reflection with retrieval-augmented large language models. Bioinformatics. 2024;40(Supplement_1):i119-i129. doi:10.1093/bioinformatics/btae238
BibTeX
@article{Jeong2024,
author = {Jeong, Minbyul and Sohn, Jiwoong and Sung, Mujeen and Kang, Jaewoo},
title = {Improving medical reasoning through retrieval and self-reflection with retrieval-augmented large language models},
journal = {Bioinformatics},
year = {2024},
volume = {40},
number = {Supplement_1},
pages = {i119--i129},
doi = {10.1093/bioinformatics/btae238}
}
EUVIMED