Biomedical knowledge graph-optimized prompt generation for large language models
Bioinformatics · 02.09.2024 · 40 (9)
DOI: 10.1093/bioinformatics/btae560 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 17.09.2024. Verfasst von Karthik Soman und 13 weiteren Autor:innen. Beteiligte Einrichtung: Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA 94158, United States. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 152-mal zitiert (Crossref) und nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science Foundation, NCATS NIH HHS.
Abstract (Originalsprache)
Motivation Large language models (LLMs) are being adopted at an unprecedented rate, yet still face challenges in knowledge-intensive domains such as biomedicine. Solutions such as pretraining and domain-specific fine-tuning add substantial computational overhead, requiring further domain-expertise. Here, we introduce a token-optimized and robust Knowledge Graph-based Retrieval Augmented Generation (KG-RAG) framework by leveraging a massive biomedical KG (SPOKE) with LLMs such as Llama-2-13b, GPT-3.5-Turbo, and GPT-4, to generate meaningful biomedical text rooted in established knowledge. Results Compared to the existing RAG technique for Knowledge Graphs, the proposed method utilizes minimal graph schema for context extraction and uses embedding methods for context pruning. This optimization in context extraction results in more than 50% reduction in token consumption without compromising the accuracy, making a cost-effective and robust RAG implementation on proprietary LLMs. KG-RAG consistently enhanced the performance of LLMs across diverse biomedical prompts by generating responses rooted in established knowledge, accompanied by accurate provenance and statistical evidence (if available) to substantiate the claims. Further benchmarking on human curated datasets, such as biomedical true/false and multiple-choice questions (MCQ), showed a remarkable 71% boost in the performance of the Llama-2 model on the challenging MCQ dataset, demonstrating the framework’s capacity to empower open-source models with fewer parameters for domain-specific questions. Furthermore, KG-RAG enhanced the performance of proprietary GPT models, such as GPT-3.5 and GPT-4. In summary, the proposed framework combines explicit and implicit knowledge of KG and LLM in a token optimized fashion, thus enhancing the adaptability of general-purpose LLMs to tackle domain-specific questions in a cost-effective fashion. Availability and implementation SPOKE KG can be accessed at https://spoke.rbvi.ucsf.edu/neighborhood.html. It can also be accessed using REST-API (https://spoke.rbvi.ucsf.edu/swagger/). KG-RAG code is made available at https://github.com/BaranziniLab/KG_RAG. Biomedical benchmark datasets used in this study are made available to the research community in the same GitHub repository.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Department of Neurology, Weill Institute for Neurosciences, University of California, San Francisco, San Francisco, CA 94158, United States.
- San Diego Supercomputer Center, University of California, San Diego, CA 92093, United States.
- Department of Pharmaceutical Chemistry, School of Pharmacy, University of California, San Francisco, CA 94158, United States.
- Institute for Systems Biology, Seattle, WA 98109, United States.
- Bakar Computational Health Sciences Institute, University of California, San Francisco, CA 94158, United States.
- Mate Bioservices, Inc. Swallowtail Ct., Brisbane, CA 94005, United States.
ORCID-Profile: John H Morris, Sui Huang, Sergio E Baranzini
Förderung
- National Science Foundation
- NCATS NIH HHS
Zitieren
APA
Soman, K., Rose, P. W., Morris, J. H., Akbas, R. E., Smith, B., Peetoom, B., Villouta-Reyes, C., Cerono, G., Shi, Y., Rizk-Jackson, A., Israni, S., Nelson, C. A., Huang, S., & Baranzini, S. E. (2024). Biomedical knowledge graph-optimized prompt generation for large language models. Bioinformatics, 40(9). https://doi.org/10.1093/bioinformatics/btae560
Vancouver
Soman K, Rose PW, Morris JH, Akbas RE, Smith B, Peetoom B, et al. Biomedical knowledge graph-optimized prompt generation for large language models. Bioinformatics. 2024;40(9). doi:10.1093/bioinformatics/btae560
BibTeX
@article{Soman2024,
author = {Soman, Karthik and Rose, Peter W and Morris, John H and Akbas, Rabia E and Smith, Brett and Peetoom, Braian and Villouta-Reyes, Catalina and Cerono, Gabriel and Shi, Yongmei and Rizk-Jackson, Angela and Israni, Sharat and Nelson, Charlotte A and Huang, Sui and Baranzini, Sergio E},
title = {Biomedical knowledge graph-optimized prompt generation for large language models},
journal = {Bioinformatics},
year = {2024},
volume = {40},
number = {9},
doi = {10.1093/bioinformatics/btae560}
}
EUVIMED