EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Federated knowledge retrieval elevates large language model performance on biomedical benchmarks

Janet Joy iD, Andrew I Su iD

GigaScience · 2026

DOI: 10.1093/gigascience/giag007 ↗

Kurz gefasst

Fachartikel in der Zeitschrift GigaScience, 2026. Verfasst von Janet Joy und 1 weiteren Autor:innen. Beteiligte Einrichtung: Department of Integrative Structural and Computational Biology, Scripps Research, 10550 N Torrey Pines Rd, La Jolla, CA, 92037, USA. Hauptthemen laut MEDLINE: Large Language Models, Biomedical Research, Natural Language Processing. Die Arbeit nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: NCATS NIH HHS, NIA NIH HHS, Scripps Research Translational Institute.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) have significantly advanced natural language processing in biomedical research; however, their reliance on implicit, statistical representations often results in factual inaccuracies or hallucinations, posing significant concerns in high-stakes biomedical contexts.

Results

To overcome these limitations, we developed BioThings Explorer-Retrieval-Augmented Generation (BTE-RAG), a Retrieval-Augmented Generation framework that integrates the reasoning capabilities of advanced language models with explicit mechanistic evidence sourced from BTE, an API federation of more than sixty authoritative biomedical knowledge sources. We systematically evaluated BTE-RAG in comparison to traditional LLM-only methods across three benchmark datasets that we created from DrugMechDB. These datasets specifically targeted gene-centric mechanisms (798 questions), metabolite effects (201 questions), and drug–biological process relationships (842 questions). On the gene-centric task, BTE-RAG increased accuracy from 51 to 75.8% for GPT-4o mini and from 69.8 to 78.6% for GPT-4o. In metabolite-focused questions, the proportion of responses with cosine similarity scores of at least 0.90 rose by 82% for GPT-4o mini and 77% for GPT-4o. While overall accuracy was consistent in the drug–biological process benchmark, the retrieval method enhanced response concordance, producing a greater than 10% increase in high-agreement answers (from 129 to 144) using GPT-4o. We additionally evaluated BTE-RAG alongside GeneGPT-based models on the GeneTuring gene–disease association benchmark and on our mechanistic gene benchmark, demonstrating that the BTE-RAG layer consistently improves accuracy relative to alternative approaches.

Conclusion

Federated knowledge retrieval provides transparent improvements in accuracy for LLMs, establishing BTE-RAG as a valuable and practical tool for mechanistic exploration and translational biomedical research.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsBiomedical ResearchNatural Language ProcessingBenchmarkingHumans

Stichwörter der Autor:innen

BioThings ExplorerDrugMechDB benchmarkingRetrieval-augmented generation (RAG)biomedical knowledge graphsfederated knowledge retrievalhallucination mitigationlarge language models (LLMs)mechanistic reasoning

Autor:innen und Institutionen

  • Department of Integrative Structural and Computational Biology, Scripps Research, 10550 N Torrey Pines Rd, La Jolla, CA, 92037, USA.

ORCID-Profile: Janet Joy, Andrew I Su

Förderung

  • NCATS NIH HHS
  • NIA NIH HHS
  • Scripps Research Translational Institute
  • National Institute on Aging
  • National Center for Advancing Translational Sciences

Zitieren

APA

Joy, J., & Su, A. I. (2026). Federated knowledge retrieval elevates large language model performance on biomedical benchmarks. GigaScience. https://doi.org/10.1093/gigascience/giag007

Vancouver

Joy J, Su AI. Federated knowledge retrieval elevates large language model performance on biomedical benchmarks. GigaScience. 2026. doi:10.1093/gigascience/giag007

BibTeX
@article{Joy2026,
  author = {Joy, Janet and Su, Andrew I},
  title = {Federated knowledge retrieval elevates large language model performance on biomedical benchmarks},
  journal = {GigaScience},
  year = {2026},
  doi = {10.1093/gigascience/giag007}
}
Kostenlos starten →