EUVIMEDEUVIMED entdecken
Open Access

GeneGPT: augmenting large language models with domain tools for improved access to biomedical information

Qiao Jin iD, Yifan Yang, Qingyu Chen iD, Zhiyong Lu iD

Bioinformatics · 01.02.2024 · 40 (2)

DOI: 10.1093/bioinformatics/btae075 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 10.02.2024. Verfasst von Qiao Jin und 3 weiteren Autor:innen. Beteiligte Einrichtung: National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, MD 20894, United States. Hauptthemen laut MEDLINE: Algorithms. Die Arbeit wurde bisher 180-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NLM NIH HHS, NIH, National Library of Medicine.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation While large language models (LLMs) have been successfully applied to various tasks, they still face challenges with hallucinations. Augmenting LLMs with domain-specific tools such as database utilities can facilitate easier and more precise access to specialized knowledge. In this article, we present GeneGPT, a novel method for teaching LLMs to use the Web APIs of the National Center for Biotechnology Information (NCBI) for answering genomics questions. Specifically, we prompt Codex to solve the GeneTuring tests with NCBI Web APIs by in-context learning and an augmented decoding algorithm that can detect and execute API calls. Results Experimental results show that GeneGPT achieves state-of-the-art performance on eight tasks in the GeneTuring benchmark with an average score of 0.83, largely surpassing retrieval-augmented LLMs such as the new Bing (0.44), biomedical LLMs such as BioMedLM (0.08) and BioGPT (0.04), as well as GPT-3 (0.16) and ChatGPT (0.12). Our further analyses suggest that: First, API demonstrations have good cross-task generalizability and are more useful than documentations for in-context learning; second, GeneGPT can generalize to longer chains of API calls and answer multi-hop questions in GeneHop, a novel dataset introduced in this work; finally, different types of errors are enriched in different tasks, providing valuable insights for future improvements. Availability and implementation The GeneGPT code and data are publicly available at https://github.com/ncbi/GeneGPT.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

AlgorithmsDatabases, FactualDocumentationLanguage

Autor:innen und Institutionen

  • National Center for Biotechnology Information, National Library of Medicine, National Institutes of Health, Bethesda, MD 20894, United States.

ORCID-Profile: Qiao Jin, Qingyu Chen, Zhiyong Lu

Förderung

  • NLM NIH HHS
  • NIH
  • National Library of Medicine

Zitieren

APA

Jin, Q., Yang, Y., Chen, Q., & Lu, Z. (2024). GeneGPT: augmenting large language models with domain tools for improved access to biomedical information. Bioinformatics, 40(2). https://doi.org/10.1093/bioinformatics/btae075

Vancouver

Jin Q, Yang Y, Chen Q, Lu Z. GeneGPT: augmenting large language models with domain tools for improved access to biomedical information. Bioinformatics. 2024;40(2). doi:10.1093/bioinformatics/btae075

BibTeX
@article{Jin2024,
  author = {Jin, Qiao and Yang, Yifan and Chen, Qingyu and Lu, Zhiyong},
  title = {GeneGPT: augmenting large language models with domain tools for improved access to biomedical information},
  journal = {Bioinformatics},
  year = {2024},
  volume = {40},
  number = {2},
  doi = {10.1093/bioinformatics/btae075}
}
Kostenlos starten →