EUVIMEDEUVIMED entdecken
Open Access

BioInstruct: instruction tuning of large language models for biomedical natural language processing

Hieu Tran, Zhichao Yang, Zonghai Yao iD, Hong Yu

Journal of the American Medical Informatics Association · 01.09.2024 · 31 (9) · 1821-1832

DOI: 10.1093/jamia/ocae122 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 04.06.2024. Verfasst von Hieu Tran und 3 weiteren Autor:innen. Beteiligte Einrichtung: Manning College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA 01003, United States. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 75-mal zitiert (Crossref) und nennt 61 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: US Department of Veterans Affairs Homeless Programs Office, NIMH NIH HHS, NIDA NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

To enhance the performance of large language models (LLMs) in biomedical natural language processing (BioNLP) by introducing a domain-specific instruction dataset and examining its impact when combined with multi-task learning principles. Materials and Methods We created the BioInstruct, comprising 25 005 instructions to instruction-tune LLMs (LLaMA 1 and 2, 7B and 13B version). The instructions were created by prompting the GPT-4 language model with 3-seed samples randomly drawn from an 80 human curated instructions. We employed Low-Rank Adaptation (LoRA) for parameter-efficient fine-tuning. We then evaluated these instruction-tuned LLMs on several BioNLP tasks, which can be grouped into 3 major categories: question answering (QA), information extraction (IE), and text generation (GEN). We also examined whether categories (eg, QA, IE, and generation) of instructions impact model performance. Results and Discussion Comparing with LLMs without instruction-tuned, our instruction-tuned LLMs demonstrated marked performance gains: 17.3% in QA on average accuracy metric, 5.7% in IE on average F1 metric, and 96% in Generation tasks on average GPT-4 score metric. Our 7B-parameter instruction-tuned LLaMA 1 model was competitive or even surpassed other LLMs in the biomedical domain that were also fine-tuned from LLaMA 1 with vast domain-specific data or a variety of tasks. Our results also show that the performance gain is significantly higher when instruction fine-tuning is conducted with closely related tasks. Our findings align with the observations of multi-task learning, suggesting the synergies between 2 tasks.

Conclusion

The BioInstruct dataset serves as a valuable resource and instruction tuned LLMs lead to the best performing BioNLP applications.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language Processing

Stichwörter der Autor:innen

information extractioninstruction tuninglarge language modelsmulti-task learningnatural language inferencequestion answeringtext generation

Autor:innen und Institutionen

  • Manning College of Information and Computer Sciences, University of Massachusetts Amherst, Amherst, MA 01003, United States.
  • Department of Medicine, University of Massachusetts Medical School, Worcester, MA 01655, United States.
  • Center for Biomedical and Health Research in Data Sciences, Miner School of Computer and Information Sciences, University of Massachusetts Lowell, Lowell, MA 01854, United States.
  • Center for Healthcare Organization and Implementation Research, VA Bedford Health Care, Bedford, MA 01730, United States.

ORCID-Profile: Zonghai Yao

Förderung

  • US Department of Veterans Affairs Homeless Programs Office
  • NIMH NIH HHS
  • NIDA NIH HHS
  • NINR NIH HHS
  • VA
  • NIH
  • HSRD VA
  • NIA NIH HHS

Zitieren

APA

Tran, H., Yang, Z., Yao, Z., & Yu, H. (2024). BioInstruct: instruction tuning of large language models for biomedical natural language processing. Journal of the American Medical Informatics Association, 31(9), 1821-1832. https://doi.org/10.1093/jamia/ocae122

Vancouver

Tran H, Yang Z, Yao Z, Yu H. BioInstruct: instruction tuning of large language models for biomedical natural language processing. Journal of the American Medical Informatics Association. 2024;31(9):1821-1832. doi:10.1093/jamia/ocae122

BibTeX
@article{Tran2024,
  author = {Tran, Hieu and Yang, Zhichao and Yao, Zonghai and Yu, Hong},
  title = {BioInstruct: instruction tuning of large language models for biomedical natural language processing},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  volume = {31},
  number = {9},
  pages = {1821--1832},
  doi = {10.1093/jamia/ocae122}
}
Kostenlos starten →