EUVIMEDEUVIMED entdecken
Open Access

Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need

Cheng Peng iD, Xi Yang, Aokun Chen, Zehao Yu, Kaleb E Smith, Anthony B Costa, Mona G Flores, Jiang Bian iD, Yonghui Wu iD

Journal of the American Medical Informatics Association · 01.09.2024 · 31 (9) · 1892-1903

DOI: 10.1093/jamia/ocae078 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 17.04.2024. Verfasst von Cheng Peng und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL 32611, United States. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Ed and Ethel Moore Alzheimer's Disease Research Program 23A09, Patient-Centered Outcomes Research Institute, NIA NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To solve major clinical natural language processing (NLP) tasks using a unified text-to-text learning architecture based on a generative large language model (LLM) via prompt tuning.

Methods

We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM, GatorTronGPT, developed using GPT-3 architecture and trained with up to 20 billion parameters. We adopted soft prompts (ie, trainable vectors) with frozen LLM, where the LLM parameters were not updated (ie, frozen) and only the vectors of soft prompts were updated, known as prompt tuning. We added additional soft prompts as a prefix to the input layer, which were optimized during the prompt tuning. We evaluated the proposed method using 7 clinical NLP tasks and compared them with previous task-specific solutions based on Transformer models. Results and Conclusion The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM. Our approach outperformed previous task-specific transformer models by ∼3% for concept extraction and 7% for relation extraction applied to social determinants of health, 3.4% for clinical concept normalization, 3.4%-10% for clinical abbreviation disambiguation, and 5.5%-9% for natural language inference. Our approach also outperformed a previously developed prompt-based machine reading comprehension (MRC) model, GatorTron-MRC, for clinical concept and relation extraction. The proposed approach can deliver the “one model for all” promise from training to deployment using a unified generative LLM.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingElectronic Health RecordsHumansMachine Learning

Stichwörter der Autor:innen

clinical natural language processinglarge language modelprompt tuningtext generationtransformer model

Autor:innen und Institutionen

  • Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL 32611, United States.
  • Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL 32610, United States.
  • NVIDIA, Santa Clara, CA 95051, United States.

ORCID-Profile: Cheng Peng, Jiang Bian, Yonghui Wu

Förderung

  • Ed and Ethel Moore Alzheimer's Disease Research Program 23A09
  • Patient-Centered Outcomes Research Institute
  • NIA NIH HHS
  • NCCDPHP CDC HHS
  • NHLBI NIH HHS
  • NIAID NIH HHS
  • UF Clinical and Translational Science Institute Biomedical Informatics Program
  • UF Health Cancer Center

Zitieren

APA

Peng, C., Yang, X., Chen, A., Yu, Z., Smith, K. E., Costa, A. B., Flores, M. G., Bian, J., & Wu, Y. (2024). Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need. Journal of the American Medical Informatics Association, 31(9), 1892-1903. https://doi.org/10.1093/jamia/ocae078

Vancouver

Peng C, Yang X, Chen A, Yu Z, Smith KE, Costa AB, et al. Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need. Journal of the American Medical Informatics Association. 2024;31(9):1892-1903. doi:10.1093/jamia/ocae078

BibTeX
@article{Peng2024,
  author = {Peng, Cheng and Yang, Xi and Chen, Aokun and Yu, Zehao and Smith, Kaleb E and Costa, Anthony B and Flores, Mona G and Bian, Jiang and Wu, Yonghui},
  title = {Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  volume = {31},
  number = {9},
  pages = {1892--1903},
  doi = {10.1093/jamia/ocae078}
}
Kostenlos starten →