Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need
Journal of the American Medical Informatics Association · 01.09.2024 · 31 (9) · 1892-1903
DOI: 10.1093/jamia/ocae078 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 17.04.2024. Verfasst von Cheng Peng und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL 32611, United States. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 35-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Ed and Ethel Moore Alzheimer's Disease Research Program 23A09, Patient-Centered Outcomes Research Institute, NIA NIH HHS.
Abstract (Originalsprache)
Objective
To solve major clinical natural language processing (NLP) tasks using a unified text-to-text learning architecture based on a generative large language model (LLM) via prompt tuning.
Methods
We formulated 7 key clinical NLP tasks as text-to-text learning and solved them using one unified generative clinical LLM, GatorTronGPT, developed using GPT-3 architecture and trained with up to 20 billion parameters. We adopted soft prompts (ie, trainable vectors) with frozen LLM, where the LLM parameters were not updated (ie, frozen) and only the vectors of soft prompts were updated, known as prompt tuning. We added additional soft prompts as a prefix to the input layer, which were optimized during the prompt tuning. We evaluated the proposed method using 7 clinical NLP tasks and compared them with previous task-specific solutions based on Transformer models. Results and Conclusion The proposed approach achieved state-of-the-art performance for 5 out of 7 major clinical NLP tasks using one unified generative LLM. Our approach outperformed previous task-specific transformer models by ∼3% for concept extraction and 7% for relation extraction applied to social determinants of health, 3.4% for clinical concept normalization, 3.4%-10% for clinical abbreviation disambiguation, and 5.5%-9% for natural language inference. Our approach also outperformed a previously developed prompt-based machine reading comprehension (MRC) model, GatorTron-MRC, for clinical concept and relation extraction. The proposed approach can deliver the “one model for all” promise from training to deployment using a unified generative LLM.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL 32611, United States.
- Cancer Informatics Shared Resource, University of Florida Health Cancer Center, Gainesville, FL 32610, United States.
- NVIDIA, Santa Clara, CA 95051, United States.
ORCID-Profile: Cheng Peng, Jiang Bian, Yonghui Wu
Förderung
- Ed and Ethel Moore Alzheimer's Disease Research Program 23A09
- Patient-Centered Outcomes Research Institute
- NIA NIH HHS
- NCCDPHP CDC HHS
- NHLBI NIH HHS
- NIAID NIH HHS
- UF Clinical and Translational Science Institute Biomedical Informatics Program
- UF Health Cancer Center
Zitieren
APA
Peng, C., Yang, X., Chen, A., Yu, Z., Smith, K. E., Costa, A. B., Flores, M. G., Bian, J., & Wu, Y. (2024). Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need. Journal of the American Medical Informatics Association, 31(9), 1892-1903. https://doi.org/10.1093/jamia/ocae078
Vancouver
Peng C, Yang X, Chen A, Yu Z, Smith KE, Costa AB, et al. Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need. Journal of the American Medical Informatics Association. 2024;31(9):1892-1903. doi:10.1093/jamia/ocae078
BibTeX
@article{Peng2024,
author = {Peng, Cheng and Yang, Xi and Chen, Aokun and Yu, Zehao and Smith, Kaleb E and Costa, Anthony B and Flores, Mona G and Bian, Jiang and Wu, Yonghui},
title = {Generative large language models are all-purpose text analytics engines: text-to-text learning is all your need},
journal = {Journal of the American Medical Informatics Association},
year = {2024},
volume = {31},
number = {9},
pages = {1892--1903},
doi = {10.1093/jamia/ocae078}
}
EUVIMED