Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research
Briefings in Bioinformatics · 25.07.2024 · 25 (5)
DOI: 10.1093/bib/bbae354 ↗Kurz gefasst
Fachartikel in der Zeitschrift Briefings in Bioinformatics, online erschienen am 23.08.2024. Verfasst von Hamed Meshkin und 7 weiteren Autor:innen. Beteiligte Einrichtung: Division of Applied Regulatory Science, Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, U…. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 41-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Center for Drug Evaluation and Research.
Abstract (Originalsprache)
Large language models (LLMs) are sophisticated AI-driven models trained on vast sources of natural language data. They are adept at generating responses that closely mimic human conversational patterns. One of the most notable examples is OpenAI's ChatGPT, which has been extensively used across diverse sectors. Despite their flexibility, a significant challenge arises as most users must transmit their data to the servers of companies operating these models. Utilizing ChatGPT or similar models online may inadvertently expose sensitive information to the risk of data breaches. Therefore, implementing LLMs that are open source and smaller in scale within a secure local network becomes a crucial step for organizations where ensuring data privacy and protection has the highest priority, such as regulatory agencies. As a feasibility evaluation, we implemented a series of open-source LLMs within a regulatory agency’s local network and assessed their performance on specific tasks involving extracting relevant clinical pharmacology information from regulatory drug labels. Our research shows that some models work well in the context of few- or zero-shot learning, achieving performance comparable, or even better than, neural network models that needed thousands of training samples. One of the models was selected to address a real-world issue of finding intrinsic factors that affect drugs' clinical exposure without any training or fine-tuning. In a dataset of over 700 000 sentences, the model showed a 78.5% accuracy rate. Our work pointed to the possibility of implementing open-source LLMs within a secure local network and using these models to perform various natural language processing tasks when large numbers of training examples are unavailable.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Applied Regulatory Science, Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, WO Bldg 64, 10903 New Hampshire Ave,…
ORCID-Profile: Hamed Meshkin
Förderung
- Center for Drug Evaluation and Research
Zitieren
APA
Meshkin, H., Zirkle, J., Arabidarrehdor, G., Chaturbedi, A., Chakravartula, S., Mann, J., Thrasher, B., & Li, Z. (2024). Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research. Briefings in Bioinformatics, 25(5). https://doi.org/10.1093/bib/bbae354
Vancouver
Meshkin H, Zirkle J, Arabidarrehdor G, Chaturbedi A, Chakravartula S, Mann J, et al. Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research. Briefings in Bioinformatics. 2024;25(5). doi:10.1093/bib/bbae354
BibTeX
@article{Meshkin2024,
author = {Meshkin, Hamed and Zirkle, Joel and Arabidarrehdor, Ghazal and Chaturbedi, Anik and Chakravartula, Shilpa and Mann, John and Thrasher, Bradlee and Li, Zhihua},
title = {Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research},
journal = {Briefings in Bioinformatics},
year = {2024},
volume = {25},
number = {5},
doi = {10.1093/bib/bbae354}
}
EUVIMED