EUVIMEDEUVIMED entdecken
Open Access

Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research

Hamed Meshkin iD, Joel Zirkle, Ghazal Arabidarrehdor, Anik Chaturbedi, Shilpa Chakravartula, John Mann, Bradlee Thrasher, Zhihua Li

Briefings in Bioinformatics · 25.07.2024 · 25 (5)

DOI: 10.1093/bib/bbae354 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Briefings in Bioinformatics, online erschienen am 23.08.2024. Verfasst von Hamed Meshkin und 7 weiteren Autor:innen. Beteiligte Einrichtung: Division of Applied Regulatory Science, Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, U…. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 41-mal zitiert (Crossref) und nennt 55 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: Center for Drug Evaluation and Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) are sophisticated AI-driven models trained on vast sources of natural language data. They are adept at generating responses that closely mimic human conversational patterns. One of the most notable examples is OpenAI's ChatGPT, which has been extensively used across diverse sectors. Despite their flexibility, a significant challenge arises as most users must transmit their data to the servers of companies operating these models. Utilizing ChatGPT or similar models online may inadvertently expose sensitive information to the risk of data breaches. Therefore, implementing LLMs that are open source and smaller in scale within a secure local network becomes a crucial step for organizations where ensuring data privacy and protection has the highest priority, such as regulatory agencies. As a feasibility evaluation, we implemented a series of open-source LLMs within a regulatory agency’s local network and assessed their performance on specific tasks involving extracting relevant clinical pharmacology information from regulatory drug labels. Our research shows that some models work well in the context of few- or zero-shot learning, achieving performance comparable, or even better than, neural network models that needed thousands of training samples. One of the models was selected to address a real-world issue of finding intrinsic factors that affect drugs' clinical exposure without any training or fine-tuning. In a dataset of over 700 000 sentences, the model showed a 78.5% accuracy rate. Our work pointed to the possibility of implementing open-source LLMs within a secure local network and using these models to perform various natural language processing tasks when large numbers of training examples are unavailable.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingHumansNeural Networks, ComputerMachine Learning

Stichwörter der Autor:innen

FDA labelsfew-shot learninglarge language modelspharmacokinetic drug–drug interactions and intrinsic factorsprompt engineeringzero-shot learning

Autor:innen und Institutionen

  • Division of Applied Regulatory Science, Office of Clinical Pharmacology, Office of Translational Sciences, Center for Drug Evaluation and Research, U.S. Food and Drug Administration, WO Bldg 64, 10903 New Hampshire Ave,…

ORCID-Profile: Hamed Meshkin

Förderung

  • Center for Drug Evaluation and Research

Zitieren

APA

Meshkin, H., Zirkle, J., Arabidarrehdor, G., Chaturbedi, A., Chakravartula, S., Mann, J., Thrasher, B., & Li, Z. (2024). Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research. Briefings in Bioinformatics, 25(5). https://doi.org/10.1093/bib/bbae354

Vancouver

Meshkin H, Zirkle J, Arabidarrehdor G, Chaturbedi A, Chakravartula S, Mann J, et al. Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research. Briefings in Bioinformatics. 2024;25(5). doi:10.1093/bib/bbae354

BibTeX
@article{Meshkin2024,
  author = {Meshkin, Hamed and Zirkle, Joel and Arabidarrehdor, Ghazal and Chaturbedi, Anik and Chakravartula, Shilpa and Mann, John and Thrasher, Bradlee and Li, Zhihua},
  title = {Harnessing large language models’ zero-shot and few-shot learning capabilities for regulatory research},
  journal = {Briefings in Bioinformatics},
  year = {2024},
  volume = {25},
  number = {5},
  doi = {10.1093/bib/bbae354}
}
Kostenlos starten →