LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction
Journal of the American Medical Informatics Association · 01.09.2024 · 31 (9) · 2010-2018
DOI: 10.1093/jamia/ocae147 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 21.06.2024. Verfasst von Huixue Zhou und 4 weiteren Autor:innen. Beteiligte Einrichtung: Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 33-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIA NIH HHS, NCCIH NIH HHS, NCI NIH HHS.
Abstract (Originalsprache)
Objective
To investigate the demonstration in large language models (LLMs) for biomedical relation extraction. This study introduces a framework comprising three types of adaptive tuning methods to assess their impacts and effectiveness. Materials and Methods Our study was conducted in two phases. Initially, we analyzed a range of demonstration components vital for LLMs’ biomedical data capabilities, including task descriptions and examples, experimenting with various combinations. Subsequently, we introduced the LLM instruction-example adaptive prompting (LEAP) framework, including instruction adaptive tuning, example adaptive tuning, and instruction-example adaptive tuning methods. This framework aims to systematically investigate both adaptive task descriptions and adaptive examples within the demonstration. We assessed the performance of the LEAP framework on the DDI, ChemProt, and BioRED datasets, employing LLMs such as Llama2-7b, Llama2-13b, and MedLLaMA_13B.
Results
Our findings indicated that Instruction + Options + Example and its expanded form substantially improved F1 scores over the standard Instruction + Options mode for zero-shot LLMs. The LEAP framework, particularly through its example adaptive prompting, demonstrated superior performance over conventional instruction tuning across all models. Notably, the MedLLAMA_13B model achieved an exceptional F1 score of 95.13 on the ChemProt dataset using this method. Significant improvements were also observed in the DDI 2013 and BioRED datasets, confirming the method’s robustness in sophisticated data extraction scenarios.
Conclusion
The LEAP framework offers a compelling strategy for enhancing LLM training strategies, steering away from extensive fine-tuning towards more dynamic and contextually enriched prompting methodologies, showcasing in biomedical relation extraction.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States.
- Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States.
ORCID-Profile: Huixue Zhou, Yongkang Xiao, Han Yang, Rui Zhang
Förderung
- NIA NIH HHS
- NCCIH NIH HHS
- NCI NIH HHS
- NIH HHS
- National Institutes of Health
- National Center for Complementary and Integrative Health
- National Institute on Aging
- National Cancer Institute
Zitieren
APA
Zhou, H., Li, M., Xiao, Y., Yang, H., & Zhang, R. (2024). LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction. Journal of the American Medical Informatics Association, 31(9), 2010-2018. https://doi.org/10.1093/jamia/ocae147
Vancouver
Zhou H, Li M, Xiao Y, Yang H, Zhang R. LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction. Journal of the American Medical Informatics Association. 2024;31(9):2010-2018. doi:10.1093/jamia/ocae147
BibTeX
@article{Zhou2024,
author = {Zhou, Huixue and Li, Mingchen and Xiao, Yongkang and Yang, Han and Zhang, Rui},
title = {LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction},
journal = {Journal of the American Medical Informatics Association},
year = {2024},
volume = {31},
number = {9},
pages = {2010--2018},
doi = {10.1093/jamia/ocae147}
}
EUVIMED