EUVIMEDEUVIMED entdecken
Open Access

LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction

Huixue Zhou iD, Mingchen Li, Yongkang Xiao iD, Han Yang iD, Rui Zhang iD

Journal of the American Medical Informatics Association · 01.09.2024 · 31 (9) · 2010-2018

DOI: 10.1093/jamia/ocae147 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 21.06.2024. Verfasst von Huixue Zhou und 4 weiteren Autor:innen. Beteiligte Einrichtung: Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States. Hauptthemen laut MEDLINE: Natural Language Processing. Die Arbeit wurde bisher 33-mal zitiert (Crossref) und nennt 41 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIA NIH HHS, NCCIH NIH HHS, NCI NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

To investigate the demonstration in large language models (LLMs) for biomedical relation extraction. This study introduces a framework comprising three types of adaptive tuning methods to assess their impacts and effectiveness. Materials and Methods Our study was conducted in two phases. Initially, we analyzed a range of demonstration components vital for LLMs’ biomedical data capabilities, including task descriptions and examples, experimenting with various combinations. Subsequently, we introduced the LLM instruction-example adaptive prompting (LEAP) framework, including instruction adaptive tuning, example adaptive tuning, and instruction-example adaptive tuning methods. This framework aims to systematically investigate both adaptive task descriptions and adaptive examples within the demonstration. We assessed the performance of the LEAP framework on the DDI, ChemProt, and BioRED datasets, employing LLMs such as Llama2-7b, Llama2-13b, and MedLLaMA_13B.

Results

Our findings indicated that Instruction + Options + Example and its expanded form substantially improved F1 scores over the standard Instruction + Options mode for zero-shot LLMs. The LEAP framework, particularly through its example adaptive prompting, demonstrated superior performance over conventional instruction tuning across all models. Notably, the MedLLAMA_13B model achieved an exceptional F1 score of 95.13 on the ChemProt dataset using this method. Significant improvements were also observed in the DDI 2013 and BioRED datasets, confirming the method’s robustness in sophisticated data extraction scenarios.

Conclusion

The LEAP framework offers a compelling strategy for enhancing LLM training strategies, steering away from extensive fine-tuning towards more dynamic and contextually enriched prompting methodologies, showcasing in biomedical relation extraction.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Natural Language ProcessingData MiningDatasets as Topic

Stichwörter der Autor:innen

biomedical relation extractioninstruction tuninginstruction-example adaptive promptinglarge language modelnatural language processing

Autor:innen und Institutionen

  • Institute for Health Informatics, University of Minnesota, Minneapolis, MN 55455, United States.
  • Division of Computational Health Sciences, Department of Surgery, University of Minnesota, Minneapolis, MN 55455, United States.

ORCID-Profile: Huixue Zhou, Yongkang Xiao, Han Yang, Rui Zhang

Förderung

  • NIA NIH HHS
  • NCCIH NIH HHS
  • NCI NIH HHS
  • NIH HHS
  • National Institutes of Health
  • National Center for Complementary and Integrative Health
  • National Institute on Aging
  • National Cancer Institute

Zitieren

APA

Zhou, H., Li, M., Xiao, Y., Yang, H., & Zhang, R. (2024). LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction. Journal of the American Medical Informatics Association, 31(9), 2010-2018. https://doi.org/10.1093/jamia/ocae147

Vancouver

Zhou H, Li M, Xiao Y, Yang H, Zhang R. LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction. Journal of the American Medical Informatics Association. 2024;31(9):2010-2018. doi:10.1093/jamia/ocae147

BibTeX
@article{Zhou2024,
  author = {Zhou, Huixue and Li, Mingchen and Xiao, Yongkang and Yang, Han and Zhang, Rui},
  title = {LEAP: LLM instruction-example adaptive prompting framework for biomedical relation extraction},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  volume = {31},
  number = {9},
  pages = {2010--2018},
  doi = {10.1093/jamia/ocae147}
}
Kostenlos starten →