Generalizable and scalable multistage biomedical concept normalization leveraging large language models
Research Synthesis Methods · 2025-05 · 16 (3) · 479-490
DOI: 10.1017/rsm.2025.9 ↗Kurz gefasst
Fachartikel in der Zeitschrift Research Synthesis Methods, online erschienen am 12.03.2025. Verfasst von Nicholas J Dobbins. Beteiligte Einrichtung: Biomedical Informatics and Data Science, Department of Medicine, Johns Hopkins University, Baltimore, MD, USA. Hauptthemen laut MEDLINE: Large Language Models, Natural Language Processing, Biomedical Research. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-ND 4.0). Gefördert von: U.S. National Library of Medicine.
Abstract (Originalsprache)
Background
Biomedical entity normalization is critical to biomedical research because the richness of free-text clinical data, such as progress notes, can often be fully leveraged only after translating words and phrases into structured and coded representations suitable for analysis. Large Language Models (LLMs), in turn, have shown great potential and high performance in a variety of natural language processing (NLP) tasks, but their application for normalization remains understudied.
Methods
We applied both proprietary and open-source LLMs in combination with several rule-based normalization systems commonly used in biomedical research. We used a two-step LLM integration approach, (1) using an LLM to generate alternative phrasings of a source utterance, and (2) to prune candidate UMLS concepts, using a variety of prompting methods. We measure results by $F_{\beta }$ , where we favor recall over precision, and F1.
Results
We evaluated a total of 5,523 concept terms and text contexts from a publicly available dataset of human-annotated biomedical abstracts. Incorporating GPT-3.5-turbo increased overall $F_{\beta }$ and F1 in normalization systems +16.5 and +16.2 (OpenAI embeddings), +9.5 and +7.3 (MetaMapLite), +13.9 and +10.9 (QuickUMLS), and +10.5 and +10.3 (BM25), while the open-source Vicuna model achieved +20.2 and +21.7 (OpenAI embeddings), +10.8 and +12.2 (MetaMapLite), +14.7 and +15 (QuickUMLS), and +15.6 and +18.7 (BM25).
Conclusions
Existing general-purpose LLMs, both propriety and open-source, can be leveraged to greatly improve normalization performance using existing tools, with no fine-tuning.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Biomedical Informatics and Data Science, Department of Medicine, Johns Hopkins University, Baltimore, MD, USA.
Förderung
- U.S. National Library of Medicine
Zitieren
APA
Dobbins, N. J. (2025). Generalizable and scalable multistage biomedical concept normalization leveraging large language models. Research Synthesis Methods, 16(3), 479-490. https://doi.org/10.1017/rsm.2025.9
Vancouver
Dobbins NJ. Generalizable and scalable multistage biomedical concept normalization leveraging large language models. Research Synthesis Methods. 2025;16(3):479-490. doi:10.1017/rsm.2025.9
BibTeX
@article{Dobbins2025,
author = {Dobbins, Nicholas J},
title = {Generalizable and scalable multistage biomedical concept normalization leveraging large language models},
journal = {Research Synthesis Methods},
year = {2025},
volume = {16},
number = {3},
pages = {479--490},
doi = {10.1017/rsm.2025.9}
}
EUVIMED