EUVIMEDEUVIMED entdecken
Open Access · CC BY-ND 4.0

Generalizable and scalable multistage biomedical concept normalization leveraging large language models

Nicholas J Dobbins

Research Synthesis Methods · 2025-05 · 16 (3) · 479-490

DOI: 10.1017/rsm.2025.9 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Research Synthesis Methods, online erschienen am 12.03.2025. Verfasst von Nicholas J Dobbins. Beteiligte Einrichtung: Biomedical Informatics and Data Science, Department of Medicine, Johns Hopkins University, Baltimore, MD, USA. Hauptthemen laut MEDLINE: Large Language Models, Natural Language Processing, Biomedical Research. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 51 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-ND 4.0). Gefördert von: U.S. National Library of Medicine.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Biomedical entity normalization is critical to biomedical research because the richness of free-text clinical data, such as progress notes, can often be fully leveraged only after translating words and phrases into structured and coded representations suitable for analysis. Large Language Models (LLMs), in turn, have shown great potential and high performance in a variety of natural language processing (NLP) tasks, but their application for normalization remains understudied.

Methods

We applied both proprietary and open-source LLMs in combination with several rule-based normalization systems commonly used in biomedical research. We used a two-step LLM integration approach, (1) using an LLM to generate alternative phrasings of a source utterance, and (2) to prune candidate UMLS concepts, using a variety of prompting methods. We measure results by $F_{\beta }$ , where we favor recall over precision, and F1.

Results

We evaluated a total of 5,523 concept terms and text contexts from a publicly available dataset of human-annotated biomedical abstracts. Incorporating GPT-3.5-turbo increased overall $F_{\beta }$ and F1 in normalization systems +16.5 and +16.2 (OpenAI embeddings), +9.5 and +7.3 (MetaMapLite), +13.9 and +10.9 (QuickUMLS), and +10.5 and +10.3 (BM25), while the open-source Vicuna model achieved +20.2 and +21.7 (OpenAI embeddings), +10.8 and +12.2 (MetaMapLite), +14.7 and +15 (QuickUMLS), and +15.6 and +18.7 (BM25).

Conclusions

Existing general-purpose LLMs, both propriety and open-source, can be leveraged to greatly improve normalization performance using existing tools, with no fine-tuning.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsNatural Language ProcessingBiomedical ResearchHumansAlgorithmsUnified Medical Language SystemData Mining

Stichwörter der Autor:innen

UMLSbiomedical conceptsentity linkinglarge language modelnormalization

Autor:innen und Institutionen

  • Biomedical Informatics and Data Science, Department of Medicine, Johns Hopkins University, Baltimore, MD, USA.

Förderung

  • U.S. National Library of Medicine

Zitieren

APA

Dobbins, N. J. (2025). Generalizable and scalable multistage biomedical concept normalization leveraging large language models. Research Synthesis Methods, 16(3), 479-490. https://doi.org/10.1017/rsm.2025.9

Vancouver

Dobbins NJ. Generalizable and scalable multistage biomedical concept normalization leveraging large language models. Research Synthesis Methods. 2025;16(3):479-490. doi:10.1017/rsm.2025.9

BibTeX
@article{Dobbins2025,
  author = {Dobbins, Nicholas J},
  title = {Generalizable and scalable multistage biomedical concept normalization leveraging large language models},
  journal = {Research Synthesis Methods},
  year = {2025},
  volume = {16},
  number = {3},
  pages = {479--490},
  doi = {10.1017/rsm.2025.9}
}
Kostenlos starten →