EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights

François Remy iD, Kris Demuynck iD, Thomas Demeester iD

Journal of the American Medical Informatics Association · 01.09.2024 · 31 (9) · 1844-1855

DOI: 10.1093/jamia/ocae029 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 27.02.2024. Verfasst von François Remy und 2 weiteren Autor:innen. Beteiligte Einrichtung: Internet and Data Science Lab, imec, Ghent University, Ghent, Belgium. Hauptthemen laut MEDLINE: Semantics, Unified Medical Language System, Natural Language Processing. Die Arbeit wurde bisher 81-mal zitiert (Crossref) und nennt 61 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: ADAM, VLAIO O&O.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

In this study, we investigate the potential of large language models (LLMs) to complement biomedical knowledge graphs in the training of semantic models for the biomedical and clinical domains. Materials and Methods Drawing on the wealth of the Unified Medical Language System knowledge graph and harnessing cutting-edge LLMs, we propose a new state-of-the-art approach for obtaining high-fidelity representations of biomedical concepts and sentences, consisting of 3 steps: an improved contrastive learning phase, a novel self-distillation phase, and a weight averaging phase.

Results

Through rigorous evaluations of diverse downstream tasks, we demonstrate consistent and substantial improvements over the previous state of the art for semantic textual similarity (STS), biomedical concept representation (BCR), and clinically named entity linking, across 15+ datasets. Besides our new state-of-the-art biomedical model for English, we also distill and release a multilingual model compatible with 50+ languages and finetuned on 7 European languages.

Discussion

Many clinical pipelines can benefit from our latest models. Our new multilingual model enables a range of languages to benefit from our advancements in biomedical semantic representation learning, opening a new avenue for bioinformatics researchers around the world. As a result, we hope to see BioLORD-2023 becoming a precious tool for future biomedical applications.

Conclusion

In this article, we introduced BioLORD-2023, a state-of-the-art model for STS and BCR designed for the clinical domain.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

SemanticsUnified Medical Language SystemNatural Language ProcessingHumans

Stichwörter der Autor:innen

biological ontologiesknowledge basesmachine learningnatural language processingsemantics

Autor:innen und Institutionen

  • Internet and Data Science Lab, imec, Ghent University, Ghent, Belgium.

ORCID-Profile: François Remy, Kris Demuynck, Thomas Demeester

Förderung

  • ADAM
  • VLAIO O&O

Zitieren

APA

Remy, F., Demuynck, K., & Demeester, T. (2024). BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights. Journal of the American Medical Informatics Association, 31(9), 1844-1855. https://doi.org/10.1093/jamia/ocae029

Vancouver

Remy F, Demuynck K, Demeester T. BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights. Journal of the American Medical Informatics Association. 2024;31(9):1844-1855. doi:10.1093/jamia/ocae029

BibTeX
@article{Remy2024,
  author = {Remy, François and Demuynck, Kris and Demeester, Thomas},
  title = {BioLORD-2023: semantic textual representations fusing large language models and clinical knowledge graph insights},
  journal = {Journal of the American Medical Informatics Association},
  year = {2024},
  volume = {31},
  number = {9},
  pages = {1844--1855},
  doi = {10.1093/jamia/ocae029}
}
Kostenlos starten →