EUVIMEDEUVIMED entdecken
Open Access

MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval

Qiao Jin iD, Won Kim, Qingyu Chen iD, Donald C Comeau, Lana Yeganova, W John Wilbur, Zhiyong Lu iD

Bioinformatics · 01.11.2023 · 39 (11)

DOI: 10.1093/bioinformatics/btad651 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioinformatics, online erschienen am 01.11.2023. Verfasst von Qiao Jin und 6 weiteren Autor:innen. Beteiligte Einrichtung: National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD, United S…. Hauptthemen laut MEDLINE: Information Storage and Retrieval, Semantics. Die Arbeit wurde bisher 197-mal zitiert (Crossref) und nennt 38 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIH Intramural Research Program, National Library of Medicine.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Motivation Information retrieval (IR) is essential in biomedical knowledge acquisition and clinical decision support. While recent progress has shown that language model encoders perform better semantic retrieval, training such models requires abundant query–article annotations that are difficult to obtain in biomedicine. As a result, most biomedical IR systems only conduct lexical matching. In response, we introduce MedCPT, a first-of-its-kind Contrastively Pre-trained Transformer model for zero-shot semantic IR in biomedicine. Results To train MedCPT, we collected an unprecedented scale of 255 million user click logs from PubMed. With such data, we use contrastive learning to train a pair of closely integrated retriever and re-ranker. Experimental results show that MedCPT sets new state-of-the-art performance on six biomedical IR tasks, outperforming various baselines including much larger models, such as GPT-3-sized cpt-text-XL. In addition, MedCPT also generates better biomedical article and sentence representations for semantic evaluations. As such, MedCPT can be readily applied to various real-world biomedical IR tasks. Availability and implementation The MedCPT code and model are available at https://github.com/ncbi/MedCPT.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Information Storage and RetrievalSemanticsLanguageNatural Language ProcessingPubMedReview Literature as Topic

Autor:innen und Institutionen

  • National Center for Biotechnology Information (NCBI), National Library of Medicine (NLM), National Institutes of Health (NIH), Bethesda, MD, United States.

ORCID-Profile: Qiao Jin, Qingyu Chen, Zhiyong Lu

Förderung

  • NIH Intramural Research Program, National Library of Medicine

Zitieren

APA

Jin, Q., Kim, W., Chen, Q., Comeau, D. C., Yeganova, L., Wilbur, W. J., & Lu, Z. (2023). MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval. Bioinformatics, 39(11). https://doi.org/10.1093/bioinformatics/btad651

Vancouver

Jin Q, Kim W, Chen Q, Comeau DC, Yeganova L, Wilbur WJ, et al. MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval. Bioinformatics. 2023;39(11). doi:10.1093/bioinformatics/btad651

BibTeX
@article{Jin2023,
  author = {Jin, Qiao and Kim, Won and Chen, Qingyu and Comeau, Donald C and Yeganova, Lana and Wilbur, W John and Lu, Zhiyong},
  title = {MedCPT: Contrastive Pre-trained Transformers with large-scale PubMed search logs for zero-shot biomedical information retrieval},
  journal = {Bioinformatics},
  year = {2023},
  volume = {39},
  number = {11},
  doi = {10.1093/bioinformatics/btad651}
}
Kostenlos starten →