EUVIMEDEUVIMED entdecken
Open Access

CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation

Qianqian Xie iD, Jeffrey Zhang, Yan Wang iD, Jimin Huang, Fongci Lin, Ruey-Ling Weng, Huan He, Qingyu Chen, Hua Xu iD

Journal of the American Medical Informatics Association · 01.10.2026 · 33 (10) · 1934-1942

DOI: 10.1093/jamia/ocag122 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 04.08.2026. Verfasst von Qianqian Xie und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics and Data Science, School of Medicine, Yale University, New Haven, CT, 06510, United States. Hauptthemen laut MEDLINE: Large Language Models, Information Storage and Retrieval, Biomedical Research. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIH HHS, National Institutes of Health.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

Accurate citation of relevant publications is essential for scientific integrity in biomedical research. Large language models (LLMs) excel at text generation but often hallucinate fabricated or inaccurate citations. Retrieval-augmented generation (RAG) can mitigate these errors, yet current approaches lack semantic precision in evidence retrieval. This study aims to develop a domain-specific RAG system for reliable, context-specific biomedical citation recommendations. Materials and Methods We introduce CiteSure, a sentence-level citation recommendation tool designed to deliver reliable, evidence-based, and context-specific references using LLMs. CiteSure utilizes a 2-stage retrieval-augmented generation (RAG) framework, combining a domain-specific dense retriever (BioLLM2Vec) and reranker (BioRankLLaMA), adapted from LLaMA3-8B-Instruct using biomedical-specific training data. CiteSure leverages the complementary strengths of retrieval and generative LLM models, ensuring factual precision and contextual alignment. We evaluated CiteSure on a curated Alzheimer’s disease dataset, comparing it to standalone LLMs and traditional retrieval-based methods.

Results

CiteSure achieved 100% factual accuracy and the highest relevance score of 77.50%, outperforming all baselines. BioLLM2Vec retrieved relevant articles with over 80% accuracy in the top 100 candidates. BioRankLLaMA consistently outperformed baseline rerankers across MAP, MRR, and Precision@5 metrics, confirming the benefit of domain-specific adaptation and contrastive fine-tuning. Discussion and Conclusion Our results demonstrate that CiteSure, built on a 2-stage retrieval-augmented generation framework, effectively integrates domain-specific retrieval with LLM-based generation to achieve substantial improvements over baseline approaches. Our work underscores the importance of domain-specific adaptation in biomedical citation recommendation and provides publicly available datasets, models, and code for support future research.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Large Language ModelsInformation Storage and RetrievalBiomedical ResearchHumansAlzheimer DiseaseGenerative Artificial Intelligence

Stichwörter der Autor:innen

biomedical citation recommendationlarge language modelsretrieval augmentation

Autor:innen und Institutionen

  • Department of Biomedical Informatics and Data Science, School of Medicine, Yale University, New Haven, CT, 06510, United States.

ORCID-Profile: Qianqian Xie, Yan Wang, Hua Xu

Förderung

  • NIH HHS
  • National Institutes of Health

Zitieren

APA

Xie, Q., Zhang, J., Wang, Y., Huang, J., Lin, F., Weng, R. L., He, H., Chen, Q., & Xu, H. (2026). CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation. Journal of the American Medical Informatics Association, 33(10), 1934-1942. https://doi.org/10.1093/jamia/ocag122

Vancouver

Xie Q, Zhang J, Wang Y, Huang J, Lin F, Weng RL, et al. CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation. Journal of the American Medical Informatics Association. 2026;33(10):1934-1942. doi:10.1093/jamia/ocag122

BibTeX
@article{Xie2026,
  author = {Xie, Qianqian and Zhang, Jeffrey and Wang, Yan and Huang, Jimin and Lin, Fongci and Weng, Ruey-Ling and He, Huan and Chen, Qingyu and Xu, Hua},
  title = {CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation},
  journal = {Journal of the American Medical Informatics Association},
  year = {2026},
  volume = {33},
  number = {10},
  pages = {1934--1942},
  doi = {10.1093/jamia/ocag122}
}
Kostenlos starten →