CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation
Journal of the American Medical Informatics Association · 01.10.2026 · 33 (10) · 1934-1942
DOI: 10.1093/jamia/ocag122 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 04.08.2026. Verfasst von Qianqian Xie und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics and Data Science, School of Medicine, Yale University, New Haven, CT, 06510, United States. Hauptthemen laut MEDLINE: Large Language Models, Information Storage and Retrieval, Biomedical Research. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIH HHS, National Institutes of Health.
Abstract (Originalsprache)
Objectives
Accurate citation of relevant publications is essential for scientific integrity in biomedical research. Large language models (LLMs) excel at text generation but often hallucinate fabricated or inaccurate citations. Retrieval-augmented generation (RAG) can mitigate these errors, yet current approaches lack semantic precision in evidence retrieval. This study aims to develop a domain-specific RAG system for reliable, context-specific biomedical citation recommendations. Materials and Methods We introduce CiteSure, a sentence-level citation recommendation tool designed to deliver reliable, evidence-based, and context-specific references using LLMs. CiteSure utilizes a 2-stage retrieval-augmented generation (RAG) framework, combining a domain-specific dense retriever (BioLLM2Vec) and reranker (BioRankLLaMA), adapted from LLaMA3-8B-Instruct using biomedical-specific training data. CiteSure leverages the complementary strengths of retrieval and generative LLM models, ensuring factual precision and contextual alignment. We evaluated CiteSure on a curated Alzheimer’s disease dataset, comparing it to standalone LLMs and traditional retrieval-based methods.
Results
CiteSure achieved 100% factual accuracy and the highest relevance score of 77.50%, outperforming all baselines. BioLLM2Vec retrieved relevant articles with over 80% accuracy in the top 100 candidates. BioRankLLaMA consistently outperformed baseline rerankers across MAP, MRR, and Precision@5 metrics, confirming the benefit of domain-specific adaptation and contrastive fine-tuning. Discussion and Conclusion Our results demonstrate that CiteSure, built on a 2-stage retrieval-augmented generation framework, effectively integrates domain-specific retrieval with LLM-based generation to achieve substantial improvements over baseline approaches. Our work underscores the importance of domain-specific adaptation in biomedical citation recommendation and provides publicly available datasets, models, and code for support future research.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biomedical Informatics and Data Science, School of Medicine, Yale University, New Haven, CT, 06510, United States.
ORCID-Profile: Qianqian Xie, Yan Wang, Hua Xu
Förderung
- NIH HHS
- National Institutes of Health
Zitieren
APA
Xie, Q., Zhang, J., Wang, Y., Huang, J., Lin, F., Weng, R. L., He, H., Chen, Q., & Xu, H. (2026). CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation. Journal of the American Medical Informatics Association, 33(10), 1934-1942. https://doi.org/10.1093/jamia/ocag122
Vancouver
Xie Q, Zhang J, Wang Y, Huang J, Lin F, Weng RL, et al. CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation. Journal of the American Medical Informatics Association. 2026;33(10):1934-1942. doi:10.1093/jamia/ocag122
BibTeX
@article{Xie2026,
author = {Xie, Qianqian and Zhang, Jeffrey and Wang, Yan and Huang, Jimin and Lin, Fongci and Weng, Ruey-Ling and He, Huan and Chen, Qingyu and Xu, Hua},
title = {CiteSure: retrieval-augmented large language models for faithful biomedical citation recommendation},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
volume = {33},
number = {10},
pages = {1934--1942},
doi = {10.1093/jamia/ocag122}
}
EUVIMED