A critical evaluation of generative query expansion on biomedical literature retrieval
Journal of the American Medical Informatics Association · 01.06.2026 · 33 (6) · 1121-1133
DOI: 10.1093/jamia/ocag037 ↗Kurz gefasst
Evaluationsstudie in der Zeitschrift Journal of the American Medical Informatics Association, online erschienen am 01.04.2026. Verfasst von Yilu Fang und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Columbia University, New York, NY, United States. Hauptthemen laut MEDLINE: Information Storage and Retrieval. Die Arbeit nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH), NLM NIH HHS, NIH HHS.
Abstract (Originalsprache)
Objective
To evaluate the effectiveness of generative query expansion for biomedical literature retrieval. Materials and Methods We thoroughly examined eight generative query expansion methods using three large language models across five datasets for biomedical literature retrieval. We further performed a quantitative analysis, including performance comparisons, rank transition analysis, and article-type effect analysis. We also conducted a qualitative examination of representative cases, from which we derived an error taxonomy.
Results
On BioASQ-Y/N, GPT-4o-based query expansion shifts Recall@10 to 0.417-0.512 and nDCG@10 to 0.358-0.479, relative to a baseline of 0.491 and 0.456. For PubMedQA, Precision@1 ranges from 0.764 to 0.876 and nDCG@10 from 0.847 to 0.931, compared with baseline values of 0.893 and 0.935. For 2019-Trec-PM, query expansion yields Recall@100 of 0.217-0.256 and nDCG@100 of 0.272-0.312, versus a baseline of 0.227 and 0.274. Similarly, for 2018-TREC-PM, Recall@100 spans 0.169-0.227 and nDCG@100 spans 0.195-0.250, relative to baseline scores of 0.164 and 0.191. For 2017-TREC-PM, Recall@100 and nDCG@100 fall within 0.111-0.139 and 0.154-0.191 under query expansion, compared with baseline metrics of 0.102 and 0.147. Both general-purpose and domain-specific Llama-based models demonstrate similar performance to GPT-4o. Discussion and Conclusion The impact of query expansion varies significantly by the expansion methods and type of evidence, but is relatively agnostic to backbone model choice. Notably, query expansion primarily affects article ranking but has a limited impact on the screening stage. Our findings underscore the unique challenges of biomedical literature retrieval and highlight the need to develop domain-specific information retrieval techniques.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Biomedical Informatics, Columbia University, New York, NY, United States.
- Department of Population Health Sciences, Weill Cornell Medicine, New York, NY, United States.
ORCID-Profile: Yilu Fang, Gongbo Zhang, Fangyi Chen, Yifan Peng, Chunhua Weng
Förderung
- National Center for Advancing Translational Sciences (NCATS) of the National Institutes of Health (NIH)
- NLM NIH HHS
- NIH HHS
- NCATS NIH HHS
- National Center for Advancing Translational Sciences
- National Institutes of Health
- National Library of Medicine
Zitieren
APA
Fang, Y., Zhang, G., Chen, F., Peng, Y., & Weng, C. (2026). A critical evaluation of generative query expansion on biomedical literature retrieval. Journal of the American Medical Informatics Association, 33(6), 1121-1133. https://doi.org/10.1093/jamia/ocag037
Vancouver
Fang Y, Zhang G, Chen F, Peng Y, Weng C. A critical evaluation of generative query expansion on biomedical literature retrieval. Journal of the American Medical Informatics Association. 2026;33(6):1121-1133. doi:10.1093/jamia/ocag037
BibTeX
@article{Fang2026,
author = {Fang, Yilu and Zhang, Gongbo and Chen, Fangyi and Peng, Yifan and Weng, Chunhua},
title = {A critical evaluation of generative query expansion on biomedical literature retrieval},
journal = {Journal of the American Medical Informatics Association},
year = {2026},
volume = {33},
number = {6},
pages = {1121--1133},
doi = {10.1093/jamia/ocag037}
}
EUVIMED