RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization
Journal of the American Medical Informatics Association · 2024
DOI: 10.1093/jamia/ocae129 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of the American Medical Informatics Association, 2024. Verfasst von Yiming Li und 22 weiteren Autor:innen. Beteiligte Einrichtung: McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States. Hauptthemen laut MEDLINE: Algorithms, Information Storage and Retrieval, PubMed. Die Arbeit wurde bisher 68-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NIH HHS, National Institute of Allergy and Infectious Diseases, National Institute of Allergy and Infectious Diseases of the National Institutes of Health.
Abstract (Originalsprache)
Objectives
Precise literature recommendation and summarization are crucial for biomedical professionals. While the latest iteration of generative pretrained transformer (GPT) incorporates 2 distinct modes—real-time search and pretrained model utilization—it encounters challenges in dealing with these tasks. Specifically, the real-time search can pinpoint some relevant articles but occasionally provides fabricated papers, whereas the pretrained model excels in generating well-structured summaries but struggles to cite specific sources. In response, this study introduces RefAI, an innovative retrieval-augmented generative tool designed to synergize the strengths of large language models (LLMs) while overcoming their limitations. Materials and Methods RefAI utilized PubMed for systematic literature retrieval, employed a novel multivariable algorithm for article recommendation, and leveraged GPT-4 turbo for summarization. Ten queries under 2 prevalent topics (“cancer immunotherapy and target therapy” and “LLMs in medicine”) were chosen as use cases and 3 established counterparts (ChatGPT-4, ScholarAI, and Gemini) as our baselines. The evaluation was conducted by 10 domain experts through standard statistical analyses for performance comparison.
Results
The overall performance of RefAI surpassed that of the baselines across 5 evaluated dimensions—relevance and quality for literature recommendation, accuracy, comprehensiveness, and reference integration for summarization, with the majority exhibiting statistically significant improvements (P-values <.05).
Discussion
RefAI demonstrated substantial improvements in literature recommendation and summarization over existing tools, addressing issues like fabricated papers, metadata inaccuracies, restricted recommendations, and poor reference integration.
Conclusion
By augmenting LLM with external resources and a novel ranking algorithm, RefAI is uniquely capable of recommending high-quality literature and generating well-structured summaries, holding the potential to meet the critical needs of biomedical professionals in navigating and synthesizing vast amounts of scientific literature.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- McWilliams School of Biomedical Informatics, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States.
- Department of Computer Science, College of Natural Sciences, University of Texas at Austin, Austin, TX 78712, United States.
- Department of Biostatistics & Data Science, School of Public Health, The University of Texas Health Science Center at Houston, Houston, TX 77030, United States.
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL 32224, United States.
- Department of Lymphoma and Myeloma, The University of Texas MD Anderson Cancer Center, Houston, TX 77030, United States.
- Department of Nanomedicine, Houston Methodist Academic Institute, Houston, TX 77030, United States.
ORCID-Profile: Yiming Li, Yifang Dang, Zenan Sun, Fang Li, Cui Tao
Förderung
- NIH HHS
- National Institute of Allergy and Infectious Diseases
- National Institute of Allergy and Infectious Diseases of the National Institutes of Health
- NIAID NIH HHS
- NLM NIH HHS
- the American Heart Association
- Cancer Prevention and Research Institute of Texas
- American Heart Association
Zitieren
APA
Li, Y., Zhao, J., Li, M., Dang, Y., Yu, E., Li, J., Sun, Z., Hussein, U., Wen, J., Abdelhameed, A. M., Mai, J., Li, S., Yu, Y., Hu, X., Yang, D., Feng, J., Li, Z., He, J., Tao, W., …, & Tao, C. (2024). RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization. Journal of the American Medical Informatics Association. https://doi.org/10.1093/jamia/ocae129
Vancouver
Li Y, Zhao J, Li M, Dang Y, Yu E, Li J, et al. RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization. Journal of the American Medical Informatics Association. 2024. doi:10.1093/jamia/ocae129
BibTeX
@article{Li2024,
author = {Li, Yiming and Zhao, Jeff and Li, Manqi and Dang, Yifang and Yu, Evan and Li, Jianfu and Sun, Zenan and Hussein, Usama and Wen, Jianguo and Abdelhameed, Ahmed M and Mai, Junhua and Li, Shenduo and Yu, Yue and Hu, Xinyue and Yang, Daowei and Feng, Jingna and Li, Zehan and He, Jianping and Tao, Wei and Duan, Tiehang and Lou, Yanyan and Li, Fang and Tao, Cui},
title = {RefAI: a GPT-powered retrieval-augmented generative tool for biomedical literature recommendation and summarization},
journal = {Journal of the American Medical Informatics Association},
year = {2024},
doi = {10.1093/jamia/ocae129}
}
EUVIMED