From Evidence to Recommendations With Large Language Models: A Feasibility Study
Journal of Evidence-Based Medicine · 2025
DOI: 10.1111/jebm.70067 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Evidence-Based Medicine, 2025. Verfasst von Weilong Zhao und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health Policy and Management, School of Public Health, Lanzhou University, Lanzhou, China. Hauptthemen laut MEDLINE: Artificial Intelligence, Practice Guidelines as Topic, Evidence-Based Medicine, Language. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: The Outstanding Young Scholars Support Program of Lanzhou University.
Abstract (Originalsprache)
Background
Formulating evidene‐based recommendations for practice guidelines is a complex process that requires substantial expertise. Artificial intelligence (AI) is promising in accelerating the guideline development process. This study evaluates the feasibility of leveraging five large language models (LLMs)—ChatGPT‐3.5, Claude‐3 sonnet, Bard, ChatGLM‐4, Kimi chat—to generate recommendations based on structured evidence, assesses their concordance, and explores the potential for AI.
Methods
The general and specific prompts were drafted and validated. We searched PubMed to include evidence‐based guidelines related to health and lifestyle. We randomly selected one recommendation from every included guideline as the sample and extracted the evidence base supporting the selected recommendations. The prompts and evidence were fed into five LLMs to generate structured recommendations.
Results
ChatGPT‐3.5 demonstrated the highest proficiency in comprehensively extracting and synthesizing evidence to formulate novel insights. Bard consistently adhered to existing guideline principles, aligning its algorithm with these tenets. Claude generated fewer topical recommendations, focusing instead on evidence analysis and mitigating irrelevant information. ChatGLM‐4 exhibited a balanced approach, combining evidence extraction with adherence to guideline principles. Kimi showed potential in generating concise and targeted recommendations. Among the six generated recommendations, average consistency ranged from 50% to 91.7%.
Conclusion
The findings of this study suggest that LLMs hold immense potential in accelerating the formulation of evidence‐based recommendations. LLMs can rapidly and comprehensively extract and synthesize relevant information from structured evidence, generating recommendations that align with the available evidence.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Health Policy and Management, School of Public Health, Lanzhou University, Lanzhou, China.
- Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China.
- School of Nursing, Peking University, Beijing, China.
- School of Nursing, Gansu University of Chinese Medicine, Lanzhou, China.
- Key Laboratory of Evidence-Based Medicine of Gansu Province, Lanzhou University, Lanzhou, China.
ORCID-Profile: Long Ge
Förderung
- The Outstanding Young Scholars Support Program of Lanzhou University
Zitieren
APA
Zhao, W., Xia, D., Ye, Z., Lai, H., Sun, M., Huang, J., Liu, J., Liu, J., & Ge, L. (2025). From Evidence to Recommendations With Large Language Models: A Feasibility Study. Journal of Evidence-Based Medicine. https://doi.org/10.1111/jebm.70067
Vancouver
Zhao W, Xia D, Ye Z, Lai H, Sun M, Huang J, et al. From Evidence to Recommendations With Large Language Models: A Feasibility Study. Journal of Evidence-Based Medicine. 2025. doi:10.1111/jebm.70067
BibTeX
@article{Zhao2025,
author = {Zhao, Weilong and Xia, Danni and Ye, Ziying and Lai, Honghao and Sun, Mingyao and Huang, Jiajie and Liu, Jiayi and Liu, Jianing and Ge, Long},
title = {From Evidence to Recommendations With Large Language Models: A Feasibility Study},
journal = {Journal of Evidence-Based Medicine},
year = {2025},
doi = {10.1111/jebm.70067}
}
EUVIMED