EUVIMEDEUVIMED entdecken
Crossref

From Evidence to Recommendations With Large Language Models: A Feasibility Study

Weilong Zhao, Danni Xia, Ziying Ye, Honghao Lai, Mingyao Sun, Jiajie Huang, Jiayi Liu, Jianing Liu, Long Ge iD

Journal of Evidence-Based Medicine · 2025

DOI: 10.1111/jebm.70067 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Evidence-Based Medicine, 2025. Verfasst von Weilong Zhao und 8 weiteren Autor:innen. Beteiligte Einrichtung: Department of Health Policy and Management, School of Public Health, Lanzhou University, Lanzhou, China. Hauptthemen laut MEDLINE: Artificial Intelligence, Practice Guidelines as Topic, Evidence-Based Medicine, Language. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: The Outstanding Young Scholars Support Program of Lanzhou University.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Formulating evidene‐based recommendations for practice guidelines is a complex process that requires substantial expertise. Artificial intelligence (AI) is promising in accelerating the guideline development process. This study evaluates the feasibility of leveraging five large language models (LLMs)—ChatGPT‐3.5, Claude‐3 sonnet, Bard, ChatGLM‐4, Kimi chat—to generate recommendations based on structured evidence, assesses their concordance, and explores the potential for AI.

Methods

The general and specific prompts were drafted and validated. We searched PubMed to include evidence‐based guidelines related to health and lifestyle. We randomly selected one recommendation from every included guideline as the sample and extracted the evidence base supporting the selected recommendations. The prompts and evidence were fed into five LLMs to generate structured recommendations.

Results

ChatGPT‐3.5 demonstrated the highest proficiency in comprehensively extracting and synthesizing evidence to formulate novel insights. Bard consistently adhered to existing guideline principles, aligning its algorithm with these tenets. Claude generated fewer topical recommendations, focusing instead on evidence analysis and mitigating irrelevant information. ChatGLM‐4 exhibited a balanced approach, combining evidence extraction with adherence to guideline principles. Kimi showed potential in generating concise and targeted recommendations. Among the six generated recommendations, average consistency ranged from 50% to 91.7%.

Conclusion

The findings of this study suggest that LLMs hold immense potential in accelerating the formulation of evidence‐based recommendations. LLMs can rapidly and comprehensively extract and synthesize relevant information from structured evidence, generating recommendations that align with the available evidence.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Artificial IntelligencePractice Guidelines as TopicEvidence-Based MedicineLanguageFeasibility StudiesHumansAlgorithmsLarge Language Models

Stichwörter der Autor:innen

ChatGPTevidence‐based decision‐makingguidelinelarge language model

Autor:innen und Institutionen

  • Department of Health Policy and Management, School of Public Health, Lanzhou University, Lanzhou, China.
  • Evidence-Based Social Science Research Center, School of Public Health, Lanzhou University, Lanzhou, China.
  • School of Nursing, Peking University, Beijing, China.
  • School of Nursing, Gansu University of Chinese Medicine, Lanzhou, China.
  • Key Laboratory of Evidence-Based Medicine of Gansu Province, Lanzhou University, Lanzhou, China.

ORCID-Profile: Long Ge

Förderung

  • The Outstanding Young Scholars Support Program of Lanzhou University

Zitieren

APA

Zhao, W., Xia, D., Ye, Z., Lai, H., Sun, M., Huang, J., Liu, J., Liu, J., & Ge, L. (2025). From Evidence to Recommendations With Large Language Models: A Feasibility Study. Journal of Evidence-Based Medicine. https://doi.org/10.1111/jebm.70067

Vancouver

Zhao W, Xia D, Ye Z, Lai H, Sun M, Huang J, et al. From Evidence to Recommendations With Large Language Models: A Feasibility Study. Journal of Evidence-Based Medicine. 2025. doi:10.1111/jebm.70067

BibTeX
@article{Zhao2025,
  author = {Zhao, Weilong and Xia, Danni and Ye, Ziying and Lai, Honghao and Sun, Mingyao and Huang, Jiajie and Liu, Jiayi and Liu, Jianing and Ge, Long},
  title = {From Evidence to Recommendations With Large Language Models: A Feasibility Study},
  journal = {Journal of Evidence-Based Medicine},
  year = {2025},
  doi = {10.1111/jebm.70067}
}
Kostenlos starten →