EUVIMEDEUVIMED entdecken
LeitlinieQuerschnittstudieOpen Access

Evaluating Large Language Models and Retrieval-Augmented Generation Enhancement for Delivering Guideline-Adherent Nutrition Information for Cardiovascular Disease Prevention: Cross-Sectional Study

Vijaya Parameswaran iD, Jenna Bernard iD, Alec Bernard iD, Neil Deo iD, Sean Tsung iD, Kalle Lyytinen iD, Christopher Sharp iD, Fatima Rodriguez iD, David J Maron iD, Rajesh Dash iD

Journal of Medical Internet Research · 07.10.2025 · 27 · e78625

DOI: 10.2196/78625 ↗

Kurz gefasst

Leitlinie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 07.10.2025. Verfasst von Vijaya Parameswaran und 9 weiteren Autor:innen. Beteiligte Einrichtung: School of Medicine, Stanford University, Palo Alto, CA, United States. Hauptthemen laut MEDLINE: Cardiovascular Diseases, Language, Guideline Adherence. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NHLBI NIH HHS.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Cardiovascular disease (CVD) remains the leading cause of death worldwide, yet many web-based sources on cardiovascular (CV) health are inaccessible. Large language models (LLMs) are increasingly used for health-related inquiries and offer an opportunity to produce accessible and scalable CV health information. However, because these models are trained on heterogeneous data, including unverified user-generated content, the quality and reliability of food and nutrition information on CVD prevention remain uncertain. Recent studies have examined LLM use in various health care applications, but their effectiveness for providing nutrition information remains understudied. Although retrieval-augmented generation (RAG) frameworks have been shown to enhance LLM consistency and accuracy, their use in delivering nutrition information for CVD prevention requires further evaluation.

Objective

To evaluate the effectiveness of off-the-shelf and RAG-enhanced LLMs in delivering guideline-adherent nutrition information for CVD prevention, we assessed 3 off-the-shelf models (ChatGPT-4o, Perplexity, and Llama 3-70B) and a Llama 3-70B+RAG model.

Methods

We curated 30 nutrition questions that comprehensively addressed CVD prevention. These were approved by a registered dietitian providing preventive cardiology services at an academic medical center and were posed 3 times to each model. We developed a 15,074-word knowledge bank incorporating the American Heart Association’s 2021 dietary guidelines and related website content to enhance Meta’s Llama 3-70B model using RAG. The model received this and a few-shot prompt as context, included citations in a Context Source section, and used vector similarity to align responses with guideline content, with the temperature parameter set to 0.5 to enhance consistency. Model responses were evaluated by 3 expert reviewers against benchmark CV guidelines for appropriateness, reliability, readability, harm, and guideline adherence. Mean scores were compared using ANOVA, with statistical significance set at P70%; P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Cardiovascular DiseasesLanguageGuideline AdherenceHumansCross-Sectional StudiesInternetLarge Language Models

Stichwörter der Autor:innen

artificial intelligencecardiovascular dietary guidelineslarge language modelsqualitative evaluationretrieval-augmented generation

Autor:innen und Institutionen

  • School of Medicine, Stanford University, Palo Alto, CA, United States.
  • Tufts University, Boston, MA, United States.
  • University of Illinois Urbana-Champaign, Champaign, IL, United States.
  • Stanford University, Palo Alto, CA, United States.
  • Case Western Reserve University, Cleveland, OH, United States.

ORCID-Profile: Vijaya Parameswaran, Jenna Bernard, Alec Bernard, Neil Deo, Sean Tsung, Kalle Lyytinen, Christopher Sharp, Fatima Rodriguez, David J Maron, Rajesh Dash

Förderung

  • NHLBI NIH HHS

Zitieren

APA

Parameswaran, V., Bernard, J., Bernard, A., Deo, N., Tsung, S., Lyytinen, K., Sharp, C., Rodriguez, F., Maron, D. J., & Dash, R. (2025). Evaluating Large Language Models and Retrieval-Augmented Generation Enhancement for Delivering Guideline-Adherent Nutrition Information for Cardiovascular Disease Prevention: Cross-Sectional Study. Journal of Medical Internet Research, 27, e78625. https://doi.org/10.2196/78625

Vancouver

Parameswaran V, Bernard J, Bernard A, Deo N, Tsung S, Lyytinen K, et al. Evaluating Large Language Models and Retrieval-Augmented Generation Enhancement for Delivering Guideline-Adherent Nutrition Information for Cardiovascular Disease Prevention: Cross-Sectional Study. Journal of Medical Internet Research. 2025;27:e78625. doi:10.2196/78625

BibTeX
@article{Parameswaran2025,
  author = {Parameswaran, Vijaya and Bernard, Jenna and Bernard, Alec and Deo, Neil and Tsung, Sean and Lyytinen, Kalle and Sharp, Christopher and Rodriguez, Fatima and Maron, David J and Dash, Rajesh},
  title = {Evaluating Large Language Models and Retrieval-Augmented Generation Enhancement for Delivering Guideline-Adherent Nutrition Information for Cardiovascular Disease Prevention: Cross-Sectional Study},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  volume = {27},
  pages = {e78625},
  doi = {10.2196/78625}
}
Kostenlos starten →