Large Language Models in Patient Health Communication for Atherosclerotic Cardiovascular Disease: Pilot Cross-Sectional Comparative Analysis
JMIR Medical Informatics · 2026
DOI: 10.2196/81422 ↗At a glance
Vergleichsstudie in JMIR Medical Informatics, 2026. Written by Pengfei Li and 9 co-authors. Institution: Department of Emergency and Critical Care Medicine, The Affiliated Suzhou Hospital of Nanjing Medical University, Suzhou Municipal Hospital, Gusu Sch…. Major MEDLINE topics: Large Language Models, Atherosclerosis, Health Communication, Cardiovascular Diseases. Cited 7 times so far (Crossref); 41 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Large language models (LLMs) have emerged as promising tools for enhancing public access to medical information, particularly for chronic diseases such as atherosclerotic cardiovascular disease (ASCVD). However, their effectiveness in patient-centered health communication remains underexplored, especially in multilingual contexts.
Objective
Our study aimed to conduct a comparative evaluation of 3 advanced LLMs—DeepSeek R1, ChatGPT-4o, and Gemini—in generating responses to ASCVD-related patient queries in both English and Chinese, assessing their performance across the domains of accuracy, completeness, and comprehensibility.
Methods
We conducted a cross-sectional evaluation based on 25 clinically validated ASCVD questions spanning 5 domains—definitions, diagnosis, treatment, prevention, and lifestyle. Each question was submitted 5 times to each of the 3 LLMs in both English and Chinese, yielding 750 responses in total, all generated under default settings to approximate real-world conditions. Three board-certified cardiologists blinded to model identity independently scored the responses using standardized Likert scales with predefined anchors. The assessment followed a rigorous multistage process that incorporated randomization, washout periods, and final consensus scoring.
Results
DeepSeek R1 achieved the highest “good response” rates (24/25, 96% in both English and Chinese), substantially outperforming ChatGPT-4o (21/25, 84%) and Gemini (12/25, 48% in English and 17/25, 68% in Chinese). DeepSeek R1 demonstrated superior median accuracy scores (6, IQR 6-6 in both languages) and completeness scores (3, IQR 2-3 in both languages) compared to the other models (P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Emergency and Critical Care Medicine, The Affiliated Suzhou Hospital of Nanjing Medical University, Suzhou Municipal Hospital, Gusu School, Nanjing Medical University, Suzhou, Jiangsu, China.
- Department of Rehabilitation, Shandong Provincial Hospital Affiliated to Shandong First Medical University, Jinan, Shandong, China.
- Department of Emergency Management, School of Health Policy & Management, Nanjing Medical University, Nanjing, Jiangsu, China.
- Department of Emergency and Critical Care Medicine, The First Affiliated Hospital of Nanjing Medical University, Nanjing, Jiangsu, China.
ORCID profiles: Pengfei Li, Yinfei Xu, Xiang Liu, Zhean Shen, Yi Wang, Xinyi Lv, Ziyi Lu, Hui Wu, Jiaqi Zhuang, Yan Chen
Cite
APA
Li, P., Xu, Y., Liu, X., Shen, Z., Wang, Y., Lv, X., Lu, Z., Wu, H., Zhuang, J., & Chen, Y. (2026). Large Language Models in Patient Health Communication for Atherosclerotic Cardiovascular Disease: Pilot Cross-Sectional Comparative Analysis. JMIR Medical Informatics. https://doi.org/10.2196/81422
Vancouver
Li P, Xu Y, Liu X, Shen Z, Wang Y, Lv X, et al. Large Language Models in Patient Health Communication for Atherosclerotic Cardiovascular Disease: Pilot Cross-Sectional Comparative Analysis. JMIR Medical Informatics. 2026. doi:10.2196/81422
BibTeX
@article{Li2026,
author = {Li, Pengfei and Xu, Yinfei and Liu, Xiang and Shen, Zhean and Wang, Yi and Lv, Xinyi and Lu, Ziyi and Wu, Hui and Zhuang, Jiaqi and Chen, Yan},
title = {Large Language Models in Patient Health Communication for Atherosclerotic Cardiovascular Disease: Pilot Cross-Sectional Comparative Analysis},
journal = {JMIR Medical Informatics},
year = {2026},
doi = {10.2196/81422}
}
EUVIMED