Assessing the Quality of AI Explanations of Atrial Fibrillation: Comparative Analysis of ChatGPT and Google Gemini
JMIR Formative Research · 2026-11-05 · 10 · e73715
DOI: 10.2196/73715 ↗At a glance
Vergleichsstudie in JMIR Formative Research, published online on 2026-11-05. Written by Jasneel Kahlam and 7 co-authors. Institution: Stony Brook Southampton Hospital, Southampton, NY, United States. Major MEDLINE topics: Atrial Fibrillation, Artificial Intelligence, Patient Education as Topic. 25 references.
Abstract (original language)
Background
Atrial fibrillation (AF) is one of the most common arrhythmias and is associated with multiple risk factors, including stroke and heart failure. Therefore, it is important that accurate and comprehensive educational resources on AF are available. Artificial intelligence (AI) platforms such as Google Gemini and ChatGPT are emerging tools for medical education. To our knowledge, this is the first study to assess the quality of these platforms for AF education.
Objective
This pilot study aimed to (1) assess the quality of ChatGPT and Google Gemini explanations of AF and its treatment, (2) compare responses from both platforms, and (3) analyze differences in interpretation between cardiologists and nonmedical professionals.
Methods
The prompt “Explain atrial fibrillation and how to treat it to a patient” was entered into both ChatGPT and Google Gemini. The responses, along with a survey created in Google Forms, were sent out to potential participants between September and October 2024. The survey questions were derived from the Patient Education Materials Assessment Tool for Printable Materials and DISCERN criteria. Each survey question was multiple choice and rated on a 5-point Likert scale, where 1 indicated strong disagreement and 5 indicated strong agreement. Averages of each answer from the survey were calculated and compared using the Wilcoxon signed-rank test.
Results
The survey was completed by 28 participants, including cardiologists and cardiology fellows and 17 nonmedical professionals (college graduates not employed in the medical field). No significant overall quality differences were observed between ChatGPT and Google Gemini. When comparing participant groups, cardiologists rated AI-generated responses higher than nonmedical professionals for perceived bias (mean 3.82 vs mean 4.33; P=.04) and for explaining consequences of no treatment (mean 2.85 vs mean 1.86; P=.005). In contrast, nonmedical professionals rated structural and organizational features more favorably than cardiologists, including visual cues (mean 3.97 vs mean 2.48; P
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Stony Brook Southampton Hospital, Southampton, NY, United States.
- Department of Internal Medicine, Department of Internal Medicine, New York Hospital Queens, Flushing, NY, United States.
- Department of Cardiology, Deborah Heart and Lung Center, Browns Mills, NJ, United States.
- Stony Brook University Hospital, Stony Brook, NY, United States.
- Brooklyn Hospital Center, Brooklyn, NY, United States.
- American Preventive Screening & Education Association, Stratford, NJ, United States.
ORCID profiles: Jasneel Kahlam, Alexander Sacher, Kyrollos Saad, David Zhang, Rohit Sekandalapuram, Quang Dai La, David F Lo, John P Reilly
Cite
APA
Kahlam, J., Sacher, A., Saad, K., Zhang, D., Sekandalapuram, R., La, Q. D., Lo, D. F., & Reilly, J. P. (2026). Assessing the Quality of AI Explanations of Atrial Fibrillation: Comparative Analysis of ChatGPT and Google Gemini. JMIR Formative Research, 10, e73715. https://doi.org/10.2196/73715
Vancouver
Kahlam J, Sacher A, Saad K, Zhang D, Sekandalapuram R, La QD, et al. Assessing the Quality of AI Explanations of Atrial Fibrillation: Comparative Analysis of ChatGPT and Google Gemini. JMIR Formative Research. 2026;10:e73715. doi:10.2196/73715
BibTeX
@article{Kahlam2026,
author = {Kahlam, Jasneel and Sacher, Alexander and Saad, Kyrollos and Zhang, David and Sekandalapuram, Rohit and La, Quang Dai and Lo, David F and Reilly, John P},
title = {Assessing the Quality of AI Explanations of Atrial Fibrillation: Comparative Analysis of ChatGPT and Google Gemini},
journal = {JMIR Formative Research},
year = {2026},
volume = {10},
pages = {e73715},
doi = {10.2196/73715}
}
EUVIMED