Comparing the Accuracy of ChatGPT-4o, DeepSeek-V3, and Gemini 2.5 Flash in Answering Frequently Asked Questions About Systemic Lupus Erythematosus: Quantitative Study
JMIR Formative Research · 08.06.2026 · 10 · e89156-e89156
DOI: 10.2196/89156 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift JMIR Formative Research, online erschienen am 08.06.2026. Verfasst von Alvina Widhani und 13 weiteren Autor:innen. Beteiligte Einrichtung: Division of Allergy and Clinical Immunology, Department of Internal Medicine, Faculty of Medicine, Universitas Indonesia, Diponegoro No. 71 Jakarta, …. Hauptthemen laut MEDLINE: Lupus Erythematosus, Systemic. Die Arbeit nennt 35 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Systemic lupus erythematosus (SLE) is a complex, fluctuating disease, creating a continuous need for reliable patient information. A prior study concluded that patients with SLE often turn to the internet, including artificial intelligence (AI) chatbots, for information regarding SLE. The rise of AI chatbots as a primary information source presents a critical challenge regarding the accuracy of the information they provide.
Objective
This study aimed to evaluate the performance of the latest generation of AI chatbots (ChatGPT-4o, DeepSeek-V3, and Gemini 2.5 Flash) in answering frequently asked questions about SLE.
Methods
Twenty-two frequently asked questions about SLE in Bahasa Indonesia (the Indonesian language) were posed to each chatbot. Responses were independently and blindly evaluated for accuracy by 5 clinical immunologists using a 4-point Likert scale. Readability was assessed using the Flesch reading ease score formula. Statistical comparisons for accuracy and readability were performed using repeated-measures ANOVA or the Friedman test, followed by the Bonferroni test for pairwise comparisons. The Spearman ρ was used to evaluate correlations among accuracy, readability, and word count.
Results
Gemini 2.5 Flash demonstrated the highest accuracy, with a mean score of 1.25 (SD 0.53), significantly outperforming ChatGPT-4o (mean 1.71, SD 0.61; P
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Allergy and Clinical Immunology, Department of Internal Medicine, Faculty of Medicine, Universitas Indonesia, Diponegoro No. 71 Jakarta, Jakarta, 10430, Indonesia, 62 622-1390-4546.
- Department of Internal Medicine, Cipto Mangunkusumo General Hospital, Jakarta, Indonesia.
- Division of Allergy and Clinical Immunology, Department of Internal Medicine, Faculty of Medicine, Andalas University, Padang, West Sumatra, Indonesia.
- Division of Rheumatology, Allergy-Immunology, Infectious Diseases, Department of Medicine, Persahabatan Hospital, Jakarta, Indonesia.
- Division of Allergy and Clinical Immunology, Department of Internal Medicine, Faculty of Medicine, Sriwijaya University, Palembang, South Sumatra, Indonesia.
- Division of Allergy and Clinical Immunology, Department of Internal Medicine, Faculty of Medicine, Universitas Airlangga, Surabaya, East Java, Indonesia.
ORCID-Profile: Alvina Widhani, Suzy Maria, Aisha Putri Chairani, Nabila Yulianingrum Adella Visco, Muhammad Faiz Amirullah Nurhadi, Lutfi Airlangga Harjoprawito, Dwitya Elvira, Irwin Tedja, Yuniza Yuniza, Deasy Fetarayani, Sukamto Koesnoe, Bramantya Wicaksana, Anshari Saifuddin Hasibuan, Evy Yunihastuti
Zitieren
APA
Widhani, A., Maria, S., Chairani, A. P., Visco, N. Y. A., Nurhadi, M. F. A., Harjoprawito, L. A., Elvira, D., Tedja, I., Yuniza, Y., Fetarayani, D., Koesnoe, S., Wicaksana, B., Hasibuan, A. S., & Yunihastuti, E. (2026). Comparing the Accuracy of ChatGPT-4o, DeepSeek-V3, and Gemini 2.5 Flash in Answering Frequently Asked Questions About Systemic Lupus Erythematosus: Quantitative Study. JMIR Formative Research, 10, e89156-e89156. https://doi.org/10.2196/89156
Vancouver
Widhani A, Maria S, Chairani AP, Visco NYA, Nurhadi MFA, Harjoprawito LA, et al. Comparing the Accuracy of ChatGPT-4o, DeepSeek-V3, and Gemini 2.5 Flash in Answering Frequently Asked Questions About Systemic Lupus Erythematosus: Quantitative Study. JMIR Formative Research. 2026;10:e89156-e89156. doi:10.2196/89156
BibTeX
@article{Widhani2026,
author = {Widhani, Alvina and Maria, Suzy and Chairani, Aisha Putri and Visco, Nabila Yulianingrum Adella and Nurhadi, Muhammad Faiz Amirullah and Harjoprawito, Lutfi Airlangga and Elvira, Dwitya and Tedja, Irwin and Yuniza, Yuniza and Fetarayani, Deasy and Koesnoe, Sukamto and Wicaksana, Bramantya and Hasibuan, Anshari Saifuddin and Yunihastuti, Evy},
title = {Comparing the Accuracy of ChatGPT-4o, DeepSeek-V3, and Gemini 2.5 Flash in Answering Frequently Asked Questions About Systemic Lupus Erythematosus: Quantitative Study},
journal = {JMIR Formative Research},
year = {2026},
volume = {10},
pages = {e89156--e89156},
doi = {10.2196/89156}
}
EUVIMED