Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios
Applied Clinical Informatics · 01.05.2026 · 17 (03) · 518-525
DOI: 10.1055/a-2899-0123 ↗Kurz gefasst
Evaluationsstudie in der Zeitschrift Applied Clinical Informatics, erschienen am 01.05.2026. Verfasst von Arif Onur Atay und 3 weiteren Autor:innen. Beteiligte Einrichtung: Torbali State Hospital, Department of Obstetrics and Gynecology, Türkiye, Izmir. Hauptthemen laut MEDLINE: Obstetrics, Gynecology, Clinical Decision-Making. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Clinicians frequently face questions that require rapid, evidence-based answers. Artificial intelligence (AI) tools are increasingly used for this purpose, yet their reliability for clinical decision-making remains uncertain.
Objectives
This study compared two generative large language model systems (ChatGPT and Gemini) and a retrieval-supported clinical platform (OpenEvidence) to determine which provides the most reliable, clear, and clinically applicable information in obstetrics, gynecology, and urogynecology.
Methods
A cross-sectional comparative design was used to evaluate ChatGPT (GPT-5), Gemini (Gemini 2.5), and the retrieval-supported platform OpenEvidence. Twenty-four clinical questions across three subspecialties were independently assessed by two blinded specialists using the Expert-Adapted DISCERN (EA-DISCERN) tool, which rates 12 quality domains on a 5-point scale. Mean ± standard deviation scores were compared across systems and clinical domains using repeated-measures analysis.
Results
OpenEvidence achieved the highest mean total score (54.0 ± 2.3), outperforming Gemini (50.3 ± 2.4) and ChatGPT (48.7 ± 2.4; p < 0.001). OpenEvidence scored significantly higher in evidence-based domains; clinical accuracy, guideline consistency, completeness, transparency, and reliability across all fields. As of this writing, Gemini ranked between the two, showing a modest advantage over ChatGPT in rationale explanation and evidence transparency, whereas both generative models scored higher in language fluency and readability. Overall, total EA-DISCERN scores ranked OpenEvidence highest, followed by Gemini, then ChatGPT. Interrater reliability for the total score was intraclass correlation coefficient [2,1] (absolute agreement = 0.391).
Conclusion
OpenEvidence provided more guideline-aligned and transparent responses, whereas ChatGPT and Gemini were generally more fluent and readable. For obstetrics and gynecology clinicians, retrieval-supported platforms may be more suitable for point-of-care verification, while generative models should be used more cautiously and with clinician oversight.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Torbali State Hospital, Department of Obstetrics and Gynecology, Türkiye, Izmir.
- Ege University Faculty of Medicine, Department of Obstetrics and Gynecology, Türkiye, Izmir.
ORCID-Profile: Arif Onur Atay
Zitieren
APA
Atay, A. O., Atay, F., Ozmen, S., & Balci, M. F. (2026). Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios. Applied Clinical Informatics, 17(03), 518-525. https://doi.org/10.1055/a-2899-0123
Vancouver
Atay AO, Atay F, Ozmen S, Balci MF. Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios. Applied Clinical Informatics. 2026;17(03):518-525. doi:10.1055/a-2899-0123
BibTeX
@article{Atay2026,
author = {Atay, Arif Onur and Atay, Feride and Ozmen, Samican and Balci, Mucahit Furkan},
title = {Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios},
journal = {Applied Clinical Informatics},
year = {2026},
volume = {17},
number = {03},
pages = {518--525},
doi = {10.1055/a-2899-0123}
}
EUVIMED