EUVIMEDEUVIMED entdecken
EvaluationsstudieOpen Access

Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios

Arif Onur Atay iD, Feride Atay, Samican Ozmen, Mucahit Furkan Balci

Applied Clinical Informatics · 01.05.2026 · 17 (03) · 518-525

DOI: 10.1055/a-2899-0123 ↗

Kurz gefasst

Evaluationsstudie in der Zeitschrift Applied Clinical Informatics, erschienen am 01.05.2026. Verfasst von Arif Onur Atay und 3 weiteren Autor:innen. Beteiligte Einrichtung: Torbali State Hospital, Department of Obstetrics and Gynecology, Türkiye, Izmir. Hauptthemen laut MEDLINE: Obstetrics, Gynecology, Clinical Decision-Making. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Clinicians frequently face questions that require rapid, evidence-based answers. Artificial intelligence (AI) tools are increasingly used for this purpose, yet their reliability for clinical decision-making remains uncertain.

Objectives

This study compared two generative large language model systems (ChatGPT and Gemini) and a retrieval-supported clinical platform (OpenEvidence) to determine which provides the most reliable, clear, and clinically applicable information in obstetrics, gynecology, and urogynecology.

Methods

A cross-sectional comparative design was used to evaluate ChatGPT (GPT-5), Gemini (Gemini 2.5), and the retrieval-supported platform OpenEvidence. Twenty-four clinical questions across three subspecialties were independently assessed by two blinded specialists using the Expert-Adapted DISCERN (EA-DISCERN) tool, which rates 12 quality domains on a 5-point scale. Mean ± standard deviation scores were compared across systems and clinical domains using repeated-measures analysis.

Results

OpenEvidence achieved the highest mean total score (54.0 ± 2.3), outperforming Gemini (50.3 ± 2.4) and ChatGPT (48.7 ± 2.4; p < 0.001). OpenEvidence scored significantly higher in evidence-based domains; clinical accuracy, guideline consistency, completeness, transparency, and reliability across all fields. As of this writing, Gemini ranked between the two, showing a modest advantage over ChatGPT in rationale explanation and evidence transparency, whereas both generative models scored higher in language fluency and readability. Overall, total EA-DISCERN scores ranked OpenEvidence highest, followed by Gemini, then ChatGPT. Interrater reliability for the total score was intraclass correlation coefficient [2,1] (absolute agreement = 0.391).

Conclusion

OpenEvidence provided more guideline-aligned and transparent responses, whereas ChatGPT and Gemini were generally more fluent and readable. For obstetrics and gynecology clinicians, retrieval-supported platforms may be more suitable for point-of-care verification, while generative models should be used more cautiously and with clinician oversight.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ObstetricsGynecologyClinical Decision-MakingLarge Language ModelsGenerative Artificial IntelligenceHumansFemale

Autor:innen und Institutionen

  • Torbali State Hospital, Department of Obstetrics and Gynecology, Türkiye, Izmir.
  • Ege University Faculty of Medicine, Department of Obstetrics and Gynecology, Türkiye, Izmir.

ORCID-Profile: Arif Onur Atay

Zitieren

APA

Atay, A. O., Atay, F., Ozmen, S., & Balci, M. F. (2026). Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios. Applied Clinical Informatics, 17(03), 518-525. https://doi.org/10.1055/a-2899-0123

Vancouver

Atay AO, Atay F, Ozmen S, Balci MF. Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios. Applied Clinical Informatics. 2026;17(03):518-525. doi:10.1055/a-2899-0123

BibTeX
@article{Atay2026,
  author = {Atay, Arif Onur and Atay, Feride and Ozmen, Samican and Balci, Mucahit Furkan},
  title = {Evaluation of ChatGPT, Gemini, and OpenEvidence in Obstetric and Gynecologic Clinical Decision Scenarios},
  journal = {Applied Clinical Informatics},
  year = {2026},
  volume = {17},
  number = {03},
  pages = {518--525},
  doi = {10.1055/a-2899-0123}
}
Kostenlos starten →