Artificial Intelligence and Gynecological Oncology: A Comparative Study of ChatGPT Omni and Gemini Pro across Repeated Intervals with Case-Scenario and Open-Ended Queries
Oncology Research and Treatment · 2025
DOI: 10.1159/000545231 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift Oncology Research and Treatment, 2025. Verfasst von Seckin Tuna Kaplan. Beteiligte Einrichtung: Division of Gynecologic Oncology, Department of Obstetrics and Gynecology, Kocaeli University, Faculty of Medicine, Izmit, Turkey. Hauptthemen laut MEDLINE: Artificial Intelligence, Genital Neoplasms, Female, Medical Oncology, Clinical Decision-Making. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 18 Literaturangaben.
Abstract (Originalsprache)
Introduction
Artificial intelligence (AI) models offer potential benefits in supporting clinical decision-making, diagnosis, and treatment. The study aimed to compare the performance of ChatGPT-4o (Omni) and Gemini Pro in answering clinical questions and case scenarios related to gynecological oncology and to assess the consistency of their long-term responses.
Methods
A two-phase comparative analysis was conducted. 700 clinical questions (350 per model) were developed and categorized into open-ended and case-scenario questions. Three months later, the same set of questions was presented again to evaluate any changes in performance for accuracy, completeness, and guideline adherence.
Results
Omni outperformed Gemini Pro across all question types (p = 0.001). Omni achieved a mean score of 5.9 for the basic open-ended questions, higher than Gemini, which had 5.1 (p = 0.001). It also maintained a clear advantage in complex, open-ended questions, scoring a mean of 5.6 than Gemini AI’s 4.2 (p = 0.001). Omni scored a mean of 5.7 for basic case scenarios, while Gemini AI lagged with a mean score of 5 (p = 0.001). Omni showed a modest improvement in complex, open-ended queries, with an increase of 0.2 points (+3.57%) (p = 0.001). Omni provided more accurate and comprehensive responses in guideline adherence than Gemini, particularly in complex cases requiring nuanced judgment and adherence to oncology protocols. Its responses aligned with the latest guidelines, including the American Society of Clinical Oncology and the National Comprehensive Cancer Network.
Conclusion
Omni is a more reliable and consistent model for answering questions related to gynecological cancers than Gemini. The stability of Omni’s performance over time highlights its potential as an effective tool for clinical applications requiring high accuracy and consistency.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Gynecologic Oncology, Department of Obstetrics and Gynecology, Kocaeli University, Faculty of Medicine, Izmit, Turkey.
Zitieren
APA
Kaplan, S. T. (2025). Artificial Intelligence and Gynecological Oncology: A Comparative Study of ChatGPT Omni and Gemini Pro across Repeated Intervals with Case-Scenario and Open-Ended Queries. Oncology Research and Treatment. https://doi.org/10.1159/000545231
Vancouver
Kaplan ST. Artificial Intelligence and Gynecological Oncology: A Comparative Study of ChatGPT Omni and Gemini Pro across Repeated Intervals with Case-Scenario and Open-Ended Queries. Oncology Research and Treatment. 2025. doi:10.1159/000545231
BibTeX
@article{Kaplan2025,
author = {Kaplan, Seckin Tuna},
title = {Artificial Intelligence and Gynecological Oncology: A Comparative Study of ChatGPT Omni and Gemini Pro across Repeated Intervals with Case-Scenario and Open-Ended Queries},
journal = {Oncology Research and Treatment},
year = {2025},
doi = {10.1159/000545231}
}
EUVIMED