Assessing the Accuracy, Completeness, and Reference Quality of GPT-4 and Google for Gynecologic Cancer Information: Comparative Quantitative Study
JMIR Cancer · 2026
DOI: 10.2196/76471 ↗Kurz gefasst
Vergleichsstudie in der Zeitschrift JMIR Cancer, 2026. Verfasst von Alexandra H Smick und 4 weiteren Autor:innen. Beteiligte Einrichtung: Gynecologic Oncology, University of California, Los Angeles, Los Angeles, CA, United States. Hauptthemen laut MEDLINE: Genital Neoplasms, Female, Search Engine, Patient Education as Topic. Die Arbeit nennt 23 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Patients with newly diagnosed gynecologic cancers often seek information online, but the quality of available resources may be inconsistent. Although GPT-4 may offer an alternative to traditional internet search engines, its performance remains largely under-studied in gynecologic oncology.
Objective
This study aimed to compare the completeness, accuracy, and reference quality of responses generated by GPT-4 with those generated by Google in clinical scenarios involving a new diagnosis of a gynecologic cancer.
Methods
Clinical scenarios representing early- and advanced-stage endometrial, ovarian, and cervical cancers were developed by gynecologic oncologists using publicly available patient education materials. Each scenario included 4 standardized questions addressing etiology, prognosis, treatment, and treatment efficacy. GPT-4 and Google were queried for each question, with new sessions for GPT-4 and private browsing for Google to minimize bias. Responses were independently evaluated by 4 gynecologic oncology experts who were blinded to each other’s ratings. Accuracy was scored on a 6-point Likert scale; completeness and reference quality were scored on 3-point scales. Reference quality was categorized as low (commercial), medium (institutional or government), or high (peer reviewed). Optional free-text reviewer comments were collected and summarized descriptively to provide context for the quantitative findings. Descriptive statistics and Wilcoxon signed-rank tests were used for analysis.
Results
Across 6 clinical scenarios and 21 standardized questions (N=84 total responses), GPT-4 outperformed Google across all evaluated domains. The median accuracy score was 6.00 (IQR 5.00-6.00) for GPT-4 and 5.00 (IQR 4.00-6.00) for Google (P=.04). The median completeness score was 3.00 (IQR 3.00-3.00) for GPT-4 and 2.00 (IQR 1.00-3.00) for Google (P=.009). Reference quality was also higher for GPT-4, with a median score of 3.00 (IQR 3.00-3.00) compared to 2.00 (IQR 2.00-2.00) for Google (P=.009). Reviewer comments noted that GPT-4 provided more accurate, comprehensive, and personalized responses, while Google returned less detailed content from general consumer health websites rather than peer-reviewed sources.
Conclusions
GPT-4 may serve as a reliable and high-quality resource for information in gynecologic oncology. Compared to Google, it delivered more accurate and complete content, with higher-quality references. Further research is needed to assess the readability and accessibility of GPT-4–generated content across diverse patient populations.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Gynecologic Oncology, University of California, Los Angeles, Los Angeles, CA, United States.
- Undergraduate College, University of California, Los Angeles, Los Angeles, CA, United States.
- Gynecologic Oncology, Olive View-UCLA Medical Center, Sylmar, CA, United States.
ORCID-Profile: Alexandra H Smick, Martins Ayoola, Rajeshree Rajpara, Isabel Lazo, Dana M Chase
Zitieren
APA
Smick, A. H., Ayoola, M., Rajpara, R., Lazo, I., & Chase, D. M. (2026). Assessing the Accuracy, Completeness, and Reference Quality of GPT-4 and Google for Gynecologic Cancer Information: Comparative Quantitative Study. JMIR Cancer. https://doi.org/10.2196/76471
Vancouver
Smick AH, Ayoola M, Rajpara R, Lazo I, Chase DM. Assessing the Accuracy, Completeness, and Reference Quality of GPT-4 and Google for Gynecologic Cancer Information: Comparative Quantitative Study. JMIR Cancer. 2026. doi:10.2196/76471
BibTeX
@article{Smick2026,
author = {Smick, Alexandra H and Ayoola, Martins and Rajpara, Rajeshree and Lazo, Isabel and Chase, Dana M},
title = {Assessing the Accuracy, Completeness, and Reference Quality of GPT-4 and Google for Gynecologic Cancer Information: Comparative Quantitative Study},
journal = {JMIR Cancer},
year = {2026},
doi = {10.2196/76471}
}
EUVIMED