EUVIMEDEUVIMED entdecken
VergleichsstudieQuerschnittstudie

A Comparison of Prostate Cancer Screening Information Quality on Standard and Advanced Versions of ChatGPT, Google Gemini, and Microsoft Copilot: A Cross-Sectional Study

Otis L Owens iD, Michael Leonard

American Journal of Health Promotion · 2025

DOI: 10.1177/08901171251316371 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift American Journal of Health Promotion, 2025. Verfasst von Otis L Owens und 1 weiteren Autor:innen. Beteiligte Einrichtung: College of Social Work, University of South Carolina, Columbia, SC, USA. Hauptthemen laut MEDLINE: Prostatic Neoplasms, Early Detection of Cancer, Consumer Health Information. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 62 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

Artificially Intelligent (AI) chatbots have the potential to produce information to support shared prostate cancer (PrCA) decision-making. Therefore, our purpose was to evaluate and compare the accuracy, completeness, readability, and credibility of responses from standard and advanced versions of popular chatbots: ChatGPT-3.5, ChatGPT-4.0, Microsoft Copilot, Microsoft Copilot Pro, Google Gemini, and Google Gemini Advanced. We also investigated whether prompting chatbots for low-literacy PrCA information would improve the readability of responses. Lastly, we determined if the responses were appropriate for African-American men, who have the worst PrCA outcomes. Approach The study used a cross-sectional approach to examine the quality of responses solicited from chatbots.

Participants

The study did not include human subjects.

Method

Eleven frequently asked PrCA questions, based on resources produced by the Centers for Disease Control and Prevention (CDC) and the American Cancer Society (ACS), were posed to each chatbot twice (once for low literacy populations). A coding/rating form containing questions with key points/answers from the ACS or CDC to facilitate the rating process. Accuracy and completeness were rated dichotomously (i.e., yes/no). Credibility was determined by whether a trustworthy medical or health-related organization was cited. Readability was determined using a Flesch-Kincaid readability score calculator that enabled chatbot responses to be entered individually. Average accuracy, completeness, credibility, and readability percentages or scores were calculated using Excel.

Results

All chatbots were accurate, but the completeness, readability, and credibility of responses varied. Soliciting low-literacy responses significantly improved readability, but sometimes at the detriment of completeness. All chatbots recognized the higher PrCA risk in African-American men and tailored screening recommendations. Microsoft Copilot Pro had the best overall performance on standard screening questions. Microsoft Copilot outperformed other chatbots on responses for low literacy populations.

Conclusions

AI chatbots are useful tools for learning about PrCA screening but should be combined with healthcare provider advice.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Prostatic NeoplasmsEarly Detection of CancerConsumer Health InformationHumansMaleCross-Sectional StudiesMiddle AgedHealth LiteracyComprehensionUnited StatesAgedBlack or African AmericanGenerative Artificial Intelligence

Stichwörter der Autor:innen

African Americanartificial intelligencecancer screeningmachine learningprostate cancershared decision making

Autor:innen und Institutionen

  • College of Social Work, University of South Carolina, Columbia, SC, USA.

ORCID-Profile: Otis L Owens

Zitieren

APA

Owens, O. L., & Leonard, M. (2025). A Comparison of Prostate Cancer Screening Information Quality on Standard and Advanced Versions of ChatGPT, Google Gemini, and Microsoft Copilot: A Cross-Sectional Study. American Journal of Health Promotion. https://doi.org/10.1177/08901171251316371

Vancouver

Owens OL, Leonard M. A Comparison of Prostate Cancer Screening Information Quality on Standard and Advanced Versions of ChatGPT, Google Gemini, and Microsoft Copilot: A Cross-Sectional Study. American Journal of Health Promotion. 2025. doi:10.1177/08901171251316371

BibTeX
@article{Owens2025,
  author = {Owens, Otis L and Leonard, Michael},
  title = {A Comparison of Prostate Cancer Screening Information Quality on Standard and Advanced Versions of ChatGPT, Google Gemini, and Microsoft Copilot: A Cross-Sectional Study},
  journal = {American Journal of Health Promotion},
  year = {2025},
  doi = {10.1177/08901171251316371}
}
Kostenlos starten →