Accuracy of ChatGPT‐4 Plus in Providing Information on Oral Cancer Management
Oral Diseases · 2025
DOI: 10.1111/odi.70110 ↗Kurz gefasst
Fachartikel in der Zeitschrift Oral Diseases, 2025. Verfasst von Busra Yilmaz und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Oral Medicine and Maxillofacial Radiology, School of Dental Medicine, Istinye University, Istanbul, Turkey. Hauptthemen laut MEDLINE: Mouth Neoplasms, Artificial Intelligence, Patient Education as Topic, Decision Support Systems, Clinical. Die Arbeit nennt 50 Literaturangaben.
Abstract (Originalsprache)
Objective
Artificial intelligence (AI)‐driven large language models, such as Chat Generative Pre‐Trained Transformer (ChatGPT)‐4 Plus, are increasingly used for patient education and clinical decision support in oral oncology, although their accuracy in oral cancer (OC) management remains uncertain. This study evaluates the accuracy of ChatGPT‐4 Plus responses to clinically relevant questions regarding OC diagnosis, treatment, recovery, and prevention.
Methods
A cross‐sectional study assessed 65 clinically relevant OC‐related questions using a paid ChatGPT‐4 Plus subscription without modifications. Three oral medicine specialists and one radiation oncologist rated accuracy on a four‐point scoring system. Interrater reliability was measured with the intraclass correlation coefficient (ICC), and chi‐square tests were used for comparisons.
Results
Among 65 questions, 63% of responses were Score 1, with none rated as Score 4. Score 1 was most frequent in Recovery (72%), followed by Treatment (62%), Prevention (60%), and Diagnosis (55%). Scores 2 and 3 responses were highest in Diagnosis (45%). Recovery had significantly higher Score 1 responses than Diagnosis ( p < 0.05), while other comparisons were not significant. ICC ranged from 0.85 to 0.93.
Conclusions
ChatGPT‐4 Plus provided accurate responses to clinically relevant OC‐related questions, particularly regarding recovery. However, diagnostic inconsistencies highlight the need for clinician oversight before integrating AI into practice.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Oral Medicine and Maxillofacial Radiology, School of Dental Medicine, Istinye University, Istanbul, Turkey.
- Department of Nanomedicine, Houston Methodist Research Institute, Houston, Texas, USA.
- Department of Oral Medicine/Oral & Maxillofacial Surgery, Atrium Health Carolinas Medical Center, Charlotte, North Carolina, USA.
- Department of Otolaryngology/Head & Neck Surgery, Wake Forest University School of Medicine, Winston-Salem, North Carolina, USA.
- Department of Radiation Oncology, Levine Cancer Institute, Atrium Health, Charlotte, North Carolina, USA.
- Department of Supportive Oncology, Levine Cancer Institute, Atrium Health, Charlotte, North Carolina, USA.
ORCID-Profile: Busra Yilmaz, Emine Nur Kahraman, Michael T Brennan, Amardeep S Grewal, Aynur Aktas
Zitieren
APA
Yilmaz, B., Kahraman, E. N., Brennan, M. T., Grewal, A. S., & Aktas, A. (2025). Accuracy of ChatGPT‐4 Plus in Providing Information on Oral Cancer Management. Oral Diseases. https://doi.org/10.1111/odi.70110
Vancouver
Yilmaz B, Kahraman EN, Brennan MT, Grewal AS, Aktas A. Accuracy of ChatGPT‐4 Plus in Providing Information on Oral Cancer Management. Oral Diseases. 2025. doi:10.1111/odi.70110
BibTeX
@article{Yilmaz2025,
author = {Yilmaz, Busra and Kahraman, Emine Nur and Brennan, Michael T and Grewal, Amardeep S and Aktas, Aynur},
title = {Accuracy of ChatGPT‐4 Plus in Providing Information on Oral Cancer Management},
journal = {Oral Diseases},
year = {2025},
doi = {10.1111/odi.70110}
}
EUVIMED