General-purpose and medically adapted large language models versus oncologist consensus for immunotherapy videos
Frontiers in Public Health · 30.10.2026 · 14
DOI: 10.3389/fpubh.2026.1962857 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Public Health, online erschienen am 30.10.2026. Verfasst von Şendağ Yaslikaya und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Medical Oncology, Giresun Training and Research Hospital, Giresun, Türkiye. Hauptthemen laut MEDLINE: Large Language Models, Immunotherapy, Video Recording, Oncologists. Die Arbeit nennt 29 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Patients receiving immune checkpoint inhibitors depend on accurate education to recognize immune-related adverse events and to know when to contact their oncology team, and many rely on online video of highly variable quality. Expert review of such material is resource-intensive, and large language models (LLMs) are increasingly proposed to support preliminary screening; whether any model agrees with specialist judgment closely enough to do so safely, and how often models pass expert-inadequate material as adequate, is unknown.
Methods
In this cross-sectional content analysis (reported per STROBE), we evaluated anonymized transcripts of 90 English-language YouTube videos on cancer immunotherapy. Three medical oncologists and four LLMs — ChatGPT-5.5, Claude Opus 4.8, Gemini 3.1 Pro, and the medically adapted MedGemma 27B — independently scored every transcript using DISCERN, the Global Quality Score (GQS), and the Patient Education Materials Assessment Tool (PEMAT) understandability and actionability domains, each model in three separate runs, with expert majority consensus as the reference standard.
Results
Agreement among oncologists was good to excellent, whereas agreement among models was limited. By expert-majority classification, 35 of 90 transcripts (38.9%) met at least three of the four adequacy criteria and 24 (26.7%) met all four. ChatGPT-5.5 deviated least from expert scores on every instrument and classified adequacy most accurately (sensitivity 94.3%, 95% CI 81.4–98.4; specificity 89.1%, 78.2–94.9; Cohen’s kappa 0.82, 0.69–0.93). Gemini and MedGemma scored systematically more leniently under the tested conditions and produced 43 and 26 false-positive adequacy classifications, respectively, passing material the oncologists judged inadequate; the medically adapted model did not improve agreement in this evaluation, and raw popularity metrics did not identify high-quality content.
Discussion
The transcript-based educational content of most sampled videos did not meet all four adequacy criteria. These findings support further evaluation of selected model–prompt combinations for preliminary review of patient-education material, but they do not establish an operating threshold for clinical use, and the leniency observed in two models means expert review cannot be removed from the process.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Medical Oncology, Giresun Training and Research Hospital, Giresun, Türkiye.
- Department of Medical Oncology, Faculty of Medicine, Çukurova University, Adana, Türkiye.
- Department of Medical Oncology, Bartın State Hospital, Ministry of Health, Bartın, Türkiye.
Zitieren
APA
Yaslikaya, Ş., Ataş İpek, Ş., & Erdem Palaz, E. (2026). General-purpose and medically adapted large language models versus oncologist consensus for immunotherapy videos. Frontiers in Public Health, 14. https://doi.org/10.3389/fpubh.2026.1962857
Vancouver
Yaslikaya Ş, Ataş İpek Ş, Erdem Palaz E. General-purpose and medically adapted large language models versus oncologist consensus for immunotherapy videos. Frontiers in Public Health. 2026;14. doi:10.3389/fpubh.2026.1962857
BibTeX
@article{Yaslikaya2026,
author = {Yaslikaya, Şendağ and Ataş İpek, Şuheda and Erdem Palaz, Esra},
title = {General-purpose and medically adapted large language models versus oncologist consensus for immunotherapy videos},
journal = {Frontiers in Public Health},
year = {2026},
volume = {14},
doi = {10.3389/fpubh.2026.1962857}
}
EUVIMED