Diagnosis of malignancy in oropharyngeal confocal laser endomicroscopy using GPT 4.0 with vision
European Archives of Oto-Rhino-Laryngology · 2024
DOI: 10.1007/s00405-024-08476-5 ↗Kurz gefasst
Fachartikel in der Zeitschrift European Archives of Oto-Rhino-Laryngology, 2024. Verfasst von Matti Sievert und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Otorhinolaryngology, Head and Neck Surgery, Friedrich Alexander University of Erlangen-Nuremberg, Erlangen University Hospital, Erlange…. Hauptthemen laut MEDLINE: Head and Neck Neoplasms. Die Arbeit wurde bisher 24-mal zitiert (Crossref) und nennt 27 Literaturangaben. Gefördert von: Deutsche Forschungsgemeinschaft.
Abstract (Originalsprache)
PURPOSE: Confocal Laser Endomicroscopy (CLE) is an imaging tool, that has demonstrated potential for intraoperative, real-time, non-invasive, microscopical assessment of surgical margins of oropharyngeal squamous cell carcinoma (OPSCC). However, interpreting CLE images remains challenging. This study investigates the application of OpenAI's Generative Pretrained Transformer (GPT) 4.0 with Vision capabilities for automated classification of CLE images in OPSCC. METHODS: CLE Images of histological confirmed SCC or healthy mucosa from a database of 12 809 CLE images from 5 patients with OPSCC were retrieved and anonymized. Using a training data set of 16 images, a validation set of 139 images, comprising SCC (83 images, 59.7%) and healthy normal mucosa (56 images, 40.3%) was classified using the application programming interface (API) of GPT4.0. The same set of images was also classified by CLE experts (two surgeons and one pathologist), who were blinded to the histology. Diagnostic metrics, the reliability of GPT and inter-rater reliability were assessed. RESULTS: Overall accuracy of the GPT model was 71.2%, the intra-rater agreement was κ = 0.837, indicating an almost perfect agreement across the three runs of GPT-generated results. Human experts achieved an accuracy of 88.5% with a substantial level of agreement (κ = 0.773). CONCLUSIONS: Though limited to a specific clinical framework, patient and image set, this study sheds light on some previously unexplored diagnostic capabilities of large language models using few-shot prompting. It suggests the model`s ability to extrapolate information and classify CLE images with minimal example data. Whether future versions of the model can achieve clinically relevant diagnostic accuracy, especially in uncurated data sets, remains to be investigated.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Otorhinolaryngology, Head and Neck Surgery, Friedrich Alexander University of Erlangen-Nuremberg, Erlangen University Hospital, Erlangen, Germany.
- Technische Hochschule Ingolstadt, Ingolstadt, Germany.
- Institute of Pathology, Friedrich-Alexander-Universität Erlangen-Nürnberg, University Hospital, Erlangen, Germany.
- Department of Artificial Intelligence in Biomedical Engineering, Friedrich-Alexander-Universität Erlangen-Nürnberg, Erlangen, Germany.
- Department of Otorhinolaryngology, Plastic and Aesthetic Operations, University Hospital Würzburg, Joseph-Schneider-Straße 11, 97080, Würzburg, Germany. Goncalves_M@ukw.de.
Förderung
- Deutsche Forschungsgemeinschaft
Zitieren
APA
Sievert, M., Aubreville, M., Mueller, S. K., Eckstein, M., Breininger, K., Iro, H., & Goncalves, M. (2024). Diagnosis of malignancy in oropharyngeal confocal laser endomicroscopy using GPT 4.0 with vision. European Archives of Oto-Rhino-Laryngology. https://doi.org/10.1007/s00405-024-08476-5
Vancouver
Sievert M, Aubreville M, Mueller SK, Eckstein M, Breininger K, Iro H, et al. Diagnosis of malignancy in oropharyngeal confocal laser endomicroscopy using GPT 4.0 with vision. European Archives of Oto-Rhino-Laryngology. 2024. doi:10.1007/s00405-024-08476-5
BibTeX
@article{Sievert2024,
author = {Sievert, Matti and Aubreville, Marc and Mueller, Sarina Katrin and Eckstein, Markus and Breininger, Katharina and Iro, Heinrich and Goncalves, Miguel},
title = {Diagnosis of malignancy in oropharyngeal confocal laser endomicroscopy using GPT 4.0 with vision},
journal = {European Archives of Oto-Rhino-Laryngology},
year = {2024},
doi = {10.1007/s00405-024-08476-5}
}
EUVIMED