EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Exploratory Evaluation of Diagnostic Accuracy and Temporal Reproducibility of Multimodal Large Language Models in the Image-Based Assessment of Oral Mucosal Lesions

Lovro Dumančić, Marko Antonio Cug, Danica Vidović Juras, Luís Monteiro, Rui Albuquerque, Vlaho Brailo

Applied Sciences · 21.04.2026 · 16 (8) · 4046

DOI: 10.3390/app16084046 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 21.04.2026. Verfasst von Lovro Dumančić und 5 weiteren Autor:innen. Die Arbeit nennt 26 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objective

The aim was to evaluate the diagnostic accuracy and temporal reproducibility of multimodal large language models (LLMs) in the image-based diagnosis of oral mucosal lesions. Materials and Methods: The study included 100 anonymized clinical photographs of oral mucosal conditions obtained from the archive of the Department of Oral Medicine, School of Dental Medicine, University of Zagreb. Images were categorized into four subgroups: physiological variations, benign mucosal lesions, oral potentially malignant disorders, and oral cancer (25 images each). Three multimodal LLMs (ChatGPT-5.1 Plus, Gemini 3 Pro, and Perplexity Pro) analyzed each image using an identical prompt and were required to provide a single most probable diagnosis based solely on visual features. To evaluate temporal reproducibility, the entire evaluation was repeated in three independent testing cycles conducted at one-month intervals. Diagnostic accuracy was compared using chi-square tests, while intra-model agreement across cycles was assessed using Fleiss’ kappa.

Results

Gemini demonstrated the highest diagnostic accuracy, reaching 78% correct responses in cycles 2 and 3, significantly outperforming ChatGPT (55–57%) and Perplexity (28–31%) (p < 0.00001). Subgroup analyses showed similar trends, with Gemini achieving the highest accuracy across most lesion categories. Intra-model agreement across cycles was moderate for ChatGPT (κ = 0.525), fair for Gemini (κ = 0.338) and Perplexity (κ = 0.409). Gemini also showed the highest proportion of responses that remained correct across all three cycles (51%).

Conclusions

Multimodal LLMs demonstrate promising diagnostic capabilities in the image-based assessment of oral mucosal lesions; however, variability in reproducibility highlights the need for cautious clinical implementation and further validation.

Zitieren

APA

Dumančić, L., Cug, M. A., Juras, D. V., Monteiro, L., Albuquerque, R., & Brailo, V. (2026). Exploratory Evaluation of Diagnostic Accuracy and Temporal Reproducibility of Multimodal Large Language Models in the Image-Based Assessment of Oral Mucosal Lesions. Applied Sciences, 16(8), 4046. https://doi.org/10.3390/app16084046

Vancouver

Dumančić L, Cug MA, Juras DV, Monteiro L, Albuquerque R, Brailo V. Exploratory Evaluation of Diagnostic Accuracy and Temporal Reproducibility of Multimodal Large Language Models in the Image-Based Assessment of Oral Mucosal Lesions. Applied Sciences. 2026;16(8):4046. doi:10.3390/app16084046

BibTeX
@article{Dumani2026,
  author = {Dumančić, Lovro and Cug, Marko Antonio and Juras, Danica Vidović and Monteiro, Luís and Albuquerque, Rui and Brailo, Vlaho},
  title = {Exploratory Evaluation of Diagnostic Accuracy and Temporal Reproducibility of Multimodal Large Language Models in the Image-Based Assessment of Oral Mucosal Lesions},
  journal = {Applied Sciences},
  year = {2026},
  volume = {16},
  number = {8},
  pages = {4046},
  doi = {10.3390/app16084046}
}
Kostenlos starten →