Diagnostic Accuracy of Commercial Large Language Models for Anogenital Skin Lesion Images: A Comparative Study of Gemini, Claude, and ChatGPT
Diagnostische Genauigkeit kommerzieller Large Language Models für Bilder anogenitaler Hautläsionen: Vergleich von Gemini, Claude und ChatGPT
The Journal of Infectious Diseases · 02.09.2026 · 234 (2) · e330-e337
DOI: 10.1093/infdis/jiag258 ↗Auf einen Blick
Für 218 Bilder anogenitaler Hautveränderungen erreichte Gemini die beste diagnostische Genauigkeit. Insgesamt war die Leistung der LLMs jedoch begrenzt, vor allem bei diagnostisch schwierigen Bildern.
Fragestellung
Wie genau sind Gemini 2.5 Pro, Claude Opus 4.1 und ChatGPT 5 Thinking bei der Diagnostik anogenitaler dermatologischer Bilder?
Methode
Vergleichende Studie mit deidentifizierten klinischen Bildern aus dem STI Atlas und weiteren öffentlich verfügbaren Quellen; durchgeführt zwischen September und November 2025.
Ergebnisse
218 Bilder. Gemini: STI-Binärklassifikation 76,2 % (95 %-KI 70,5–81,9); Top-1 39,0 % (95 %-KI 32,7–45,7); Top-3 54,6 % (95 %-KI 47,9–61,1); Top-5 60,6 % (95 %-KI 53,9–66,9).
Einschränkungen
Keine der LLMs erkannte Mpox korrekt; bei diagnostisch schwierigen Bildern sank die Top-5-Genauigkeit auf 29,2 %–40,0 %.
Schlussfolgerung der Autor:innen
LLMs zeigten eine begrenzte Genauigkeit bei der Diagnose anogenitaler dermatologischer Erkrankungen. Gemini war am besten, erreichte aber nur 39,0 % für die Top-1-Diagnose.
Kernaussagen
- Gemini war bei der STI-Binärklassifikation am besten
- Top-1-Diagnose mit Gemini: 39,0 %
- Bei schwierigen Bildern sank die Top-5-Genauigkeit deutlich
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Vergleichsstudie in der Zeitschrift The Journal of Infectious Diseases, online erschienen am 09.05.2026. Verfasst von Nyi Nyi Soe und 7 weiteren Autor:innen. Beteiligte Einrichtung: Melbourne Sexual Health Centre, Bayside Health, Melbourne, Victoria, Australia. Hauptthemen laut MEDLINE: Large Language Models, Sexually Transmitted Diseases, Skin Diseases, Anus Diseases. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Health and Medical Research Council.
Abstract (Originalsprache)
Background
Diagnosing anogenital dermatological conditions often requires specialist expertise that is unavailable in many clinical settings. Large language models (LLMs) are increasingly accessible to clinicians, but their diagnostic accuracy for anogenital dermatology has not been evaluated. We evaluated the diagnostic accuracy of 3 LLMs (Gemini 2.5 Pro, Claude Opus 4.1, and ChatGPT 5 Thinking) for anogenital dermatological conditions.
Methods
This study was conducted between September and November 2025, using deidentified clinical images of anogenital conditions from the STI Atlas (stiatlas.org) and other publicly available sources. Primary outcomes were correct classification of images identified as sexually transmitted infections (STIs) vs non-STIs and the inclusion of the correct diagnosis among the LLMs’ top-ranked (top-1), top-3, or top-5 differential diagnoses.
Results
Among 218 images, Gemini achieved the highest accuracy for STI binary classification (76.2% [95% CI, 70.5%–81.9%]) and differential diagnosis (top-1, 39.0% [95% CI, 32.7%–45.7%]; top-3, 54.6% [95% CI, 47.9%–61.1%]; top-5, 60.6% [95% CI, 53.9%–66.9%]), followed by ChatGPT and Claude. In subgroup analysis, all LLMs showed substantially reduced accuracy for diagnostically challenging images (top-5 accuracy range, 29.2%–40.0%). Gemini consistently outperformed Claude across most subgroups (P < .05). None of the LLMs could identify any mpox correctly.
Conclusions
LLMs showed limited accuracy for diagnosing anogenital dermatological conditions, particularly for challenging images. The best-performing model (Gemini) achieved only 39.0% for top-1 diagnosis, indicating that current LLMs cannot reliably diagnose anogenital conditions. These tools may support supervised clinical triage but need further validation before routine clinical use. Future studies should compare LLMs with clinicians and explore how they can assist clinical diagnosis.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Melbourne Sexual Health Centre, Bayside Health, Melbourne, Victoria, Australia.
- School of Translational Medicine, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Victoria, Australia.
- Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, Victoria, Australia.
ORCID-Profile: Nyi Nyi Soe, Phyu Mon Latt, David Lee, Ei T Aung, Ryan Horn, Jason J Ong
Förderung
- National Health and Medical Research Council
Zitieren
APA
Soe, N. N., Latt, P. M., Lee, D., Aung, E. T., Horn, R., Ong, J. J., Fairley, C. K., & Chow, E. P. F. (2026). Diagnostic Accuracy of Commercial Large Language Models for Anogenital Skin Lesion Images: A Comparative Study of Gemini, Claude, and ChatGPT. The Journal of Infectious Diseases, 234(2), e330-e337. https://doi.org/10.1093/infdis/jiag258
Vancouver
Soe NN, Latt PM, Lee D, Aung ET, Horn R, Ong JJ, et al. Diagnostic Accuracy of Commercial Large Language Models for Anogenital Skin Lesion Images: A Comparative Study of Gemini, Claude, and ChatGPT. The Journal of Infectious Diseases. 2026;234(2):e330-e337. doi:10.1093/infdis/jiag258
BibTeX
@article{Soe2026,
author = {Soe, Nyi Nyi and Latt, Phyu Mon and Lee, David and Aung, Ei T and Horn, Ryan and Ong, Jason J and Fairley, Christopher K and Chow, Eric P F},
title = {Diagnostic Accuracy of Commercial Large Language Models for Anogenital Skin Lesion Images: A Comparative Study of Gemini, Claude, and ChatGPT},
journal = {The Journal of Infectious Diseases},
year = {2026},
volume = {234},
number = {2},
pages = {e330--e337},
doi = {10.1093/infdis/jiag258}
}
EUVIMED