Validation of a Dermatology-Focused Multimodal Large Language Model in Classification of Pigmented Skin Lesions
Diagnostics · 2025-11-06 · 15 (21) · 2808
DOI: 10.3390/diagnostics15212808 ↗At a glance
Journal article in Diagnostics, published online on 2025-11-06. Written by Joshua Mijares and 3 co-authors. Institution: Department of Dermatology, Indiana University School of Medicine, Indianapolis, IN 46202, USA. Cited 5 times so far (Crossref); 37 references. The full text is freely available (open access, CC BY 4.0). Funded by: NHLBI NIH HHS, NIH HHS, Short-Term Training Program in Biomedical Sciences Grant.
Abstract (original language)
Background
Artificial intelligence (AI) has shown significant promise in augmenting diagnostic capabilities across medical specialties. Recent advancements in generative AI allow for synthesis and interpretation of complex clinical data including imaging and patient history to assess disease risk.
Objective
To evaluate the diagnostic performance of a dermatology-trained multimodal large language model (DermFlow, Delaware, USA) in assessing malignancy risk of pigmented skin lesions.
Methods
This retrospective study utilized data from 59 patients with 68 biopsy-proven pigmented skin lesions seen at Indiana University clinics from February 2023 to May 2025. De-identified patient histories and clinical images were input into DermFlow, and clinical images only were input into Claude Sonnet 4 (Claude) to generate differential diagnoses. Clinician pre-operative diagnoses were extracted from the clinical note. Assessments were compared to histopathologic diagnoses (gold standard).
Results
Among 68 clinically concerning pigmented lesions, DermFlow achieved 47.1% top diagnosis accuracy and 92.6% any-diagnosis accuracy, with F1 = 0.948, sensitivity 93.9%, and specificity 89.5% (balanced accuracy 91.7%). Claude had 8.8% top diagnosis and 73.5% any-diagnosis accuracy, F1 = 0.816, sensitivity 81.6%, specificity 52.6% (balanced accuracy 67.1%). Clinicians achieved 38.2% top diagnosis and 72.1% any-diagnosis accuracy, F1 = 0.776, sensitivity 67.3%, specificity 84.2% (balanced accuracy 75.8%). DermFlow recommended biopsy in 95.6% of cases vs. 82.4% for Claude, with multiple pairwise differences favoring DermFlow (p < 0.05).
Conclusions
DermFlow demonstrated comparable or superior diagnostic performance to clinicians and superior performance to Claude in evaluating pigmented skin lesions. Although additional data must be gathered to further validate the model in real clinical settings, these initial findings suggest potential utility for dermatology-trained AI models in clinical practice, particularly in settings with limited dermatologist availability.
Subjects
Author keywords
Authors and institutions
- Department of Dermatology, Indiana University School of Medicine, Indianapolis, IN 46202, USA.
- Ronald O. Perelman Department of Dermatology, New York University Grossman School of Medicine, New York, NY 10016, USA.
ORCID profiles: Joshua Mijares
Funding
- NHLBI NIH HHS
- NIH HHS
- Short-Term Training Program in Biomedical Sciences Grant
- National Institutes of Health
Cite
APA
Mijares, J., Jairath, N., Zhang, A., & Que, S. K. T. (2025). Validation of a Dermatology-Focused Multimodal Large Language Model in Classification of Pigmented Skin Lesions. Diagnostics, 15(21), 2808. https://doi.org/10.3390/diagnostics15212808
Vancouver
Mijares J, Jairath N, Zhang A, Que SKT. Validation of a Dermatology-Focused Multimodal Large Language Model in Classification of Pigmented Skin Lesions. Diagnostics. 2025;15(21):2808. doi:10.3390/diagnostics15212808
BibTeX
@article{Mijares2025,
author = {Mijares, Joshua and Jairath, Neil and Zhang, Andrew and Que, Syril Keena T},
title = {Validation of a Dermatology-Focused Multimodal Large Language Model in Classification of Pigmented Skin Lesions},
journal = {Diagnostics},
year = {2025},
volume = {15},
number = {21},
pages = {2808},
doi = {10.3390/diagnostics15212808}
}
EUVIMED