Evaluating Sex and Age Biases in Multimodal Large Language Models for Skin Disease Identification from Dermatoscopic Images
Health Data Science · 2025-01 · 5
DOI: 10.34133/hds.0256 ↗Kurz gefasst
Fachartikel in der Zeitschrift Health Data Science, 2025-04. Verfasst von Zhiyu Wan und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NHGRI NIH HHS, National Institutes of Health.
Abstract (Originalsprache)
Background
Multimodal large language models (LLMs) have shown potential in various health-related fields. However, many healthcare studies have raised concerns about the reliability and biases of LLMs in healthcare applications.
Methods
To explore the practical application of multimodal LLMs in skin disease identification, and to evaluate sex and age biases, we tested the performance of 2 popular multimodal LLMs, ChatGPT-4 and LLaVA-1.6, across diverse sex and age groups using a subset of a large dermatoscopic dataset containing around 10,000 images and 3 skin diseases (melanoma, melanocytic nevi, and benign keratosis-like lesions).
Results
In comparison to 3 deep learning models (VGG16, ResNet50, and Model Derm) based on convolutional neural network (CNN), one vision transformer model (Swin-B), we found that ChatGPT-4 and LLaVA-1.6 demonstrated overall accuracies that were 3% and 23% higher (and F1-scores that were 4% and 34% higher), respectively, than the best performing CNN-based baseline while maintaining accuracies that were 38% and 26% lower (and F1-scores that were 38% and 19% lower), respectively, than Swin-B. Meanwhile, ChatGPT-4 is generally unbiased in identifying these skin diseases across sex and age groups, while LLaVA-1.6 is generally unbiased across age groups, in contrast to Swin-B, which is biased in identifying melanocytic nevi.
Conclusions
This study suggests the usefulness and fairness of LLMs in dermatological applications, aiding physicians and practitioners with diagnostic recommendations and patient screening. To further verify and evaluate the reliability and fairness of LLMs in healthcare, experiments using larger and more diverse datasets need to be performed in the future.
Autor:innen und Institutionen
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA.
- School of Biomedical Engineering, ShanghaiTech University, Shanghai, China.
- Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA.
- Department of Computer Science, Vanderbilt University, Nashville, TN, USA.
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA.
ORCID-Profile: Zhiyu Wan, Shunxing Bao, Qian Wang, Bradley A Malin
Förderung
- NHGRI NIH HHS
- National Institutes of Health
Zitieren
APA
Wan, Z., Guo, Y., Bao, S., Wang, Q., & Malin, B. A. (2025). Evaluating Sex and Age Biases in Multimodal Large Language Models for Skin Disease Identification from Dermatoscopic Images. Health Data Science, 5. https://doi.org/10.34133/hds.0256
Vancouver
Wan Z, Guo Y, Bao S, Wang Q, Malin BA. Evaluating Sex and Age Biases in Multimodal Large Language Models for Skin Disease Identification from Dermatoscopic Images. Health Data Science. 2025;5. doi:10.34133/hds.0256
BibTeX
@article{Wan2025,
author = {Wan, Zhiyu and Guo, Yuhang and Bao, Shunxing and Wang, Qian and Malin, Bradley A},
title = {Evaluating Sex and Age Biases in Multimodal Large Language Models for Skin Disease Identification from Dermatoscopic Images},
journal = {Health Data Science},
year = {2025},
volume = {5},
doi = {10.34133/hds.0256}
}
EUVIMED