EUVIMEDEUVIMED entdecken
Open Access

Evaluating Sex and Age Biases in Multimodal Large Language Models for Skin Disease Identification from Dermatoscopic Images

Zhiyu Wan iD, Yuhang Guo, Shunxing Bao iD, Qian Wang iD, Bradley A Malin iD

Health Data Science · 2025-01 · 5

DOI: 10.34133/hds.0256 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Health Data Science, 2025-04. Verfasst von Zhiyu Wan und 4 weiteren Autor:innen. Beteiligte Einrichtung: Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 37 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar. Gefördert von: NHGRI NIH HHS, National Institutes of Health.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Multimodal large language models (LLMs) have shown potential in various health-related fields. However, many healthcare studies have raised concerns about the reliability and biases of LLMs in healthcare applications.

Methods

To explore the practical application of multimodal LLMs in skin disease identification, and to evaluate sex and age biases, we tested the performance of 2 popular multimodal LLMs, ChatGPT-4 and LLaVA-1.6, across diverse sex and age groups using a subset of a large dermatoscopic dataset containing around 10,000 images and 3 skin diseases (melanoma, melanocytic nevi, and benign keratosis-like lesions).

Results

In comparison to 3 deep learning models (VGG16, ResNet50, and Model Derm) based on convolutional neural network (CNN), one vision transformer model (Swin-B), we found that ChatGPT-4 and LLaVA-1.6 demonstrated overall accuracies that were 3% and 23% higher (and F1-scores that were 4% and 34% higher), respectively, than the best performing CNN-based baseline while maintaining accuracies that were 38% and 26% lower (and F1-scores that were 38% and 19% lower), respectively, than Swin-B. Meanwhile, ChatGPT-4 is generally unbiased in identifying these skin diseases across sex and age groups, while LLaVA-1.6 is generally unbiased across age groups, in contrast to Swin-B, which is biased in identifying melanocytic nevi.

Conclusions

This study suggests the usefulness and fairness of LLMs in dermatological applications, aiding physicians and practitioners with diagnostic recommendations and patient screening. To further verify and evaluate the reliability and fairness of LLMs in healthcare, experiments using larger and more diverse datasets need to be performed in the future.

Autor:innen und Institutionen

  • Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA.
  • School of Biomedical Engineering, ShanghaiTech University, Shanghai, China.
  • Department of Electrical and Computer Engineering, Vanderbilt University, Nashville, TN, USA.
  • Department of Computer Science, Vanderbilt University, Nashville, TN, USA.
  • Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA.

ORCID-Profile: Zhiyu Wan, Shunxing Bao, Qian Wang, Bradley A Malin

Förderung

  • NHGRI NIH HHS
  • National Institutes of Health

Zitieren

APA

Wan, Z., Guo, Y., Bao, S., Wang, Q., & Malin, B. A. (2025). Evaluating Sex and Age Biases in Multimodal Large Language Models for Skin Disease Identification from Dermatoscopic Images. Health Data Science, 5. https://doi.org/10.34133/hds.0256

Vancouver

Wan Z, Guo Y, Bao S, Wang Q, Malin BA. Evaluating Sex and Age Biases in Multimodal Large Language Models for Skin Disease Identification from Dermatoscopic Images. Health Data Science. 2025;5. doi:10.34133/hds.0256

BibTeX
@article{Wan2025,
  author = {Wan, Zhiyu and Guo, Yuhang and Bao, Shunxing and Wang, Qian and Malin, Bradley A},
  title = {Evaluating Sex and Age Biases in Multimodal Large Language Models for Skin Disease Identification from Dermatoscopic Images},
  journal = {Health Data Science},
  year = {2025},
  volume = {5},
  doi = {10.34133/hds.0256}
}
Kostenlos starten →