EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Systematic Failure of Vision Transformers in Imbalanced Skin Lesion Classification

Serra Aksoy, Pinar Demircioglu, Ismail Bogrekci

Dermato · 11.06.2026 · 6 (2) · 22

DOI: 10.3390/dermato6020022 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Dermato, online erschienen am 11.06.2026. Verfasst von Serra Aksoy und 2 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Scientific Research Projects Commission of Aydin Adnan Menderes University, Research Projects Supporting Publication Continuity.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background/Objectives: Vision Transformers (ViTs) have demonstrated impressive performance in dealing with large-scale natural image datasets. They have started to be used in medical image classification problems as well. However, how they behave under real-world conditions, such as data scarcity and extreme class imbalance, has not been well investigated. In this study, we examine the feasibility of using a standard Vision Transformer Base model that learned from scratch how to classify skin lesion images into multiple classes using the ISIC 2019 dataset.

Methods

The Vision Transformer architecture was trained from scratch using stratified splitting of the data, class-balanced cross-entropy loss, multi-seed initialization, and control of hyperparameters such as patch size and dropout rate. The evaluation of the Vision Transformer architecture was performed using a hold-out test set with metrics such as accuracy, macro-F1, weighted-F1, and analysis of the confusion matrix.

Results

Across all configurations, the training exhibited substantial instability and consistent overfitting behavior, with an average accuracy gap between validation and test sets of 22.7%. Test accuracy ranged from 8.0% to 37.8%, showing high sensitivity to initialization. For minority classes, the F1-score remained very low (F1 < 0.05) even though the classes were balanced in the loss function.

Conclusions

The results indicate that a standard ViT-Base model trained from scratch can exhibit pronounced instability and a tendency toward majority-class bias when applied to multi-class skin lesion classification under conditions of extreme class imbalance and data scarcity. The findings point to the limitations of using simple transformer models without pre-training or other forms of inductive bias in scarce data settings.

Förderung

  • Scientific Research Projects Commission of Aydin Adnan Menderes University, Research Projects Supporting Publication Continuity

Zitieren

APA

Aksoy, S., Demircioglu, P., & Bogrekci, I. (2026). Systematic Failure of Vision Transformers in Imbalanced Skin Lesion Classification. Dermato, 6(2), 22. https://doi.org/10.3390/dermato6020022

Vancouver

Aksoy S, Demircioglu P, Bogrekci I. Systematic Failure of Vision Transformers in Imbalanced Skin Lesion Classification. Dermato. 2026;6(2):22. doi:10.3390/dermato6020022

BibTeX
@article{Aksoy2026,
  author = {Aksoy, Serra and Demircioglu, Pinar and Bogrekci, Ismail},
  title = {Systematic Failure of Vision Transformers in Imbalanced Skin Lesion Classification},
  journal = {Dermato},
  year = {2026},
  volume = {6},
  number = {2},
  pages = {22},
  doi = {10.3390/dermato6020022}
}
Kostenlos starten →