EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Ensemble-Based Multimodal Deep Learning for Precise Skin Cancer Diagnosis: Integrating Clinical Imagery with Patient Metadata

Wyssem Fathallah, M’hamed Abid, Mourad Mars, Hedi Sakli

Technologies · 02.05.2026 · 14 (5) · 277

DOI: 10.3390/technologies14050277 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Technologies, online erschienen am 02.05.2026. Verfasst von Wyssem Fathallah und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Umm al-Qura University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rising incidence of skin cancer necessitates scalable and accurate diagnostic tools. While dermoscopy-based systems have achieved expert-level performance, clinical smartphone images pose challenges due to variability in lighting, resolution, and artifacts. Recent advances in multimodal deep learning have shown promise, yet most approaches rely on simple feature concatenation or single-model classifiers, limiting their ability to capture complex cross-modal interactions. This study aims to bridge the diagnostic gap in resource-limited settings by developing a robust multimodal framework that synergizes clinical smartphone images with structured patient metadata for automated skin cancer classification. We propose a novel hybrid architecture integrating a Swin Transformer V2 (SwinV2-Tiny) for hierarchical visual feature extraction and a Denoising Autoencoder (DAE) with PCA for robust metadata embedding. These heterogeneous modalities are fused via a Gated Attention Mechanism that dynamically weighs feature importance across streams. Classification is performed by a Heterogeneous Meta-Stack Ensemble comprising CatBoost, LightGBM, XGBoost, and Logistic Regression, designed to maximize calibration and generalization across imbalanced classes. Evaluated on the PAD-UFES-20 dataset (2298 clinical smartphone images, six diagnostic classes), the proposed framework achieves state-of-the-art performance with a macro-averaged F1-score of 0.977, accuracy of 0.978, and an AUC of 0.990. It significantly outperforms unimodal baselines and existing multimodal methods, demonstrating superior sensitivity (0.974) and precision (0.981), particularly for underrepresented malignant classes like Melanoma (F1: 0.995) and Squamous Cell Carcinoma (F1: 0.960). The integration of clinical metadata with advanced visual embeddings via gated attention significantly enhances diagnostic reliability. Comprehensive ablation studies confirm the contribution of each architectural component. This framework offers a viable pathway for deploying high-precision, AI-driven dermatological screening tools on standard smartphone devices.

Förderung

  • Umm al-Qura University

Zitieren

APA

Fathallah, W., Abid, M., Mars, M., & Sakli, H. (2026). Ensemble-Based Multimodal Deep Learning for Precise Skin Cancer Diagnosis: Integrating Clinical Imagery with Patient Metadata. Technologies, 14(5), 277. https://doi.org/10.3390/technologies14050277

Vancouver

Fathallah W, Abid M, Mars M, Sakli H. Ensemble-Based Multimodal Deep Learning for Precise Skin Cancer Diagnosis: Integrating Clinical Imagery with Patient Metadata. Technologies. 2026;14(5):277. doi:10.3390/technologies14050277

BibTeX
@article{Fathallah2026,
  author = {Fathallah, Wyssem and Abid, M’hamed and Mars, Mourad and Sakli, Hedi},
  title = {Ensemble-Based Multimodal Deep Learning for Precise Skin Cancer Diagnosis: Integrating Clinical Imagery with Patient Metadata},
  journal = {Technologies},
  year = {2026},
  volume = {14},
  number = {5},
  pages = {277},
  doi = {10.3390/technologies14050277}
}
Kostenlos starten →