EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Visual and lesion descriptor fusion using cross-attention for skin disease diagnosis

Maira Afzal, Jamal Hussain Shah, Rabia Saleem, Muhammad Usman Younas, Ali Tahir, Mutaz Elradi S Saeed

Frontiers in Oncology · 21.08.2026 · 16

DOI: 10.3389/fonc.2026.1885137 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Frontiers in Oncology, online erschienen am 21.08.2026. Verfasst von Maira Afzal und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, COMSATS University Islamabad, Wah Cantt, Pakistan. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Introduction

Accurate and early diagnosis of skin disease remains a significant challenge in clinical dermatology due to variation in skin tones and visual similarity among disease lesion types. Dermatologists rely on both clinical expertise and advanced diagnostic tools to achieve reliable diagnosis. To assist in the early diagnosis, many researchers have proposed traditional computer vision methods; however, these approaches remain limited due to the wide range of conditions and the overlapping of early-stage symptoms. Recognizing these limitations, deep learning models have been explored to improve performance, but they still face challenges in effective feature extraction and have weak generalization due to limited datasets.

Methods

To address this, a cross-attention-based framework that integrates visual with structured lesion descriptor derived from dermoscopic attributes is proposed to enhance early-stage skin disease diagnosis. Visual features are extracted using a fine-tuned transformer, while descriptor features are encoded using BERT-based language models. Furthermore, the ISIC 2018 dataset is enriched with structured lesion descriptors automatically generated from dermoscopic images using the ABCD rule. The symptom descriptors provide semantic representations for multimodal framework instead of an independent clinical modality.

Results

The experimental results demonstrate that the proposed framework outperforms well compared to unimodal baselines, where CNN-based methods achieved 85% to 93% accuracy, while transformer-based pre-trained models reach 94.77%. In contrast, the proposed model achieves a peak accuracy of 96.87% accuracy, which highlights its effectiveness for skin disease classification.

Discussion

Unlike the traditional image-only system, the proposed model analyzed both visual features with their descriptive structured lesion descriptors that enhance interpretability and provide clinically meaningful diagnostic support.

Schlagwörter

Stichwörter der Autor:innen

BERTexplainable artificial intelligence (XAI)medical imagingmultimodalskin diseasevisual transformer (ViT)

Autor:innen und Institutionen

  • Department of Computer Science, COMSATS University Islamabad, Wah Cantt, Pakistan.
  • Department of Information Technology, Government College University Faisalabad, Faisalabad, Pakistan.
  • Department of Computer Science, College of Engineering and Computer Science, Jazan University, Jazan, Saudi Arabia.
  • Department of Computer Science, Nile Valley University, Khartoum, Sudan.

Zitieren

APA

Afzal, M., Shah, J. H., Saleem, R., Younas, M. U., Tahir, A., & Saeed, M. E. S. (2026). Visual and lesion descriptor fusion using cross-attention for skin disease diagnosis. Frontiers in Oncology, 16. https://doi.org/10.3389/fonc.2026.1885137

Vancouver

Afzal M, Shah JH, Saleem R, Younas MU, Tahir A, Saeed MES. Visual and lesion descriptor fusion using cross-attention for skin disease diagnosis. Frontiers in Oncology. 2026;16. doi:10.3389/fonc.2026.1885137

BibTeX
@article{Afzal2026,
  author = {Afzal, Maira and Shah, Jamal Hussain and Saleem, Rabia and Younas, Muhammad Usman and Tahir, Ali and Saeed, Mutaz Elradi S},
  title = {Visual and lesion descriptor fusion using cross-attention for skin disease diagnosis},
  journal = {Frontiers in Oncology},
  year = {2026},
  volume = {16},
  doi = {10.3389/fonc.2026.1885137}
}
Kostenlos starten →