Visual and lesion descriptor fusion using cross-attention for skin disease diagnosis
Frontiers in Oncology · 21.08.2026 · 16
DOI: 10.3389/fonc.2026.1885137 ↗Kurz gefasst
Fachartikel in der Zeitschrift Frontiers in Oncology, online erschienen am 21.08.2026. Verfasst von Maira Afzal und 5 weiteren Autor:innen. Beteiligte Einrichtung: Department of Computer Science, COMSATS University Islamabad, Wah Cantt, Pakistan. Die Arbeit nennt 37 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Introduction
Accurate and early diagnosis of skin disease remains a significant challenge in clinical dermatology due to variation in skin tones and visual similarity among disease lesion types. Dermatologists rely on both clinical expertise and advanced diagnostic tools to achieve reliable diagnosis. To assist in the early diagnosis, many researchers have proposed traditional computer vision methods; however, these approaches remain limited due to the wide range of conditions and the overlapping of early-stage symptoms. Recognizing these limitations, deep learning models have been explored to improve performance, but they still face challenges in effective feature extraction and have weak generalization due to limited datasets.
Methods
To address this, a cross-attention-based framework that integrates visual with structured lesion descriptor derived from dermoscopic attributes is proposed to enhance early-stage skin disease diagnosis. Visual features are extracted using a fine-tuned transformer, while descriptor features are encoded using BERT-based language models. Furthermore, the ISIC 2018 dataset is enriched with structured lesion descriptors automatically generated from dermoscopic images using the ABCD rule. The symptom descriptors provide semantic representations for multimodal framework instead of an independent clinical modality.
Results
The experimental results demonstrate that the proposed framework outperforms well compared to unimodal baselines, where CNN-based methods achieved 85% to 93% accuracy, while transformer-based pre-trained models reach 94.77%. In contrast, the proposed model achieves a peak accuracy of 96.87% accuracy, which highlights its effectiveness for skin disease classification.
Discussion
Unlike the traditional image-only system, the proposed model analyzed both visual features with their descriptive structured lesion descriptors that enhance interpretability and provide clinically meaningful diagnostic support.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Computer Science, COMSATS University Islamabad, Wah Cantt, Pakistan.
- Department of Information Technology, Government College University Faisalabad, Faisalabad, Pakistan.
- Department of Computer Science, College of Engineering and Computer Science, Jazan University, Jazan, Saudi Arabia.
- Department of Computer Science, Nile Valley University, Khartoum, Sudan.
Zitieren
APA
Afzal, M., Shah, J. H., Saleem, R., Younas, M. U., Tahir, A., & Saeed, M. E. S. (2026). Visual and lesion descriptor fusion using cross-attention for skin disease diagnosis. Frontiers in Oncology, 16. https://doi.org/10.3389/fonc.2026.1885137
Vancouver
Afzal M, Shah JH, Saleem R, Younas MU, Tahir A, Saeed MES. Visual and lesion descriptor fusion using cross-attention for skin disease diagnosis. Frontiers in Oncology. 2026;16. doi:10.3389/fonc.2026.1885137
BibTeX
@article{Afzal2026,
author = {Afzal, Maira and Shah, Jamal Hussain and Saleem, Rabia and Younas, Muhammad Usman and Tahir, Ali and Saeed, Mutaz Elradi S},
title = {Visual and lesion descriptor fusion using cross-attention for skin disease diagnosis},
journal = {Frontiers in Oncology},
year = {2026},
volume = {16},
doi = {10.3389/fonc.2026.1885137}
}
EUVIMED