EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

DINOv3–SAM: Fusing Semantic and Anatomical Foundation Models for Robust Medical Image Classification

Saeed Al Gharib, F Dornaika, J Charafeddine, S Haddad

Journal of Imaging Informatics in Medicine · 02.11.2026

DOI: 10.1007/s10278-026-02405-0 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Imaging Informatics in Medicine, online erschienen am 02.11.2026. Verfasst von Saeed Al Gharib und 3 weiteren Autor:innen. Beteiligte Einrichtung: University of the Basque Country UPV/EHU, San Sebastian, Spain. Die Arbeit nennt 38 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Robust medical image classification remains a persistent challenge due to heterogeneous acquisition protocols, label noise, and subtle inter-class variability. In chest radiography, these difficulties are compounded by shortcut learning from non-anatomical artifacts and projection variability, which severely undermine generalization. To address these limitations, we propose DINOv3–SAM, a dual-encoder framework that synergistically integrates complementary visual representations. The architecture combines a self-supervised DINOv3 vision transformer, which captures global semantic context, with the Segment Anything Model (SAM) image encoder, which provides anatomy-aware spatial features. The two feature streams are spatially aligned and adaptively refined through a Convolutional Block Attention Module (CBAM), which emphasizes informative channels and diagnostically relevant regions prior to classification. Notably, only the final DINOv3 layers are fine-tuned while SAM remains frozen, yielding a computationally efficient framework with only 5.3 million trainable parameters. We evaluate DINOv3–SAM across five public datasets spanning diverse modalities and diagnostic tasks: pediatric pneumonia classification, cardiomegaly detection, large-scale RSNA pneumonia detection, COVID-19 radiography, and dermatological lesion classification. A unified stratified train/validation/test protocol ensures consistent and fair evaluation across both binary and multi-class settings. Each configuration was trained three times with different random seeds, and comprehensive comparisons against reproduced CNN baselines and published state-of-the-art methods, supported by statistical significance testing, demonstrate improvements across most datasets and metrics. The proposed framework achieves a mean accuracy of 94.94% (±3.06) on pediatric pneumonia, 97.24% (±0.77) accuracy (AUC 0.998 ±0.001) on COVID-19 radiography, and a mean AUC of 0.8705 (±0.0445) on the challenging RSNA benchmark. These results underscore the efficacy of fusing self-supervised semantic representations with segmentation-informed anatomical cues to enhance robustness against dataset bias, acquisition variability, and cross-domain shifts. The proposed approach offers a reliable foundation for multi-dataset and multi-modality medical image classification in research-oriented decision-support systems.

Schlagwörter

Stichwörter der Autor:innen

AUROCCBAM attentionCOVID-19CardiomegalyChest X-ray analysisCross-domain generalizationDINOv3Dermatology imagingDual-encoder fusionMacro-F1Medical image classificationMulti-dataset learningPneumoniaSegment Anything Model (SAM)Vision transformers

Autor:innen und Institutionen

  • University of the Basque Country UPV/EHU, San Sebastian, Spain.
  • University of the Basque Country UPV/EHU, San Sebastian, Spain. fadi.dornaika@ehu.eus.
  • IKERBASQUE, Basque Foundation for Science, Bilbao, Spain. fadi.dornaika@ehu.eus.
  • De Vinci Higher Education, De Vinci Research Center, Paris, France.
  • Balamand University, Kourah, Lebanon.

Zitieren

APA

Al Gharib, S., Dornaika, F., Charafeddine, J., & Haddad, S. (2026). DINOv3–SAM: Fusing Semantic and Anatomical Foundation Models for Robust Medical Image Classification. Journal of Imaging Informatics in Medicine. https://doi.org/10.1007/s10278-026-02405-0

Vancouver

Al Gharib S, Dornaika F, Charafeddine J, Haddad S. DINOv3–SAM: Fusing Semantic and Anatomical Foundation Models for Robust Medical Image Classification. Journal of Imaging Informatics in Medicine. 2026. doi:10.1007/s10278-026-02405-0

BibTeX
@article{AlGharib2026,
  author = {Al Gharib, Saeed and Dornaika, F and Charafeddine, J and Haddad, S},
  title = {DINOv3–SAM: Fusing Semantic and Anatomical Foundation Models for Robust Medical Image Classification},
  journal = {Journal of Imaging Informatics in Medicine},
  year = {2026},
  doi = {10.1007/s10278-026-02405-0}
}
Kostenlos starten →