EUVIMEDAtrasti EUVIMED
Open access · CC BY 4.0

Leakage-Aware and Reproducible Evaluation of Deep Learning Models for Multi-Class Oral Disease Image Classification

A. M. Mutawa, Yacoub Yousef Altarakemah, Karthiga Thirupathy

Computers · 2026-11-13 · 15 (11) · 830

DOI: 10.3390/computers15110830 ↗

At a glance

Journal article in Computers, published online on 2026-11-13. Written by A. M. Mutawa and 2 co-authors. The full text is freely available (open access, CC BY 4.0). Funded by: Kuwait Foundation for the Advancement of Sciences.

SantraukaBibliografiniai duomenysCiteRelated studies

Abstract (original language)

Duplicate and near-duplicate images can compromise deep learning evaluation through cross-partition leakage. This study investigated duplicate-aware evaluation and reproducibility using 667 public oral photographs representing Caries (219), Mouth Ulcer (265), and Tooth Discoloration (183). Exact and near-duplicate content was examined using cryptographic and perceptual hashing, supplemented by human-reference assessment of 378 image pairs. The automated audit initially identified 580 duplicate-content groups, refined to 564 following human validation. A corrected group-disjoint split (466 training, 101 validation, 100 test images) was used to evaluate ImageNet-pretrained ResNet50, EfficientNet-B3, and ViT-B/16 across three seeds. Performance, calibration, and uncertainty were assessed using a standardized protocol and 10,000 paired duplicate-group bootstrap resamples. ViT-B/16 achieved the highest mean accuracy (92.67% ± 1.15 percentage points) and macro-F1 (0.9194 ± 0.0138), followed by ResNet50 (92.00% ± 2.00; 0.9135 ± 0.0213) and EfficientNet-B3 (83.33% ± 2.08; 0.8263 ± 0.0224). The ViT-B/16–ResNet50 accuracy difference was 0.67 percentage points (95% confidence interval: −3.33 to 4.67), indicating uncertain relative superiority. Exploratory evaluation on 1786 externally annotated caries-positive images yielded mean sensitivities of 87.38%, 74.25%, and 69.65% for EfficientNet-B3, ResNet50, and ViT-B/16, respectively, with substantial seed variability. These findings support human-validated duplicate auditing, group-disjoint partitioning, multi-seed testing, and paired uncertainty analysis. Limited disease coverage, unavailable patient identifiers, and positive-only external evaluation constrain conclusions regarding multiclass generalizability and clinical applicability.

Funding

  • Kuwait Foundation for the Advancement of Sciences

Cite

APA

Mutawa, A. M., Altarakemah, Y. Y., & Thirupathy, K. (2026). Leakage-Aware and Reproducible Evaluation of Deep Learning Models for Multi-Class Oral Disease Image Classification. Computers, 15(11), 830. https://doi.org/10.3390/computers15110830

Vancouver

Mutawa AM, Altarakemah YY, Thirupathy K. Leakage-Aware and Reproducible Evaluation of Deep Learning Models for Multi-Class Oral Disease Image Classification. Computers. 2026;15(11):830. doi:10.3390/computers15110830

BibTeX
@article{Mutawa2026,
  author = {Mutawa, A. M. and Altarakemah, Yacoub Yousef and Thirupathy, Karthiga},
  title = {Leakage-Aware and Reproducible Evaluation of Deep Learning Models for Multi-Class Oral Disease Image Classification},
  journal = {Computers},
  year = {2026},
  volume = {15},
  number = {11},
  pages = {830},
  doi = {10.3390/computers15110830}
}
Start for free →