EUVIMEDEUVIMED entdecken
Crossref

What are we really measuring? Rethinking dataset bias in web-scale natural image collections via unsupervised semantic clustering

Amir Hossein Saleknia, Mohammad Sabokrou

Neurocomputing · 2026-07 · 686 · 133679

DOI: 10.1016/j.neucom.2026.133679 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Neurocomputing, 2026-07. Verfasst von Amir Hossein Saleknia und 1 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In computer vision, a prevailing method for quantifying dataset bias is to train a model to distinguish between datasets. High classification accuracy is then interpreted as evidence of meaningful semantic differences. This approach assumes that standard image augmentations successfully suppress low-level, non-semantic cues, and that any remaining performance must therefore reflect true semantic divergence. We demonstrate that this fundamental assumption is flawed within the domain of large-scale natural image collections. High classification accuracy is often driven by resolution-based artifacts, which are structural fingerprints arising from native image resolution distributions and interpolation effects during resizing. These artifacts form robust, dataset-specific signatures that persist despite conventional image corruptions. Through controlled experiments, we show that models achieve strong dataset classification even on non-semantic, procedurally generated images, proving their reliance on superficial cues. To address this issue, we revisit this decades-old idea of dataset separability, but not with supervised classification. Instead, we introduce an unsupervised approach that measures true semantic separability. Our framework directly assesses semantic similarity by clustering semantically-rich features from foundational vision models, deliberately bypassing supervised classification on dataset labels. When applied to major web-scale datasets, the primary focus of this work, the high separability reported by supervised methods largely vanishes, with clustering accuracy dropping to near-chance levels. This reveals that conventional classification-based evaluation systematically overstates semantic bias by an overwhelming margin.

Zitieren

APA

Saleknia, A. H., & Sabokrou, M. (2026). What are we really measuring? Rethinking dataset bias in web-scale natural image collections via unsupervised semantic clustering. Neurocomputing, 686, 133679. https://doi.org/10.1016/j.neucom.2026.133679

Vancouver

Saleknia AH, Sabokrou M. What are we really measuring? Rethinking dataset bias in web-scale natural image collections via unsupervised semantic clustering. Neurocomputing. 2026;686:133679. doi:10.1016/j.neucom.2026.133679

BibTeX
@article{Saleknia2026,
  author = {Saleknia, Amir Hossein and Sabokrou, Mohammad},
  title = {What are we really measuring? Rethinking dataset bias in web-scale natural image collections via unsupervised semantic clustering},
  journal = {Neurocomputing},
  year = {2026},
  volume = {686},
  pages = {133679},
  doi = {10.1016/j.neucom.2026.133679}
}
Kostenlos starten →