What are we really measuring? Rethinking dataset bias in web-scale natural image collections via unsupervised semantic clustering
Neurocomputing · 2026-07 · 686 · 133679
DOI: 10.1016/j.neucom.2026.133679 ↗Kurz gefasst
Fachartikel in der Zeitschrift Neurocomputing, 2026-07. Verfasst von Amir Hossein Saleknia und 1 weiteren Autor:innen. Die Arbeit nennt 44 Literaturangaben.
Abstract (Originalsprache)
In computer vision, a prevailing method for quantifying dataset bias is to train a model to distinguish between datasets. High classification accuracy is then interpreted as evidence of meaningful semantic differences. This approach assumes that standard image augmentations successfully suppress low-level, non-semantic cues, and that any remaining performance must therefore reflect true semantic divergence. We demonstrate that this fundamental assumption is flawed within the domain of large-scale natural image collections. High classification accuracy is often driven by resolution-based artifacts, which are structural fingerprints arising from native image resolution distributions and interpolation effects during resizing. These artifacts form robust, dataset-specific signatures that persist despite conventional image corruptions. Through controlled experiments, we show that models achieve strong dataset classification even on non-semantic, procedurally generated images, proving their reliance on superficial cues. To address this issue, we revisit this decades-old idea of dataset separability, but not with supervised classification. Instead, we introduce an unsupervised approach that measures true semantic separability. Our framework directly assesses semantic similarity by clustering semantically-rich features from foundational vision models, deliberately bypassing supervised classification on dataset labels. When applied to major web-scale datasets, the primary focus of this work, the high separability reported by supervised methods largely vanishes, with clustering accuracy dropping to near-chance levels. This reveals that conventional classification-based evaluation systematically overstates semantic bias by an overwhelming margin.
Zitieren
APA
Saleknia, A. H., & Sabokrou, M. (2026). What are we really measuring? Rethinking dataset bias in web-scale natural image collections via unsupervised semantic clustering. Neurocomputing, 686, 133679. https://doi.org/10.1016/j.neucom.2026.133679
Vancouver
Saleknia AH, Sabokrou M. What are we really measuring? Rethinking dataset bias in web-scale natural image collections via unsupervised semantic clustering. Neurocomputing. 2026;686:133679. doi:10.1016/j.neucom.2026.133679
BibTeX
@article{Saleknia2026,
author = {Saleknia, Amir Hossein and Sabokrou, Mohammad},
title = {What are we really measuring? Rethinking dataset bias in web-scale natural image collections via unsupervised semantic clustering},
journal = {Neurocomputing},
year = {2026},
volume = {686},
pages = {133679},
doi = {10.1016/j.neucom.2026.133679}
}
EUVIMED