Finding Non-Redundant Simpson's Paradox in Multidimensional Data
Proceedings of the VLDB Endowment · 01.05.2026 · 19 (9) · 2466-2479
DOI: 10.14778/3819518.3819564 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 19.08.2026. Verfasst von Yi Yang und 3 weiteren Autor:innen. Die Arbeit nennt 59 Literaturangaben.
Abstract (Originalsprache)
Simpson's paradox has broad impact across many scientific domains. Existing detection methods overlook a key issue: many detected paradoxes may be redundant , arising from equivalent data subsets, identical subpopulation partitions, or correlated outcome variables, thereby obscure insights and increase computational cost. In this paper, we present a framework for finding non-redundant Simpson's paradoxes by formalizing three sources of redundancy—sibling child, separator, and statistic equivalence—and showing that pair-wise redundancy forms an equivalence relation. We further propose a concise representation that groups redundant paradoxes and develop efficient algorithms combining depth-first population materialization with redundancy-aware discovery. Experiments on real and synthetic datasets show that redundancy is prevalent (over 40% in some cases), while our methods scale to millions of records, achieve up to 6.72× speedup over brute-force approaches and identify robust paradoxes, enabling efficient discovery, compact summarization, and clear interpretation in multidimensional data.
Zitieren
APA
Yang, Y., Pei, J., Yang, J., & Xie, J. (2026). Finding Non-Redundant Simpson's Paradox in Multidimensional Data. Proceedings of the VLDB Endowment, 19(9), 2466-2479. https://doi.org/10.14778/3819518.3819564
Vancouver
Yang Y, Pei J, Yang J, Xie J. Finding Non-Redundant Simpson's Paradox in Multidimensional Data. Proceedings of the VLDB Endowment. 2026;19(9):2466-2479. doi:10.14778/3819518.3819564
BibTeX
@article{Yang2026,
author = {Yang, Yi and Pei, Jian and Yang, Jun and Xie, Jichun},
title = {Finding Non-Redundant Simpson's Paradox in Multidimensional Data},
journal = {Proceedings of the VLDB Endowment},
year = {2026},
volume = {19},
number = {9},
pages = {2466--2479},
doi = {10.14778/3819518.3819564}
}
EUVIMED