EUVIMEDEUVIMED entdecken
Crossref

Finding Non-Redundant Simpson's Paradox in Multidimensional Data

Yi Yang, Jian Pei, Jun Yang, Jichun Xie

Proceedings of the VLDB Endowment · 01.05.2026 · 19 (9) · 2466-2479

DOI: 10.14778/3819518.3819564 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 19.08.2026. Verfasst von Yi Yang und 3 weiteren Autor:innen. Die Arbeit nennt 59 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Simpson's paradox has broad impact across many scientific domains. Existing detection methods overlook a key issue: many detected paradoxes may be redundant , arising from equivalent data subsets, identical subpopulation partitions, or correlated outcome variables, thereby obscure insights and increase computational cost. In this paper, we present a framework for finding non-redundant Simpson's paradoxes by formalizing three sources of redundancy—sibling child, separator, and statistic equivalence—and showing that pair-wise redundancy forms an equivalence relation. We further propose a concise representation that groups redundant paradoxes and develop efficient algorithms combining depth-first population materialization with redundancy-aware discovery. Experiments on real and synthetic datasets show that redundancy is prevalent (over 40% in some cases), while our methods scale to millions of records, achieve up to 6.72× speedup over brute-force approaches and identify robust paradoxes, enabling efficient discovery, compact summarization, and clear interpretation in multidimensional data.

Zitieren

APA

Yang, Y., Pei, J., Yang, J., & Xie, J. (2026). Finding Non-Redundant Simpson's Paradox in Multidimensional Data. Proceedings of the VLDB Endowment, 19(9), 2466-2479. https://doi.org/10.14778/3819518.3819564

Vancouver

Yang Y, Pei J, Yang J, Xie J. Finding Non-Redundant Simpson's Paradox in Multidimensional Data. Proceedings of the VLDB Endowment. 2026;19(9):2466-2479. doi:10.14778/3819518.3819564

BibTeX
@article{Yang2026,
  author = {Yang, Yi and Pei, Jian and Yang, Jun and Xie, Jichun},
  title = {Finding Non-Redundant Simpson's Paradox in Multidimensional Data},
  journal = {Proceedings of the VLDB Endowment},
  year = {2026},
  volume = {19},
  number = {9},
  pages = {2466--2479},
  doi = {10.14778/3819518.3819564}
}
Kostenlos starten →