EUVIMEDEUVIMED entdecken
Crossref

Robust Query Driven Cardinality Estimation under Changing Workloads

Parimarjan Negi, Ziniu Wu, Andreas Kipf, Nesime Tatbul, Ryan Marcus, Sam Madden, Tim Kraska, Mohammad Alizadeh

Proceedings of the VLDB Endowment · 2023-02 · 16 (6) · 1520-1533

DOI: 10.14778/3583140.3583164 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 20.04.2023. Verfasst von Parimarjan Negi und 7 weiteren Autor:innen. Die Arbeit wurde bisher 82-mal zitiert (Crossref) und nennt 46 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Query driven cardinality estimation models learn from a historical log of queries. They are lightweight, having low storage requirements, fast inference and training, and are easily adaptable for any kind of query. Unfortunately, such models can suffer unpredictably bad performance under workload drift, i.e., if the query pattern or data changes. This makes them unreliable and hard to deploy. We analyze the reasons why models become unpredictable due to workload drift, and introduce modifications to the query representation and neural network training techniques to make query-driven models robust to the effects of workload drift. First, we emulate workload drift in queries involving some unseen tables or columns by randomly masking out some table or column features during training. This forces the model to make predictions with missing query information, relying more on robust features based on up-to-date DBMS statistics that are useful even when query or data drift happens. Second, we introduce join bitmaps, which extends sampling-based features to be consistent across joins using ideas from sideways information passing. Finally, we show how both of these ideas can be adapted to handle data updates. We show significantly greater generalization than past works across different workloads and databases. For instance, a model trained with our techniques on a simple workload (JOBLight-train), with 40 k synthetically generated queries of at most 3 tables each, is able to generalize to the much more complex Join Order Benchmark, which include queries with up to 16 tables, and improve query runtimes by 2× over PostgreSQL. We show similar robustness results with data updates, and across other workloads. We discuss the situations where we expect, and see, improvements, as well as more challenging workload drift scenarios where these techniques do not improve much over PostgreSQL. However, even in the most challenging scenarios, our models never perform worse than PostgreSQL, while standard query driven models can get much worse than PostgreSQL.

Zitieren

APA

Negi, P., Wu, Z., Kipf, A., Tatbul, N., Marcus, R., Madden, S., Kraska, T., & Alizadeh, M. (2023). Robust Query Driven Cardinality Estimation under Changing Workloads. Proceedings of the VLDB Endowment, 16(6), 1520-1533. https://doi.org/10.14778/3583140.3583164

Vancouver

Negi P, Wu Z, Kipf A, Tatbul N, Marcus R, Madden S, et al. Robust Query Driven Cardinality Estimation under Changing Workloads. Proceedings of the VLDB Endowment. 2023;16(6):1520-1533. doi:10.14778/3583140.3583164

BibTeX
@article{Negi2023,
  author = {Negi, Parimarjan and Wu, Ziniu and Kipf, Andreas and Tatbul, Nesime and Marcus, Ryan and Madden, Sam and Kraska, Tim and Alizadeh, Mohammad},
  title = {Robust Query Driven Cardinality Estimation under Changing Workloads},
  journal = {Proceedings of the VLDB Endowment},
  year = {2023},
  volume = {16},
  number = {6},
  pages = {1520--1533},
  doi = {10.14778/3583140.3583164}
}
Kostenlos starten →