EUVIMEDEUVIMED entdecken
Crossref

SQLStorm: Taking Database Benchmarking into the LLM Era

Tobias Schmidt, Viktor Leis, Peter Boncz, Thomas Neumann

Proceedings of the VLDB Endowment · 2025-07 · 18 (11) · 4144-4157

DOI: 10.14778/3749646.3749683 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the VLDB Endowment, online erschienen am 04.09.2025. Verfasst von Tobias Schmidt und 3 weiteren Autor:innen. Die Arbeit wurde bisher 22-mal zitiert (Crossref) und nennt 70 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In this paper, we introduce a new methodology for constructing database benchmarks using Large Language Models (LLMs), as well as SQLStorm v1.0, a concrete benchmark on a real-world dataset of three sizes (1 GB, 12 GB, 220 GB) consisting of over 18 K queries. This methodology of using AI to generate query workloads breaks new ground, not only in its ability to cheaply ($15) generate huge volumes (22 MB) of realistic queries but especially because it greatly expands the amount of SQL functionality and query constructions that is covered, compared to human-written SQL benchmarks such as TPC-H, TPC-DS, and JOB. The use cases of SQLStorm that we think will advance data systems most are: (i) improving SQL compatibility between systems, (ii) increasing system quality by identifying crashes/errors and fixing those, (iii) improving cardinality estimators and query optimizers, by identifying trends and opportunities (queries where other systems do much better), as well as (iv) overall system performance, both in terms of speed and robustness.

Zitieren

APA

Schmidt, T., Leis, V., Boncz, P., & Neumann, T. (2025). SQLStorm: Taking Database Benchmarking into the LLM Era. Proceedings of the VLDB Endowment, 18(11), 4144-4157. https://doi.org/10.14778/3749646.3749683

Vancouver

Schmidt T, Leis V, Boncz P, Neumann T. SQLStorm: Taking Database Benchmarking into the LLM Era. Proceedings of the VLDB Endowment. 2025;18(11):4144-4157. doi:10.14778/3749646.3749683

BibTeX
@article{Schmidt2025,
  author = {Schmidt, Tobias and Leis, Viktor and Boncz, Peter and Neumann, Thomas},
  title = {SQLStorm: Taking Database Benchmarking into the LLM Era},
  journal = {Proceedings of the VLDB Endowment},
  year = {2025},
  volume = {18},
  number = {11},
  pages = {4144--4157},
  doi = {10.14778/3749646.3749683}
}
Kostenlos starten →