EUVIMEDEUVIMED entdecken
Crossref

AUTOMATED DETECTION AND COMPARATIVE ANALYSIS OF SQL ANTIPATTERNS IN TEXT-TO-SQL DATASETS

V. M. Sabadosh, V. M. Kotsovsky

Ukrainian Journal of Information Technology · 2026 · 8 (1) · 135-143

DOI: 10.23939/ujit2026.01.135 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Ukrainian Journal of Information Technology, 2026. Verfasst von V. M. Sabadosh und 1 weiteren Autor:innen. Die Arbeit nennt 16 Literaturangaben.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study examines the quality of SQL queries in Text-to-SQL datasets, focusing on antipatterns in gold-standard queries used to train and evaluate large language models (LLMs). While such queries may be syntactically valid and executable, they can still contain structural flaws that affect correctness, portability, and benchmark reliability. Since these flaws do not prevent successful execution, they often go undetected by conventional validation methods — creating what we call a correctness paradox. A rule-based static analysis framework was designed and implemented for detecting 14 SQL antipatterns in Text-to-SQL datasets. The rules operate on abstract syntax trees produced by a dialect-aware SQL parser. Each detected antipattern is classified into one of four severity levels (Critical, High, Medium, Low) using a configurable profile for SQLite, PostgreSQL, and other RDBMSs. The framework was applied to six partitions from three major Text-to-SQL benchmarks: Spider (11,840 queries), BIRD (10,962 queries), and the Gretel synthetic dataset (100,000 queries). Overall, 82.5–89.7% of queries were antipattern-free, and 16,554 antipattern occurrences across 14 types were detected. Spider was dominated by Missing GROUP BY, BIRD by Function in WHERE, whereas Gretel exhibited the broadest defect spectrum and was the only dataset containing 16 syntactically invalid SQL queries. Manual verification of all Critical-severity cases confirmed that the vast majority represented genuine defects; a small number of borderline cases were retained due to the framework's deliberately conservative detection policy. A comparative evaluation on a representative sample of 1,237 queries showed that, while our framework detected antipatterns across all 14 classes, the competing tool identified the expected antipattern in only 58.6% of cases, covering just 4 classes. For Cartesian products specifically, the competing tool reached only 29.5% recall, missing eight distinct syntactic patterns. Based on these findings, we recommend integrating automated antipattern detection into benchmark quality assurance workflows, prioritizing and fixing Critical-severity cases across the analysed datasets, correcting the 16 syntactically invalid SQL queries identified in the Gretel dataset, and resolving Missing GROUP BY patterns that undermine cross-engine portability. To support adoption of these recommendations, the framework is released as an open-source tool.

Zitieren

APA

Sabadosh, V. M., & Kotsovsky, V. M. (2026). AUTOMATED DETECTION AND COMPARATIVE ANALYSIS OF SQL ANTIPATTERNS IN TEXT-TO-SQL DATASETS. Ukrainian Journal of Information Technology, 8(1), 135-143. https://doi.org/10.23939/ujit2026.01.135

Vancouver

Sabadosh VM, Kotsovsky VM. AUTOMATED DETECTION AND COMPARATIVE ANALYSIS OF SQL ANTIPATTERNS IN TEXT-TO-SQL DATASETS. Ukrainian Journal of Information Technology. 2026;8(1):135-143. doi:10.23939/ujit2026.01.135

BibTeX
@article{Sabadosh2026,
  author = {Sabadosh, V. M. and Kotsovsky, V. M.},
  title = {AUTOMATED DETECTION AND COMPARATIVE ANALYSIS OF SQL ANTIPATTERNS IN TEXT-TO-SQL DATASETS},
  journal = {Ukrainian Journal of Information Technology},
  year = {2026},
  volume = {8},
  number = {1},
  pages = {135--143},
  doi = {10.23939/ujit2026.01.135}
}
Kostenlos starten →