EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Making SQL Executable with Execution-Guided Text-to-SQL: A Reproducible Study on WikiSQL-Style Queries

Eric Chau, Matt Peters

Artificial Intelligence and Machine Learning Review · 04.07.2025 · 6 (3) · 1-17

DOI: 10.69987/aimlr.2025.60301 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Artificial Intelligence and Machine Learning Review, online erschienen am 04.07.2025. Verfasst von Eric Chau und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Natural-language interfaces to relational databases require models that generate SQL which is not only syntactically valid, but also executable and semantically faithful to the user question. In practice, text-to-SQL systems often fail for reasons that are invisible to purely string-based evaluation: a query may differ from the gold SQL yet still execute to the correct answer, or it may match the gold form while being brittle to schema and value variation. This paper investigates how simple execution feedback can be used to make generated SQL executable and to improve answer accuracy. We present a controlled comparison of three decoding settings that mirror common LLM prompting pipelines: (i) prompt-only decoding with minimal schema awareness, (ii) schema-linking decoding that explicitly maps question spans to table columns and cell values, and (iii) execution-guided decoding that executes candidate SQL programs and repairs empty/invalid results using value-level feedback. Experiments follow the WikiSQL task formulation introduced by Seq2SQL [1] (single-table SELECT with optional aggregation and up to three WHERE conditions). We evaluate on the standard split sizes (56,355/8,421/15,878) and report both exact-match (EM) and execution accuracy (EX). On the test split of our WikiSQL-style reproduction, prompt-only decoding reaches 30.02% EX, schema-linking reaches 82.84% EX, and execution-guided decoding reaches 84.13% EX, a +1.28 absolute improvement over schema-linking. Detailed breakdowns show that execution feedback primarily reduces condition-value errors and improves robustness as query complexity increases. The study highlights when execution-based filtering/repair is beneficial, and provides a reproducible baseline for evaluating “make-it-executable” decoding strategies.

Zitieren

APA

Chau, E., & Peters, M. (2025). Making SQL Executable with Execution-Guided Text-to-SQL: A Reproducible Study on WikiSQL-Style Queries. Artificial Intelligence and Machine Learning Review, 6(3), 1-17. https://doi.org/10.69987/aimlr.2025.60301

Vancouver

Chau E, Peters M. Making SQL Executable with Execution-Guided Text-to-SQL: A Reproducible Study on WikiSQL-Style Queries. Artificial Intelligence and Machine Learning Review. 2025;6(3):1-17. doi:10.69987/aimlr.2025.60301

BibTeX
@article{Chau2025,
  author = {Chau, Eric and Peters, Matt},
  title = {Making SQL Executable with Execution-Guided Text-to-SQL: A Reproducible Study on WikiSQL-Style Queries},
  journal = {Artificial Intelligence and Machine Learning Review},
  year = {2025},
  volume = {6},
  number = {3},
  pages = {1--17},
  doi = {10.69987/aimlr.2025.60301}
}
Kostenlos starten →