Making SQL Executable with Execution-Guided Text-to-SQL: A Reproducible Study on WikiSQL-Style Queries
Artificial Intelligence and Machine Learning Review · 04.07.2025 · 6 (3) · 1-17
DOI: 10.69987/aimlr.2025.60301 ↗Kurz gefasst
Fachartikel in der Zeitschrift Artificial Intelligence and Machine Learning Review, online erschienen am 04.07.2025. Verfasst von Eric Chau und 1 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Natural-language interfaces to relational databases require models that generate SQL which is not only syntactically valid, but also executable and semantically faithful to the user question. In practice, text-to-SQL systems often fail for reasons that are invisible to purely string-based evaluation: a query may differ from the gold SQL yet still execute to the correct answer, or it may match the gold form while being brittle to schema and value variation. This paper investigates how simple execution feedback can be used to make generated SQL executable and to improve answer accuracy. We present a controlled comparison of three decoding settings that mirror common LLM prompting pipelines: (i) prompt-only decoding with minimal schema awareness, (ii) schema-linking decoding that explicitly maps question spans to table columns and cell values, and (iii) execution-guided decoding that executes candidate SQL programs and repairs empty/invalid results using value-level feedback. Experiments follow the WikiSQL task formulation introduced by Seq2SQL [1] (single-table SELECT with optional aggregation and up to three WHERE conditions). We evaluate on the standard split sizes (56,355/8,421/15,878) and report both exact-match (EM) and execution accuracy (EX). On the test split of our WikiSQL-style reproduction, prompt-only decoding reaches 30.02% EX, schema-linking reaches 82.84% EX, and execution-guided decoding reaches 84.13% EX, a +1.28 absolute improvement over schema-linking. Detailed breakdowns show that execution feedback primarily reduces condition-value errors and improves robustness as query complexity increases. The study highlights when execution-based filtering/repair is beneficial, and provides a reproducible baseline for evaluating “make-it-executable” decoding strategies.
Zitieren
APA
Chau, E., & Peters, M. (2025). Making SQL Executable with Execution-Guided Text-to-SQL: A Reproducible Study on WikiSQL-Style Queries. Artificial Intelligence and Machine Learning Review, 6(3), 1-17. https://doi.org/10.69987/aimlr.2025.60301
Vancouver
Chau E, Peters M. Making SQL Executable with Execution-Guided Text-to-SQL: A Reproducible Study on WikiSQL-Style Queries. Artificial Intelligence and Machine Learning Review. 2025;6(3):1-17. doi:10.69987/aimlr.2025.60301
BibTeX
@article{Chau2025,
author = {Chau, Eric and Peters, Matt},
title = {Making SQL Executable with Execution-Guided Text-to-SQL: A Reproducible Study on WikiSQL-Style Queries},
journal = {Artificial Intelligence and Machine Learning Review},
year = {2025},
volume = {6},
number = {3},
pages = {1--17},
doi = {10.69987/aimlr.2025.60301}
}
EUVIMED