Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors
Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 3722-3745
DOI: 10.1145/3808171 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Jiawei Shen und 8 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Shanghai Special Program for Promoting High-Quality Industrial Development, CCF-Huawei Populus Grove Fund.
Abstract (Originalsprache)
Large language models (LLMs) have been adopted for text-to-SQL tasks, utilizing their in-context learning (ICL) capability to translate natural language questions into SQL queries. However, such a technique faces correctness problems. In this paper, we conduct the first comprehensive study of text-to-SQL errors of ICL-based techniques. Our study covers four representative ICL-based techniques, five basic repairing methods, two benchmarks, and two LLM settings. We find that text-to-SQL errors are widespread and summarize 27 error types of 7 categories. We also find that existing repairing attempts have limited correctness improvement while having high computational overhead and many mis-repairs. Based on these findings, we propose MapleDoctor, a novel text-to-SQL error detection and repairing framework. The evaluation demonstrates that MapleDoctor outperforms existing solutions by repairing 13.8% more queries with a negligible number of mis-repairs and reducing 67.4% repair latency. The artifact is publicly available at GitHub.
Förderung
- National Natural Science Foundation of China
- Shanghai Special Program for Promoting High-Quality Industrial Development
- CCF-Huawei Populus Grove Fund
- Shanghai Driverless Train Control System Engineering Technology Research Center
Zitieren
APA
Shen, J., Wan, C., Qiao, R., Zou, J., Xu, H., Shao, Y., Zhang, Y., Miao, W., & Pu, G. (2026). Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors. Proceedings of the ACM on Software Engineering, 3(FSE), 3722-3745. https://doi.org/10.1145/3808171
Vancouver
Shen J, Wan C, Qiao R, Zou J, Xu H, Shao Y, et al. Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors. Proceedings of the ACM on Software Engineering. 2026;3(FSE):3722-3745. doi:10.1145/3808171
BibTeX
@article{Shen2026,
author = {Shen, Jiawei and Wan, Chengcheng and Qiao, Ruoyi and Zou, Jiazhen and Xu, Hang and Shao, Yuchen and Zhang, Yueling and Miao, Weikai and Pu, Geguang},
title = {Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors},
journal = {Proceedings of the ACM on Software Engineering},
year = {2026},
volume = {3},
number = {FSE},
pages = {3722--3745},
doi = {10.1145/3808171}
}
EUVIMED