EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors

Jiawei Shen, Chengcheng Wan, Ruoyi Qiao, Jiazhen Zou, Hang Xu, Yuchen Shao, Yueling Zhang, Weikai Miao, Geguang Pu

Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 3722-3745

DOI: 10.1145/3808171 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Jiawei Shen und 8 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 80 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Shanghai Special Program for Promoting High-Quality Industrial Development, CCF-Huawei Populus Grove Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large language models (LLMs) have been adopted for text-to-SQL tasks, utilizing their in-context learning (ICL) capability to translate natural language questions into SQL queries. However, such a technique faces correctness problems. In this paper, we conduct the first comprehensive study of text-to-SQL errors of ICL-based techniques. Our study covers four representative ICL-based techniques, five basic repairing methods, two benchmarks, and two LLM settings. We find that text-to-SQL errors are widespread and summarize 27 error types of 7 categories. We also find that existing repairing attempts have limited correctness improvement while having high computational overhead and many mis-repairs. Based on these findings, we propose MapleDoctor, a novel text-to-SQL error detection and repairing framework. The evaluation demonstrates that MapleDoctor outperforms existing solutions by repairing 13.8% more queries with a negligible number of mis-repairs and reducing 67.4% repair latency. The artifact is publicly available at GitHub.

Förderung

  • National Natural Science Foundation of China
  • Shanghai Special Program for Promoting High-Quality Industrial Development
  • CCF-Huawei Populus Grove Fund
  • Shanghai Driverless Train Control System Engineering Technology Research Center

Zitieren

APA

Shen, J., Wan, C., Qiao, R., Zou, J., Xu, H., Shao, Y., Zhang, Y., Miao, W., & Pu, G. (2026). Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors. Proceedings of the ACM on Software Engineering, 3(FSE), 3722-3745. https://doi.org/10.1145/3808171

Vancouver

Shen J, Wan C, Qiao R, Zou J, Xu H, Shao Y, et al. Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors. Proceedings of the ACM on Software Engineering. 2026;3(FSE):3722-3745. doi:10.1145/3808171

BibTeX
@article{Shen2026,
  author = {Shen, Jiawei and Wan, Chengcheng and Qiao, Ruoyi and Zou, Jiazhen and Xu, Hang and Shao, Yuchen and Zhang, Yueling and Miao, Weikai and Pu, Geguang},
  title = {Understanding, Detecting, and Repairing Real-World In-Context-Learning-Based Text-to-SQL Errors},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2026},
  volume = {3},
  number = {FSE},
  pages = {3722--3745},
  doi = {10.1145/3808171}
}
Kostenlos starten →