EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

ACME: Automated Clause Mapping Engine for Testing Emerging Database Systems

Yuancheng Jiang, Jianing Wang, Chuqi Zhang, Roland H. C. Yap, Zhenkai Liang, Manuel Rigger

Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 853-875

DOI: 10.1145/3797134 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Yuancheng Jiang und 5 weiteren Autor:innen. Die Arbeit nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Research Foundation, Singapore, and Cyber Security Agency of Singapore, Ministry of Education (MOE), Singapore.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A growing number of emerging database management systems, such as time-series and streaming database systems, have been developed to support specialized workloads with enhanced performance and functionality. However, these systems are often less mature than traditional relational database systems, making them more prone to logic bugs and internal errors affecting correctness and reliability. To address this, we propose an enhanced differential testing framework designed for emerging SQL-like database systems. Our key insight is that many of these systems are conceptually extensions of relational database systems, allowing us to uncover bugs by comparing query results with those from more robust and mature relational database systems. To bridge the differences in syntax and semantics between emerging and relational database systems, we leverage Large Language Models (LLMs) to make differential testing more effective by discovering clause mappings that translate system-specific features in emerging database systems into equivalent SQL expressions. Our approach proceeds in three steps: (i) analyzing the syntax and semantics of queries with runtime errors to reason on clause mappings using LLMs; (ii) validating the generated clause mappings by executing test queries and re-prompting upon validation failures; and (iii) generating semantically equivalent, yet syntactically diverse queries to broaden the coverage of differential testing. We implemented this approach in a tool called ACME and applied it to four widely used emerging database systems, uncovering 59 previously unknown bugs, including 17 logic bugs and 42 internal errors. Of these, 52 have been fixed and 5 confirmed by vendors. Our evaluation demonstrates that ACME enhances LLM reliability through query validation. Furthermore, the evaluation shows the effectiveness of differential testing even when using local models or a limited online token budget. Our results demonstrate the practicality and effectiveness of ACME in improving the robustness and accuracy of emerging database systems through scalable, LLM-assisted differential testing.

Förderung

  • National Research Foundation, Singapore, and Cyber Security Agency of Singapore
  • Ministry of Education (MOE), Singapore

Zitieren

APA

Jiang, Y., Wang, J., Zhang, C., Yap, R. H. C., Liang, Z., & Rigger, M. (2026). ACME: Automated Clause Mapping Engine for Testing Emerging Database Systems. Proceedings of the ACM on Software Engineering, 3(FSE), 853-875. https://doi.org/10.1145/3797134

Vancouver

Jiang Y, Wang J, Zhang C, Yap RHC, Liang Z, Rigger M. ACME: Automated Clause Mapping Engine for Testing Emerging Database Systems. Proceedings of the ACM on Software Engineering. 2026;3(FSE):853-875. doi:10.1145/3797134

BibTeX
@article{Jiang2026,
  author = {Jiang, Yuancheng and Wang, Jianing and Zhang, Chuqi and Yap, Roland H. C. and Liang, Zhenkai and Rigger, Manuel},
  title = {ACME: Automated Clause Mapping Engine for Testing Emerging Database Systems},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2026},
  volume = {3},
  number = {FSE},
  pages = {853--875},
  doi = {10.1145/3797134}
}
Kostenlos starten →