Effective REST APIs Testing with Error Message Analysis
Proceedings of the ACM on Software Engineering · 22.06.2025 · 2 (ISSTA) · 1978-2000
DOI: 10.1145/3728964 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 22.06.2025. Verfasst von Lixin Xu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 62 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation of Jiangsu Province, Primary Research and Development Plan of Jiangsu Province.
Abstract (Originalsprache)
REST APIs are essential for building modern enterprise systems, but effectively testing them remains challenging, particularly due to difficulties in inferring constraints from specifications. Current testing approaches typically use feedback from HTTP status codes to guide input generation. However, they overlook valuable information available in the accompanying error messages, reducing their effectiveness in exploring the APIs’ input spaces. In this paper, we propose EmRest , a black-box testing approach that leverages error message analysis to enhance both valid and exceptional test input generation for REST APIs. For each operation under test, EmRest first identifies all possible value assignment strategies for each of its input parameters. It then repeatedly applies combinatorial testing to sample test inputs based on these strategies, and statistically analyzes the error messages (of 400-range status code) received to infer and exclude invalid combinations of value assignment strategies (i.e., constraints of the input space). Additionally, EmRest seeks to mutate valid value assignment strategies that are finally identified to generate test inputs for exceptional testing. The error messages (of 500-range status code) received are categorized to identify bug-prone operations, for which more testing resources are allocated. Our experimental results on 16 real-world REST APIs demonstrates the effectiveness of EmRest . It achieves higher operation coverage than state-of-the-art approaches in 50% of APIs, and detects 226 unique bugs undetected by other approaches.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation of Jiangsu Province
- Primary Research and Development Plan of Jiangsu Province
Zitieren
APA
Xu, L., Wu, H., Pan, Z., Xu, T., Wang, S., Niu, X., & Nie, C. (2025). Effective REST APIs Testing with Error Message Analysis. Proceedings of the ACM on Software Engineering, 2(ISSTA), 1978-2000. https://doi.org/10.1145/3728964
Vancouver
Xu L, Wu H, Pan Z, Xu T, Wang S, Niu X, et al. Effective REST APIs Testing with Error Message Analysis. Proceedings of the ACM on Software Engineering. 2025;2(ISSTA):1978-2000. doi:10.1145/3728964
BibTeX
@article{Xu2025,
author = {Xu, Lixin and Wu, Huayao and Pan, Zhenyu and Xu, Tongtong and Wang, Shaohua and Niu, Xintao and Nie, Changhai},
title = {Effective REST APIs Testing with Error Message Analysis},
journal = {Proceedings of the ACM on Software Engineering},
year = {2025},
volume = {2},
number = {ISSTA},
pages = {1978--2000},
doi = {10.1145/3728964}
}
EUVIMED