Reverse Engineering of Chatbot Behaviour for Automated Testing
SN Computer Science · 20.08.2026 · 7 (7)
DOI: 10.1007/s42979-026-05237-5 ↗Kurz gefasst
Fachartikel in der Zeitschrift SN Computer Science, online erschienen am 20.08.2026. Verfasst von Iván Sotillo del Horno und 3 weiteren Autor:innen. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Universidad Autónoma de Madrid.
Abstract (Originalsprache)
Conversational agents—or chatbots—are increasingly being used to access all sorts of services, like citizen services in city halls, customer support, or shopping. Moreover, recent advances in generative artificial intelligence are prompting the integration of conversational assistants into many applications, like programming IDEs, office automation software, or operating systems. Given the prominence of these agents, their correctness is a rising concern. However, automated and robust testing techniques for conversational systems are still needed. In this paper, we present a reverse engineering technique for extracting a behavioural model of a deployed chatbot (i.e., treated as a black-box) through the automated exploration of its functionality via Large Language Models. The extracted model is used for automated testing by generating testing conversation profiles, which a user simulator employs to conduct focused conversations with the chatbot-under-test. We describe our supporting tool Tracer , available as an integrated chatbot testing web platform at http://miso.ii.uam.es:8081/ . We report on three evaluations that show that our exploration technique can yield an accurate model of the chatbot-under-test (via coverage analysis), the subsequent testing can discover existing errors in the chatbot (via mutation testing), and the exploration is precise in discovering functionality in real-world chatbots in production built with heterogeneous technologies.
Förderung
- Universidad Autónoma de Madrid
Zitieren
APA
Horno, I. S. D., Pozzo, A. D., Guerra, E., & Lara, J. D. (2026). Reverse Engineering of Chatbot Behaviour for Automated Testing. SN Computer Science, 7(7). https://doi.org/10.1007/s42979-026-05237-5
Vancouver
Horno ISD, Pozzo AD, Guerra E, Lara JD. Reverse Engineering of Chatbot Behaviour for Automated Testing. SN Computer Science. 2026;7(7). doi:10.1007/s42979-026-05237-5
BibTeX
@article{Horno2026,
author = {Horno, Iván Sotillo del and Pozzo, Alejandro del and Guerra, Esther and Lara, Juan de},
title = {Reverse Engineering of Chatbot Behaviour for Automated Testing},
journal = {SN Computer Science},
year = {2026},
volume = {7},
number = {7},
doi = {10.1007/s42979-026-05237-5}
}
EUVIMED