EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Reverse Engineering of Chatbot Behaviour for Automated Testing

Iván Sotillo del Horno, Alejandro del Pozzo, Esther Guerra, Juan de Lara

SN Computer Science · 20.08.2026 · 7 (7)

DOI: 10.1007/s42979-026-05237-5 ↗

Kurz gefasst

Fachartikel in der Zeitschrift SN Computer Science, online erschienen am 20.08.2026. Verfasst von Iván Sotillo del Horno und 3 weiteren Autor:innen. Die Arbeit nennt 49 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Universidad Autónoma de Madrid.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Conversational agents—or chatbots—are increasingly being used to access all sorts of services, like citizen services in city halls, customer support, or shopping. Moreover, recent advances in generative artificial intelligence are prompting the integration of conversational assistants into many applications, like programming IDEs, office automation software, or operating systems. Given the prominence of these agents, their correctness is a rising concern. However, automated and robust testing techniques for conversational systems are still needed. In this paper, we present a reverse engineering technique for extracting a behavioural model of a deployed chatbot (i.e., treated as a black-box) through the automated exploration of its functionality via Large Language Models. The extracted model is used for automated testing by generating testing conversation profiles, which a user simulator employs to conduct focused conversations with the chatbot-under-test. We describe our supporting tool Tracer , available as an integrated chatbot testing web platform at http://miso.ii.uam.es:8081/ . We report on three evaluations that show that our exploration technique can yield an accurate model of the chatbot-under-test (via coverage analysis), the subsequent testing can discover existing errors in the chatbot (via mutation testing), and the exploration is precise in discovering functionality in real-world chatbots in production built with heterogeneous technologies.

Förderung

  • Universidad Autónoma de Madrid

Zitieren

APA

Horno, I. S. D., Pozzo, A. D., Guerra, E., & Lara, J. D. (2026). Reverse Engineering of Chatbot Behaviour for Automated Testing. SN Computer Science, 7(7). https://doi.org/10.1007/s42979-026-05237-5

Vancouver

Horno ISD, Pozzo AD, Guerra E, Lara JD. Reverse Engineering of Chatbot Behaviour for Automated Testing. SN Computer Science. 2026;7(7). doi:10.1007/s42979-026-05237-5

BibTeX
@article{Horno2026,
  author = {Horno, Iván Sotillo del and Pozzo, Alejandro del and Guerra, Esther and Lara, Juan de},
  title = {Reverse Engineering of Chatbot Behaviour for Automated Testing},
  journal = {SN Computer Science},
  year = {2026},
  volume = {7},
  number = {7},
  doi = {10.1007/s42979-026-05237-5}
}
Kostenlos starten →