ViBR: Automated Bug Replay from Video-Based Reports using Vision-Language Models
Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 3275-3297
DOI: 10.1145/3808152 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Sidong Feng und 5 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Bug reports play a critical role in software maintenance by helping users convey encountered issues to developers. Recently, GUI screen capture videos have gained popularity as a bug reporting artifact due to their ease of use and ability to retain rich contextual information. However, automatically reproducing bugs from such recordings remains a significant challenge. Existing methods often rely on fragile image-processing heuristics, explicit touch indicators, or pre-constructed UI transition graphs, which require non-trivial instrumentation and app-specific setup. This paper presents ViBR, a lightweight and fully automated approach that reproduces bugs directly from GUI recordings. Specifically, ViBR combines CLIP-based embedding similarity for action boundary segmentation with Vision-Language Models (VLMs) for region-aware GUI state comparison and guided bug replay. Experimental results show that ViBR successfully reproduces 72% of bug recordings, significantly outperforming state-of-the-art baselines and ablation variants.
Zitieren
APA
Feng, S., Wang, D., Tomic, N., Yu, T., Aleti, A., & Chen, C. (2026). ViBR: Automated Bug Replay from Video-Based Reports using Vision-Language Models. Proceedings of the ACM on Software Engineering, 3(FSE), 3275-3297. https://doi.org/10.1145/3808152
Vancouver
Feng S, Wang D, Tomic N, Yu T, Aleti A, Chen C. ViBR: Automated Bug Replay from Video-Based Reports using Vision-Language Models. Proceedings of the ACM on Software Engineering. 2026;3(FSE):3275-3297. doi:10.1145/3808152
BibTeX
@article{Feng2026,
author = {Feng, Sidong and Wang, Dingbang and Tomic, Nikola and Yu, Tingting and Aleti, Aldeida and Chen, Chunyang},
title = {ViBR: Automated Bug Replay from Video-Based Reports using Vision-Language Models},
journal = {Proceedings of the ACM on Software Engineering},
year = {2026},
volume = {3},
number = {FSE},
pages = {3275--3297},
doi = {10.1145/3808152}
}
EUVIMED