EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ViBR: Automated Bug Replay from Video-Based Reports using Vision-Language Models

Sidong Feng, Dingbang Wang, Nikola Tomic, Tingting Yu, Aldeida Aleti, Chunyang Chen

Proceedings of the ACM on Software Engineering · 30.06.2026 · 3 (FSE) · 3275-3297

DOI: 10.1145/3808152 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the ACM on Software Engineering, online erschienen am 30.06.2026. Verfasst von Sidong Feng und 5 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Bug reports play a critical role in software maintenance by helping users convey encountered issues to developers. Recently, GUI screen capture videos have gained popularity as a bug reporting artifact due to their ease of use and ability to retain rich contextual information. However, automatically reproducing bugs from such recordings remains a significant challenge. Existing methods often rely on fragile image-processing heuristics, explicit touch indicators, or pre-constructed UI transition graphs, which require non-trivial instrumentation and app-specific setup. This paper presents ViBR, a lightweight and fully automated approach that reproduces bugs directly from GUI recordings. Specifically, ViBR combines CLIP-based embedding similarity for action boundary segmentation with Vision-Language Models (VLMs) for region-aware GUI state comparison and guided bug replay. Experimental results show that ViBR successfully reproduces 72% of bug recordings, significantly outperforming state-of-the-art baselines and ablation variants.

Zitieren

APA

Feng, S., Wang, D., Tomic, N., Yu, T., Aleti, A., & Chen, C. (2026). ViBR: Automated Bug Replay from Video-Based Reports using Vision-Language Models. Proceedings of the ACM on Software Engineering, 3(FSE), 3275-3297. https://doi.org/10.1145/3808152

Vancouver

Feng S, Wang D, Tomic N, Yu T, Aleti A, Chen C. ViBR: Automated Bug Replay from Video-Based Reports using Vision-Language Models. Proceedings of the ACM on Software Engineering. 2026;3(FSE):3275-3297. doi:10.1145/3808152

BibTeX
@article{Feng2026,
  author = {Feng, Sidong and Wang, Dingbang and Tomic, Nikola and Yu, Tingting and Aleti, Aldeida and Chen, Chunyang},
  title = {ViBR: Automated Bug Replay from Video-Based Reports using Vision-Language Models},
  journal = {Proceedings of the ACM on Software Engineering},
  year = {2026},
  volume = {3},
  number = {FSE},
  pages = {3275--3297},
  doi = {10.1145/3808152}
}
Kostenlos starten →