EUVIMEDEUVIMED entdecken
Crossref

SOR-BDNet: Semantic-Optical Representation for Boundary-Aware Video Anomaly Detection with GPT-4o

Yi Sun, Bryan W. Scotney, Xiushan Nie, Xingbo Liu, Shuai Zhang, Lanting Qiu

ACM Transactions on Multimedia Computing, Communications, and Applications · 31.05.2026 · 22 (5) · 1-23

DOI: 10.1145/3801156 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 21.04.2026. Verfasst von Yi Sun und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 62 Literaturangaben. Gefördert von: British Telecom Ireland Innovation Centre, Invest Northern Ireland (Invest NI) and BT, Major Fundamental Research Project of Shandong.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In recent years, Video Anomaly Detection (VAD) has shifted from conventional appearance-based modeling to semantically driven frameworks empowered by LLMs. Traditional reconstruction- and prediction-based methods, relying on motion or appearance patterns learned from normal data, often misclassify previously unseen yet semantically normal events as anomalies. To address this limitation, we propose SOR-BDNet (Semantic-Optical Representation with Boundary Detection Network), an annotation-free multimodal VAD framework that jointly leverages visual appearance and motion dynamics to generate interpretable semantic representations at the frame level. Specifically, we employ RAFT to estimate dense motion fields and concatenate the resulting flow maps with RGB images to form unified spatiotemporal inputs. These fused representations are fed into a GPT-4o-based module that generates semantic captions capturing object semantics and motion cues. Anomalies are detected by measuring semantic deviations from a memory bank constructed from normal captions. To further refine temporal boundaries, we design a boundary refinement module that integrates visual continuity constraints with contrastive feature learning based on a Swin Transformer backbone. Extensive experiments on four challenging benchmarks—UCSD-Ped2, Avenue, ShanghaiTech, and UCF-Crime—demonstrate that SOR-BDNet achieves frame-level accuracies of 97.96%, 82.86%, 87.36%, and 85.64%, respectively. These results highlight the robustness and scalability of the proposed framework, while significantly improving interpretability and generalization across diverse real-world surveillance scenarios. The source code and pretrained models are available at https://github.com/syi-coder/SOR-BDNet-Semantic-Optical-Representation-for-Boundary-Aware-Video-Anomaly-Detection-with-GPT-4o .

Förderung

  • British Telecom Ireland Innovation Centre
  • Invest Northern Ireland (Invest NI) and BT
  • Major Fundamental Research Project of Shandong

Zitieren

APA

Sun, Y., Scotney, B. W., Nie, X., Liu, X., Zhang, S., & Qiu, L. (2026). SOR-BDNet: Semantic-Optical Representation for Boundary-Aware Video Anomaly Detection with GPT-4o. ACM Transactions on Multimedia Computing, Communications, and Applications, 22(5), 1-23. https://doi.org/10.1145/3801156

Vancouver

Sun Y, Scotney BW, Nie X, Liu X, Zhang S, Qiu L. SOR-BDNet: Semantic-Optical Representation for Boundary-Aware Video Anomaly Detection with GPT-4o. ACM Transactions on Multimedia Computing, Communications, and Applications. 2026;22(5):1-23. doi:10.1145/3801156

BibTeX
@article{Sun2026,
  author = {Sun, Yi and Scotney, Bryan W. and Nie, Xiushan and Liu, Xingbo and Zhang, Shuai and Qiu, Lanting},
  title = {SOR-BDNet: Semantic-Optical Representation for Boundary-Aware Video Anomaly Detection with GPT-4o},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2026},
  volume = {22},
  number = {5},
  pages = {1--23},
  doi = {10.1145/3801156}
}
Kostenlos starten →