EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

MT-CMVAD: A Multi-Modal Transformer Framework for Cross-Modal Video Anomaly Detection

Hantao Ding, Shengfeng Lou, Hairong Ye, Yanbing Chen

Applied Sciences · 16.06.2025 · 15 (12) · 6773

DOI: 10.3390/app15126773 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 16.06.2025. Verfasst von Hantao Ding und 3 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 45 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video anomaly detection (VAD) faces significant challenges in multimodal semantic alignment and long-term temporal modeling within open surveillance scenarios. Existing methods are often plagued by modality discrepancies and fragmented temporal reasoning. To address these issues, we introduce MT-CMVAD, a hierarchically structured Transformer architecture that makes two key technical contributions: (1) A Context-Aware Dynamic Fusion Module that leverages cross-modal attention with learnable gating coefficients to effectively bridge the gap between RGB and optical flow modalities through adaptive feature recalibration, significantly enhancing fusion performance; (2) A Multi-Scale Spatiotemporal Transformer that establishes global-temporal dependencies via dilated attention mechanisms while preserving local spatial semantics through pyramidal feature aggregation. To address the sparse anomaly supervision dilemma, we propose a hybrid learning objective that integrates dual-stream reconstruction loss with prototype-based contrastive discrimination, enabling the joint optimization of pattern restoration and discriminative representation learning. Our extensive experiments on the UCF-Crime, UBI-Fights, and UBnormal datasets demonstrate state-of-the-art performance, achieving AUC scores of 98.9%, 94.7%, and 82.9%, respectively. The explicit spatiotemporal encoding scheme further improves temporal alignment accuracy by 2.4%, contributing to enhanced anomaly localization and overall detection accuracy. Additionally, the proposed framework achieves a 14.3% reduction in FLOPs and demonstrates 18.7% faster convergence during training, highlighting its practical value for real-world deployment. Our optimized window-shift attention mechanism also reduces computational complexity, making MT-CMVAD a robust and efficient solution for safety-critical video understanding tasks.

Zitieren

APA

Ding, H., Lou, S., Ye, H., & Chen, Y. (2025). MT-CMVAD: A Multi-Modal Transformer Framework for Cross-Modal Video Anomaly Detection. Applied Sciences, 15(12), 6773. https://doi.org/10.3390/app15126773

Vancouver

Ding H, Lou S, Ye H, Chen Y. MT-CMVAD: A Multi-Modal Transformer Framework for Cross-Modal Video Anomaly Detection. Applied Sciences. 2025;15(12):6773. doi:10.3390/app15126773

BibTeX
@article{Ding2025,
  author = {Ding, Hantao and Lou, Shengfeng and Ye, Hairong and Chen, Yanbing},
  title = {MT-CMVAD: A Multi-Modal Transformer Framework for Cross-Modal Video Anomaly Detection},
  journal = {Applied Sciences},
  year = {2025},
  volume = {15},
  number = {12},
  pages = {6773},
  doi = {10.3390/app15126773}
}
Kostenlos starten →