EUVIMEDEUVIMED entdecken
Crossref

Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language

Dicong Wang, Qilong Wang, Qinghua Hu, Kaijun Wu

IEEE Transactions on Instrumentation and Measurement · 2025 · 74 · 1-12

DOI: 10.1109/tim.2025.3578702 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Instrumentation and Measurement, 2025. Verfasst von Dicong Wang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation Key Project of Gansu Province, Inner Mongolia Key Research and Development and Achievement Transformation Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The deep learning-based anomaly detection methods using visual sensors generally rely on a single modality or variants as raw signal inputs, which severely limits expressiveness and adaptability. The evolution of multimodal and visual-language pretrained models is shaping new possibilities in video anomaly detection (VAD). So, how to efficiently leverage them to achieve reliable multimodal VAD presents a significant challenge worth investigating. In this work, we propose a novel dual-stream multimodal VAD network, which integrates coarse-grained and fine-grained streams combining video, audio, and text modalities. First, in the coarse-grained stream, we perform cross-modal fusion of audio features with temporally modeled visual features, utilizing contrastive optimization to achieve more accurate coarse-grained results. In the fine-grained stream, we constructed abnormal-aware context prompts (ACPs) by integrating visual information and prior knowledge related to anomalous events into the text modality. Through the “coarse-support-fine” strategy, we further enhanced the model’s ability to discriminate fine-grained anomalies. Our method achieved optimal performance in experiments on two large-scale anomaly datasets, demonstrating its effectiveness and superiority. It supports the development of highly robust intelligent monitoring systems and promotes the potential applications of multimodal VAD across industrial monitoring, public safety, smart cities, and so on.

Förderung

  • National Natural Science Foundation of China
  • Natural Science Foundation Key Project of Gansu Province
  • Inner Mongolia Key Research and Development and Achievement Transformation Project
  • Key Research and Development Project of Lanzhou Jiaotong University
  • Key Talent Project of Gansu Province

Zitieren

APA

Wang, D., Wang, Q., Hu, Q., & Wu, K. (2025). Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language. IEEE Transactions on Instrumentation and Measurement, 74, 1-12. https://doi.org/10.1109/tim.2025.3578702

Vancouver

Wang D, Wang Q, Hu Q, Wu K. Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language. IEEE Transactions on Instrumentation and Measurement. 2025;74:1-12. doi:10.1109/tim.2025.3578702

BibTeX
@article{Wang2025,
  author = {Wang, Dicong and Wang, Qilong and Hu, Qinghua and Wu, Kaijun},
  title = {Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language},
  journal = {IEEE Transactions on Instrumentation and Measurement},
  year = {2025},
  volume = {74},
  pages = {1--12},
  doi = {10.1109/tim.2025.3578702}
}
Kostenlos starten →