Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language
IEEE Transactions on Instrumentation and Measurement · 2025 · 74 · 1-12
DOI: 10.1109/tim.2025.3578702 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Instrumentation and Measurement, 2025. Verfasst von Dicong Wang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 69 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Natural Science Foundation Key Project of Gansu Province, Inner Mongolia Key Research and Development and Achievement Transformation Project.
Abstract (Originalsprache)
The deep learning-based anomaly detection methods using visual sensors generally rely on a single modality or variants as raw signal inputs, which severely limits expressiveness and adaptability. The evolution of multimodal and visual-language pretrained models is shaping new possibilities in video anomaly detection (VAD). So, how to efficiently leverage them to achieve reliable multimodal VAD presents a significant challenge worth investigating. In this work, we propose a novel dual-stream multimodal VAD network, which integrates coarse-grained and fine-grained streams combining video, audio, and text modalities. First, in the coarse-grained stream, we perform cross-modal fusion of audio features with temporally modeled visual features, utilizing contrastive optimization to achieve more accurate coarse-grained results. In the fine-grained stream, we constructed abnormal-aware context prompts (ACPs) by integrating visual information and prior knowledge related to anomalous events into the text modality. Through the “coarse-support-fine” strategy, we further enhanced the model’s ability to discriminate fine-grained anomalies. Our method achieved optimal performance in experiments on two large-scale anomaly datasets, demonstrating its effectiveness and superiority. It supports the development of highly robust intelligent monitoring systems and promotes the potential applications of multimodal VAD across industrial monitoring, public safety, smart cities, and so on.
Förderung
- National Natural Science Foundation of China
- Natural Science Foundation Key Project of Gansu Province
- Inner Mongolia Key Research and Development and Achievement Transformation Project
- Key Research and Development Project of Lanzhou Jiaotong University
- Key Talent Project of Gansu Province
Zitieren
APA
Wang, D., Wang, Q., Hu, Q., & Wu, K. (2025). Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language. IEEE Transactions on Instrumentation and Measurement, 74, 1-12. https://doi.org/10.1109/tim.2025.3578702
Vancouver
Wang D, Wang Q, Hu Q, Wu K. Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language. IEEE Transactions on Instrumentation and Measurement. 2025;74:1-12. doi:10.1109/tim.2025.3578702
BibTeX
@article{Wang2025,
author = {Wang, Dicong and Wang, Qilong and Hu, Qinghua and Wu, Kaijun},
title = {Multimodal VAD: Visual Anomaly Detection in Intelligent Monitoring System via Audio-Vision-Language},
journal = {IEEE Transactions on Instrumentation and Measurement},
year = {2025},
volume = {74},
pages = {1--12},
doi = {10.1109/tim.2025.3578702}
}
EUVIMED