EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ASTRA: Joint Latent Representation and Knowledge Transfer for Multimodal Anomaly Detection

Youngbin Kim, Taeyeong Kim, Wongyu Lee, Myeongah Cho

IEEE Access · 2026 · 14 · 109706-109725

DOI: 10.1109/access.2026.3710522 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Youngbin Kim und 3 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: BK21 FOUR program of Graduate School, Kyung Hee University, Korea Planning & Evaluation Institute of Industrial Technology, Korea government.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Weakly supervised video anomaly detection (WSVAD) aims to localize frame-level anomalies using only video-level labels. Recent vision–language-based methods have achieved promising results, but they mainly rely on visual cues and thus remain limited when visual evidence is ambiguous or degraded. Audio-visual VAD methods have attempted to address this limitation; however, existing approaches often rely on one-way knowledge distillation or simple modality fusion, which can cause modality bias and limit effective cross-modal learning. To address these issues, we propose ASTRA: Joint Latent Representation and Knowledge Transfer for Multimodal Anomaly Detection, a weakly supervised audio-visual anomaly detection framework. ASTRA consists of two main components. First, Multimodal Interactive Knowledge Transfer (MINT) performs prediction-level knowledge transfer among visual, audio, and audio-visual branches, enabling bidirectional audio-visual interaction and fusion-guided visual refinement. Second, Modality-Invariant Semantic Alignment (MISA) aligns visual and audio anomaly representations in a shared semantic space while separating them from normal and background patterns. Through these components, ASTRA learns complementary and semantically consistent multimodal representations for anomaly detection. Comprehensive experiments on multiple WSVAD benchmarks demonstrate that ASTRA improves coarse-grained anomaly detection, fine-grained temporal localization, and cross-domain generalization compared with existing methods.

Förderung

  • BK21 FOUR program of Graduate School, Kyung Hee University
  • Korea Planning & Evaluation Institute of Industrial Technology
  • Korea government

Zitieren

APA

Kim, Y., Kim, T., Lee, W., & Cho, M. (2026). ASTRA: Joint Latent Representation and Knowledge Transfer for Multimodal Anomaly Detection. IEEE Access, 14, 109706-109725. https://doi.org/10.1109/access.2026.3710522

Vancouver

Kim Y, Kim T, Lee W, Cho M. ASTRA: Joint Latent Representation and Knowledge Transfer for Multimodal Anomaly Detection. IEEE Access. 2026;14:109706-109725. doi:10.1109/access.2026.3710522

BibTeX
@article{Kim2026,
  author = {Kim, Youngbin and Kim, Taeyeong and Lee, Wongyu and Cho, Myeongah},
  title = {ASTRA: Joint Latent Representation and Knowledge Transfer for Multimodal Anomaly Detection},
  journal = {IEEE Access},
  year = {2026},
  volume = {14},
  pages = {109706--109725},
  doi = {10.1109/access.2026.3710522}
}
Kostenlos starten →