Audio and Video Scene Classification with Cross-Modal Attention Mechanism
Big Data and Cognitive Computing · 15.07.2026 · 10 (7) · 237
DOI: 10.3390/bdcc10070237 ↗Kurz gefasst
Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 15.07.2026. Verfasst von Mingze Xia und 2 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Scene classification aims to identify scene categories by analyzing environmental information. In real-world scenes, scene features are primarily captured through acoustic and visual modalities. However, environmental complexity and information diversity pose significant challenges to classification performance. To address the issues of insufficient information interaction and inadequate feature fusion in traditional segmented fusion methods for audio–visual data, this paper proposes an audio–visual scene classification method based on a cross-modal attention mechanism. The fusion mechanism of multi-modal features is investigated, and scene classification performance is enhanced by optimizing the feature fusion strategy. The proposed method consists of three components: a cross-modal attention module, a gating unit, and a residual connection. The cross-modal attention module achieves adaptive feature alignment by establishing dynamic correlations between audio and visual features. The multi-modal gating unit employs an adaptive gating mechanism to dynamically adjust the contribution weight of each modality, thereby alleviating the information loss problem commonly encountered in traditional methods. The residual connection module preserves the original modality features to prevent information degradation. The model’s performance is evaluated through testing and validation on real-scene audio–visual datasets. Multiple sets of experimental results on these datasets demonstrate that the proposed cross-modal attention method achieves a significant improvement in classification accuracy.
Zitieren
APA
Xia, M., Yin, G., & Dong, Y. (2026). Audio and Video Scene Classification with Cross-Modal Attention Mechanism. Big Data and Cognitive Computing, 10(7), 237. https://doi.org/10.3390/bdcc10070237
Vancouver
Xia M, Yin G, Dong Y. Audio and Video Scene Classification with Cross-Modal Attention Mechanism. Big Data and Cognitive Computing. 2026;10(7):237. doi:10.3390/bdcc10070237
BibTeX
@article{Xia2026,
author = {Xia, Mingze and Yin, Guisheng and Dong, Yuxin},
title = {Audio and Video Scene Classification with Cross-Modal Attention Mechanism},
journal = {Big Data and Cognitive Computing},
year = {2026},
volume = {10},
number = {7},
pages = {237},
doi = {10.3390/bdcc10070237}
}
EUVIMED