EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Audio and Video Scene Classification with Cross-Modal Attention Mechanism

Mingze Xia, Guisheng Yin, Yuxin Dong

Big Data and Cognitive Computing · 15.07.2026 · 10 (7) · 237

DOI: 10.3390/bdcc10070237 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Big Data and Cognitive Computing, online erschienen am 15.07.2026. Verfasst von Mingze Xia und 2 weiteren Autor:innen. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Scene classification aims to identify scene categories by analyzing environmental information. In real-world scenes, scene features are primarily captured through acoustic and visual modalities. However, environmental complexity and information diversity pose significant challenges to classification performance. To address the issues of insufficient information interaction and inadequate feature fusion in traditional segmented fusion methods for audio–visual data, this paper proposes an audio–visual scene classification method based on a cross-modal attention mechanism. The fusion mechanism of multi-modal features is investigated, and scene classification performance is enhanced by optimizing the feature fusion strategy. The proposed method consists of three components: a cross-modal attention module, a gating unit, and a residual connection. The cross-modal attention module achieves adaptive feature alignment by establishing dynamic correlations between audio and visual features. The multi-modal gating unit employs an adaptive gating mechanism to dynamically adjust the contribution weight of each modality, thereby alleviating the information loss problem commonly encountered in traditional methods. The residual connection module preserves the original modality features to prevent information degradation. The model’s performance is evaluated through testing and validation on real-scene audio–visual datasets. Multiple sets of experimental results on these datasets demonstrate that the proposed cross-modal attention method achieves a significant improvement in classification accuracy.

Zitieren

APA

Xia, M., Yin, G., & Dong, Y. (2026). Audio and Video Scene Classification with Cross-Modal Attention Mechanism. Big Data and Cognitive Computing, 10(7), 237. https://doi.org/10.3390/bdcc10070237

Vancouver

Xia M, Yin G, Dong Y. Audio and Video Scene Classification with Cross-Modal Attention Mechanism. Big Data and Cognitive Computing. 2026;10(7):237. doi:10.3390/bdcc10070237

BibTeX
@article{Xia2026,
  author = {Xia, Mingze and Yin, Guisheng and Dong, Yuxin},
  title = {Audio and Video Scene Classification with Cross-Modal Attention Mechanism},
  journal = {Big Data and Cognitive Computing},
  year = {2026},
  volume = {10},
  number = {7},
  pages = {237},
  doi = {10.3390/bdcc10070237}
}
Kostenlos starten →