Guiding Audio-Visual Question Answering with Collective Question Reasoning
International Journal of Computer Vision · 2025-10 · 133 (10) · 6912-6929
DOI: 10.1007/s11263-025-02510-7 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 03.07.2025. Verfasst von Baoqi Pei und 8 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 78 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key R&D Program of China, Shanghai Committee of Science and Technology.
Abstract (Originalsprache)
Audio-Visual Question Answering (AVQA) requires the model to answer questions with complex dynamic audio-visual information. Prior works on this task mainly consider only using single question-answer pairs during training, overlooking the rich semantic associations between questions. In this work, we propose a novel Collective Question-Guided Network (CoQo), which accepts multiple question-answer pairs as input and leverages the reasoning over these questions to assist the model training process. The core module is the proposed Question Guided Transformer (QGT), which uses collective question reasoning to perform question-guided feature extraction. Since multiple question-answer pairs are not always available, especially during inference, our QGT uses a set of learnable tokens to learn the collective information from multiple questions during training. At inference time, these learnable tokens bring additional reasoning information even when only one question is used as input. We employ QGT in both spatial and temporal dimensions to extract question-related features effectively and efficiently. To better capture detailed audio-visual associations, we train the model in a finer level by distinguishing feature pairs of different questions within the same video. Extensive experiments demonstrate that our method can achieve state-of-the-art performance on three AVQA datasets while reducing training time significantly. We also observe strong performances of our method on three VQA benchmarks. Detailed ablation studies further confirm the effectiveness of our proposed collective question reasoning scheme, both quantitatively and qualitatively.
Förderung
- National Key R&D Program of China
- Shanghai Committee of Science and Technology
Zitieren
APA
Pei, B., Huang, Y., Chen, G., Xu, J., Wang, Y., Wang, L., Lu, T., Qiao, Y., & Wu, F. (2025). Guiding Audio-Visual Question Answering with Collective Question Reasoning. International Journal of Computer Vision, 133(10), 6912-6929. https://doi.org/10.1007/s11263-025-02510-7
Vancouver
Pei B, Huang Y, Chen G, Xu J, Wang Y, Wang L, et al. Guiding Audio-Visual Question Answering with Collective Question Reasoning. International Journal of Computer Vision. 2025;133(10):6912-6929. doi:10.1007/s11263-025-02510-7
BibTeX
@article{Pei2025,
author = {Pei, Baoqi and Huang, Yifei and Chen, Guo and Xu, Jilan and Wang, Yali and Wang, Limin and Lu, Tong and Qiao, Yu and Wu, Fei},
title = {Guiding Audio-Visual Question Answering with Collective Question Reasoning},
journal = {International Journal of Computer Vision},
year = {2025},
volume = {133},
number = {10},
pages = {6912--6929},
doi = {10.1007/s11263-025-02510-7}
}
EUVIMED