Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering
IEEE Transactions on Big Data · 2025-06 · 11 (3) · 1346-1356
DOI: 10.1109/tbdata.2024.3453750 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Big Data, 2025-06. Verfasst von Heng Liu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
Accurately locating the question-related regions in one given image is crucial for visual question answering (VQA). The current approaches suffer two limitations: (1) Dividing one image into multiple regions may lose parts of semantic information and original relationships between regions; (2) Choosing only one or all image regions to predict the answer may correspondingly result in the insufficiency or redundancy of information. Therefore, how to effectively mine the relationship between image regions and choose the relevant image regions are vital. In this paper, we propose a novel M ulti-granularity feature interaction and M ulti-region selection-based triplet VQA model (M2TVQA). To tackle the first limitation, we propose the multi-granularity feature interaction strategy that adaptively supplements the global coarse-granularity features with the regional fine-granularity features. To overcome the second limitation, we design the Top- $K$ K learning strategy to adaptively select $K$ K most relevant image regions to the question, even if the selected regions are far away in space. Such a strategy can select as many relevant image regions as possible and reduce introducing noise. Finally, we construct the multi-modality triplet to predict the answer of VQA. Extended experiments on two public outside knowledge datasets OK-VQA and KRVQA verify the effectiveness of the proposed model.
Förderung
- National Key R&D Program of China
- National Natural Science Foundation of China
Zitieren
APA
Liu, H., Wang, B., Sun, Y., Gao, J., Li, X., Hu, Y., & Yin, B. (2025). Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering. IEEE Transactions on Big Data, 11(3), 1346-1356. https://doi.org/10.1109/tbdata.2024.3453750
Vancouver
Liu H, Wang B, Sun Y, Gao J, Li X, Hu Y, et al. Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering. IEEE Transactions on Big Data. 2025;11(3):1346-1356. doi:10.1109/tbdata.2024.3453750
BibTeX
@article{Liu2025,
author = {Liu, Heng and Wang, Boyue and Sun, Yanfeng and Gao, Junbin and Li, Xiaoyan and Hu, Yongli and Yin, Baocai},
title = {Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering},
journal = {IEEE Transactions on Big Data},
year = {2025},
volume = {11},
number = {3},
pages = {1346--1356},
doi = {10.1109/tbdata.2024.3453750}
}
EUVIMED