EUVIMEDEUVIMED entdecken
Crossref

Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering

Heng Liu, Boyue Wang, Yanfeng Sun, Junbin Gao, Xiaoyan Li, Yongli Hu, Baocai Yin

IEEE Transactions on Big Data · 2025-06 · 11 (3) · 1346-1356

DOI: 10.1109/tbdata.2024.3453750 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Big Data, 2025-06. Verfasst von Heng Liu und 6 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 47 Literaturangaben. Gefördert von: National Key R&D Program of China, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurately locating the question-related regions in one given image is crucial for visual question answering (VQA). The current approaches suffer two limitations: (1) Dividing one image into multiple regions may lose parts of semantic information and original relationships between regions; (2) Choosing only one or all image regions to predict the answer may correspondingly result in the insufficiency or redundancy of information. Therefore, how to effectively mine the relationship between image regions and choose the relevant image regions are vital. In this paper, we propose a novel M ulti-granularity feature interaction and M ulti-region selection-based triplet VQA model (M2TVQA). To tackle the first limitation, we propose the multi-granularity feature interaction strategy that adaptively supplements the global coarse-granularity features with the regional fine-granularity features. To overcome the second limitation, we design the Top- $K$ K learning strategy to adaptively select $K$ K most relevant image regions to the question, even if the selected regions are far away in space. Such a strategy can select as many relevant image regions as possible and reduce introducing noise. Finally, we construct the multi-modality triplet to predict the answer of VQA. Extended experiments on two public outside knowledge datasets OK-VQA and KRVQA verify the effectiveness of the proposed model.

Förderung

  • National Key R&D Program of China
  • National Natural Science Foundation of China

Zitieren

APA

Liu, H., Wang, B., Sun, Y., Gao, J., Li, X., Hu, Y., & Yin, B. (2025). Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering. IEEE Transactions on Big Data, 11(3), 1346-1356. https://doi.org/10.1109/tbdata.2024.3453750

Vancouver

Liu H, Wang B, Sun Y, Gao J, Li X, Hu Y, et al. Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering. IEEE Transactions on Big Data. 2025;11(3):1346-1356. doi:10.1109/tbdata.2024.3453750

BibTeX
@article{Liu2025,
  author = {Liu, Heng and Wang, Boyue and Sun, Yanfeng and Gao, Junbin and Li, Xiaoyan and Hu, Yongli and Yin, Baocai},
  title = {Multi-Granularity Feature Interaction and Multi-Region Selection Based Triplet Visual Question Answering},
  journal = {IEEE Transactions on Big Data},
  year = {2025},
  volume = {11},
  number = {3},
  pages = {1346--1356},
  doi = {10.1109/tbdata.2024.3453750}
}
Kostenlos starten →