EUVIMEDEUVIMED entdecken
Crossref

Question Type-Aware Debiasing for Test-Time Visual Question Answering Model Adaptation

Jin Liu, Jialong Xie, Fengyu Zhou, Shengfeng He

IEEE Transactions on Circuits and Systems for Video Technology · 2024-11 · 34 (11) · 10805-10816

DOI: 10.1109/tcsvt.2024.3410041 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2024-11. Verfasst von Jin Liu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 45 Literaturangaben. Gefördert von: Key Research and Development Project of Shandong Province, Jinan “20 New Colleges and Universities” Funded Scientific Research Leader Studio, Central Government Guiding Local Science and Technology Development Foundation of Shandong Province.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In Visual Question Answering (VQA), addressing language prior bias, where models excessively rely on superficial correlations between questions and answers, is crucial. This issue becomes more pronounced in real-world applications with diverse domains and varied question-answer distributions during testing. To tackle this challenge, Test-time Adaptation (TTA) has emerged, allowing pre-trained VQA models to adapt using unlabeled test samples. Current state-of-the-art models select reliable test samples based on fixed entropy thresholds and employ self-supervised debiasing techniques. However, these methods struggle with diverse answer spaces linked to different question types and may fail to identify biased samples that still leverage relevant visual context. In this paper, we propose Question type-guided Entropy Minimization and Debiasing (QED) as a solution for test-time VQA model adaptation. Our approach involves adaptive entropy minimization based on question types to improve the identification of fine-grained and unreliable samples. Additionally, we generate negative samples for each test sample and label them as biased if their answer entropy change rate significantly differs from positive test samples, subsequently removing them. We evaluate our approach on two public benchmarks, VQA-CP v2, and VQA-CP v1, and achieve new state-of-the-art results, with overall accuracy rates of 48.13% and 46.18%, respectively.

Förderung

  • Key Research and Development Project of Shandong Province
  • Jinan “20 New Colleges and Universities” Funded Scientific Research Leader Studio
  • Central Government Guiding Local Science and Technology Development Foundation of Shandong Province
  • Jinan City and University Cooperation Development Strategy Project
  • Guangdong Natural Science Funds for Distinguished Young Scholar
  • National Research Foundation Singapore under the AI Singapore Program
  • Meituan Academy of Robotics Shenzhen

Zitieren

APA

Liu, J., Xie, J., Zhou, F., & He, S. (2024). Question Type-Aware Debiasing for Test-Time Visual Question Answering Model Adaptation. IEEE Transactions on Circuits and Systems for Video Technology, 34(11), 10805-10816. https://doi.org/10.1109/tcsvt.2024.3410041

Vancouver

Liu J, Xie J, Zhou F, He S. Question Type-Aware Debiasing for Test-Time Visual Question Answering Model Adaptation. IEEE Transactions on Circuits and Systems for Video Technology. 2024;34(11):10805-10816. doi:10.1109/tcsvt.2024.3410041

BibTeX
@article{Liu2024,
  author = {Liu, Jin and Xie, Jialong and Zhou, Fengyu and He, Shengfeng},
  title = {Question Type-Aware Debiasing for Test-Time Visual Question Answering Model Adaptation},
  journal = {IEEE Transactions on Circuits and Systems for Video Technology},
  year = {2024},
  volume = {34},
  number = {11},
  pages = {10805--10816},
  doi = {10.1109/tcsvt.2024.3410041}
}
Kostenlos starten →