EUVIMEDEUVIMED entdecken
Crossref

Robust Visual Question Answering: Datasets, Methods, and Future Challenges

Jie Ma, Pinghui Wang, Dechen Kong, Zewei Wang, Jun Liu, Hongbin Pei, Junzhou Zhao

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-08 · 46 (8) · 5575-5594

DOI: 10.1109/tpami.2024.3366154 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-08. Verfasst von Jie Ma und 6 weiteren Autor:innen. Die Arbeit wurde bisher 54-mal zitiert (Crossref) und nennt 198 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Natural Science Basic Research Program of Shaanxi.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual question answering requires a system to provide an accurate natural language answer given an image and a natural language question. However, it is widely recognized that previous generic VQA methods often tend to memorize biases present in the training data rather than learning proper behaviors, such as grounding images before predicting answers. Therefore, these methods usually achieve high in-distribution but poor out-of-distribution performance. In recent years, various datasets and debiasing methods have been proposed to evaluate and enhance the VQA robustness, respectively. This paper provides the first comprehensive survey focused on this emerging fashion. Specifically, we first provide an overview of the development process of datasets from in-distribution and out-of-distribution perspectives. Then, we examine the evaluation metrics employed by these datasets. Third, we propose a typology that presents the development process, similarities and differences, robustness comparison, and technical features of existing debiasing methods. Furthermore, we analyze and discuss the robustness of representative vision-and-language pre-training models on VQA. Finally, through a thorough review of the available literature and experimental analysis, we discuss the key areas for future research from various viewpoints.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Natural Science Basic Research Program of Shaanxi
  • Youth Innovation Team of Shaanxi Universities
  • Shaanxi Undergraduate and Higher Education Teaching Reform Research Program

Zitieren

APA

Ma, J., Wang, P., Kong, D., Wang, Z., Liu, J., Pei, H., & Zhao, J. (2024). Robust Visual Question Answering: Datasets, Methods, and Future Challenges. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8), 5575-5594. https://doi.org/10.1109/tpami.2024.3366154

Vancouver

Ma J, Wang P, Kong D, Wang Z, Liu J, Pei H, et al. Robust Visual Question Answering: Datasets, Methods, and Future Challenges. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(8):5575-5594. doi:10.1109/tpami.2024.3366154

BibTeX
@article{Ma2024,
  author = {Ma, Jie and Wang, Pinghui and Kong, Dechen and Wang, Zewei and Liu, Jun and Pei, Hongbin and Zhao, Junzhou},
  title = {Robust Visual Question Answering: Datasets, Methods, and Future Challenges},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {8},
  pages = {5575--5594},
  doi = {10.1109/tpami.2024.3366154}
}
Kostenlos starten →