EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage

Federico Becattini, Pietro Bongini, Luana Bulla, Alberto Del Bimbo, Ludovica Marinucci, Misael Mongiovì, Valentina Presutti

ACM Transactions on Multimedia Computing, Communications, and Applications · 30.11.2023 · 19 (6) · 1-20

DOI: 10.1145/3590773 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Multimedia Computing, Communications, and Applications, online erschienen am 12.07.2023. Verfasst von Federico Becattini und 6 weiteren Autor:innen. Die Arbeit wurde bisher 23-mal zitiert (Crossref) und nennt 69 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual question answering has recently been settled as a fundamental multi-modal reasoning task of artificial intelligence that allows users to get information about visual content by asking questions in natural language. In the cultural heritage domain, this task can contribute to assisting visitors in museums and cultural sites, thus increasing engagement. However, the development of visual question answering models for cultural heritage is prevented by the lack of suitable large-scale datasets. To meet this demand, we built a large-scale heterogeneous and multilingual (Italian and English) dataset for cultural heritage that comprises approximately 500K Italian cultural assets and 6.5M question-answer pairs. We propose a novel formulation of the task that requires reasoning over both the visual content and an associated natural language description, and present baselines for this task. Results show that the current state of the art is reasonably effective but still far from satisfactory; therefore, further research in this area is recommended. Nonetheless, we also present a holistic baseline to address visual and contextual questions and foster future research on the topic.

Zitieren

APA

Becattini, F., Bongini, P., Bulla, L., Bimbo, A. D., Marinucci, L., Mongiovì, M., & Presutti, V. (2023). VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(6), 1-20. https://doi.org/10.1145/3590773

Vancouver

Becattini F, Bongini P, Bulla L, Bimbo AD, Marinucci L, Mongiovì M, et al. VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(6):1-20. doi:10.1145/3590773

BibTeX
@article{Becattini2023,
  author = {Becattini, Federico and Bongini, Pietro and Bulla, Luana and Bimbo, Alberto Del and Marinucci, Ludovica and Mongiovì, Misael and Presutti, Valentina},
  title = {VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2023},
  volume = {19},
  number = {6},
  pages = {1--20},
  doi = {10.1145/3590773}
}
Kostenlos starten →