EUVIMEDScopri EUVIMED
Open access · CC BY 4.0

VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage

Federico Becattini, Pietro Bongini, Luana Bulla, Alberto Del Bimbo, Ludovica Marinucci, Misael Mongiovì, Valentina Presutti

ACM Transactions on Multimedia Computing, Communications, and Applications · 2023-11-30 · 19 (6) · 1-20

DOI: 10.1145/3590773 ↗

At a glance

Journal article in ACM Transactions on Multimedia Computing, Communications, and Applications, published online on 2023-07-12. Written by Federico Becattini and 6 co-authors. Cited 23 times so far (Crossref); 69 references. The full text is freely available (open access, CC BY 4.0).

AbstractDati bibliograficiCiteRelated studies

Abstract (original language)

Visual question answering has recently been settled as a fundamental multi-modal reasoning task of artificial intelligence that allows users to get information about visual content by asking questions in natural language. In the cultural heritage domain, this task can contribute to assisting visitors in museums and cultural sites, thus increasing engagement. However, the development of visual question answering models for cultural heritage is prevented by the lack of suitable large-scale datasets. To meet this demand, we built a large-scale heterogeneous and multilingual (Italian and English) dataset for cultural heritage that comprises approximately 500K Italian cultural assets and 6.5M question-answer pairs. We propose a novel formulation of the task that requires reasoning over both the visual content and an associated natural language description, and present baselines for this task. Results show that the current state of the art is reasonably effective but still far from satisfactory; therefore, further research in this area is recommended. Nonetheless, we also present a holistic baseline to address visual and contextual questions and foster future research on the topic.

Cite

APA

Becattini, F., Bongini, P., Bulla, L., Bimbo, A. D., Marinucci, L., Mongiovì, M., & Presutti, V. (2023). VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage. ACM Transactions on Multimedia Computing, Communications, and Applications, 19(6), 1-20. https://doi.org/10.1145/3590773

Vancouver

Becattini F, Bongini P, Bulla L, Bimbo AD, Marinucci L, Mongiovì M, et al. VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage. ACM Transactions on Multimedia Computing, Communications, and Applications. 2023;19(6):1-20. doi:10.1145/3590773

BibTeX
@article{Becattini2023,
  author = {Becattini, Federico and Bongini, Pietro and Bulla, Luana and Bimbo, Alberto Del and Marinucci, Ludovica and Mongiovì, Misael and Presutti, Valentina},
  title = {VISCOUNTH: A Large-scale Multilingual Visual Question Answering Dataset for Cultural Heritage},
  journal = {ACM Transactions on Multimedia Computing, Communications, and Applications},
  year = {2023},
  volume = {19},
  number = {6},
  pages = {1--20},
  doi = {10.1145/3590773}
}
Start for free →