Multimodal metadata assignment for cultural heritage artifacts
Multimedia Systems · 2023-04 · 29 (2) · 847-869
DOI: 10.1007/s00530-022-01025-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Multimedia Systems, online erschienen am 21.11.2022. Verfasst von Luis Rei und 7 weiteren Autor:innen. Die Arbeit wurde bisher 30-mal zitiert (Crossref) und nennt 64 Literaturangaben. Gefördert von: European Commission.
Abstract (Originalsprache)
We develop a multimodal classifier for the cultural heritage domain using a late fusion approach and introduce a novel dataset. The three modalities are Image, Text, and Tabular data. We based the image classifier on a ResNet convolutional neural network architecture and the text classifier on a multilingual transformer architecture (XML-Roberta). Both are trained as multitask classifiers. Tabular data and late fusion are handled by Gradient Tree Boosting. We also show how we leveraged a specific data model and taxonomy in a Knowledge Graph to create the dataset and to store classification results.
Förderung
- European Commission
Zitieren
APA
Rei, L., Mladenic, D., Dorozynski, M., Rottensteiner, F., Schleider, T., Troncy, R., Lozano, J. S., & Salvatella, M. G. (2023). Multimodal metadata assignment for cultural heritage artifacts. Multimedia Systems, 29(2), 847-869. https://doi.org/10.1007/s00530-022-01025-2
Vancouver
Rei L, Mladenic D, Dorozynski M, Rottensteiner F, Schleider T, Troncy R, et al. Multimodal metadata assignment for cultural heritage artifacts. Multimedia Systems. 2023;29(2):847-869. doi:10.1007/s00530-022-01025-2
BibTeX
@article{Rei2023,
author = {Rei, Luis and Mladenic, Dunja and Dorozynski, Mareike and Rottensteiner, Franz and Schleider, Thomas and Troncy, Raphaël and Lozano, Jorge Sebastián and Salvatella, Mar Gaitán},
title = {Multimodal metadata assignment for cultural heritage artifacts},
journal = {Multimedia Systems},
year = {2023},
volume = {29},
number = {2},
pages = {847--869},
doi = {10.1007/s00530-022-01025-2}
}
EUVIMED