Match me if you can: aligning historical posters and newspaper articles via multimodal deep learning
Digital Scholarship in the Humanities · 19.06.2026
DOI: 10.1093/llc/fqag088 ↗Kurz gefasst
Fachartikel in der Zeitschrift Digital Scholarship in the Humanities, online erschienen am 19.06.2026. Verfasst von Fan Xu und 2 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Gefördert von: PRIDE program of the Luxembourg National Research Fund, Deep Data Science of Digital History, Doctoral Training Unit and by the FNR.
Abstract (Originalsprache)
Multimodal deep learning (MDL) has potential for digital history (DH) research, from simple tasks such as image or text classification, to more sophisticated ones such as information retrieval (IR). We propose a novel IR method based on vision language models (VLMs), a class of MDL models, to extract embeddings from images and text to find connections among unrelated image and text datasets that are challenging to find using conventional approaches such as keyword search. We evaluate our method with a proof-of-concept case study on debates surrounding the civil usage of nuclear power in Europe, based on poster images from the Landelijk Kernenergie Archief website and German newspaper articles found in the Impresso corpus, where both datasets are assumed to share some common themes. We find that poster images and newspaper articles can be automatically matched from independent sources based on representations extracted by pre-trained VLMs. We further find that VLMs achieve better performance when translating newspaper articles to English or summarizing them in English. Ultimately, our work bridges the gap of cross-modal and cross-lingual IR for DH, showing real-world applicability of MDL and opening new avenues in DH research. Our code and data are available at https://github.com/StevenXuf/MatchMeIfYouCan.
Förderung
- PRIDE program of the Luxembourg National Research Fund
- Deep Data Science of Digital History
- Doctoral Training Unit and by the FNR
- Swiss National Science Foundation
Zitieren
APA
Xu, F., Düring, M., & Leiva, L. A. (2026). Match me if you can: aligning historical posters and newspaper articles via multimodal deep learning. Digital Scholarship in the Humanities. https://doi.org/10.1093/llc/fqag088
Vancouver
Xu F, Düring M, Leiva LA. Match me if you can: aligning historical posters and newspaper articles via multimodal deep learning. Digital Scholarship in the Humanities. 2026. doi:10.1093/llc/fqag088
BibTeX
@article{Xu2026,
author = {Xu, Fan and Düring, Marten and Leiva, Luis A},
title = {Match me if you can: aligning historical posters and newspaper articles via multimodal deep learning},
journal = {Digital Scholarship in the Humanities},
year = {2026},
doi = {10.1093/llc/fqag088}
}
EUVIMED