Application of Computer Vision to the Automated Extraction of Metadata from Natural History Specimen Labels: A Case Study on Herbarium Specimens
Plants · 17.02.2026 · 15 (4) · 637
DOI: 10.3390/plants15040637 ↗Kurz gefasst
Fachartikel in der Zeitschrift Plants, online erschienen am 17.02.2026. Verfasst von Jacopo Zacchigna und 6 weiteren Autor:innen. Beteiligte Einrichtung: Department of Mathematics, Informatics and Geosciences, University of Trieste, Via Weiss 2, 34127 Trieste, Italy. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 42 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Metadata extraction from natural history collection labels is a pivotal task for the online publication of digitized specimens. However, given the scale of these collections—which are estimated to host more than 2 billion specimens worldwide, including ca. 400 million herbarium specimens—manual metadata extraction is an extremely time-consuming task. Thus, automated data extraction from digital images of specimens and their labels therefore is a promising application of state-of-the-art computer vision techniques. Extracting information from herbarium specimen labels normally involves three main steps: text segmentation, multilingual and handwriting recognition, and data parsing. The primary bottleneck in this workflow lies in the limitations of Optical Character Recognition (OCR) systems. This study explores how the general knowledge embedded in multimodal Transformer models can be transferred to the specific task of herbarium specimen label digitization. The final goal is to develop an easy-to-use, end-to-end solution to mitigate the limitations of classic OCR approaches while offering greater flexibility to adapt to different label formats. Donut-base, a pre-trained visual document understanding (VDU) transformer, was the base model selected for fine-tuning. A dataset from the University of Pisa served as a test bed. The initial attempt achieved an accuracy of 85%, measured using the Tree Edit Distance (TED), demonstrating the feasibility of fine-tuning for this task. Cases with low accuracies were also investigated to identify limitations of the approach. In particular, specimens with multiple labels, especially if combining handwritten and typewritten text, proved to be the most challenging. Strategies aimed at addressing these weaknesses are discussed.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Mathematics, Informatics and Geosciences, University of Trieste, Via Weiss 2, 34127 Trieste, Italy.
- Department of Life Sciences, University of Trieste, Via Licio Giorgieri 10, 34127 Trieste, Italy.
- Department of Engineering and Architecture, University of Trieste, Via Alfonso Valerio 6/1, 34127 Trieste, Italy.
- Botanic Garden and Museum, University Museum System, University of Pisa, Via Ghini 13, 56126 Pisa, Italy.
- PLANTSEED Lab, Department of Biology, University of Pisa, Via Derna 1, 56126 Pisa, Italy.
- Centro Interuniversitario per la Biodiversità Vegetale Big Data-PLANT DATA, Department of Biological, Geological and Environmental Sciences, Alma Mater Studiorum University of Bologna, Via Zamboni 33, 40126 Bologna, Ita…
ORCID-Profile: Weiwei Liu, Felice Andrea Pellegrino, Adriano Peron, Francesco Roma-Marzio, Lorenzo Peruzzi, Stefano Martellos
Zitieren
APA
Zacchigna, J., Liu, W., Pellegrino, F. A., Peron, A., Roma-Marzio, F., Peruzzi, L., & Martellos, S. (2026). Application of Computer Vision to the Automated Extraction of Metadata from Natural History Specimen Labels: A Case Study on Herbarium Specimens. Plants, 15(4), 637. https://doi.org/10.3390/plants15040637
Vancouver
Zacchigna J, Liu W, Pellegrino FA, Peron A, Roma-Marzio F, Peruzzi L, et al. Application of Computer Vision to the Automated Extraction of Metadata from Natural History Specimen Labels: A Case Study on Herbarium Specimens. Plants. 2026;15(4):637. doi:10.3390/plants15040637
BibTeX
@article{Zacchigna2026,
author = {Zacchigna, Jacopo and Liu, Weiwei and Pellegrino, Felice Andrea and Peron, Adriano and Roma-Marzio, Francesco and Peruzzi, Lorenzo and Martellos, Stefano},
title = {Application of Computer Vision to the Automated Extraction of Metadata from Natural History Specimen Labels: A Case Study on Herbarium Specimens},
journal = {Plants},
year = {2026},
volume = {15},
number = {4},
pages = {637},
doi = {10.3390/plants15040637}
}
EUVIMED