Quantitative analysis of datasets and their usage patterns in language models
Zagadnienia Informacji Naukowej - Studia Informacyjne · 01.12.2025 · 63 (1A) · 88-109
DOI: 10.36702/zin2025.01.05e ↗Kurz gefasst
Fachartikel in der Zeitschrift Zagadnienia Informacji Naukowej - Studia Informacyjne, online erschienen am 01.12.2025. Verfasst von Adam Jachimczyk. Die Arbeit nennt 58 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Purpose/Thesis: The growing popularity of language models implies increasing interest in the datasets used for their training. This analysis aligns with the research trend focused on such datasets, concentrating on the frequency of their use and patterns of application across different types of language models. Approach/Methods: A total of 56,762 language models from the Hugging Face repository were analyzed. The study involved identifying the datasets used and determining their frequency of use in model training. It also examined the diversity of dataset usage among model authors and the degree of dataset usage across various language model categories Results and Conclusions: Most models were trained using only a single dataset. It was also observed that categories with fewer models exhibited greater diversity in the datasets used for training. Originality/Value: The analysis reveals certain patterns concerning the degree of diversity in dataset usage, depending on the author and the category of the language model.
Zitieren
APA
Jachimczyk, A. (2025). Quantitative analysis of datasets and their usage patterns in language models. Zagadnienia Informacji Naukowej - Studia Informacyjne, 63(1A), 88-109. https://doi.org/10.36702/zin2025.01.05e
Vancouver
Jachimczyk A. Quantitative analysis of datasets and their usage patterns in language models. Zagadnienia Informacji Naukowej - Studia Informacyjne. 2025;63(1A):88-109. doi:10.36702/zin2025.01.05e
BibTeX
@article{Jachimczyk2025,
author = {Jachimczyk, Adam},
title = {Quantitative analysis of datasets and their usage patterns in language models},
journal = {Zagadnienia Informacji Naukowej - Studia Informacyjne},
year = {2025},
volume = {63},
number = {1A},
pages = {88--109},
doi = {10.36702/zin2025.01.05e}
}
EUVIMED