PU-Based Quality Classifier for LLM Training Texts
Applied Sciences · 27.07.2026 · 16 (15) · 7481
DOI: 10.3390/app16157481 ↗Kurz gefasst
Fachartikel in der Zeitschrift Applied Sciences, online erschienen am 27.07.2026. Verfasst von Kacper Paczutkowski und 1 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
In this study, we examine positive-unlabeled (PU) learning for classifying Polish texts intended for large language model training. We evaluate six PU methods, Naive, Clust, Strict-LassClust, Non-Strict-LassClust, LassoJoint, and Spy, under both classic and MVC non-SCAR labeling schemes. Across the benchmark, the best F1 reaches 0.906 on an open subtitles corpus under classic labeling and 0.900 under MVC labeling; the largest absolute gain over the Naive baseline is 0.811 F1 points on ulotki medyczne and 0.758 on plwiki in the classic setting. The experimental workflow supports multiple labeling rates, c calc, and repeated random seeds, and evaluates models with AUC, PR-AUC, and F1 as the main metrics. Our study focuses on six SpeakLeash datasets spanning different domains: plwiki, open subtitles corpus, job offers corpus, ISAP corpus, ulotki medyczne, and wolne lektury corpus. For each dataset, we construct a binary PU setup from the quality field, where HIGH is treated as positive and LOW is negative, while other values are removed. The resulting preprocessed tables retain only the selected linguistic and structural features, rescale numeric variables to the [0, 1] range, and remove the raw text column.
Zitieren
APA
Paczutkowski, K., & Furmańczyk, K. (2026). PU-Based Quality Classifier for LLM Training Texts. Applied Sciences, 16(15), 7481. https://doi.org/10.3390/app16157481
Vancouver
Paczutkowski K, Furmańczyk K. PU-Based Quality Classifier for LLM Training Texts. Applied Sciences. 2026;16(15):7481. doi:10.3390/app16157481
BibTeX
@article{Paczutkowski2026,
author = {Paczutkowski, Kacper and Furmańczyk, Konrad},
title = {PU-Based Quality Classifier for LLM Training Texts},
journal = {Applied Sciences},
year = {2026},
volume = {16},
number = {15},
pages = {7481},
doi = {10.3390/app16157481}
}
EUVIMED