EUVIMEDEUVIMED entdecken
Crossref

Data cleaning and spam filtering methods in the TAWOS database

Márk Szabó, Ádám Kovács, Gábor Kusper

Annales Mathematicae et Informaticae · 2026 · 63 · 132-148

DOI: 10.33039/ami.2026.06.004 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Annales Mathematicae et Informaticae, online erschienen am 17.07.2026. Verfasst von Márk Szabó und 2 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

. The TAWOS dataset is a large relational database of issue-track-ing data mined from open-source projects. While it is valuable for empirical software engineering, the issue texts also contain spam, placeholders, and off-topic noise that can distort downstream analytics and fine-tuning tasks. We present a four-stage filtering pipeline for cleaning TAWOS and focus on a validity scoring step that assigns each issue a 0–100 score from its title and description. We compare a deterministic rule-based classifier, OwnMetrics, against four local small language models (Llama-3.1-8B, Mistral-7B, Phi-3.5-mini, and Gemma-3-4B) prompted for JSON scores. Evaluation first uses a near-balanced labeled benchmark of 947 GitHub issues (481 spam/noise and 466 legitimate issues), collected from moderator-locked spam issues and legitimate issues from popular repositories. On this GitHub-based threshold-selection benchmark, OwnMetrics obtains the highest accuracy, 91.0%, at threshold 75, while the best LLM configurations reach 89.2% (Gemma-3-4B) and 89.1% (Mistral-7B). A separate manually labeled in-domain validation on 300 Jira/TAWOS issues (39 spam

Zitieren

APA

Szabó, M., Kovács, Á., & Kusper, G. (2026). Data cleaning and spam filtering methods in the TAWOS database. Annales Mathematicae et Informaticae, 63, 132-148. https://doi.org/10.33039/ami.2026.06.004

Vancouver

Szabó M, Kovács Á, Kusper G. Data cleaning and spam filtering methods in the TAWOS database. Annales Mathematicae et Informaticae. 2026;63:132-148. doi:10.33039/ami.2026.06.004

BibTeX
@article{Szab2026,
  author = {Szabó, Márk and Kovács, Ádám and Kusper, Gábor},
  title = {Data cleaning and spam filtering methods in the TAWOS database},
  journal = {Annales Mathematicae et Informaticae},
  year = {2026},
  volume = {63},
  pages = {132--148},
  doi = {10.33039/ami.2026.06.004}
}
Kostenlos starten →