EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Lightweight Approach for Adaptation of Language Models for Cybersecurity Named Entity Recognition

Smita Srivastava, Deepa Gupta, Biswajit Paul, Shubhashisa Sahoo

Journal of Cybersecurity and Privacy · 01.11.2026 · 6 (6) · 188

DOI: 10.3390/jcp6060188 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Cybersecurity and Privacy, online erschienen am 01.11.2026. Verfasst von Smita Srivastava und 3 weiteren Autor:innen. Die Arbeit nennt 83 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Amrita Vishwa Vidyapeetham.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Cybersecurity Named Entity Recognition (CyberNER) commonly relies on fine-tuning general-purpose language models (LMs), whose WordPiece tokenizers fragment compound technical identifiers such as function names and version strings. Training from scratch with a domain-specific tokenizer addresses this at high compute cost, continual pretraining usually retains the original tokenizer, and both must be repeated as domain vocabulary evolves. We propose a lower-cost alternative: fusing a small set of domain terms into a general-purpose LM’s vocabulary before fine-tuning. Candidate terms are grouped by WordPiece split count and ranked by TF-IDF within each group, and the vocabulary budget is located across twenty configurations. On the auto-labeled STUCCO-CNC corpus (13.8 K sentences, 0.81 M tokens), fusing 250 terms that split into five or more pieces into BERT-large-cased raises 5-fold cross-validated F1 from 96.66 ± 0.25 to 96.91 ± 0.15 (best TF-IDF configuration). On the 80/20 split used to select this configuration, F1 rises from 97.20 to 97.63, exceeding four of five domain-pretrained encoders, including CySecBERT (97.45), and coming within 0.20 F1 of SecureBERT (97.83). Unlike SecureBERT, whose continual pretraining used about 1.07 billion tokens, the approach needs no pretraining run and fine-tunes in about 1.5 h (measured) on a single 24 GB GPU. Representation diagnostics (t-SNE, cluster-validity indices, cluster-wise silhouette and attention entropy) show fused-term clusters with no negatively scored members. On the manually annotated CyberNER-STIX corpus (∼23.5 K sentences, four datasets harmonized to STIX 2.1), evaluated only after the configuration was fixed on STUCCO-CNC, fusion improves entity-level F1 by about 1.0 point (1.57 points at token level), and the best configuration (80.10) exceeds the only published result on this corpus (72.30, BERT-base-cased), largely owing to the larger backbone. Fusion therefore improves on the baseline on both corpora, but the selection rule outperforms random selection only on CyberNER-STIX, so its benefit is dataset-dependent. All gains from fusion are small and indicative rather than statistically established.

Förderung

  • Amrita Vishwa Vidyapeetham

Zitieren

APA

Srivastava, S., Gupta, D., Paul, B., & Sahoo, S. (2026). A Lightweight Approach for Adaptation of Language Models for Cybersecurity Named Entity Recognition. Journal of Cybersecurity and Privacy, 6(6), 188. https://doi.org/10.3390/jcp6060188

Vancouver

Srivastava S, Gupta D, Paul B, Sahoo S. A Lightweight Approach for Adaptation of Language Models for Cybersecurity Named Entity Recognition. Journal of Cybersecurity and Privacy. 2026;6(6):188. doi:10.3390/jcp6060188

BibTeX
@article{Srivastava2026,
  author = {Srivastava, Smita and Gupta, Deepa and Paul, Biswajit and Sahoo, Shubhashisa},
  title = {A Lightweight Approach for Adaptation of Language Models for Cybersecurity Named Entity Recognition},
  journal = {Journal of Cybersecurity and Privacy},
  year = {2026},
  volume = {6},
  number = {6},
  pages = {188},
  doi = {10.3390/jcp6060188}
}
Kostenlos starten →