EUVIMEDEUVIMED entdecken
Crossref

A Comparative Evaluation of Large Language Models for Named Entity Recognition in Cyber Threat Intelligence

Aykhan Huseynli

International Journal of Advanced Computer Science and Applications · 2026 · 17 (6)

DOI: 10.14569/ijacsa.2026.0170614 ↗

Kurz gefasst

Fachartikel in der Zeitschrift International Journal of Advanced Computer Science and Applications, 2026. Verfasst von Aykhan Huseynli.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Cyber Threat Intelligence reports combine analytical prose with dense technical indicators, making structured entity extraction a challenging but operationally valuable task. This study presents a comparative evaluation of three large language models – Claude Sonnet 4.6, GPT-5.4, and LLaMA 4 Scout – on a manu-ally annotated corpus of 21 real-world CTI reports across 15 entity types and 1284 ground truth instances. This study evalu-ates zero-shot and few-shot prompting conditions and studies the effect of iterative prompt refinement, focusing on explicit format constraints for cryptographic hash entities. Results show that Claude Sonnet 4.6 and GPT-5.4 achieve comparable perfor-mance under zero-shot conditions, with LLaMA 4 Scout trailing by a substantial margin. Few-shot prompting consistently reduc-es hallucination rates, but yields mixed F1 results, with exemplar cardinality emerging as a critical and underappreciated design factor. Entity extraction difficulty varies substantially across types, with technical indicator categories showing near-perfect performance and semantic categories such as tool and target sector posing the greatest challenges across all evaluated mod-els.

Zitieren

APA

Huseynli, A. (2026). A Comparative Evaluation of Large Language Models for Named Entity Recognition in Cyber Threat Intelligence. International Journal of Advanced Computer Science and Applications, 17(6). https://doi.org/10.14569/ijacsa.2026.0170614

Vancouver

Huseynli A. A Comparative Evaluation of Large Language Models for Named Entity Recognition in Cyber Threat Intelligence. International Journal of Advanced Computer Science and Applications. 2026;17(6). doi:10.14569/ijacsa.2026.0170614

BibTeX
@article{Huseynli2026,
  author = {Huseynli, Aykhan},
  title = {A Comparative Evaluation of Large Language Models for Named Entity Recognition in Cyber Threat Intelligence},
  journal = {International Journal of Advanced Computer Science and Applications},
  year = {2026},
  volume = {17},
  number = {6},
  doi = {10.14569/ijacsa.2026.0170614}
}
Kostenlos starten →