Evaluating large language models effectiveness for flow-based intrusion detection: a comparative study with ML and DL baselines
Artificial Intelligence Review · 09.01.2026 · 59 (2)
DOI: 10.1007/s10462-025-11432-2 ↗Kurz gefasst
Fachartikel in der Zeitschrift Artificial Intelligence Review, online erschienen am 09.01.2026. Verfasst von Lorena Mehavilla und 3 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 44 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Universidad de Zaragoza.
Abstract (Originalsprache)
This paper presents the first systematic benchmark evaluating Large Language Models (LLMs), specifically GPT-2, GPT-Neo-125M, and LLaMA-3.2-1B, as standalone classifiers for intrusion detection, covering both binary and multiclass classification tasks, using structured Zeek logs derived from the CIC IoT 2023 dataset. We compare their performance against established and widely used Machine Learning (XGBoost, Random Forest, Decision Tree) and Deep Learning models (MLP, GRU, LeNet-5) across key evaluation metrics: detection effectiveness (precision, recall and F1-score), inference speed, and resource consumption. All models are consistently trained and rigorously evaluated on the CIC IoT 2023 dataset, ensuring fair, reproducible, and transparent comparisons. Our findings indicate that while LLMs achieve strong F1-score exceeding 95%, and do not fully utilize available GPU resources, they still do not outperform top-performing ML models. Notably XGBoost achieves a higher F1-score of 96.96%, using only 4% of the available CPU. These results emphasize the practical trade-offs between detection capability, inference efficiency, and hardware requirements when applying LLMs in flow-based IDS contexts, particularly in resource-constrained environments such as IoT or edge deployments.
Förderung
- Universidad de Zaragoza
Zitieren
APA
Mehavilla, L., Rodríguez, M., García, J., & Alesanco, Á. (2026). Evaluating large language models effectiveness for flow-based intrusion detection: a comparative study with ML and DL baselines. Artificial Intelligence Review, 59(2). https://doi.org/10.1007/s10462-025-11432-2
Vancouver
Mehavilla L, Rodríguez M, García J, Alesanco Á. Evaluating large language models effectiveness for flow-based intrusion detection: a comparative study with ML and DL baselines. Artificial Intelligence Review. 2026;59(2). doi:10.1007/s10462-025-11432-2
BibTeX
@article{Mehavilla2026,
author = {Mehavilla, Lorena and Rodríguez, María and García, José and Alesanco, Álvaro},
title = {Evaluating large language models effectiveness for flow-based intrusion detection: a comparative study with ML and DL baselines},
journal = {Artificial Intelligence Review},
year = {2026},
volume = {59},
number = {2},
doi = {10.1007/s10462-025-11432-2}
}
EUVIMED