Benchmarking and Cross-Dataset Evaluation of AI-Based Intrusion Detection Systems for Smart City IoT Networks
Computers · 2026-05-26 · 15 (6) · 340
DOI: 10.3390/computers15060340 ↗At a glance
Journal article in Computers, published online on 2026-05-26. Written by Ahlam Alghamdi and 1 co-authors. 43 references. The full text is freely available (open access, CC BY 4.0).
Abstract (original language)
The rapid expansion of Internet of Things (IoT) infrastructures in smart city environments has increased the demand for reliable intrusion detection systems (IDS). However, many existing studies rely on single-dataset evaluations and inconsistent experimental settings, which can lead to overly optimistic performance estimates. In this study, we propose a standardized benchmarking framework for evaluating artificial intelligence-based IDS across heterogeneous IoT datasets, including CIC-IoT 2023, BoT-IoT, and N-BaIoT. Multiple classical machine learning and deep learning models are evaluated under a unified preprocessing pipeline and a consistent evaluation protocol. A hybrid CNN–BiLSTM–Attention architecture is also implemented as a reference model within this framework. While several models achieve near-perfect performance under intra-dataset evaluation, cross-dataset experiments reveal substantial performance degradation and unstable metric behavior under distribution shifts. These results highlight the limitations of dataset-specific optimization and emphasize the necessity of cross-dataset validation for realistic IoT intrusion detection evaluation. All experiments are conducted under a binary intrusion detection setting (benign vs. attack) to enable consistent comparison across datasets. Consequently, the reported results reflect binary detection performance and do not capture attack-type discrimination.
Cite
APA
Alghamdi, A., & Dardouri, S. (2026). Benchmarking and Cross-Dataset Evaluation of AI-Based Intrusion Detection Systems for Smart City IoT Networks. Computers, 15(6), 340. https://doi.org/10.3390/computers15060340
Vancouver
Alghamdi A, Dardouri S. Benchmarking and Cross-Dataset Evaluation of AI-Based Intrusion Detection Systems for Smart City IoT Networks. Computers. 2026;15(6):340. doi:10.3390/computers15060340
BibTeX
@article{Alghamdi2026,
author = {Alghamdi, Ahlam and Dardouri, Samia},
title = {Benchmarking and Cross-Dataset Evaluation of AI-Based Intrusion Detection Systems for Smart City IoT Networks},
journal = {Computers},
year = {2026},
volume = {15},
number = {6},
pages = {340},
doi = {10.3390/computers15060340}
}
EUVIMED