Feature-Level Analysis of Data Leakage in the IoT-Flock ICU Dataset for Healthcare Network Intrusion Detection
International Journal on Advanced Science, Engineering and Information Technology · 25.10.2026 · 16 (5) · 1957-1968
DOI: 10.18517/ijaseit.16.5.12289 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal on Advanced Science, Engineering and Information Technology, online erschienen am 25.10.2026. Verfasst von Ahmed Hammood Shlaka Al-Shammaria und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Machine-learning intrusion detectors for the Internet of Medical Things are often benchmarked on the IoT-Flock ICU dataset, on which accuracies of 96 to 100% are routinely reported. This study examines whether these scores reflect attack behavior or data-generation artifacts. We propose a Leakage-Aware Feature Audit that combines a protocol-semantic taxonomy of the 40 non-constant packet fields with four indices: single-feature separability, value purity, the uncertainty coefficient, and a cross-capture transfer gap. We prove bounds linking purity and train-test duplication to trivially attainable accuracy. It is complemented by four evaluation protocols (random, de-duplicated, temporal, and leave-one-attack-capture-out), seven classifiers, and counterfactual stress tests on 188,694 packets. After reproducing the original results, we found that benign and attack traffic differ in addresses, capture clocks, TCP/IP stack fingerprints, capture filters, and link-layer overhead. Addresses, stack fields, or capture time alone yield an MCC of 0.961 to 1.000. Giving attack packets the benign stack values reduced the recall of fingerprint-based models from 1.000 to 0.413 to 0.435, and a compound header-overhead value derived from two behavioral length fields separated the classes perfectly. After these channels were removed and the capture filter harmonized, random-split models reached the accuracy ceiling of the remaining behavioral view (0.955; MCC 0.885). On unseen attack captures, the best classifier reached an MCC of 0.516, and the mean over seven classifiers was 0.384. Published near-perfect scores therefore largely measure capture conditions. We provide a leakage-controlled benchmark protocol and a checklist for evaluating emulated IoMT datasets.
Zitieren
APA
Al-Shammaria, A. H. S., Obaida, A. H., & Almuslimawib, A. M. A. (2026). Feature-Level Analysis of Data Leakage in the IoT-Flock ICU Dataset for Healthcare Network Intrusion Detection. International Journal on Advanced Science, Engineering and Information Technology, 16(5), 1957-1968. https://doi.org/10.18517/ijaseit.16.5.12289
Vancouver
Al-Shammaria AHS, Obaida AH, Almuslimawib AMA. Feature-Level Analysis of Data Leakage in the IoT-Flock ICU Dataset for Healthcare Network Intrusion Detection. International Journal on Advanced Science, Engineering and Information Technology. 2026;16(5):1957-1968. doi:10.18517/ijaseit.16.5.12289
BibTeX
@article{AlShammaria2026,
author = {Al-Shammaria, Ahmed Hammood Shlaka and Obaida, Aqeel Hakim and Almuslimawib, Ali Mohammad Abdulhussein},
title = {Feature-Level Analysis of Data Leakage in the IoT-Flock ICU Dataset for Healthcare Network Intrusion Detection},
journal = {International Journal on Advanced Science, Engineering and Information Technology},
year = {2026},
volume = {16},
number = {5},
pages = {1957--1968},
doi = {10.18517/ijaseit.16.5.12289}
}
EUVIMED