Category-Invariance-Guided Feature Selection (CIFS): Diagnosing and Mitigating the Generalization Gap in Flow-Based VPN Traffic Classification
European Journal of Computer Science and Information Technology · 01.01.2026 · 14 (5) · 46-69
DOI: 10.37745/ejcsit.2013/vol14n54669 ↗Kurz gefasst
Fachartikel in der Zeitschrift European Journal of Computer Science and Information Technology, online erschienen am 30.07.2026. Verfasst von Rabby Zil Jalal Kanon und 2 weiteren Autor:innen.
Abstract (Originalsprache)
Prior studies on VPN traffic classification using the ISCX VPN-nonVPN dataset report high accuracy (often above 90%) under conventional random train-test splits. This study investigates whether such performance reflects genuine generalization. After deduplication and 20-seed application-disjoint evaluation, Random Forest's accuracy falls from 92.3% to between 35% and 67% when tested on withheld, unseen application categories, and for four of seven categories falls at or below a trivial majority-class baseline. Three independent sanity checks confirm this gap is genuine rather than a data-leakage artifact. A feature ablation study shows the gap is driven by category-specific feature interactions rather than a single overfitting mechanism.As a secondary, partial mitigation, we propose Category-Invariance-Guided Feature Selection (CIFS), a lightweight feature-selection method that removes application-revealing signal from features. On Random Forest, CIFS significantly improves accuracy in two of seven categories after Bonferroni correction (BROWSING, MAIL). However, an identical evaluation on Decision Tree shows no significant benefit and one significant harm, indicating CIFS's benefit is classifier-specific rather than universal.
Zitieren
APA
Kanon, R. Z. J., Fahim, M. S. I., & Sohan, K. (2026). Category-Invariance-Guided Feature Selection (CIFS): Diagnosing and Mitigating the Generalization Gap in Flow-Based VPN Traffic Classification. European Journal of Computer Science and Information Technology, 14(5), 46-69. https://doi.org/10.37745/ejcsit.2013/vol14n54669
Vancouver
Kanon RZJ, Fahim MSI, Sohan K. Category-Invariance-Guided Feature Selection (CIFS): Diagnosing and Mitigating the Generalization Gap in Flow-Based VPN Traffic Classification. European Journal of Computer Science and Information Technology. 2026;14(5):46-69. doi:10.37745/ejcsit.2013/vol14n54669
BibTeX
@article{Kanon2026,
author = {Kanon, Rabby Zil Jalal and Fahim, Md Shoriful Islam and Sohan, Khaledujjaman},
title = {Category-Invariance-Guided Feature Selection (CIFS): Diagnosing and Mitigating the Generalization Gap in Flow-Based VPN Traffic Classification},
journal = {European Journal of Computer Science and Information Technology},
year = {2026},
volume = {14},
number = {5},
pages = {46--69},
doi = {10.37745/ejcsit.2013/vol14n54669}
}
EUVIMED