ByteScope: Detection-Aware Learning for Robust Visual Malware Recognition Under Leakage-Audited, Open-World Evaluation
Machine Learning and Knowledge Extraction · 01.11.2026 · 8 (11) · 360
DOI: 10.3390/make8110360 ↗Kurz gefasst
Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 01.11.2026. Verfasst von Arsanchai Sukkuea und 3 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Walailak University.
Abstract (Originalsprache)
Visual malware recognition classifies byte-plot images of executables and reports near-perfect accuracy, yet it is rarely evaluated the way a deployed detector would be. We present ByteScope, a framework that adds a cosine-margin family classifier, a supervised contrastive embedding for rejecting unknown families, and a class-balanced malware detection head calibrated at a fixed false-positive rate to an image backbone, together with a leakage-audited, open-world evaluation protocol, and we evaluate it on MaleVis against nine convolutional neural network (CNN), Vision Transformer and hybrid backbones with three seeds. A perceptual-hash audit, confirmed by pixel-level, embedding-based and visual inspection, shows that 50.6% of the test images and 70.0% of the malware have a same-class near-duplicate in the training split. On Swin-T, ByteScope reaches the highest mean 26-class accuracy (89.3%) and de-duplicated accuracy (78.4%), matches the best baseline in unknown-family detection (area under the receiver operating characteristic curve, AUROC, 0.925 against 0.922), and halves the evasion rate of its backbone under image-domain benign padding at a validation-calibrated threshold (15.2% against 36.0%) by making the operating point more stable, an effect that we trace to the normalized cosine classifier; at matched test false-positive rates, robustness is similar. At a 1% false-positive rate, neither ByteScope nor any of the nine baselines detects more than 3% of malware on average, whereas several variants of our hybrid CNN–Transformer architecture exceed this level (6.8 to 48.1%, with a large variance across seeds), and roughly 52 benign validation images cannot certify any rate below 5.6%. We release the code, the near-duplicate list and the splits.
Förderung
- Walailak University
Zitieren
APA
Sukkuea, A., Akkajit, P., Tharu, N. K., & Wannawas, N. (2026). ByteScope: Detection-Aware Learning for Robust Visual Malware Recognition Under Leakage-Audited, Open-World Evaluation. Machine Learning and Knowledge Extraction, 8(11), 360. https://doi.org/10.3390/make8110360
Vancouver
Sukkuea A, Akkajit P, Tharu NK, Wannawas N. ByteScope: Detection-Aware Learning for Robust Visual Malware Recognition Under Leakage-Audited, Open-World Evaluation. Machine Learning and Knowledge Extraction. 2026;8(11):360. doi:10.3390/make8110360
BibTeX
@article{Sukkuea2026,
author = {Sukkuea, Arsanchai and Akkajit, Pensiri and Tharu, Nanda Kumar and Wannawas, Nat},
title = {ByteScope: Detection-Aware Learning for Robust Visual Malware Recognition Under Leakage-Audited, Open-World Evaluation},
journal = {Machine Learning and Knowledge Extraction},
year = {2026},
volume = {8},
number = {11},
pages = {360},
doi = {10.3390/make8110360}
}
EUVIMED