CyberRoBERTa: A Lightweight Transformer-Based Language Model for IoT Intrusion Detection
Sensors · 07.10.2026 · 26 (19) · 6326
DOI: 10.3390/s26196326 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 07.10.2026. Verfasst von Dhai Alhafi und 1 weiteren Autor:innen. Beteiligte Einrichtung: Computer Science Department, College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Ara…. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 35 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
With the accelerated proliferation of the Internet of Things (IoT) and its widespread use in sensitive areas such as healthcare, homes, and factories, security has become an increasingly critical concern. IoT remains one of the most challenging domains to secure, lacking the manageable scale, well-maintained software, and homogeneous environments of easily secured systems. Large language models (LLMs) have recently been applied to improve IoT security; however, most applications treat LLMs as “off-the-shelf” tools, creating a domain mismatch because model representations, tokenizers, and data collators are designed for natural language rather than network traffic. In this work, we present CyberRoBERTa, built specifically to test whether departing from off-the-shelf LLM components, through a domain-specific Unigram tokenizer and an IoT-specialized data collator, yields a genuine advantage for IoT intrusion detection. We instantiate this framework as a compact, 10.2-million-parameter model designed for fully self-contained, on-device operation. Using CICIoT2023, TON_IoT, and Edge-IIoTset, the model achieves a binary classification F1-score of 96% and a multiclass F1-score of 89.54%. We further validate these results in three ways: a comparison against a classical machine learning baseline shows comparable accuracy; isolating the tokenizer’s contribution from pretraining scale shows its benefit holds under matched pretraining conditions; and evaluating class-imbalance interventions shows improved detection on the most data-scarce class.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Computer Science Department, College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh 11432, Saudi Arabia.
ORCID-Profile: Dhai Alhafi, Fatmah Alanazi
Zitieren
APA
Alhafi, D., & Alanazi, F. (2026). CyberRoBERTa: A Lightweight Transformer-Based Language Model for IoT Intrusion Detection. Sensors, 26(19), 6326. https://doi.org/10.3390/s26196326
Vancouver
Alhafi D, Alanazi F. CyberRoBERTa: A Lightweight Transformer-Based Language Model for IoT Intrusion Detection. Sensors. 2026;26(19):6326. doi:10.3390/s26196326
BibTeX
@article{Alhafi2026,
author = {Alhafi, Dhai and Alanazi, Fatmah},
title = {CyberRoBERTa: A Lightweight Transformer-Based Language Model for IoT Intrusion Detection},
journal = {Sensors},
year = {2026},
volume = {26},
number = {19},
pages = {6326},
doi = {10.3390/s26196326}
}
EUVIMED