EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

6G-Bench: An Open Benchmark for Semantic Communication and Network-Level Reasoning With Foundation Models in AI-Native 6G Networks

Mohamed Amine Ferrag, Abderrahmane Lakas, Mérouane Debbah

IEEE Open Journal of the Communications Society · 2026 · 7 · 3305-3330

DOI: 10.1109/ojcoms.2026.3680457 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Open Journal of the Communications Society, 2026. Verfasst von Mohamed Amine Ferrag und 2 weiteren Autor:innen. Die Arbeit wurde bisher 9-mal zitiert (Crossref) und nennt 50 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: United Arab Emirates University Program for Advanced Research, Khalifa University of Science and Technology.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Emerging sixth-generation (6G) networks are increasingly envisioned as AI-native, intent-driven systems in which foundation models act as high-level reasoning and coordination layers above standardized network functions. However, existing evaluations of large language models (LLMs) in wireless and networking domains largely focus on isolated tasks or treat networks as numeric constraints, leaving network-level semantic reasoning over intent, policy, trust, and multi-agent coordination insufficiently explored. This paper introduces 6G-Bench, an open benchmark for evaluating semantic communication and network-level reasoning in AI-native 6G networks. 6G-Bench defines a taxonomy of 30 decision-making tasks (T1–T30) extracted from ongoing 6G and AI-agent standardization activities in 3GPP, IETF, ETSI, ITU-T, and the O-RAN Alliance, and organizes them into five standardization-aligned capability categories. Starting from 113,475 scenarios, we generate a balanced pool of 10,000 very-hard multiple-choice questions using task-conditioned prompts that enforce multi-step quantitative reasoning under uncertainty and worst-case regret minimization over multi-turn horizons. After automated filtering and expert human validation, 3,722 questions are retained as a high-confidence evaluation set, while the full pool is released to support training and fine-tuning of 6G-specialized models. Using 6G-Bench, we evaluate 22 foundation models spanning dense and mixture-of-experts architectures, short- and long-context designs (up to 1M tokens), and both open-weight and proprietary systems. Across models, deterministic single-shot accuracy (pass@1) ranges from 0.22 to 0.82, highlighting substantial variation in semantic reasoning capability. Leading models achieve intent and policy reasoning accuracy in the range 0.87–0.89, while selective robustness analysis on reasoning-intensive tasks shows pass@5 values ranging from 0.20 to 0.91. To support open science and reproducibility, we release the 6G-Bench dataset on IEEEDataport: https://dx.doi.org/10.21227/c8pt-hc87

Förderung

  • United Arab Emirates University Program for Advanced Research
  • Khalifa University of Science and Technology

Zitieren

APA

Ferrag, M. A., Lakas, A., & Debbah, M. (2026). 6G-Bench: An Open Benchmark for Semantic Communication and Network-Level Reasoning With Foundation Models in AI-Native 6G Networks. IEEE Open Journal of the Communications Society, 7, 3305-3330. https://doi.org/10.1109/ojcoms.2026.3680457

Vancouver

Ferrag MA, Lakas A, Debbah M. 6G-Bench: An Open Benchmark for Semantic Communication and Network-Level Reasoning With Foundation Models in AI-Native 6G Networks. IEEE Open Journal of the Communications Society. 2026;7:3305-3330. doi:10.1109/ojcoms.2026.3680457

BibTeX
@article{Ferrag2026,
  author = {Ferrag, Mohamed Amine and Lakas, Abderrahmane and Debbah, Mérouane},
  title = {6G-Bench: An Open Benchmark for Semantic Communication and Network-Level Reasoning With Foundation Models in AI-Native 6G Networks},
  journal = {IEEE Open Journal of the Communications Society},
  year = {2026},
  volume = {7},
  pages = {3305--3330},
  doi = {10.1109/ojcoms.2026.3680457}
}
Kostenlos starten →