EUVIMEDEUVIMED entdecken
Crossref

Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission

Seungeun Oh, Jinhyuk Kim, Jihong Park, Seung-Woo Ko, Jinho Choi, Tony Q. S. Quek, Seong-Lyun Kim

IEEE Transactions on Communications · 2026 · 74 · 10658-10675

DOI: 10.1109/tcomm.2026.3700463 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Communications, 2026. Verfasst von Seungeun Oh und 6 weiteren Autor:innen. Die Arbeit nennt 53 Literaturangaben. Gefördert von: Institute of Information & Communications Technology Planning & Evaluation, Korea government, Development of 5G-A vRAN Research Platform, 20%.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

To support emerging language-based applications using dispersed and heterogeneous computing resources, the hybrid language model (HLM) offers a promising architecture, where an on-device small language model (SLM) generates draft tokens that are validated and corrected by a remote large language model (LLM). However, the original HLM suffers from substantial communication overhead, as the LLM requires the SLM to upload the full vocabulary distribution for each token. Moreover, both communication and computation resources are wasted when the LLM validates tokens that are highly likely to be accepted. To overcome these limitations, we propose communication-efficient and uncertainty-aware HLM (CU-HLM). In CU-HLM, the SLM transmits truncated vocabulary distributions only when its output uncertainty is high. We validate the feasibility of this opportunistic transmission by discovering a strong correlation between SLM’s uncertainty and LLM’s rejection probability. Furthermore, we theoretically derive optimal uncertainty thresholds and optimal vocabulary truncation strategies. Simulation results show that, compared to standard HLM, CU-HLM achieves up to 206 $\times $ higher token throughput by skipping 74.8% transmissions with 97.4% vocabulary compression, while maintaining 97.4% accuracy.

Förderung

  • Institute of Information & Communications Technology Planning & Evaluation
  • Korea government
  • Development of 5G-A vRAN Research Platform, 20%
  • Agency for Science, Technology and Research
  • YKCS Open RAN Global Collaboration Center
  • IITP-ITRC Grant
  • National Research Foundation, Singapore, and the Infocomm Media Development Authority under its Communications and Connectivity Bridging Funding Initiative
  • National Research Foundation of Korea (NRF) Grant

Zitieren

APA

Oh, S., Kim, J., Park, J., Ko, S., Choi, J., Quek, T. Q. S., & Kim, S. (2026). Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission. IEEE Transactions on Communications, 74, 10658-10675. https://doi.org/10.1109/tcomm.2026.3700463

Vancouver

Oh S, Kim J, Park J, Ko S, Choi J, Quek TQS, et al. Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission. IEEE Transactions on Communications. 2026;74:10658-10675. doi:10.1109/tcomm.2026.3700463

BibTeX
@article{Oh2026,
  author = {Oh, Seungeun and Kim, Jinhyuk and Park, Jihong and Ko, Seung-Woo and Choi, Jinho and Quek, Tony Q. S. and Kim, Seong-Lyun},
  title = {Communication-Efficient Hybrid Language Model via Uncertainty-Aware Opportunistic and Compressed Transmission},
  journal = {IEEE Transactions on Communications},
  year = {2026},
  volume = {74},
  pages = {10658--10675},
  doi = {10.1109/tcomm.2026.3700463}
}
Kostenlos starten →