EUVIMEDEUVIMED entdecken
Crossref

Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach

Tantan Zhao, Fan Li, Xinyu Huang, Yiqun Liu, Arumugam Nallanathan

IEEE Transactions on Communications · 2026 · 74 · 11157-11173

DOI: 10.1109/tcomm.2026.3707756 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Communications, 2026. Verfasst von Tantan Zhao und 4 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Gefördert von: National Natural Science Foundation of China, China Postdoctoral Science Foundation, Sichuan Science and Technology Program.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multimodal semantic communication is a promising paradigm for enabling immersive intelligent services in sixth-generation (6G) networks, yet existing studies lack flexible support for arbitrary modality combinations and lightweight semantic knowledge modeling suitable for edge deployment. In this paper, we investigate a lightweight large AI model (LAM)-empowered composable multimodal semantic communication (CMSC) framework. Text is adopted as a semantic bridge to unify heterogeneous modalities, while a flexible and learnable modality composition weighting mechanism enables arbitrary combinations of image, audio, and video inputs to be aggregated into a communication-oriented semantic representation. To enhance semantic robustness, a bottleneck-aware lightweight semantic knowledge base is constructed by leveraging a frozen large language model with visual prompts, where raw images or middle video frames are jointly used with textual semantics to mitigate semantic ambiguity and compensate for semantic degradation caused by wireless channel impairments. The enhanced semantics are encoded and transmitted over wireless channels, and multimodal signals are reconstructed at the receiver via diffusion-based generation. Extensive experiments on real-world datasets show that CMSC achieves a higher compression rate while maintaining comparable transmission accuracy under both arbitrary and typical multimodal combinations, demonstrating its effectiveness for flexible and lightweight multimodal semantic communication in 6G ubiquitous intelligence.

Förderung

  • National Natural Science Foundation of China
  • China Postdoctoral Science Foundation
  • Sichuan Science and Technology Program
  • XJTU Research Fund for AI Science

Zitieren

APA

Zhao, T., Li, F., Huang, X., Liu, Y., & Nallanathan, A. (2026). Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach. IEEE Transactions on Communications, 74, 11157-11173. https://doi.org/10.1109/tcomm.2026.3707756

Vancouver

Zhao T, Li F, Huang X, Liu Y, Nallanathan A. Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach. IEEE Transactions on Communications. 2026;74:11157-11173. doi:10.1109/tcomm.2026.3707756

BibTeX
@article{Zhao2026,
  author = {Zhao, Tantan and Li, Fan and Huang, Xinyu and Liu, Yiqun and Nallanathan, Arumugam},
  title = {Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach},
  journal = {IEEE Transactions on Communications},
  year = {2026},
  volume = {74},
  pages = {11157--11173},
  doi = {10.1109/tcomm.2026.3707756}
}
Kostenlos starten →