Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach
IEEE Transactions on Communications · 2026 · 74 · 11157-11173
DOI: 10.1109/tcomm.2026.3707756 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Communications, 2026. Verfasst von Tantan Zhao und 4 weiteren Autor:innen. Die Arbeit nennt 42 Literaturangaben. Gefördert von: National Natural Science Foundation of China, China Postdoctoral Science Foundation, Sichuan Science and Technology Program.
Abstract (Originalsprache)
Multimodal semantic communication is a promising paradigm for enabling immersive intelligent services in sixth-generation (6G) networks, yet existing studies lack flexible support for arbitrary modality combinations and lightweight semantic knowledge modeling suitable for edge deployment. In this paper, we investigate a lightweight large AI model (LAM)-empowered composable multimodal semantic communication (CMSC) framework. Text is adopted as a semantic bridge to unify heterogeneous modalities, while a flexible and learnable modality composition weighting mechanism enables arbitrary combinations of image, audio, and video inputs to be aggregated into a communication-oriented semantic representation. To enhance semantic robustness, a bottleneck-aware lightweight semantic knowledge base is constructed by leveraging a frozen large language model with visual prompts, where raw images or middle video frames are jointly used with textual semantics to mitigate semantic ambiguity and compensate for semantic degradation caused by wireless channel impairments. The enhanced semantics are encoded and transmitted over wireless channels, and multimodal signals are reconstructed at the receiver via diffusion-based generation. Extensive experiments on real-world datasets show that CMSC achieves a higher compression rate while maintaining comparable transmission accuracy under both arbitrary and typical multimodal combinations, demonstrating its effectiveness for flexible and lightweight multimodal semantic communication in 6G ubiquitous intelligence.
Förderung
- National Natural Science Foundation of China
- China Postdoctoral Science Foundation
- Sichuan Science and Technology Program
- XJTU Research Fund for AI Science
Zitieren
APA
Zhao, T., Li, F., Huang, X., Liu, Y., & Nallanathan, A. (2026). Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach. IEEE Transactions on Communications, 74, 11157-11173. https://doi.org/10.1109/tcomm.2026.3707756
Vancouver
Zhao T, Li F, Huang X, Liu Y, Nallanathan A. Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach. IEEE Transactions on Communications. 2026;74:11157-11173. doi:10.1109/tcomm.2026.3707756
BibTeX
@article{Zhao2026,
author = {Zhao, Tantan and Li, Fan and Huang, Xinyu and Liu, Yiqun and Nallanathan, Arumugam},
title = {Composable Multimodal Semantic Communication: A Lightweight Large AI Model Approach},
journal = {IEEE Transactions on Communications},
year = {2026},
volume = {74},
pages = {11157--11173},
doi = {10.1109/tcomm.2026.3707756}
}
EUVIMED