SIMAC: A Semantic-Driven Integrated Multimodal Sensing and Communication Framework
IEEE Journal on Selected Areas in Communications · 2026 · 44 · 673-688
DOI: 10.1109/jsac.2025.3610398 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Journal on Selected Areas in Communications, 2026. Verfasst von Yubo Peng und 5 weiteren Autor:innen. Die Arbeit wurde bisher 74-mal zitiert (Crossref) und nennt 37 Literaturangaben. Gefördert von: Jiangsu Major Project on Fundamental Research, Gusu Innovation Project, High-Tech District of Suzhou City.
Abstract (Originalsprache)
Traditional unimodal sensing faces limitations in accuracy and capability, and its decoupled implementation with communication systems increases latency in bandwidth-constrained environments. Additionally, single-task-oriented sensing systems fail to address users’ diverse demands. To overcome these challenges, we propose a semantic-driven integrated multimodal sensing and communication (SIMAC) framework. This framework leverages a joint source-channel coding architecture to achieve simultaneous sensing, decoding, and transmission of sensing results. Specifically, SIMAC first introduces a multimodal semantic fusion (MSF) network, which employs two extractors to extract semantic information from radar signals and images, respectively. MSF then applies cross-attention mechanisms to fuse these unimodal features and generate multimodal semantic representations. Secondly, we present a large language model (LLM)-based semantic encoder (LSE), where relevant communication parameters and multimodal semantics are mapped into a unified latent space and input to the LLM, enabling channel-adaptive semantic encoding. Thirdly, a task-oriented sensing semantic decoder (SSD) is proposed, in which different decoded heads are designed according to the specific needs of tasks. Simultaneously, a multi-task learning strategy is introduced to train the SIMAC framework, achieving diverse sensing services. Finally, experimental simulations demonstrate that the proposed framework achieves diverse and higher-accuracy sensing services.
Förderung
- Jiangsu Major Project on Fundamental Research
- Gusu Innovation Project
- High-Tech District of Suzhou City
- National Natural Science Foundation of China
- Major Program Project of Xiangjiang Laboratory
- Qiyuan Laboratory Innovation Fund
Zitieren
APA
Peng, Y., Xiang, L., Yang, K., Jiang, F., Wang, K., & Wu, D. O. (2026). SIMAC: A Semantic-Driven Integrated Multimodal Sensing and Communication Framework. IEEE Journal on Selected Areas in Communications, 44, 673-688. https://doi.org/10.1109/jsac.2025.3610398
Vancouver
Peng Y, Xiang L, Yang K, Jiang F, Wang K, Wu DO. SIMAC: A Semantic-Driven Integrated Multimodal Sensing and Communication Framework. IEEE Journal on Selected Areas in Communications. 2026;44:673-688. doi:10.1109/jsac.2025.3610398
BibTeX
@article{Peng2026,
author = {Peng, Yubo and Xiang, Luping and Yang, Kun and Jiang, Feibo and Wang, Kezhi and Wu, Dapeng Oliver},
title = {SIMAC: A Semantic-Driven Integrated Multimodal Sensing and Communication Framework},
journal = {IEEE Journal on Selected Areas in Communications},
year = {2026},
volume = {44},
pages = {673--688},
doi = {10.1109/jsac.2025.3610398}
}
EUVIMED