EUVIMEDEUVIMED entdecken
Crossref

SIMAC: A Semantic-Driven Integrated Multimodal Sensing and Communication Framework

Yubo Peng, Luping Xiang, Kun Yang, Feibo Jiang, Kezhi Wang, Dapeng Oliver Wu

IEEE Journal on Selected Areas in Communications · 2026 · 44 · 673-688

DOI: 10.1109/jsac.2025.3610398 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Journal on Selected Areas in Communications, 2026. Verfasst von Yubo Peng und 5 weiteren Autor:innen. Die Arbeit wurde bisher 74-mal zitiert (Crossref) und nennt 37 Literaturangaben. Gefördert von: Jiangsu Major Project on Fundamental Research, Gusu Innovation Project, High-Tech District of Suzhou City.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Traditional unimodal sensing faces limitations in accuracy and capability, and its decoupled implementation with communication systems increases latency in bandwidth-constrained environments. Additionally, single-task-oriented sensing systems fail to address users’ diverse demands. To overcome these challenges, we propose a semantic-driven integrated multimodal sensing and communication (SIMAC) framework. This framework leverages a joint source-channel coding architecture to achieve simultaneous sensing, decoding, and transmission of sensing results. Specifically, SIMAC first introduces a multimodal semantic fusion (MSF) network, which employs two extractors to extract semantic information from radar signals and images, respectively. MSF then applies cross-attention mechanisms to fuse these unimodal features and generate multimodal semantic representations. Secondly, we present a large language model (LLM)-based semantic encoder (LSE), where relevant communication parameters and multimodal semantics are mapped into a unified latent space and input to the LLM, enabling channel-adaptive semantic encoding. Thirdly, a task-oriented sensing semantic decoder (SSD) is proposed, in which different decoded heads are designed according to the specific needs of tasks. Simultaneously, a multi-task learning strategy is introduced to train the SIMAC framework, achieving diverse sensing services. Finally, experimental simulations demonstrate that the proposed framework achieves diverse and higher-accuracy sensing services.

Förderung

  • Jiangsu Major Project on Fundamental Research
  • Gusu Innovation Project
  • High-Tech District of Suzhou City
  • National Natural Science Foundation of China
  • Major Program Project of Xiangjiang Laboratory
  • Qiyuan Laboratory Innovation Fund

Zitieren

APA

Peng, Y., Xiang, L., Yang, K., Jiang, F., Wang, K., & Wu, D. O. (2026). SIMAC: A Semantic-Driven Integrated Multimodal Sensing and Communication Framework. IEEE Journal on Selected Areas in Communications, 44, 673-688. https://doi.org/10.1109/jsac.2025.3610398

Vancouver

Peng Y, Xiang L, Yang K, Jiang F, Wang K, Wu DO. SIMAC: A Semantic-Driven Integrated Multimodal Sensing and Communication Framework. IEEE Journal on Selected Areas in Communications. 2026;44:673-688. doi:10.1109/jsac.2025.3610398

BibTeX
@article{Peng2026,
  author = {Peng, Yubo and Xiang, Luping and Yang, Kun and Jiang, Feibo and Wang, Kezhi and Wu, Dapeng Oliver},
  title = {SIMAC: A Semantic-Driven Integrated Multimodal Sensing and Communication Framework},
  journal = {IEEE Journal on Selected Areas in Communications},
  year = {2026},
  volume = {44},
  pages = {673--688},
  doi = {10.1109/jsac.2025.3610398}
}
Kostenlos starten →