Synchronous Multi-Modal Semantic Communication System With Packet-Level Coding
IEEE Transactions on Wireless Communications · 2025-05 · 24 (5) · 3684-3697
DOI: 10.1109/twc.2025.3534995 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Wireless Communications, 2025-05. Verfasst von Yun Tian und 4 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
Although the semantic communication with joint semantic-channel coding design has shown promising performance in transmitting data of different modalities over physical layer channels, the synchronization and packet-level forward error correction (FEC) of multimodal semantics have not been well studied. Synchronizing multimodal features in both the semantic and time domains is challenging due to the independent design of semantic encoders. In this paper, we take the facial video and speech transmission as an example and propose a Synchronous Multi-modal Semantic Communication System with Packet-Level Coding (SyncSC). To achieve semantic and time synchronization, 3D Morphable Mode (3DMM) coefficients and text are transmitted as semantics. We propose a semantic codec that achieves similar reconstruction quality with lower bandwidth. The visual-guided speech synthesis is designed to synchronize video, text and speech. We propose a packet-Level FEC method for video semantics, called PacSC, that maintains visual quality even at high packet loss rates. For text packets, a text packet loss concealment module, called TextPC, based on Bidirectional Encoder Representations from Transformers (BERT) is proposed, which improves the performance of traditional FEC methods. Simulation results show that SyncSC reduces transmission overhead while ensuring high-quality synchronous transmission of video and speech over the packet loss network.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
Zitieren
APA
Tian, Y., Ying, J., Qin, Z., Jin, Y., & Tao, X. (2025). Synchronous Multi-Modal Semantic Communication System With Packet-Level Coding. IEEE Transactions on Wireless Communications, 24(5), 3684-3697. https://doi.org/10.1109/twc.2025.3534995
Vancouver
Tian Y, Ying J, Qin Z, Jin Y, Tao X. Synchronous Multi-Modal Semantic Communication System With Packet-Level Coding. IEEE Transactions on Wireless Communications. 2025;24(5):3684-3697. doi:10.1109/twc.2025.3534995
BibTeX
@article{Tian2025,
author = {Tian, Yun and Ying, Jingkai and Qin, Zhijin and Jin, Ye and Tao, Xiaoming},
title = {Synchronous Multi-Modal Semantic Communication System With Packet-Level Coding},
journal = {IEEE Transactions on Wireless Communications},
year = {2025},
volume = {24},
number = {5},
pages = {3684--3697},
doi = {10.1109/twc.2025.3534995}
}
EUVIMED