Rate-Adaptive Coding Mechanism for Semantic Communications With Multi-Modal Data
IEEE Transactions on Communications · 2024-03 · 72 (3) · 1385-1400
DOI: 10.1109/tcomm.2023.3335977 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Communications, 2024-03. Verfasst von Yangshuo He und 2 weiteren Autor:innen. Die Arbeit wurde bisher 46-mal zitiert (Crossref) und nennt 39 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China.
Abstract (Originalsprache)
Recently, the ever-increasing demand for bandwidth in multi-modal communication systems requires a paradigm shift. Powered by deep learning, semantic communications are applied to multi-modal scenarios to boost communication efficiency and save communication resources. However, the existing end-to-end neural network (NN) based framework without the channel encoder/decoder is incompatible with modern digital communication systems. Moreover, most end-to-end designs are task-specific and require re- design and re- training for new tasks, which limits their applications. In this paper, we propose a distributed multi-modal semantic communication framework incorporating the conventional channel encoder/decoder. We adopt NN-based semantic encoder and decoder to extract correlated semantic information contained in different modalities, including speech, text, and image. Based on the proposed framework, we further establish a general rate-adaptive coding mechanism for various types of multi-modal semantic tasks. In particular, we utilize unequal error protection based on semantic importance, which is derived by evaluating the distortion bound of each modality. We further formulate and solve an optimization problem that aims at minimizing inference delay while maintaining inference accuracy for semantic tasks. Numerical results show that the proposed mechanism fares better than both conventional communication and existing semantic communication systems in terms of task performance, inference delay, and deployment complexity.
Förderung
- National Key Research and Development Program of China
- National Natural Science Foundation of China
Zitieren
APA
He, Y., Yu, G., & Cai, Y. (2024). Rate-Adaptive Coding Mechanism for Semantic Communications With Multi-Modal Data. IEEE Transactions on Communications, 72(3), 1385-1400. https://doi.org/10.1109/tcomm.2023.3335977
Vancouver
He Y, Yu G, Cai Y. Rate-Adaptive Coding Mechanism for Semantic Communications With Multi-Modal Data. IEEE Transactions on Communications. 2024;72(3):1385-1400. doi:10.1109/tcomm.2023.3335977
BibTeX
@article{He2024,
author = {He, Yangshuo and Yu, Guanding and Cai, Yunlong},
title = {Rate-Adaptive Coding Mechanism for Semantic Communications With Multi-Modal Data},
journal = {IEEE Transactions on Communications},
year = {2024},
volume = {72},
number = {3},
pages = {1385--1400},
doi = {10.1109/tcomm.2023.3335977}
}
EUVIMED