Fine-Grained Audio-Visual Aided Haptic Signal Reconstruction
IEEE Signal Processing Letters · 2024 · 31 · 1349-1353
DOI: 10.1109/lsp.2024.3398366 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Signal Processing Letters, 2024. Verfasst von Zhenyu Yang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 14 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Open Foundation of Anhui Province Key Laboratory of Intelligent Building and Building Energy Saving, Jiangsu Provincial Qinglan.
Abstract (Originalsprache)
Multi-modal services, integrating haptic modality into audio-visual applications, will further improve user's interactive and immersive experience. However, the quality of haptic signals in multi-modal services is very sensitive to transmission interference. Even in certain virtual scenarios, it is necessary to generate “virtual” haptic signals. Although several cross-modal reconstruction schemes are used to guarantee the quality of haptic signals, the weakly-supervised and weakly-paired issues still exist. To resolve this problem, a fine-grained audio-visual aided haptic signal reconstruction (FGHR) is proposed. Firstly, feature extraction with subcategory label exploration is designed by using the cross-modal transfer-based deep clustering network (CT-DCN). Then, the fusion features with shared semantic learning is performed under the triplet constraint. Finally, haptic signal reconstruction is realized by imposing the fine-grained semantic constraints. Experimental results show the effectiveness of the proposed scheme.
Förderung
- National Natural Science Foundation of China
- Open Foundation of Anhui Province Key Laboratory of Intelligent Building and Building Energy Saving
- Jiangsu Provincial Qinglan
- Special Fund for Urban and Rural Construction and Development in Jiangsu Province
Zitieren
APA
Yang, Z., Wang, H., Shi, Y., Ye, L., & Wei, X. (2024). Fine-Grained Audio-Visual Aided Haptic Signal Reconstruction. IEEE Signal Processing Letters, 31, 1349-1353. https://doi.org/10.1109/lsp.2024.3398366
Vancouver
Yang Z, Wang H, Shi Y, Ye L, Wei X. Fine-Grained Audio-Visual Aided Haptic Signal Reconstruction. IEEE Signal Processing Letters. 2024;31:1349-1353. doi:10.1109/lsp.2024.3398366
BibTeX
@article{Yang2024,
author = {Yang, Zhenyu and Wang, Haoyu and Shi, Yingying and Ye, Lei and Wei, Xin},
title = {Fine-Grained Audio-Visual Aided Haptic Signal Reconstruction},
journal = {IEEE Signal Processing Letters},
year = {2024},
volume = {31},
pages = {1349--1353},
doi = {10.1109/lsp.2024.3398366}
}
EUVIMED