EUVIMEDEUVIMED entdecken
Crossref

Fine-Grained Audio-Visual Aided Haptic Signal Reconstruction

Zhenyu Yang, Haoyu Wang, Yingying Shi, Lei Ye, Xin Wei

IEEE Signal Processing Letters · 2024 · 31 · 1349-1353

DOI: 10.1109/lsp.2024.3398366 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Signal Processing Letters, 2024. Verfasst von Zhenyu Yang und 4 weiteren Autor:innen. Die Arbeit wurde bisher 7-mal zitiert (Crossref) und nennt 14 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Open Foundation of Anhui Province Key Laboratory of Intelligent Building and Building Energy Saving, Jiangsu Provincial Qinglan.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi-modal services, integrating haptic modality into audio-visual applications, will further improve user's interactive and immersive experience. However, the quality of haptic signals in multi-modal services is very sensitive to transmission interference. Even in certain virtual scenarios, it is necessary to generate “virtual” haptic signals. Although several cross-modal reconstruction schemes are used to guarantee the quality of haptic signals, the weakly-supervised and weakly-paired issues still exist. To resolve this problem, a fine-grained audio-visual aided haptic signal reconstruction (FGHR) is proposed. Firstly, feature extraction with subcategory label exploration is designed by using the cross-modal transfer-based deep clustering network (CT-DCN). Then, the fusion features with shared semantic learning is performed under the triplet constraint. Finally, haptic signal reconstruction is realized by imposing the fine-grained semantic constraints. Experimental results show the effectiveness of the proposed scheme.

Förderung

  • National Natural Science Foundation of China
  • Open Foundation of Anhui Province Key Laboratory of Intelligent Building and Building Energy Saving
  • Jiangsu Provincial Qinglan
  • Special Fund for Urban and Rural Construction and Development in Jiangsu Province

Zitieren

APA

Yang, Z., Wang, H., Shi, Y., Ye, L., & Wei, X. (2024). Fine-Grained Audio-Visual Aided Haptic Signal Reconstruction. IEEE Signal Processing Letters, 31, 1349-1353. https://doi.org/10.1109/lsp.2024.3398366

Vancouver

Yang Z, Wang H, Shi Y, Ye L, Wei X. Fine-Grained Audio-Visual Aided Haptic Signal Reconstruction. IEEE Signal Processing Letters. 2024;31:1349-1353. doi:10.1109/lsp.2024.3398366

BibTeX
@article{Yang2024,
  author = {Yang, Zhenyu and Wang, Haoyu and Shi, Yingying and Ye, Lei and Wei, Xin},
  title = {Fine-Grained Audio-Visual Aided Haptic Signal Reconstruction},
  journal = {IEEE Signal Processing Letters},
  year = {2024},
  volume = {31},
  pages = {1349--1353},
  doi = {10.1109/lsp.2024.3398366}
}
Kostenlos starten →