EUVIMEDEUVIMED entdecken
Crossref

Spatial-frequency attention-based optical and scene flow with cross-modal knowledge distillation

Youjie Zhou, Runyu Jiao, Zhonghan Tao, Xichang Liang, Yi Wan

The Visual Computer · 2025-04 · 41 (6) · 4183-4198

DOI: 10.1007/s00371-024-03654-2 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Visual Computer, online erschienen am 09.10.2024. Verfasst von Youjie Zhou und 4 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 44 Literaturangaben. Gefördert von: China government project, Key Technology R&D Program of Shandong.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper studies the problem of multimodal fusion for optical and scene flow from RGB and depth images, or point clouds. Previous methods fuse multimodal information in “early-fusion” or “late-fusion” strategies, in which an attention mechanism is employed to address the problem of optical and scene flow estimation when RGB information is unreliable. Such attentive approaches either suffer from substantial computational and time complexities or lose the inherent characteristics of features due to downsampling. To address this issue, we propose a novel multimodal fusion approach named SFRAFT, which utilizes Fourier transform to build the spatial-frequency domain transformed self-attention and cross-attention. With the novel attentive mechanism, our approach can extract informative features more efficiently and effectively. We further enhance information exchange between the two modalities by incorporating multi-scale knowledge distillation. Experimental results on Flythings3D and KITTI show that our SFRAFT achieves the best performance with low computational and time complexity. We also prove the strong ability of our approach for flow estimation on our real-world dataset. We release the code and datasets at https://doi.org/10.5281/zenodo.12697968.

Förderung

  • China government project
  • Key Technology R&D Program of Shandong

Zitieren

APA

Zhou, Y., Jiao, R., Tao, Z., Liang, X., & Wan, Y. (2025). Spatial-frequency attention-based optical and scene flow with cross-modal knowledge distillation. The Visual Computer, 41(6), 4183-4198. https://doi.org/10.1007/s00371-024-03654-2

Vancouver

Zhou Y, Jiao R, Tao Z, Liang X, Wan Y. Spatial-frequency attention-based optical and scene flow with cross-modal knowledge distillation. The Visual Computer. 2025;41(6):4183-4198. doi:10.1007/s00371-024-03654-2

BibTeX
@article{Zhou2025,
  author = {Zhou, Youjie and Jiao, Runyu and Tao, Zhonghan and Liang, Xichang and Wan, Yi},
  title = {Spatial-frequency attention-based optical and scene flow with cross-modal knowledge distillation},
  journal = {The Visual Computer},
  year = {2025},
  volume = {41},
  number = {6},
  pages = {4183--4198},
  doi = {10.1007/s00371-024-03654-2}
}
Kostenlos starten →