EUVIMEDEUVIMED entdecken
Crossref

MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos

Junyi Ma, Xieyuanli Chen, Wentao Bao, Jingyi Xu, Hesheng Wang

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-03 · 48 (3) · 3250-3267

DOI: 10.1109/tpami.2025.3633453 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-03. Verfasst von Junyi Ma und 4 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 104 Literaturangaben. Gefördert von: National Key R&D Program of China, Shanghai Municipal Science and Technology Major Project, National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Understanding human intentions and actions through egocentric videos is important on the path to embodied artificial intelligence. As a branch of egocentric vision techniques, hand trajectory prediction plays a vital role in comprehending human motion patterns, benefiting downstream tasks in extended reality and robot manipulation. However, capturing high-level human intentions consistent with reasonable temporal causality is challenging when only egocentric videos are available. This difficulty is exacerbated under camera egomotion interference and the absence of affordance labels to explicitly guide the optimization of hand waypoint distribution. In this work, we propose a novel hand trajectory prediction method dubbed MADiff, which forecasts future hand waypoints with diffusion models. The devised denoising operation in the latent space is achieved by our proposed motion-aware Mamba, where the camera wearer's egomotion is integrated to achieve motion-driven selective scan (MDSS). To discern the relationship between hands and scenarios without explicit affordance supervision, we leverage a foundation model that fuses visual and language features to capture high-level semantics from video clips. Comprehensive experiments conducted on five public datasets with the existing and our new evaluation metrics demonstrate that MADiff predicts comparably reasonable hand trajectories compared to the state-of-the-art baselines.

Förderung

  • National Key R&D Program of China
  • Shanghai Municipal Science and Technology Major Project
  • National Natural Science Foundation of China

Zitieren

APA

Ma, J., Chen, X., Bao, W., Xu, J., & Wang, H. (2026). MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(3), 3250-3267. https://doi.org/10.1109/tpami.2025.3633453

Vancouver

Ma J, Chen X, Bao W, Xu J, Wang H. MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(3):3250-3267. doi:10.1109/tpami.2025.3633453

BibTeX
@article{Ma2026,
  author = {Ma, Junyi and Chen, Xieyuanli and Bao, Wentao and Xu, Jingyi and Wang, Hesheng},
  title = {MADiff: Motion-Aware Mamba Diffusion Models for Hand Trajectory Prediction on Egocentric Videos},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2026},
  volume = {48},
  number = {3},
  pages = {3250--3267},
  doi = {10.1109/tpami.2025.3633453}
}
Kostenlos starten →