Language-4D Cross-Boosting for Generalized Zero-Shot 6DoF Tracking and 3D Reconstruction
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2026-10 · 48 (10) · 13496-13514
DOI: 10.1109/tpami.2026.3711593 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026-10. Verfasst von Jingtao Sun und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 95 Literaturangaben. Gefördert von: National Research Foundation, Singapore.
Abstract (Originalsprache)
A more universal category-level pose tracking and reconstruction approach should be robust to both seen and unseen classes. However, the performance of existing category-level algorithms degrades for classes that are unseen in the training set. To learn a discriminative model that can achieve strong performance in Generalized Zero-Shot Learning (GZSL) settings, we introduce L4D-Track++, a unified framework for zero-shot category-level pose tracking and shape reconstruction that leverages multi-level language captions. To address the bias problem, namely the out-of-distribution gap between source (seen) and target (unseen) domains, we first propose a Multi-Level Language Conditional Feature Generation module to produce synthetic pairwise embeddings for unseen classes. Furthermore, to increase inter-class distance, decrease intra-class distance, and bridge the gap between seen and unseen category domains, we propose a language-guided multi-modal fusion strategy to construct the final fused inter-frame pairwise embeddings for downstream tracking and reconstruction tasks by taking both real and synthetic pairwise embeddings as inputs. This strategy consists of two main components: Category-Related Mutual Information-based Optimization and Modality-Related Conditional Mutual Information Maximization. Experimental results on five public datasets demonstrate that our method is effective and achieves significant improvements in GZSL performance.
Förderung
- National Research Foundation, Singapore
Zitieren
APA
Sun, J., Wang, Y., Zhao, J., Zhang, Y., Liu, M., & Shou, M. Z. (2026). Language-4D Cross-Boosting for Generalized Zero-Shot 6DoF Tracking and 3D Reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 48(10), 13496-13514. https://doi.org/10.1109/tpami.2026.3711593
Vancouver
Sun J, Wang Y, Zhao J, Zhang Y, Liu M, Shou MZ. Language-4D Cross-Boosting for Generalized Zero-Shot 6DoF Tracking and 3D Reconstruction. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2026;48(10):13496-13514. doi:10.1109/tpami.2026.3711593
BibTeX
@article{Sun2026,
author = {Sun, Jingtao and Wang, Yaonan and Zhao, Jiawen and Zhang, Yike and Liu, Min and Shou, Mike Zheng},
title = {Language-4D Cross-Boosting for Generalized Zero-Shot 6DoF Tracking and 3D Reconstruction},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2026},
volume = {48},
number = {10},
pages = {13496--13514},
doi = {10.1109/tpami.2026.3711593}
}
EUVIMED