Investigating Synthetic-to-Real Transfer Robustness for Stereo Matching and Optical Flow Estimation
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-10 · 47 (10) · 9113-9129
DOI: 10.1109/tpami.2025.3584847 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-10. Verfasst von Jiawei Zhang und 7 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 96 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities, China, China Postdoctoral Science Foundation.
Abstract (Originalsprache)
With advancements in robust stereo matching and optical flow estimation networks, models pre-trained on synthetic data demonstrate strong robustness to unseen domains. However, their robustness can be seriously degraded when fine-tuning them in real-world scenarios. This paper investigates fine-tuning stereo matching and optical flow estimation networks without compromising their robustness to unseen domains. Specifically, we divide the pixels into consistent and inconsistent regions by comparing Ground Truth (GT) with Pseudo Label (PL) and demonstrate that the imbalance learning of consistent and inconsistent regions in GT causes robustness degradation. Based on our analysis, we propose the DKT framework, which utilizes PL to balance the learning of different regions in GT. The core idea is to utilize an exponential moving average (EMA) teacher to measure what the student network has learned and dynamically adjust the learning regions. We further propose the DKT++ framework, which improves target-domain performances and network robustness by applying slow-fast update teachers to generate more accurate PL, introducing the unlabeled data and synthetic data. We integrate our frameworks with state-of-the-art networks and evaluate their effectiveness on several real-world datasets. Extensive experiments show that our method effectively preserves the robustness of stereo matching and optical flow networks during fine-tuning.
Förderung
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities, China
- China Postdoctoral Science Foundation
- Natural Science Foundation of Sichuan Province
- Central Universities Basic Research Operating Costs Project
- JST Moonshot R&D
Zitieren
APA
Zhang, J., Li, J., Huang, L., Luo, H., Yu, X., Gu, L., Zheng, J., & Bai, X. (2025). Investigating Synthetic-to-Real Transfer Robustness for Stereo Matching and Optical Flow Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(10), 9113-9129. https://doi.org/10.1109/tpami.2025.3584847
Vancouver
Zhang J, Li J, Huang L, Luo H, Yu X, Gu L, et al. Investigating Synthetic-to-Real Transfer Robustness for Stereo Matching and Optical Flow Estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(10):9113-9129. doi:10.1109/tpami.2025.3584847
BibTeX
@article{Zhang2025,
author = {Zhang, Jiawei and Li, Jiahe and Huang, Lei and Luo, Haonan and Yu, Xiaohan and Gu, Lin and Zheng, Jin and Bai, Xiao},
title = {Investigating Synthetic-to-Real Transfer Robustness for Stereo Matching and Optical Flow Estimation},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2025},
volume = {47},
number = {10},
pages = {9113--9129},
doi = {10.1109/tpami.2025.3584847}
}
EUVIMED