HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-09 · 46 (9) · 6431-6442
DOI: 10.1109/tpami.2024.3379457 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-09. Verfasst von Ting Yao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 54-mal zitiert (Crossref) und nennt 83 Literaturangaben.
Abstract (Originalsprache)
The hybrid deep models of Vision Transformer (ViT) and Convolution Neural Network (CNN) have emerged as a powerful class of backbones for vision tasks. Scaling up the input resolution of such hybrid backbones naturally strengthes model capacity, but inevitably suffers from heavy computational cost that scales quadratically. Instead, we present a new hybrid backbone with HIgh-Resolution Inputs (namely HIRI-ViT), that upgrades prevalent four-stage ViT to five-stage ViT tailored for high-resolution inputs. HIRI-ViT is built upon the seminal idea of decomposing the typical CNN operations into two parallel CNN branches in a cost-efficient manner. One high-resolution branch directly takes primary high-resolution features as inputs, but uses less convolution operations. The other low-resolution branch first performs down-sampling and then utilizes more convolution operations over such low-resolution features. Experiments on both recognition task (ImageNet-1K dataset) and dense prediction tasks (COCO and ADE20 K datasets) demonstrate the superiority of HIRI-ViT. More remarkably, under comparable computational cost ( ∼ 5.0 GFLOPs), HIRI-ViT achieves to-date the best published Top-1 accuracy of 84.3% on ImageNet with 448×448 inputs, which absolutely improves 83.4% of iFormer-S by 0.9% with 224×224 inputs.
Zitieren
APA
Yao, T., Li, Y., Pan, Y., & Mei, T. (2024). HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(9), 6431-6442. https://doi.org/10.1109/tpami.2024.3379457
Vancouver
Yao T, Li Y, Pan Y, Mei T. HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(9):6431-6442. doi:10.1109/tpami.2024.3379457
BibTeX
@article{Yao2024,
author = {Yao, Ting and Li, Yehao and Pan, Yingwei and Mei, Tao},
title = {HIRI-ViT: Scaling Vision Transformer With High Resolution Inputs},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {9},
pages = {6431--6442},
doi = {10.1109/tpami.2024.3379457}
}
EUVIMED