TCFormer: Visual Recognition via Token Clustering Transformer
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 9521-9535
DOI: 10.1109/tpami.2024.3425768 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Wang Zeng und 7 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 78 Literaturangaben. Gefördert von: Centre for Perceptual and Interactive Intelligence, Innovation and Technology Commission, Research Impact Fund.
Abstract (Originalsprache)
Transformers are widely used in computer vision areas and have achieved remarkable success. Most state-of-the-art approaches split images into regular grids and represent each grid region with a vision token. However, fixed token distribution disregards the semantic meaning of different image regions, resulting in sub-optimal performance. To address this issue, we propose the Token Clustering Transformer (TCFormer), which generates dynamic vision tokens based on semantic meaning. Our dynamic tokens possess two crucial characteristics: (1) Representing image regions with similar semantic meanings using the same vision token, even if those regions are not adjacent, and (2) concentrating on regions with valuable details and represent them using fine tokens. Through extensive experimentation across various applications, including image classification, human pose estimation, semantic segmentation, and object detection, we demonstrate the effectiveness of our TCFormer.
Förderung
- Centre for Perceptual and Interactive Intelligence
- Innovation and Technology Commission
- Research Impact Fund
- Hong Kong RGC. XiaogangWang is a PI of CPII
- InnoHK
Zitieren
APA
Zeng, W., Jin, S., Xu, L., Liu, W., Qian, C., Ouyang, W., Luo, P., & Wang, X. (2024). TCFormer: Visual Recognition via Token Clustering Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 9521-9535. https://doi.org/10.1109/tpami.2024.3425768
Vancouver
Zeng W, Jin S, Xu L, Liu W, Qian C, Ouyang W, et al. TCFormer: Visual Recognition via Token Clustering Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):9521-9535. doi:10.1109/tpami.2024.3425768
BibTeX
@article{Zeng2024,
author = {Zeng, Wang and Jin, Sheng and Xu, Lumin and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping and Wang, Xiaogang},
title = {TCFormer: Visual Recognition via Token Clustering Transformer},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {12},
pages = {9521--9535},
doi = {10.1109/tpami.2024.3425768}
}
EUVIMED