EUVIMEDEUVIMED entdecken
Crossref

TCFormer: Visual Recognition via Token Clustering Transformer

Wang Zeng, Sheng Jin, Lumin Xu, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo, Xiaogang Wang

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 9521-9535

DOI: 10.1109/tpami.2024.3425768 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Wang Zeng und 7 weiteren Autor:innen. Die Arbeit wurde bisher 21-mal zitiert (Crossref) und nennt 78 Literaturangaben. Gefördert von: Centre for Perceptual and Interactive Intelligence, Innovation and Technology Commission, Research Impact Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Transformers are widely used in computer vision areas and have achieved remarkable success. Most state-of-the-art approaches split images into regular grids and represent each grid region with a vision token. However, fixed token distribution disregards the semantic meaning of different image regions, resulting in sub-optimal performance. To address this issue, we propose the Token Clustering Transformer (TCFormer), which generates dynamic vision tokens based on semantic meaning. Our dynamic tokens possess two crucial characteristics: (1) Representing image regions with similar semantic meanings using the same vision token, even if those regions are not adjacent, and (2) concentrating on regions with valuable details and represent them using fine tokens. Through extensive experimentation across various applications, including image classification, human pose estimation, semantic segmentation, and object detection, we demonstrate the effectiveness of our TCFormer.

Förderung

  • Centre for Perceptual and Interactive Intelligence
  • Innovation and Technology Commission
  • Research Impact Fund
  • Hong Kong RGC. XiaogangWang is a PI of CPII
  • InnoHK

Zitieren

APA

Zeng, W., Jin, S., Xu, L., Liu, W., Qian, C., Ouyang, W., Luo, P., & Wang, X. (2024). TCFormer: Visual Recognition via Token Clustering Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 9521-9535. https://doi.org/10.1109/tpami.2024.3425768

Vancouver

Zeng W, Jin S, Xu L, Liu W, Qian C, Ouyang W, et al. TCFormer: Visual Recognition via Token Clustering Transformer. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):9521-9535. doi:10.1109/tpami.2024.3425768

BibTeX
@article{Zeng2024,
  author = {Zeng, Wang and Jin, Sheng and Xu, Lumin and Liu, Wentao and Qian, Chen and Ouyang, Wanli and Luo, Ping and Wang, Xiaogang},
  title = {TCFormer: Visual Recognition via Token Clustering Transformer},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {12},
  pages = {9521--9535},
  doi = {10.1109/tpami.2024.3425768}
}
Kostenlos starten →