EUVIMEDEUVIMED entdecken
Crossref

AToM: Adaptive Token Merging for Efficient Acceleration of Vision Transformer

Jaekang Shin, Myeonggu Kang, Yunki Han, Junyoung Park, Lee-Sup Kim

IEEE Transactions on Computers · 2025-05 · 74 (5) · 1620-1633

DOI: 10.1109/tc.2025.3540638 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Computers, 2025-05. Verfasst von Jaekang Shin und 4 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 43 Literaturangaben. Gefördert von: MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program, IITP.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recently, Vision Transformers (ViTs) have set a new standard in computer vision (CV), showing unparalleled image processing performance. However, their substantial computational requirements hinder practical deployment, especially on resource-limited devices common in CV applications. Token merging has emerged as a solution, condensing tokens with similar features to cut computational and memory demands. Yet, existing applications on ViTs often miss the mark in token compression, with rigid merging strategies and a lack of in-depth analysis of ViT merging characteristics. To overcome these issues, this paper introduces Adaptive Token Merging (AToM), a comprehensive algorithm-architecture co-design for accelerating ViTs. The AToM algorithm employs an image-adaptive, fine-grained merging strategy, significantly boosting computational efficiency. We also optimize the merging and unmerging processes to minimize overhead, employing techniques like First-Come-First-Merge mapping and Linear Distance Calculation. On the hardware side, the AToM architecture is tailor-made to exploit the AToM algorithm's benefits, with specialized engines for efficient merge and unmerge operations. Our pipeline architecture ensures end-to-end ViT processing, minimizing latency and memory overhead from the AToM algorithm. Across various hardware platforms including CPU, EdgeGPU, and GPU, AToM achieves average end-to-end speedups of 10.9 $\boldsymbol{\times}$ × , 7.7 $\boldsymbol{\times}$ × , and 5.4 $\boldsymbol{\times}$ × , alongside energy savings of 24.9 $\boldsymbol{\times}$ × , 1.8 $\boldsymbol{\times}$ × , and 16.7 $\boldsymbol{\times}$ × . Moreover, AToM offers 1.2 $\boldsymbol{\times}$ × 1.9 $\boldsymbol{\times}$ × higher effective throughput compared to existing transformer accelerators.

Förderung

  • MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program
  • IITP

Zitieren

APA

Shin, J., Kang, M., Han, Y., Park, J., & Kim, L. (2025). AToM: Adaptive Token Merging for Efficient Acceleration of Vision Transformer. IEEE Transactions on Computers, 74(5), 1620-1633. https://doi.org/10.1109/tc.2025.3540638

Vancouver

Shin J, Kang M, Han Y, Park J, Kim L. AToM: Adaptive Token Merging for Efficient Acceleration of Vision Transformer. IEEE Transactions on Computers. 2025;74(5):1620-1633. doi:10.1109/tc.2025.3540638

BibTeX
@article{Shin2025,
  author = {Shin, Jaekang and Kang, Myeonggu and Han, Yunki and Park, Junyoung and Kim, Lee-Sup},
  title = {AToM: Adaptive Token Merging for Efficient Acceleration of Vision Transformer},
  journal = {IEEE Transactions on Computers},
  year = {2025},
  volume = {74},
  number = {5},
  pages = {1620--1633},
  doi = {10.1109/tc.2025.3540638}
}
Kostenlos starten →