EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Transformer-Based Visual Segmentation: A Survey

Xiangtai Li, Henghui Ding, Haobo Yuan, Wenwei Zhang, Jiangmiao Pang, Guangliang Cheng, Kai Chen, Ziwei Liu, Chen Change Loy

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 10138-10163

DOI: 10.1109/tpami.2024.3434373 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Xiangtai Li und 8 weiteren Autor:innen. Die Arbeit wurde bisher 221-mal zitiert (Crossref) und nennt 374 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Alan Turing Institute, Industry Alignment Fund Industry Collaboration Projects, Funding Initiative and Singapore MOE AcRF Tier 1.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Visual segmentation seeks to partition images, video frames, or point clouds into multiple segments or groups. This technique has numerous real-world applications, such as autonomous driving, image editing, robot sensing, and medical analysis. Over the past decade, deep learning-based methods have made remarkable strides in this area. Recently, transformers, a type of neural network based on self-attention originally designed for natural language processing, have considerably surpassed previous convolutional or recurrent approaches in various vision processing tasks. Specifically, vision transformers offer robust, unified, and even simpler solutions for various segmentation tasks. This survey provides a thorough overview of transformer-based visual segmentation, summarizing recent advancements. We first review the background, encompassing problem definitions, datasets, and prior convolutional methods. Next, we summarize a meta-architecture that unifies all recent transformer-based approaches. Based on this meta-architecture, we examine various method designs, including modifications to the meta-architecture and associated applications. We also present several specific subfields, including 3D point cloud segmentation, foundation model tuning, domain-aware segmentation, efficient segmentation, and medical segmentation. Additionally, we compile and re-evaluate the reviewed methods on several well-established datasets. Finally, we identify open challenges in this field and propose directions for future research.

Förderung

  • Alan Turing Institute
  • Industry Alignment Fund Industry Collaboration Projects
  • Funding Initiative and Singapore MOE AcRF Tier 1

Zitieren

APA

Li, X., Ding, H., Yuan, H., Zhang, W., Pang, J., Cheng, G., Chen, K., Liu, Z., & Loy, C. C. (2024). Transformer-Based Visual Segmentation: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 10138-10163. https://doi.org/10.1109/tpami.2024.3434373

Vancouver

Li X, Ding H, Yuan H, Zhang W, Pang J, Cheng G, et al. Transformer-Based Visual Segmentation: A Survey. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):10138-10163. doi:10.1109/tpami.2024.3434373

BibTeX
@article{Li2024,
  author = {Li, Xiangtai and Ding, Henghui and Yuan, Haobo and Zhang, Wenwei and Pang, Jiangmiao and Cheng, Guangliang and Chen, Kai and Liu, Ziwei and Loy, Chen Change},
  title = {Transformer-Based Visual Segmentation: A Survey},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {12},
  pages = {10138--10163},
  doi = {10.1109/tpami.2024.3434373}
}
Kostenlos starten →