EUVIMEDEUVIMED entdecken
Crossref

Cross-Image Pixel Contrasting for Semantic Segmentation

Tianfei Zhou, Wenguan Wang

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-08 · 46 (8) · 5398-5412

DOI: 10.1109/tpami.2024.3367952 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-08. Verfasst von Tianfei Zhou und 1 weiteren Autor:innen. Die Arbeit wurde bisher 88-mal zitiert (Crossref) und nennt 137 Literaturangaben. Gefördert von: Fundamental Research Funds for the Central Universities, National Natural Science Foundation of China, CCF-Tencent Open Fund.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This work studies the problem of image semantic segmentation. Current approaches focus mainly on mining "local" context, i.e., dependencies between pixels within individual images, by specifically-designed, context aggregation modules (e.g., dilated convolution, neural attention) or structure-aware optimization objectives (e.g., IoU-like loss). However, they ignore "global" context of the training data, i.e., rich semantic relations between pixels across different images. Inspired by recent advance in unsupervised contrastive representation learning, we propose a pixel-wise contrastive algorithm, dubbed as PiCo, for semantic segmentation in the fully supervised learning setting. The core idea is to enforce pixel embeddings belonging to a same semantic class to be more similar than embeddings from different classes. It raises a pixel-wise metric learning paradigm for semantic segmentation, by explicitly exploring the structures of labeled pixels, which were rarely studied before. Our training algorithm is compatible with modern segmentation solutions without extra overhead during testing. We experimentally show that, with famous segmentation models (i.e., DeepLabV3, HRNet, OCRNet, SegFormer, Segmenter, MaskFormer) and backbones (i.e., MobileNet, ResNet, HRNet, MiT, ViT), our algorithm brings consistent performance improvements across diverse datasets (i.e., Cityscapes, ADE20 K, PASCAL-Context, COCO-Stuff, CamVid). We expect that this work will encourage our community to rethink the current de facto training paradigm in semantic segmentation.

Förderung

  • Fundamental Research Funds for the Central Universities
  • National Natural Science Foundation of China
  • CCF-Tencent Open Fund

Zitieren

APA

Zhou, T., & Wang, W. (2024). Cross-Image Pixel Contrasting for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8), 5398-5412. https://doi.org/10.1109/tpami.2024.3367952

Vancouver

Zhou T, Wang W. Cross-Image Pixel Contrasting for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(8):5398-5412. doi:10.1109/tpami.2024.3367952

BibTeX
@article{Zhou2024,
  author = {Zhou, Tianfei and Wang, Wenguan},
  title = {Cross-Image Pixel Contrasting for Semantic Segmentation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {8},
  pages = {5398--5412},
  doi = {10.1109/tpami.2024.3367952}
}
Kostenlos starten →