EUVIMEDEUVIMED entdecken
Crossref

Spatial Frequency Modulation for Semantic Segmentation

Linwei Chen, Ying Fu, Lin Gu, Dezhi Zheng, Jifeng Dai

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2025-11 · 47 (11) · 9767-9784

DOI: 10.1109/tpami.2025.3592621 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2025-11. Verfasst von Linwei Chen und 4 weiteren Autor:innen. Die Arbeit wurde bisher 43-mal zitiert (Crossref) und nennt 132 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Beijing Natural Science Foundation, Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

High spatial frequency information, including fine details like textures, significantly contributes to the accuracy of semantic segmentation. However, according to the Nyquist-Shannon Sampling Theorem, high-frequency components are vulnerable to aliasing or distortion when propagating through downsampling layers such as strided-convolution. Here, we propose a novel Spatial Frequency Modulation (SFM) that modulates high-frequency features to a lower frequency before downsampling and then demodulates them back during upsampling. Specifically, we implement modulation through adaptive resampling (ARS) and design a lightweight add-on that can densely sample the high-frequency areas to scale up the signal, thereby lowering its frequency in accordance with the Frequency Scaling Property. We also propose Multi-Scale Adaptive Upsampling (MSAU) to demodulate the modulated feature and recover high-frequency information through non-uniform upsampling This module further improves segmentation by explicitly exploiting information interaction between densely and sparsely resampled areas at multiple scales. Both modules can seamlessly integrate with various architectures, extending from convolutional neural networks to transformers. Feature visualization and analysis demonstrate that our method effectively alleviates aliasing while successfully retaining details after demodulation. As a result, the proposed approach considerably enhances existing state-of-the-art segmentation models (e.g., Mask2Former-Swin-T +1.5 mIoU, InternImage-T +1.4 mIoU on ADE20 K). Furthermore, ARS also enhances the performance of powerful Deformable Convolution (+0.8 mIoU on Cityscapes) by maintaining relative positional order during non-uniform sampling. Finally, we validate the broad applicability and effectiveness of SFM by extending it to image classification, adversarial robustness, instance segmentation, and panoptic segmentation tasks.

Förderung

  • National Natural Science Foundation of China
  • Beijing Natural Science Foundation
  • Fundamental Research Funds for the Central Universities
  • JST Moonshot R&D, Japan

Zitieren

APA

Chen, L., Fu, Y., Gu, L., Zheng, D., & Dai, J. (2025). Spatial Frequency Modulation for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(11), 9767-9784. https://doi.org/10.1109/tpami.2025.3592621

Vancouver

Chen L, Fu Y, Gu L, Zheng D, Dai J. Spatial Frequency Modulation for Semantic Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2025;47(11):9767-9784. doi:10.1109/tpami.2025.3592621

BibTeX
@article{Chen2025,
  author = {Chen, Linwei and Fu, Ying and Gu, Lin and Zheng, Dezhi and Dai, Jifeng},
  title = {Spatial Frequency Modulation for Semantic Segmentation},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2025},
  volume = {47},
  number = {11},
  pages = {9767--9784},
  doi = {10.1109/tpami.2025.3592621}
}
Kostenlos starten →