EUVIMEDEUVIMED entdecken
Crossref

OGP-Net: Optical Guidance Meets Pixel-Level Contrastive Distillation for Robust Multi-Modal and Missing Modality Segmentation

Aniruddh Sikdar, Jayant Teotia, Suresh Sundaram

Proceedings of the AAAI Conference on Artificial Intelligence · 11.04.2025 · 39 (7) · 6922-6930

DOI: 10.1609/aaai.v39i7.32743 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 11.04.2025. Verfasst von Aniruddh Sikdar und 2 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Enhancing the performance of semantic segmentation models with multi-spectral images (RGB-IR) is crucial, particularly for low-light and adverse environments. While multi-modal fusion techniques aim to learn cross-modality features for generating fused images or engage in knowledge distillation, they often treat multi-modal and missing modality scenarios as separate challenges, which is not an optimal approach. To address this, a novel multi-modal fusion approach called Optically-Guided Pixel-level contrastive learning Network (OGP-Net) is proposed, which uses Distillation with Multi-View Contrastive (DMC) and Distillation for Uni-modal Re- tention (DUR) to maintain the correlation between modality-shared and modality-specific features. DMC aligns the uni-modal features by projecting the semantic information across modalities into a unified latent space, ensuring that the feature maps retain multi-modal representations. Pixel-level multi-view contrastive learning is introduced to enable modality-invariant representation learning. To retain modality-specific information, DUR is proposed, which distills detailed textures from RGB images into the optical branch of OGP-Net. Additionally, the Gated Spectral Unit (GSU) is integrated into the framework to eliminate the need for manual tuning and avoid forced feature alignment. Comprehensive experiments show that OGP-Net outperforms state-of-the-art models in multi-modal and missing modality scenarios across three public benchmarking datasets. It achieves quicker convergence and learns efficiently from limited training samples.

Zitieren

APA

Sikdar, A., Teotia, J., & Sundaram, S. (2025). OGP-Net: Optical Guidance Meets Pixel-Level Contrastive Distillation for Robust Multi-Modal and Missing Modality Segmentation. Proceedings of the AAAI Conference on Artificial Intelligence, 39(7), 6922-6930. https://doi.org/10.1609/aaai.v39i7.32743

Vancouver

Sikdar A, Teotia J, Sundaram S. OGP-Net: Optical Guidance Meets Pixel-Level Contrastive Distillation for Robust Multi-Modal and Missing Modality Segmentation. Proceedings of the AAAI Conference on Artificial Intelligence. 2025;39(7):6922-6930. doi:10.1609/aaai.v39i7.32743

BibTeX
@article{Sikdar2025,
  author = {Sikdar, Aniruddh and Teotia, Jayant and Sundaram, Suresh},
  title = {OGP-Net: Optical Guidance Meets Pixel-Level Contrastive Distillation for Robust Multi-Modal and Missing Modality Segmentation},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2025},
  volume = {39},
  number = {7},
  pages = {6922--6930},
  doi = {10.1609/aaai.v39i7.32743}
}
Kostenlos starten →