FaNe: Towards Fine-Grained Cross-Modal Contrast with False-Negative Reduction and Text-Conditioned Sparse Attention
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (15) · 12681-12689
DOI: 10.1609/aaai.v40i15.38264 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Peng Zhang und 4 weiteren Autor:innen.
Abstract (Originalsprache)
Medical vision-language pre-training (VLP) offers significant potential for advancing medical image understanding by leveraging paired image-report data. However, existing methods are limited by False Negatives (FaNe) induced by semantically similar texts and insufficient fine-grained cross-modal alignment. To address these limitations, we propose FaNe, a semantic-enhanced VLP framework. To mitigate false negatives, we introduce a semantic-aware positive pair mining strategy based on text-text similarity with adaptive normalization. Furthermore, we design a text-conditioned sparse attention pooling module to enable fine-grained image-text alignment through localized visual representations guided by textual cues. To strengthen intra-modal discrimination, we develop a hard-negative aware contrastive loss that adaptively reweights semantically similar negatives. Extensive experiments on five downstream medical imaging benchmarks demonstrate that FaNe achieves state-of-the-art performance across image classification, object detection, and semantic segmentation, validating the effectiveness of our framework.
Zitieren
APA
Zhang, P., Lai, Z., Chen, W., Wu, X., & Kong, H. (2026). FaNe: Towards Fine-Grained Cross-Modal Contrast with False-Negative Reduction and Text-Conditioned Sparse Attention. Proceedings of the AAAI Conference on Artificial Intelligence, 40(15), 12681-12689. https://doi.org/10.1609/aaai.v40i15.38264
Vancouver
Zhang P, Lai Z, Chen W, Wu X, Kong H. FaNe: Towards Fine-Grained Cross-Modal Contrast with False-Negative Reduction and Text-Conditioned Sparse Attention. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(15):12681-12689. doi:10.1609/aaai.v40i15.38264
BibTeX
@article{Zhang2026,
author = {Zhang, Peng and Lai, Zhihui and Chen, Wenting and Wu, Xu and Kong, Heng},
title = {FaNe: Towards Fine-Grained Cross-Modal Contrast with False-Negative Reduction and Text-Conditioned Sparse Attention},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {15},
pages = {12681--12689},
doi = {10.1609/aaai.v40i15.38264}
}
EUVIMED