Mask2Anomaly: Mask Transformer for Universal Open-Set Segmentation
IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-12 · 46 (12) · 9286-9302
DOI: 10.1109/tpami.2024.3419055 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-12. Verfasst von Shyam Nandan Rai und 3 weiteren Autor:innen. Die Arbeit wurde bisher 19-mal zitiert (Crossref) und nennt 68 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Union Next-GenerationEU, FAIR - Future Artificial Intelligence Research, European Lighthouse on Secure and Safe AI–ELSA, HORIZON EU.
Abstract (Originalsprache)
Segmenting unknown or anomalous object instances is a critical task in autonomous driving applications, and it is approached traditionally as a per-pixel classification problem. However, reasoning individually about each pixel without considering their contextual semantics results in high uncertainty around the objects' boundaries and numerous false positives. We propose a paradigm change by shifting from a per-pixel classification to a mask classification. Our mask-based method, Mask2Anomaly, demonstrates the feasibility of integrating a mask-classification architecture to jointly address anomaly segmentation, open-set semantic segmentation, and open-set panoptic segmentation. Mask2Anomaly includes several technical novelties that are designed to improve the detection of anomalies/unknown objects: i) a global masked attention module to focus individually on the foreground and background regions; ii) a mask contrastive learning that maximizes the margin between an anomaly and known classes; iii) a mask refinement solution to reduce false positives; and iv) a novel approach to mine unknown instances based on the mask- architecture properties. By comprehensive qualitative and qualitative evaluation, we show Mask2Anomaly achieves new state-of-the-art results across the benchmarks of anomaly segmentation, open-set semantic segmentation, and open-set panoptic segmentation.
Förderung
- Union Next-GenerationEU
- FAIR - Future Artificial Intelligence Research
- European Lighthouse on Secure and Safe AI–ELSA, HORIZON EU
Zitieren
APA
Rai, S. N., Cermelli, F., Caputo, B., & Masone, C. (2024). Mask2Anomaly: Mask Transformer for Universal Open-Set Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12), 9286-9302. https://doi.org/10.1109/tpami.2024.3419055
Vancouver
Rai SN, Cermelli F, Caputo B, Masone C. Mask2Anomaly: Mask Transformer for Universal Open-Set Segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(12):9286-9302. doi:10.1109/tpami.2024.3419055
BibTeX
@article{Rai2024,
author = {Rai, Shyam Nandan and Cermelli, Fabio and Caputo, Barbara and Masone, Carlo},
title = {Mask2Anomaly: Mask Transformer for Universal Open-Set Segmentation},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2024},
volume = {46},
number = {12},
pages = {9286--9302},
doi = {10.1109/tpami.2024.3419055}
}
EUVIMED