From Easy to Hard: Learning Curricular Shape-Aware Features for Robust Panoptic Scene Graph Generation
International Journal of Computer Vision · 2025-01 · 133 (1) · 489-508
DOI: 10.1007/s11263-024-02190-9 ↗Kurz gefasst
Fachartikel in der Zeitschrift International Journal of Computer Vision, online erschienen am 05.08.2024. Verfasst von Hanrong Shi und 4 weiteren Autor:innen. Die Arbeit wurde bisher 6-mal zitiert (Crossref) und nennt 76 Literaturangaben. Gefördert von: National Key Research & Development Project of China, National Natural Science Foundation of China, Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
Panoptic Scene Graph Generation (PSG) aims to generate a comprehensive graph-structure representation based on panoptic segmentation masks. Despite remarkable progress in PSG, almost all existing methods neglect the importance of shape-aware features, which inherently focus on the contours and boundaries of objects. To bridge this gap, we propose a model-agnostic Curricular shApe-aware FEature (CAFE) learning strategy for PSG. Specifically, we incorporate shape-aware features (i.e., mask features and boundary features) into PSG, moving beyond reliance solely on bbox features. Furthermore, drawing inspiration from human cognition, we propose to integrate shape-aware features in an easy-to-hard manner. To achieve this, we categorize the predicates into three groups based on cognition learning difficulty and correspondingly divide the training process into three stages. Each stage utilizes a specialized relation classifier to distinguish specific groups of predicates. As the learning difficulty of predicates increases, these classifiers are equipped with features of ascending complexity. We also incorporate knowledge distillation to retain knowledge acquired in earlier stages. Due to its model-agnostic nature, CAFE can be seamlessly incorporated into any PSG model. Extensive experiments and ablations on two PSG tasks under both robust and zero-shot PSG have attested to the superiority and robustness of our proposed CAFE, which outperforms existing state-of-the-art methods by a large margin.
Förderung
- National Key Research & Development Project of China
- National Natural Science Foundation of China
- Fundamental Research Funds for the Central Universities
- HKUST Special Support for Young Faculty
- HKUST Sports Science and Technology Research Grant
Zitieren
APA
Shi, H., Li, L., Xiao, J., Zhuang, Y., & Chen, L. (2025). From Easy to Hard: Learning Curricular Shape-Aware Features for Robust Panoptic Scene Graph Generation. International Journal of Computer Vision, 133(1), 489-508. https://doi.org/10.1007/s11263-024-02190-9
Vancouver
Shi H, Li L, Xiao J, Zhuang Y, Chen L. From Easy to Hard: Learning Curricular Shape-Aware Features for Robust Panoptic Scene Graph Generation. International Journal of Computer Vision. 2025;133(1):489-508. doi:10.1007/s11263-024-02190-9
BibTeX
@article{Shi2025,
author = {Shi, Hanrong and Li, Lin and Xiao, Jun and Zhuang, Yueting and Chen, Long},
title = {From Easy to Hard: Learning Curricular Shape-Aware Features for Robust Panoptic Scene Graph Generation},
journal = {International Journal of Computer Vision},
year = {2025},
volume = {133},
number = {1},
pages = {489--508},
doi = {10.1007/s11263-024-02190-9}
}
EUVIMED