Self-Supervised Learning for Open-Set Scene Graph Generation by Unknown-Object and Relationship Pseudo-Labels
IEEE Access · 2026 · 14 · 70880-70892
DOI: 10.1109/access.2026.3690725 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2026. Verfasst von Motoharu Sonogashira und 2 weiteren Autor:innen. Die Arbeit nennt 46 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Japan Society for the Promotion of Science (JSPS) KAKENHI.
Abstract (Originalsprache)
Open-set scene graph generation (SGG) is a recent problem in the field of computer vision, which aims to detect relationships of known and unknown objects in an image, thereby enabling detailed understanding of complex scenes required by various real-world applications. Previous studies on open-set SGG have attempted to discover unknown objects mistakenly detected as known by pretrained closed-set models. However, such a testing-time-only approach tends to overlook unknown objects dissimilar to known ones and thus fails to detect their relationships. In this paper, we propose a novel technique for open-set SGG based on a self-supervised learning approach for training an open-set SGG model. The methodology of this technique is centered around relationship-aware pseudo-labeling, which generates pseudo-labels of unknown objects and their relationships from intermediate predictions during training, thereby training the model to explicitly learn the relationships of unknown objects. We experimentally evaluated the proposed training-time technique in comparison with previous testing-time techniques in terms of standard evaluation metrics of open-set SGG. As the result, the proposed technique consistently outperformed the previous ones when combined with each SGG model, thereby demonstrating the effectiveness of our pseudo-label-based approach, which makes full use of training-time information on unknown objects and their relationships. Future directions include automation of hyperparameter tuning, performance improvement by combination with other models and training techniques, and extension to related tasks such as open-world SGG.
Förderung
- Japan Society for the Promotion of Science (JSPS) KAKENHI
Zitieren
APA
Sonogashira, M., Iiyama, M., & Kawanishi, Y. (2026). Self-Supervised Learning for Open-Set Scene Graph Generation by Unknown-Object and Relationship Pseudo-Labels. IEEE Access, 14, 70880-70892. https://doi.org/10.1109/access.2026.3690725
Vancouver
Sonogashira M, Iiyama M, Kawanishi Y. Self-Supervised Learning for Open-Set Scene Graph Generation by Unknown-Object and Relationship Pseudo-Labels. IEEE Access. 2026;14:70880-70892. doi:10.1109/access.2026.3690725
BibTeX
@article{Sonogashira2026,
author = {Sonogashira, Motoharu and Iiyama, Masaaki and Kawanishi, Yasutomo},
title = {Self-Supervised Learning for Open-Set Scene Graph Generation by Unknown-Object and Relationship Pseudo-Labels},
journal = {IEEE Access},
year = {2026},
volume = {14},
pages = {70880--70892},
doi = {10.1109/access.2026.3690725}
}
EUVIMED