CF-MoP: counterfactual mixture-of-prototypes for scene graph generation under long-tailed supervision
Journal of King Saud University Computer and Information Sciences · 2026-08 · 38 (6)
DOI: 10.1007/s44443-026-01086-7 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of King Saud University Computer and Information Sciences, online erschienen am 14.08.2026. Verfasst von Funing Xie und 2 weiteren Autor:innen. Die Arbeit nennt 55 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Jiangsu Provincial Graduate Student Research Innovation Program, National Natural Science Foundation of China.
Abstract (Originalsprache)
Scene graph generation suffers from severe long-tailed predicate distributions, where models are biased toward frequent predicates at the expense of rare but semantically meaningful ones. Through causal analysis, we identify the empirical frequency distribution as a confounder that distorts prototype learning, causing learned representations to encode statistical regularities rather than visual semantics. Existing methods operate on representations already shaped by frequency-dominated training, where prediction-level corrections leave the underlying prototype geometry unchanged, making debiasing inherently constrained by the biased representational space they rely upon. In this paper, we propose CF-MoP (CounterFactual Mixture-of-Prototypes), a representation-level debiasing framework inspired by causal analysis. CF-MoP maintains multiple prototype sets via bootstrap aggregation to cultivate heterogeneous frequency sensitivity across heads, providing a diversified representational basis for frequency-debiased aggregation. Furthermore, an information-guided momentum adaptation strategy allocates stronger update signals to rare-class prototypes, ensuring sufficient representational support under sparse supervision. At inference time, frequency sensitivity scores are estimated to identify and down-weight frequency-sensitive prototype heads, producing frequency-debiased predictions through counterfactual-inspired aggregation. Extensive experiments on Visual Genome, GQA200, and Open Images V6 show that CF-MoP achieves consistent mean Recall gains over comparable two-stage unbiased SGG methods across PredCls and SGCls settings, while maintaining competitive performance on frequent relationships, all without requiring balanced training data.
Förderung
- Jiangsu Provincial Graduate Student Research Innovation Program
- National Natural Science Foundation of China
Zitieren
APA
Xie, F., Song, W., & Liu, F. (2026). CF-MoP: counterfactual mixture-of-prototypes for scene graph generation under long-tailed supervision. Journal of King Saud University Computer and Information Sciences, 38(6). https://doi.org/10.1007/s44443-026-01086-7
Vancouver
Xie F, Song W, Liu F. CF-MoP: counterfactual mixture-of-prototypes for scene graph generation under long-tailed supervision. Journal of King Saud University Computer and Information Sciences. 2026;38(6). doi:10.1007/s44443-026-01086-7
BibTeX
@article{Xie2026,
author = {Xie, Funing and Song, Wanru and Liu, Feng},
title = {CF-MoP: counterfactual mixture-of-prototypes for scene graph generation under long-tailed supervision},
journal = {Journal of King Saud University Computer and Information Sciences},
year = {2026},
volume = {38},
number = {6},
doi = {10.1007/s44443-026-01086-7}
}
EUVIMED