EUVIMEDEUVIMED entdecken
Crossref

Generating Images of Rare Concepts Using Pre-trained Diffusion Models

Dvir Samuel, Rami Ben-Ari, Simon Raviv, Nir Darshan, Gal Chechik

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (5) · 4695-4703

DOI: 10.1609/aaai.v38i5.28270 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Dvir Samuel und 4 weiteren Autor:innen. Die Arbeit wurde bisher 32-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Text-to-image diffusion models can synthesize high quality images, but they have various limitations. Here we highlight a common failure mode of these models, namely, generating uncommon concepts and structured concepts like hand palms. We show that their limitation is partly due to the long-tail nature of their training data: web-crawled data sets are strongly unbalanced, causing models to under-represent concepts from the tail of the distribution. We characterize the effect of unbalanced training data on text-to-image models and offer a remedy. We show that rare concepts can be correctly generated by carefully selecting suitable generation seeds in the noise space, using a small reference set of images, a technique that we call SeedSelect. SeedSelect does not require retraining or finetuning the diffusion model. We assess the faithfulness, quality and diversity of SeedSelect in creating rare objects and generating complex formations like hand images, and find it consistently achieves superior performance. We further show the advantage of SeedSelect in semantic data augmentation. Generating semantically appropriate images can successfully improve performance in few-shot recognition benchmarks, for classes from the head and from the tail of the training data of diffusion models.

Zitieren

APA

Samuel, D., Ben-Ari, R., Raviv, S., Darshan, N., & Chechik, G. (2024). Generating Images of Rare Concepts Using Pre-trained Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence, 38(5), 4695-4703. https://doi.org/10.1609/aaai.v38i5.28270

Vancouver

Samuel D, Ben-Ari R, Raviv S, Darshan N, Chechik G. Generating Images of Rare Concepts Using Pre-trained Diffusion Models. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(5):4695-4703. doi:10.1609/aaai.v38i5.28270

BibTeX
@article{Samuel2024,
  author = {Samuel, Dvir and Ben-Ari, Rami and Raviv, Simon and Darshan, Nir and Chechik, Gal},
  title = {Generating Images of Rare Concepts Using Pre-trained Diffusion Models},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {5},
  pages = {4695--4703},
  doi = {10.1609/aaai.v38i5.28270}
}
Kostenlos starten →