EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

CISA: Context Substitution for Image Semantics Augmentation

Sergey Nesteruk, Ilya Zherebtsov, Svetlana Illarionova, Dmitrii Shadrin, Andrey Somov, Sergey V. Bezzateev, Tatiana Yelina, Vladimir Denisenko, Ivan Oseledets

Mathematics · 11.04.2023 · 11 (8) · 1818

DOI: 10.3390/math11081818 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Mathematics, online erschienen am 11.04.2023. Verfasst von Sergey Nesteruk und 8 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 61 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Large datasets catalyze the rapid expansion of deep learning and computer vision. At the same time, in many domains, there is a lack of training data, which may become an obstacle for the practical application of deep computer vision models. To overcome this problem, it is popular to apply image augmentation. When a dataset contains instance segmentation masks, it is possible to apply instance-level augmentation. It operates by cutting an instance from the original image and pasting to new backgrounds. This article challenges a dataset with the same objects present in various domains. We introduce the Context Substitution for Image Semantics Augmentation framework (CISA), which is focused on choosing good background images. We compare several ways to find backgrounds that match the context of the test set, including Contrastive Language–Image Pre-Training (CLIP) image retrieval and diffusion image generation. We prove that our augmentation method is effective for classification, segmentation, and object detection with different dataset complexity and different model types. The average percentage increase in accuracy across all the tasks on a fruits and vegetables recognition dataset is 4.95%. Moreover, we show that the Fréchet Inception Distance (FID) metrics has a strong correlation with model accuracy, and it can help to choose better backgrounds without model training. The average negative correlation between model accuracy and the FID between the augmented and test datasets is 0.55 in our experiments.

Zitieren

APA

Nesteruk, S., Zherebtsov, I., Illarionova, S., Shadrin, D., Somov, A., Bezzateev, S. V., Yelina, T., Denisenko, V., & Oseledets, I. (2023). CISA: Context Substitution for Image Semantics Augmentation. Mathematics, 11(8), 1818. https://doi.org/10.3390/math11081818

Vancouver

Nesteruk S, Zherebtsov I, Illarionova S, Shadrin D, Somov A, Bezzateev SV, et al. CISA: Context Substitution for Image Semantics Augmentation. Mathematics. 2023;11(8):1818. doi:10.3390/math11081818

BibTeX
@article{Nesteruk2023,
  author = {Nesteruk, Sergey and Zherebtsov, Ilya and Illarionova, Svetlana and Shadrin, Dmitrii and Somov, Andrey and Bezzateev, Sergey V. and Yelina, Tatiana and Denisenko, Vladimir and Oseledets, Ivan},
  title = {CISA: Context Substitution for Image Semantics Augmentation},
  journal = {Mathematics},
  year = {2023},
  volume = {11},
  number = {8},
  pages = {1818},
  doi = {10.3390/math11081818}
}
Kostenlos starten →