EUVIMEDEUVIMED entdecken
Crossref

COSMOS: Catching Out-of-Context Image Misuse Using Self-Supervised Learning

Shivangi Aneja, Chris Bregler, Matthias Niessner

Proceedings of the AAAI Conference on Artificial Intelligence · 26.06.2023 · 37 (12) · 14084-14092

DOI: 10.1609/aaai.v37i12.26648 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 26.06.2023. Verfasst von Shivangi Aneja und 2 weiteren Autor:innen. Die Arbeit wurde bisher 29-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Despite the recent attention to DeepFakes, one of the most prevalent ways to mislead audiences on social media is the use of unaltered images in a new but false context. We propose a new method that automatically highlights out-of-context image and text pairs, for assisting fact-checkers. Our key insight is to leverage the grounding of images with text to distinguish out-of-context scenarios that cannot be disambiguated with language alone. We propose a self-supervised training strategy where we only need a set of captioned images. At train time, our method learns to selectively align individual objects in an image with textual claims, without explicit supervision. At test time, we check if both captions correspond to the same object(s) in the image but are semantically different, which allows us to make fairly accurate out-of-context predictions. Our method achieves 85% out-of-context detection accuracy. To facilitate benchmarking of this task, we create a large-scale dataset of 200K images with 450K textual captions from a variety of news websites, blogs, and social media posts

Zitieren

APA

Aneja, S., Bregler, C., & Niessner, M. (2023). COSMOS: Catching Out-of-Context Image Misuse Using Self-Supervised Learning. Proceedings of the AAAI Conference on Artificial Intelligence, 37(12), 14084-14092. https://doi.org/10.1609/aaai.v37i12.26648

Vancouver

Aneja S, Bregler C, Niessner M. COSMOS: Catching Out-of-Context Image Misuse Using Self-Supervised Learning. Proceedings of the AAAI Conference on Artificial Intelligence. 2023;37(12):14084-14092. doi:10.1609/aaai.v37i12.26648

BibTeX
@article{Aneja2023,
  author = {Aneja, Shivangi and Bregler, Chris and Niessner, Matthias},
  title = {COSMOS: Catching Out-of-Context Image Misuse Using Self-Supervised Learning},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2023},
  volume = {37},
  number = {12},
  pages = {14084--14092},
  doi = {10.1609/aaai.v37i12.26648}
}
Kostenlos starten →