EUVIMEDEUVIMED entdecken
Crossref

DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity

Yeying Jin, Wei Ye, Wenhan Yang, Yuan Yuan, Robby T. Tan

Proceedings of the AAAI Conference on Artificial Intelligence · 24.03.2024 · 38 (3) · 2634-2642

DOI: 10.1609/aaai.v38i3.28041 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 24.03.2024. Verfasst von Yeying Jin und 4 weiteren Autor:innen. Die Arbeit wurde bisher 47-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Removing soft and self shadows that lack clear boundaries from a single image is still challenging. Self shadows are shadows that are cast on the object itself. Most existing methods rely on binary shadow masks, without considering the ambiguous boundaries of soft and self shadows. In this paper, we present DeS3, a method that removes hard, soft and self shadows based on adaptive attention and ViT similarity. Our novel ViT similarity loss utilizes features extracted from a pre-trained Vision Transformer. This loss helps guide the reverse sampling towards recovering scene structures. Our adaptive attention is able to differentiate shadow regions from the underlying objects, as well as shadow regions from the object casting the shadow. This capability enables DeS3 to better recover the structures of objects even when they are partially occluded by shadows. Different from existing methods that rely on constraints during the training phase, we incorporate the ViT similarity during the sampling stage. Our method outperforms state-of-the-art methods on the SRD, AISTD, LRSS, USR and UIUC datasets, removing hard, soft, and self shadows robustly. Specifically, our method outperforms the SOTA method by 16% of the RMSE of the whole image on the LRSS dataset.

Zitieren

APA

Jin, Y., Ye, W., Yang, W., Yuan, Y., & Tan, R. T. (2024). DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity. Proceedings of the AAAI Conference on Artificial Intelligence, 38(3), 2634-2642. https://doi.org/10.1609/aaai.v38i3.28041

Vancouver

Jin Y, Ye W, Yang W, Yuan Y, Tan RT. DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity. Proceedings of the AAAI Conference on Artificial Intelligence. 2024;38(3):2634-2642. doi:10.1609/aaai.v38i3.28041

BibTeX
@article{Jin2024,
  author = {Jin, Yeying and Ye, Wei and Yang, Wenhan and Yuan, Yuan and Tan, Robby T.},
  title = {DeS3: Adaptive Attention-Driven Self and Soft Shadow Removal Using ViT Similarity},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2024},
  volume = {38},
  number = {3},
  pages = {2634--2642},
  doi = {10.1609/aaai.v38i3.28041}
}
Kostenlos starten →