EUVIMEDEUVIMED entdecken
Crossref

MagicPaint: Operate Anything for Image Inpainting with Diffusion Model

Qinhong Yang, Dongdong Chen, Qi Chu, Tao Gong, Qiankun Liu, Zhentao Tan, Xulin Li, Huamin Feng, Nenghai Yu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (14) · 11667-11675

DOI: 10.1609/aaai.v40i14.38151 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Qinhong Yang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent diffusion-based models have significantly improved inpainting quality. However, existing methods struggle with multi-task inpainting due to conflicting optimization objectives, and current datasets are typically limited to task-specific scenarios, hindering joint training. To address these challenges, we propose MagicPaint, a unified diffusion-based inpainting model that supports object addition, removal, and unconditional inpainting across both text and image modalities. MagicPaint semantically decouples operation types and target content by learnable tokens in MMToken Module, effectively reconciling conflicting optimization objectives and enabling robust multi-task, multi-modal inpainting. Besides, a novel inpainting paradigm named MagicMask, encodes operating intent directly into the mask and applies a mask loss for spatially precise supervision. In addition, existing inpainting datasets are insufficient for multi-task and multi-modal scenarios, limiting the capability of inpainting models. Thus, we further introduce a new dataset comprising 2.1M image tuples. It is dedicatedly designed to support diverse inpainting scenarios and significantly improves upon existing datasets, particularly in object removal. Through efforts from both model and data perspectives, MagicPaint enables users to operate anything—add, remove or inpaint content which is specified through either text or image modalities in a seamless and unified manner. Extensive experiments demonstrate that MagicPaint achieves state-of-the-art performance across three key tasks (i.e., text-guided addition, image-guided addition, and object removal) and produces outputs with superior visual consistency and contextual fidelity compared to existing methods.

Zitieren

APA

Yang, Q., Chen, D., Chu, Q., Gong, T., Liu, Q., Tan, Z., Li, X., Feng, H., & Yu, N. (2026). MagicPaint: Operate Anything for Image Inpainting with Diffusion Model. Proceedings of the AAAI Conference on Artificial Intelligence, 40(14), 11667-11675. https://doi.org/10.1609/aaai.v40i14.38151

Vancouver

Yang Q, Chen D, Chu Q, Gong T, Liu Q, Tan Z, et al. MagicPaint: Operate Anything for Image Inpainting with Diffusion Model. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(14):11667-11675. doi:10.1609/aaai.v40i14.38151

BibTeX
@article{Yang2026,
  author = {Yang, Qinhong and Chen, Dongdong and Chu, Qi and Gong, Tao and Liu, Qiankun and Tan, Zhentao and Li, Xulin and Feng, Huamin and Yu, Nenghai},
  title = {MagicPaint: Operate Anything for Image Inpainting with Diffusion Model},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {14},
  pages = {11667--11675},
  doi = {10.1609/aaai.v40i14.38151}
}
Kostenlos starten →