MagicPaint: Operate Anything for Image Inpainting with Diffusion Model
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (14) · 11667-11675
DOI: 10.1609/aaai.v40i14.38151 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Qinhong Yang und 8 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).
Abstract (Originalsprache)
Recent diffusion-based models have significantly improved inpainting quality. However, existing methods struggle with multi-task inpainting due to conflicting optimization objectives, and current datasets are typically limited to task-specific scenarios, hindering joint training. To address these challenges, we propose MagicPaint, a unified diffusion-based inpainting model that supports object addition, removal, and unconditional inpainting across both text and image modalities. MagicPaint semantically decouples operation types and target content by learnable tokens in MMToken Module, effectively reconciling conflicting optimization objectives and enabling robust multi-task, multi-modal inpainting. Besides, a novel inpainting paradigm named MagicMask, encodes operating intent directly into the mask and applies a mask loss for spatially precise supervision. In addition, existing inpainting datasets are insufficient for multi-task and multi-modal scenarios, limiting the capability of inpainting models. Thus, we further introduce a new dataset comprising 2.1M image tuples. It is dedicatedly designed to support diverse inpainting scenarios and significantly improves upon existing datasets, particularly in object removal. Through efforts from both model and data perspectives, MagicPaint enables users to operate anything—add, remove or inpaint content which is specified through either text or image modalities in a seamless and unified manner. Extensive experiments demonstrate that MagicPaint achieves state-of-the-art performance across three key tasks (i.e., text-guided addition, image-guided addition, and object removal) and produces outputs with superior visual consistency and contextual fidelity compared to existing methods.
Zitieren
APA
Yang, Q., Chen, D., Chu, Q., Gong, T., Liu, Q., Tan, Z., Li, X., Feng, H., & Yu, N. (2026). MagicPaint: Operate Anything for Image Inpainting with Diffusion Model. Proceedings of the AAAI Conference on Artificial Intelligence, 40(14), 11667-11675. https://doi.org/10.1609/aaai.v40i14.38151
Vancouver
Yang Q, Chen D, Chu Q, Gong T, Liu Q, Tan Z, et al. MagicPaint: Operate Anything for Image Inpainting with Diffusion Model. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(14):11667-11675. doi:10.1609/aaai.v40i14.38151
BibTeX
@article{Yang2026,
author = {Yang, Qinhong and Chen, Dongdong and Chu, Qi and Gong, Tao and Liu, Qiankun and Tan, Zhentao and Li, Xulin and Feng, Huamin and Yu, Nenghai},
title = {MagicPaint: Operate Anything for Image Inpainting with Diffusion Model},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {14},
pages = {11667--11675},
doi = {10.1609/aaai.v40i14.38151}
}
EUVIMED