LiveMatte: Dynamic Scene Background Restoration and Selective Portrait Patch Enhancement
IEEE Transactions on Circuits and Systems for Video Technology · 2026-08 · 36 (8) · 12311-12326
DOI: 10.1109/tcsvt.2026.3680937 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Circuits and Systems for Video Technology, 2026-08. Verfasst von Jiayu Sun und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 89 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Real-time and accurate portrait matting in videos is a challenging problem in computer vision research. Recent approaches have explored incorporating prior conditions for accurate inference. Notably, some methods ask the user to provide the background image, which requires extra effort from the user to capture the background image and is limited to videos with static backgrounds only. We note that real-time video motion segmentation methods often train a background model to detect the foreground. Our insight of this work is that if we can directly restore the background content from the input video as a prior, we may be able to achieve more precise portrait matting. In addition, this approach could potentially work even with dynamic backgrounds, without requiring additional user input. However, automatically restoring the background content is not straightforward due to the difficulty in distinguishing between foreground and background. While it may seem that stationary pixel values represent the background, these values can vary across frames. To this end, we propose a novel dynamic scene background restoration (DSBR) module that learns a background model by accumulating background content from each input video frame. It restores the current background content, which serves as a matting prior for alpha prediction of the subsequent frame. DSBR is extremely lightweight and can be easily integrated into existing matting models. Based on it, we present a real-time portrait video matting framework, LiveMatte. To more efficiently process high-resolution videos, we also introduce a selective portrait patch enhancement (SPPE) module. Extensive experiments and user studies demonstrate that our method is better and faster than existing methods.
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Sun, J., Ke, Z., Zhang, L., Lu, H., & Lau, R. W. H. (2026). LiveMatte: Dynamic Scene Background Restoration and Selective Portrait Patch Enhancement. IEEE Transactions on Circuits and Systems for Video Technology, 36(8), 12311-12326. https://doi.org/10.1109/tcsvt.2026.3680937
Vancouver
Sun J, Ke Z, Zhang L, Lu H, Lau RWH. LiveMatte: Dynamic Scene Background Restoration and Selective Portrait Patch Enhancement. IEEE Transactions on Circuits and Systems for Video Technology. 2026;36(8):12311-12326. doi:10.1109/tcsvt.2026.3680937
BibTeX
@article{Sun2026,
author = {Sun, Jiayu and Ke, Zhanghan and Zhang, Lihe and Lu, Huchuan and Lau, Rynson W. H.},
title = {LiveMatte: Dynamic Scene Background Restoration and Selective Portrait Patch Enhancement},
journal = {IEEE Transactions on Circuits and Systems for Video Technology},
year = {2026},
volume = {36},
number = {8},
pages = {12311--12326},
doi = {10.1109/tcsvt.2026.3680937}
}
EUVIMED