Face Swapping for Low-Resolution and Occluded Images In-the-Wild
IEEE Access · 2024 · 12 · 91383-91395
DOI: 10.1109/access.2024.3421528 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Access, 2024. Verfasst von Jaehyun Park und 3 weiteren Autor:innen. Die Arbeit wurde bisher 3-mal zitiert (Crossref) und nennt 43 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Ministry of Science and ICT, South Korea, BK21 FOUR program of the Education and Research Program for Future ICT Pioneers, Seoul National University.
Abstract (Originalsprache)
Safeguarding personal identity in various surveillance videos, dashcams, and on-street videos is crucial. One way to do this is to detect faces and blur them, but a better solution is to replace them with non-existent ones to maintain the naturalness of the videos. While face swapping methods have already been used in the media industry with high-quality faces, it is challenging to apply them for identity protection to faces in-the-wild where faces are often occluded and of low-resolution. Therefore, we propose a new framework for face swapping specifically designed to work with face images taken in real-world scenarios, making it useful as a privacy protection method. To tackle the issue of low-resolution images, we introduce a Cross-Resolution Contrastive Loss (CRCL) technique, which allows our neural network model to be trained using triplets of varying resolutions. This enables the model to learn and use identity information across different resolutions, thereby improving its accuracy. We also propose a plug-and-play framework that can be easily applied to existing face swapping models to handle occlusions. By explicit swapping of facial features and filling of occluded regions, our framework provides a more seamless blend. To demonstrate the effectiveness of our method in handling faces in-the-wild, we create an occluded VGGFace2 dataset consisting of face images augmented with various facial masks and hand occlusions. Through quantitative and qualitative assessments on this dataset, our proposed method demonstrates robust performance under low-resolution or occluded scenarios. Significant improvements are made in the quality of swapped faces while preserving their identity and attributes, highlighting the effectiveness of our framework in advancing face swapping as a reliable privacy protection measure.
Förderung
- Ministry of Science and ICT, South Korea
- BK21 FOUR program of the Education and Research Program for Future ICT Pioneers, Seoul National University
Zitieren
APA
Park, J., Kang, W., Koo, H. I., & Cho, N. I. (2024). Face Swapping for Low-Resolution and Occluded Images In-the-Wild. IEEE Access, 12, 91383-91395. https://doi.org/10.1109/access.2024.3421528
Vancouver
Park J, Kang W, Koo HI, Cho NI. Face Swapping for Low-Resolution and Occluded Images In-the-Wild. IEEE Access. 2024;12:91383-91395. doi:10.1109/access.2024.3421528
BibTeX
@article{Park2024,
author = {Park, Jaehyun and Kang, Wonjun and Koo, Hyung Il and Cho, Nam Ik},
title = {Face Swapping for Low-Resolution and Occluded Images In-the-Wild},
journal = {IEEE Access},
year = {2024},
volume = {12},
pages = {91383--91395},
doi = {10.1109/access.2024.3421528}
}
EUVIMED