EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Contrast-Phys+: Unsupervised and Weakly-Supervised Video-Based Remote Physiological Measurement via Spatiotemporal Contrast

Zhaodong Sun, Xiaobai Li

IEEE Transactions on Pattern Analysis and Machine Intelligence · 2024-08 · 46 (8) · 5835-5851

DOI: 10.1109/tpami.2024.3367910 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, 2024-08. Verfasst von Zhaodong Sun und 1 weiteren Autor:innen. Die Arbeit wurde bisher 72-mal zitiert (Crossref) und nennt 83 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Research Council of Finland, ICT 2023 project TrustFace, University of Oulu and Research Council of Finland Profi 7.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Video-based remote physiological measurement utilizes facial videos to measure the blood volume change signal, which is also called remote photoplethysmography (rPPG). Supervised methods for rPPG measurements have been shown to achieve good performance. However, the drawback of these methods is that they require facial videos with ground truth (GT) physiological signals, which are often costly and difficult to obtain. In this paper, we propose Contrast-Phys+, a method that can be trained in both unsupervised and weakly-supervised settings. We employ a 3DCNN model to generate multiple spatiotemporal rPPG signals and incorporate prior knowledge of rPPG into a contrastive loss function. We further incorporate the GT signals into contrastive learning to adapt to partial or misaligned labels. The contrastive loss encourages rPPG/GT signals from the same video to be grouped together, while pushing those from different videos apart. We evaluate our methods on five publicly available datasets that include both RGB and Near-infrared videos. Contrast-Phys+ outperforms the state-of-the-art supervised methods, even when using partially available or misaligned GT signals, or no labels at all. Additionally, we highlight the advantages of our methods in terms of computational efficiency, noise robustness, and generalization.

Förderung

  • Research Council of Finland
  • ICT 2023 project TrustFace
  • University of Oulu and Research Council of Finland Profi 7
  • Infotech Oulu
  • University of Oulu, Finland

Zitieren

APA

Sun, Z., & Li, X. (2024). Contrast-Phys+: Unsupervised and Weakly-Supervised Video-Based Remote Physiological Measurement via Spatiotemporal Contrast. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8), 5835-5851. https://doi.org/10.1109/tpami.2024.3367910

Vancouver

Sun Z, Li X. Contrast-Phys+: Unsupervised and Weakly-Supervised Video-Based Remote Physiological Measurement via Spatiotemporal Contrast. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2024;46(8):5835-5851. doi:10.1109/tpami.2024.3367910

BibTeX
@article{Sun2024,
  author = {Sun, Zhaodong and Li, Xiaobai},
  title = {Contrast-Phys+: Unsupervised and Weakly-Supervised Video-Based Remote Physiological Measurement via Spatiotemporal Contrast},
  journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year = {2024},
  volume = {46},
  number = {8},
  pages = {5835--5851},
  doi = {10.1109/tpami.2024.3367910}
}
Kostenlos starten →