Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild
Sensors · 2024
DOI: 10.3390/s24030775 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Walaa Othman und 4 weiteren Autor:innen. Beteiligte Einrichtung: St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 199178 St. Petersburg, Russia. Hauptthemen laut MEDLINE: Blood Volume, Electric Power Supplies. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Russian State Research.
Abstract (Originalsprache)
Heart rate is an essential vital sign to evaluate human health. Remote heart monitoring using cheaply available devices has become a necessity in the twenty-first century to prevent any unfortunate situation caused by the hectic pace of life. In this paper, we propose a new method based on the transformer architecture with a multi-skip connection biLSTM decoder to estimate heart rate remotely from videos. Our method is based on the skin color variation caused by the change in blood volume in its surface. The presented heart rate estimation framework consists of three main steps: (1) the segmentation of the facial region of interest (ROI) based on the landmarks obtained by 3DDFA; (2) the extraction of the spatial and global features; and (3) the estimation of the heart rate value from the obtained features based on the proposed method. This paper investigates which feature extractor performs better by captioning the change in skin color related to the heart rate as well as the optimal number of frames needed to achieve better accuracy. Experiments were conducted using two publicly available datasets (LGI-PPGI and Vision for Vitals) and our own in-the-wild dataset (12 videos collected by four drivers). The experiments showed that our approach achieved better results than the previously published methods, making it the new state of the art on these datasets.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 199178 St. Petersburg, Russia.
- Information Technology and Programming Faculty, ITMO University, 191002 St. Petersburg, Russia.
ORCID-Profile: Walaa Othman, Alexey Kashevnik, Nikolay Shilov, Dmitry Ryumin
Förderung
- Russian State Research
Zitieren
APA
Othman, W., Kashevnik, A., Ali, A., Shilov, N., & Ryumin, D. (2024). Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild. Sensors. https://doi.org/10.3390/s24030775
Vancouver
Othman W, Kashevnik A, Ali A, Shilov N, Ryumin D. Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild. Sensors. 2024. doi:10.3390/s24030775
BibTeX
@article{Othman2024,
author = {Othman, Walaa and Kashevnik, Alexey and Ali, Ammar and Shilov, Nikolay and Ryumin, Dmitry},
title = {Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild},
journal = {Sensors},
year = {2024},
doi = {10.3390/s24030775}
}
EUVIMED