EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild

Walaa Othman iD, Alexey Kashevnik iD, Ammar Ali, Nikolay Shilov iD, Dmitry Ryumin iD

Sensors · 2024

DOI: 10.3390/s24030775 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, 2024. Verfasst von Walaa Othman und 4 weiteren Autor:innen. Beteiligte Einrichtung: St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 199178 St. Petersburg, Russia. Hauptthemen laut MEDLINE: Blood Volume, Electric Power Supplies. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 36 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Russian State Research.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Heart rate is an essential vital sign to evaluate human health. Remote heart monitoring using cheaply available devices has become a necessity in the twenty-first century to prevent any unfortunate situation caused by the hectic pace of life. In this paper, we propose a new method based on the transformer architecture with a multi-skip connection biLSTM decoder to estimate heart rate remotely from videos. Our method is based on the skin color variation caused by the change in blood volume in its surface. The presented heart rate estimation framework consists of three main steps: (1) the segmentation of the facial region of interest (ROI) based on the landmarks obtained by 3DDFA; (2) the extraction of the spatial and global features; and (3) the estimation of the heart rate value from the obtained features based on the proposed method. This paper investigates which feature extractor performs better by captioning the change in skin color related to the heart rate as well as the optimal number of frames needed to achieve better accuracy. Experiments were conducted using two publicly available datasets (LGI-PPGI and Vision for Vitals) and our own in-the-wild dataset (12 videos collected by four drivers). The experiments showed that our approach achieved better results than the previously published methods, making it the new state of the art on these datasets.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Blood VolumeElectric Power SuppliesHumansHeart RateFaceVideotape RecordingImage Processing, Computer-Assisted

Stichwörter der Autor:innen

machine learningremote heart rate estimationvideo analysisvital signs

Autor:innen und Institutionen

  • St. Petersburg Federal Research Center of the Russian Academy of Sciences (SPC RAS), 199178 St. Petersburg, Russia.
  • Information Technology and Programming Faculty, ITMO University, 191002 St. Petersburg, Russia.

ORCID-Profile: Walaa Othman, Alexey Kashevnik, Nikolay Shilov, Dmitry Ryumin

Förderung

  • Russian State Research

Zitieren

APA

Othman, W., Kashevnik, A., Ali, A., Shilov, N., & Ryumin, D. (2024). Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild. Sensors. https://doi.org/10.3390/s24030775

Vancouver

Othman W, Kashevnik A, Ali A, Shilov N, Ryumin D. Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild. Sensors. 2024. doi:10.3390/s24030775

BibTeX
@article{Othman2024,
  author = {Othman, Walaa and Kashevnik, Alexey and Ali, Ammar and Shilov, Nikolay and Ryumin, Dmitry},
  title = {Remote Heart Rate Estimation Based on Transformer with Multi-Skip Connection Decoder: Method and Evaluation in the Wild},
  journal = {Sensors},
  year = {2024},
  doi = {10.3390/s24030775}
}
Kostenlos starten →