EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth

Laixin Peng iD, Tiansheng Liu iD, Bingwei He iD

Sensors · 11.06.2026 · 26 (12) · 3730

DOI: 10.3390/s26123730 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 11.06.2026. Verfasst von Laixin Peng und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou 350108, China. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fujian Provincial Department of Science and Technology University-Industry-Research Joint Innovation Project, Fujian Provincial Health Technology Project, High-level Talent Training Program of Fujian Cancer Hospital.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Stable three-dimensional hand landmark reconstruction using low-cost RGB-D sensors is important for human–computer interaction, robot teleoperation, and vision-based motion analysis. RGB-based hand landmark detectors provide stable semantic 2D landmarks, but their depth output is not a metric measurement in the physical camera coordinate system. Stereo cameras can provide metric depth, but direct landmark-level back-projection is sensitive to invalid pixels, local depth holes, boundary noise, and partial occlusion. To address these problems, this paper presents a lightweight RGB-D sensing front-end that combines MediaPipe semantic hand landmarks with ZED2 stereo depth. The proposed pipeline detects 21 semantic hand landmarks in the RGB image, obtains landmark-level metric depth from the aligned ZED2 depth map using local median sampling, reconstructs 3D landmarks by camera back-projection, and further applies exponential moving average filtering and a bone-length consistency constraint. Experiments were conducted on a self-collected SVO dataset containing 13 hand actions and 26 recorded sequences, and an additional checkerboard-based reference-distance validation was performed to evaluate the metric depth sampling and 3D back-projection component. Compared with single-pixel sampling, the 5×5 local median strategy slightly increased the valid-depth ratio from 0.9731 to 0.9738 and reduced the temporal smoothness metric from 1.7163 mm to 1.6902 mm. To further justify the temporal filtering choice, an additional comparison with the 1 Euro Filter was conducted using the reconstructed win5 trajectories. The 1 Euro Filter produced stronger smoothing, reducing the temporal smoothness metric to 0.196 mm, but also reduced the path-length ratio to 0.484, indicating substantial motion attenuation. EMA0.7 was therefore retained as a more balanced setting, reducing the temporal smoothness metric to 0.826 mm while maintaining a path-length ratio of 0.803. The BL0.5 bone-length constraint reduced the bone-length standard deviation from 2.0727 mm to 1.1995 mm with limited trajectory modification. The final configuration provides a practical low-cost RGB-D front-end for stable 3D hand landmark reconstruction under controlled indoor conditions.

Schlagwörter

Stichwörter der Autor:innen

3D hand landmark reconstructionMediaPipeRGB-D sensingZED2bone-length constrainthand motion trackingsensor fusionstereo depthtemporal filtering

Autor:innen und Institutionen

  • School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou 350108, China.
  • Fujian Engineering Research Center of Joint Intelligent Medical Engineering, Fuzhou University, Fuzhou 350108, China.

ORCID-Profile: Laixin Peng, Tiansheng Liu, Bingwei He

Förderung

  • Fujian Provincial Department of Science and Technology University-Industry-Research Joint Innovation Project
  • Fujian Provincial Health Technology Project
  • High-level Talent Training Program of Fujian Cancer Hospital
  • Fujian Provincial Department of Science and Technology University–Industry–Research Joint Innovation Project

Zitieren

APA

Peng, L., Liu, T., & He, B. (2026). A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth. Sensors, 26(12), 3730. https://doi.org/10.3390/s26123730

Vancouver

Peng L, Liu T, He B. A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth. Sensors. 2026;26(12):3730. doi:10.3390/s26123730

BibTeX
@article{Peng2026,
  author = {Peng, Laixin and Liu, Tiansheng and He, Bingwei},
  title = {A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {12},
  pages = {3730},
  doi = {10.3390/s26123730}
}
Kostenlos starten →