A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth
Sensors · 11.06.2026 · 26 (12) · 3730
DOI: 10.3390/s26123730 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 11.06.2026. Verfasst von Laixin Peng und 2 weiteren Autor:innen. Beteiligte Einrichtung: School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou 350108, China. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Fujian Provincial Department of Science and Technology University-Industry-Research Joint Innovation Project, Fujian Provincial Health Technology Project, High-level Talent Training Program of Fujian Cancer Hospital.
Abstract (Originalsprache)
Stable three-dimensional hand landmark reconstruction using low-cost RGB-D sensors is important for human–computer interaction, robot teleoperation, and vision-based motion analysis. RGB-based hand landmark detectors provide stable semantic 2D landmarks, but their depth output is not a metric measurement in the physical camera coordinate system. Stereo cameras can provide metric depth, but direct landmark-level back-projection is sensitive to invalid pixels, local depth holes, boundary noise, and partial occlusion. To address these problems, this paper presents a lightweight RGB-D sensing front-end that combines MediaPipe semantic hand landmarks with ZED2 stereo depth. The proposed pipeline detects 21 semantic hand landmarks in the RGB image, obtains landmark-level metric depth from the aligned ZED2 depth map using local median sampling, reconstructs 3D landmarks by camera back-projection, and further applies exponential moving average filtering and a bone-length consistency constraint. Experiments were conducted on a self-collected SVO dataset containing 13 hand actions and 26 recorded sequences, and an additional checkerboard-based reference-distance validation was performed to evaluate the metric depth sampling and 3D back-projection component. Compared with single-pixel sampling, the 5×5 local median strategy slightly increased the valid-depth ratio from 0.9731 to 0.9738 and reduced the temporal smoothness metric from 1.7163 mm to 1.6902 mm. To further justify the temporal filtering choice, an additional comparison with the 1 Euro Filter was conducted using the reconstructed win5 trajectories. The 1 Euro Filter produced stronger smoothing, reducing the temporal smoothness metric to 0.196 mm, but also reduced the path-length ratio to 0.484, indicating substantial motion attenuation. EMA0.7 was therefore retained as a more balanced setting, reducing the temporal smoothness metric to 0.826 mm while maintaining a path-length ratio of 0.803. The BL0.5 bone-length constraint reduced the bone-length standard deviation from 2.0727 mm to 1.1995 mm with limited trajectory modification. The final configuration provides a practical low-cost RGB-D front-end for stable 3D hand landmark reconstruction under controlled indoor conditions.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- School of Mechanical Engineering and Automation, Fuzhou University, Fuzhou 350108, China.
- Fujian Engineering Research Center of Joint Intelligent Medical Engineering, Fuzhou University, Fuzhou 350108, China.
ORCID-Profile: Laixin Peng, Tiansheng Liu, Bingwei He
Förderung
- Fujian Provincial Department of Science and Technology University-Industry-Research Joint Innovation Project
- Fujian Provincial Health Technology Project
- High-level Talent Training Program of Fujian Cancer Hospital
- Fujian Provincial Department of Science and Technology University–Industry–Research Joint Innovation Project
Zitieren
APA
Peng, L., Liu, T., & He, B. (2026). A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth. Sensors, 26(12), 3730. https://doi.org/10.3390/s26123730
Vancouver
Peng L, Liu T, He B. A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth. Sensors. 2026;26(12):3730. doi:10.3390/s26123730
BibTeX
@article{Peng2026,
author = {Peng, Laixin and Liu, Tiansheng and He, Bingwei},
title = {A Low-Cost RGB-D Sensing Front-End for Stable 3D Hand Landmark Reconstruction Using MediaPipe and ZED2 Stereo Depth},
journal = {Sensors},
year = {2026},
volume = {26},
number = {12},
pages = {3730},
doi = {10.3390/s26123730}
}
EUVIMED