Low-Power FPGA-Based Differentiable Mesh Rendering for On-Device Dense Photometric Pose Refinement
Chips · 09.11.2026 · 5 (4) · 38
DOI: 10.3390/chips5040038 ↗Kurz gefasst
Fachartikel in der Zeitschrift Chips, online erschienen am 09.11.2026. Verfasst von Emanuel Trabes und 1 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Union’s Horizon 2020 research and innovation programme, C2W programme.
Abstract (Originalsprache)
Differentiable rasterization enables image inversion by providing gradients of image-based losses with respect to camera and scene parameters, and has become a practical foundation for inverse graphics and photometric estimation on GPUs. However, the real-time deployment of differentiable rendering in embedded perception remains challenging due to the high cost of per-pixel sampling and gradient computation. In this work, we present an nvdiffrast-inspired differentiable rasterization engine implemented on FPGA and demonstrate its use in a simplified dense photometric pose-refinement task. Our design follows the modular philosophy of nvdiffrast while specializing it for deterministic, low-latency hardware execution. We implement point-sampled triangle rasterization with hard z-buffer visibility, perspective-correct interpolation, and an analytic antialiasing stage that yields stable, non-zero gradients at silhouettes and occlusion boundaries. Unlike GPU pipelines where texturing behavior may be hardware defined, our FPGA implementation controls the texture sampler, enabling a fused rasterize–interpolate–texture operator with a matched backward pass. Beyond pixelwise gradients, we introduce an on-device accumulation scheme that directly forms Gauss–Newton normal-equation terms, enabling second-order pose updates with substantially fewer iterations than gradient descent. We validate forward rendering and derivatives against a GLSL reference implementation, and report FPGA resource utilization and runtime. The core rasterization and analytic antialiasing pipeline achieves 2.5 ms per frame, while the full textured keyframe-projection operator runs in 8 ms per frame on an AMD Kria KV260. In a dense photometric pose-refinement evaluation on synthetic and real sequences, the proposed Jacobian accumulation reduces the number of optimization iterations required for convergence, improving time-to-solution despite the added cost of texturing. Together with a post-implementation on-chip power estimate of 3.662 W and an estimated 29.3 mJ per full-resolution differentiable rendering iteration, these results provide an implementation-level estimate of the accelerator’s power envelope for real-time photometric tracking and refinement.
Förderung
- European Union’s Horizon 2020 research and innovation programme
- C2W programme
Zitieren
APA
Trabes, E., & Sukuyama, C. V. (2026). Low-Power FPGA-Based Differentiable Mesh Rendering for On-Device Dense Photometric Pose Refinement. Chips, 5(4), 38. https://doi.org/10.3390/chips5040038
Vancouver
Trabes E, Sukuyama CV. Low-Power FPGA-Based Differentiable Mesh Rendering for On-Device Dense Photometric Pose Refinement. Chips. 2026;5(4):38. doi:10.3390/chips5040038
BibTeX
@article{Trabes2026,
author = {Trabes, Emanuel and Sukuyama, Carlos Valderrama},
title = {Low-Power FPGA-Based Differentiable Mesh Rendering for On-Device Dense Photometric Pose Refinement},
journal = {Chips},
year = {2026},
volume = {5},
number = {4},
pages = {38},
doi = {10.3390/chips5040038}
}
EUVIMED