EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Low-Power FPGA-Based Differentiable Mesh Rendering for On-Device Dense Photometric Pose Refinement

Emanuel Trabes, Carlos Valderrama Sukuyama

Chips · 09.11.2026 · 5 (4) · 38

DOI: 10.3390/chips5040038 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Chips, online erschienen am 09.11.2026. Verfasst von Emanuel Trabes und 1 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Union’s Horizon 2020 research and innovation programme, C2W programme.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Differentiable rasterization enables image inversion by providing gradients of image-based losses with respect to camera and scene parameters, and has become a practical foundation for inverse graphics and photometric estimation on GPUs. However, the real-time deployment of differentiable rendering in embedded perception remains challenging due to the high cost of per-pixel sampling and gradient computation. In this work, we present an nvdiffrast-inspired differentiable rasterization engine implemented on FPGA and demonstrate its use in a simplified dense photometric pose-refinement task. Our design follows the modular philosophy of nvdiffrast while specializing it for deterministic, low-latency hardware execution. We implement point-sampled triangle rasterization with hard z-buffer visibility, perspective-correct interpolation, and an analytic antialiasing stage that yields stable, non-zero gradients at silhouettes and occlusion boundaries. Unlike GPU pipelines where texturing behavior may be hardware defined, our FPGA implementation controls the texture sampler, enabling a fused rasterize–interpolate–texture operator with a matched backward pass. Beyond pixelwise gradients, we introduce an on-device accumulation scheme that directly forms Gauss–Newton normal-equation terms, enabling second-order pose updates with substantially fewer iterations than gradient descent. We validate forward rendering and derivatives against a GLSL reference implementation, and report FPGA resource utilization and runtime. The core rasterization and analytic antialiasing pipeline achieves 2.5 ms per frame, while the full textured keyframe-projection operator runs in 8 ms per frame on an AMD Kria KV260. In a dense photometric pose-refinement evaluation on synthetic and real sequences, the proposed Jacobian accumulation reduces the number of optimization iterations required for convergence, improving time-to-solution despite the added cost of texturing. Together with a post-implementation on-chip power estimate of 3.662 W and an estimated 29.3 mJ per full-resolution differentiable rendering iteration, these results provide an implementation-level estimate of the accelerator’s power envelope for real-time photometric tracking and refinement.

Förderung

  • European Union’s Horizon 2020 research and innovation programme
  • C2W programme

Zitieren

APA

Trabes, E., & Sukuyama, C. V. (2026). Low-Power FPGA-Based Differentiable Mesh Rendering for On-Device Dense Photometric Pose Refinement. Chips, 5(4), 38. https://doi.org/10.3390/chips5040038

Vancouver

Trabes E, Sukuyama CV. Low-Power FPGA-Based Differentiable Mesh Rendering for On-Device Dense Photometric Pose Refinement. Chips. 2026;5(4):38. doi:10.3390/chips5040038

BibTeX
@article{Trabes2026,
  author = {Trabes, Emanuel and Sukuyama, Carlos Valderrama},
  title = {Low-Power FPGA-Based Differentiable Mesh Rendering for On-Device Dense Photometric Pose Refinement},
  journal = {Chips},
  year = {2026},
  volume = {5},
  number = {4},
  pages = {38},
  doi = {10.3390/chips5040038}
}
Kostenlos starten →