EUVIMEDEUVIMED entdecken
Crossref

Thinking Fast, Thinking Slow: Adaptive Multimodal Transformer-based Sensor Fusion for Depth Estimation on Ultra-low-power MCUs

Luca Crupi, Lorenzo Lamberti, Giovanni Badaracco, Daniele Allegri, Alessandro Giusti, Daniele Palossi

IEEE Sensors Journal · 2026 · 1-1

DOI: 10.1109/jsen.2026.3713809 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Sensors Journal, 2026. Verfasst von Luca Crupi und 5 weiteren Autor:innen. Gefördert von: SNSF RoboMix2 project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Artificial intelligence (AI)-based multimodal sensor fusion is a relevant topic gaining ever more traction across ultra-low-power (ULP) embedded and cyber-physical systems, as it improves reliability, accuracy, and robustness under real-world constraints. However, adding more and more sensors to ultra-constrained sub-100 mW platforms requires balancing energy consumption against prediction accuracy. To achieve this ambitious goal, we present a novel adaptive AI methodology that combines multimodal sensor fusion (camera, ultrasound, and Time-of-Flight sensors) with a lightweight recurrent Transformer-based architecture (688 k parameters). We address the depth map estimation task with a mechanism that combines token propagation across iterations with incremental sensor utilization. At each iteration, a confidence-based gating mechanism dynamically decides whether to continue the computation by adding progressively richer but more power-demanding sensors as input. Token propagation ensures temporal consistency by forwarding context features across time. To deploy our algorithm and test a first real-world prototype, we design a novel printed circuit board featuring all three sensors, coupled with an ULP GWT GAP9 multicore System-on-Chip. When comparing our adaptive system against the same pipeline using all sensors and iterations on the NYUv2 dataset, we lose only 4.8% of the {\delta}1 accuracy in exchange for 90% energy saving (2.44 mJ/frame). Finally, our adaptive method marks only 5.6% lower {\delta}1 accuracy than MobileDepth despite using 9x fewer parameters. Compared with a state-of-the-art model also running on GAP9, our method improves {\delta}1 accuracy by 31.8% thanks to our adaptive sensor fusion while operating within the same average power budget (~400 mW).

Förderung

  • SNSF RoboMix2 project

Zitieren

APA

Crupi, L., Lamberti, L., Badaracco, G., Allegri, D., Giusti, A., & Palossi, D. (2026). Thinking Fast, Thinking Slow: Adaptive Multimodal Transformer-based Sensor Fusion for Depth Estimation on Ultra-low-power MCUs. IEEE Sensors Journal, 1-1. https://doi.org/10.1109/jsen.2026.3713809

Vancouver

Crupi L, Lamberti L, Badaracco G, Allegri D, Giusti A, Palossi D. Thinking Fast, Thinking Slow: Adaptive Multimodal Transformer-based Sensor Fusion for Depth Estimation on Ultra-low-power MCUs. IEEE Sensors Journal. 2026:1-1. doi:10.1109/jsen.2026.3713809

BibTeX
@article{Crupi2026,
  author = {Crupi, Luca and Lamberti, Lorenzo and Badaracco, Giovanni and Allegri, Daniele and Giusti, Alessandro and Palossi, Daniele},
  title = {Thinking Fast, Thinking Slow: Adaptive Multimodal Transformer-based Sensor Fusion for Depth Estimation on Ultra-low-power MCUs},
  journal = {IEEE Sensors Journal},
  year = {2026},
  pages = {1--1},
  doi = {10.1109/jsen.2026.3713809}
}
Kostenlos starten →