Neuromorphic Camera Denoising Using Graph Neural Network-Driven Transformers
IEEE Transactions on Neural Networks and Learning Systems · 2024-03 · 35 (3) · 4110-4124
DOI: 10.1109/tnnls.2022.3201830 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Neural Networks and Learning Systems, 2024-03. Verfasst von Yusra Alkendi und 5 weiteren Autor:innen. Die Arbeit wurde bisher 52-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Khalifa University of Science and Technology, Aerospace Research and Innovation Center (ARIC), which is jointly funded by STRATA Manufacturing PJSC (a Mubadala company) and the Khalifa University of Scienc…, Khalifa University of Science and Technology under the Faculty Start Up.
Abstract (Originalsprache)
Neuromorphic vision is a bio-inspired technology that has triggered a paradigm shift in the computer vision community and is serving as a key enabler for a wide range of applications. This technology has offered significant advantages, including reduced power consumption, reduced processing needs, and communication speedups. However, neuromorphic cameras suffer from significant amounts of measurement noise. This noise deteriorates the performance of neuromorphic event-based perception and navigation algorithms. In this article, we propose a novel noise filtration algorithm to eliminate events that do not represent real log-intensity variations in the observed scene. We employ a graph neural network (GNN)-driven transformer algorithm, called GNN-Transformer, to classify every active event pixel in the raw stream into real log-intensity variation or noise. Within the GNN, a message-passing framework, referred to as EventConv, is carried out to reflect the spatiotemporal correlation among the events while preserving their asynchronous nature. We also introduce the known-object ground-truth labeling (KoGTL) approach for generating approximate ground-truth labels of event streams under various illumination conditions. KoGTL is used to generate labeled datasets, from experiments recorded in challenging lighting conditions, including moon light. These datasets are used to train and extensively test our proposed algorithm. When tested on unseen datasets, the proposed algorithm outperforms state-of-the-art methods by at least 8.8% in terms of filtration accuracy. Additional tests are also conducted on publicly available datasets (ETH Zürich Color-DAVIS346 datasets) to demonstrate the generalization capabilities of the proposed algorithm in the presence of illumination variations and different motion dynamics. Compared to state-of-the-art solutions, qualitative results verified the superior capability of the proposed algorithm to eliminate noise while preserving meaningful events in the scene.
Förderung
- Khalifa University of Science and Technology
- Aerospace Research and Innovation Center (ARIC), which is jointly funded by STRATA Manufacturing PJSC (a Mubadala company) and the Khalifa University of Scienc…
- Khalifa University of Science and Technology under the Faculty Start Up
Zitieren
APA
Alkendi, Y., Azzam, R., Ayyad, A., Javed, S., Seneviratne, L., & Zweiri, Y. (2024). Neuromorphic Camera Denoising Using Graph Neural Network-Driven Transformers. IEEE Transactions on Neural Networks and Learning Systems, 35(3), 4110-4124. https://doi.org/10.1109/tnnls.2022.3201830
Vancouver
Alkendi Y, Azzam R, Ayyad A, Javed S, Seneviratne L, Zweiri Y. Neuromorphic Camera Denoising Using Graph Neural Network-Driven Transformers. IEEE Transactions on Neural Networks and Learning Systems. 2024;35(3):4110-4124. doi:10.1109/tnnls.2022.3201830
BibTeX
@article{Alkendi2024,
author = {Alkendi, Yusra and Azzam, Rana and Ayyad, Abdulla and Javed, Sajid and Seneviratne, Lakmal and Zweiri, Yahya},
title = {Neuromorphic Camera Denoising Using Graph Neural Network-Driven Transformers},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
year = {2024},
volume = {35},
number = {3},
pages = {4110--4124},
doi = {10.1109/tnnls.2022.3201830}
}
EUVIMED