Ultra-High Temporal Resolution Visual Reconstruction From a Fovea-Like Spike Camera via Spiking Neuron Model
IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.01.2023 · 45 (1) · 1233-1249
DOI: 10.1109/tpami.2022.3146140 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.01.2023. Verfasst von Lin Zhu und 4 weiteren Autor:innen. Hauptthemen laut MEDLINE: Algorithms, Neurons. Die Arbeit wurde bisher 28-mal zitiert (Crossref) und nennt 53 Literaturangaben. Gefördert von: National Natural Science Foundation of China.
Abstract (Originalsprache)
Neuromorphic vision sensor is a new bio-inspired imaging paradigm emerged in recent years. It uses the asynchronous spike signals instead of the traditional frame-based manner to achieve ultra-high speed sampling. Unlike the dynamic vision sensor (DVS) that perceives movement by imitating the retinal periphery, the spike camera was developed recently to perceive fine textures by simulating a small retinal region called the fovea. For this new type of neuromorphic camera, how to reconstruct ultra-high speed visual images from spike data becomes an important yet challenging issue in visual scene perception, analysis, and recognition applications. In this paper, a bio-inspired visual reconstruction framework for the spike camera is proposed for the first time. Its core idea is to use the biologically inspired adaptive adjustment mechanisms, combined with the spatiotemporal spike information extracted by the proposed model, to reconstruct the full texture of natural scenes in an ultra-high temporal resolution. Specifically, the proposed model consists of a motion local excitation layer, a spike refining layer and a visual reconstruction layer motivated by the bio-realistic leaky integrate-and-fire (LIF) neurons and synapse connection with spike-timing dependent plasticity (STDP) rule. To evaluate the performance, a spike dataset was constructed for normal and high-speed scenes in real-world recorded by the spike camera. The experimental results show that the proposed approach can reconstruct the visual images with 40,000 frames per second in both normal and high-speed scenes, while achieving high dynamic range and high image quality.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Förderung
- National Natural Science Foundation of China
Zitieren
APA
Zhu, L., Dong, S., Li, J., Huang, T., & Tian, Y. (2023). Ultra-High Temporal Resolution Visual Reconstruction From a Fovea-Like Spike Camera via Spiking Neuron Model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1), 1233-1249. https://doi.org/10.1109/tpami.2022.3146140
Vancouver
Zhu L, Dong S, Li J, Huang T, Tian Y. Ultra-High Temporal Resolution Visual Reconstruction From a Fovea-Like Spike Camera via Spiking Neuron Model. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(1):1233-1249. doi:10.1109/tpami.2022.3146140
BibTeX
@article{Zhu2023,
author = {Zhu, Lin and Dong, Siwei and Li, Jianing and Huang, Tiejun and Tian, Yonghong},
title = {Ultra-High Temporal Resolution Visual Reconstruction From a Fovea-Like Spike Camera via Spiking Neuron Model},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2023},
volume = {45},
number = {1},
pages = {1233--1249},
doi = {10.1109/tpami.2022.3146140}
}
EUVIMED