Recurrent Neural Networks for Snapshot Compressive Imaging
IEEE Transactions on Pattern Analysis and Machine Intelligence · 01.02.2023 · 45 (2) · 2264-2281
DOI: 10.1109/tpami.2022.3161934 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Pattern Analysis and Machine Intelligence, erschienen am 01.02.2023. Verfasst von Ziheng Cheng und 6 weiteren Autor:innen. Die Arbeit wurde bisher 116-mal zitiert (Crossref) und nennt 77 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Shaanxi Youth Innovation Team Project, Higher Education Discipline Innovation Project.
Abstract (Originalsprache)
Conventional high-speed and spectral imaging systems are expensive and they usually consume a significant amount of memory and bandwidth to save and transmit the high-dimensional data. By contrast, snapshot compressive imaging (SCI), where multiple sequential frames are coded by different masks and then summed to a single measurement, is a promising idea to use a 2-dimensional camera to capture 3-dimensional scenes. In this paper, we consider the reconstruction problem in SCI, i.e., recovering a series of scenes from a compressed measurement. Specifically, the measurement and modulation masks are fed into our proposed network, dubbed BIdirectional Recurrent Neural networks with Adversarial Training (BIRNAT) to reconstruct the desired frames. BIRNAT employs a deep convolutional neural network with residual blocks and self-attention to reconstruct the first frame, based on which a bidirectional recurrent neural network is utilized to sequentially reconstruct the following frames. Moreover, we build an extended BIRNAT-color algorithm for color videos aiming at joint reconstruction and demosaicing. Extensive results on both video and spectral, simulation and real data from three SCI cameras demonstrate the superior performance of BIRNAT.
Förderung
- National Natural Science Foundation of China
- Shaanxi Youth Innovation Team Project
- Higher Education Discipline Innovation Project
- Program for Oversea Talent by Chinese Central Government
- Westlake foundation and Lochn Optics
Zitieren
APA
Cheng, Z., Chen, B., Lu, R., Wang, Z., Zhang, H., Meng, Z., & Yuan, X. (2023). Recurrent Neural Networks for Snapshot Compressive Imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(2), 2264-2281. https://doi.org/10.1109/tpami.2022.3161934
Vancouver
Cheng Z, Chen B, Lu R, Wang Z, Zhang H, Meng Z, et al. Recurrent Neural Networks for Snapshot Compressive Imaging. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2023;45(2):2264-2281. doi:10.1109/tpami.2022.3161934
BibTeX
@article{Cheng2023,
author = {Cheng, Ziheng and Chen, Bo and Lu, Ruiying and Wang, Zhengjue and Zhang, Hao and Meng, Ziyi and Yuan, Xin},
title = {Recurrent Neural Networks for Snapshot Compressive Imaging},
journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence},
year = {2023},
volume = {45},
number = {2},
pages = {2264--2281},
doi = {10.1109/tpami.2022.3161934}
}
EUVIMED