RadarFormer: End-to-End Human Perception With Through-Wall Radar and Transformers
IEEE Transactions on Neural Networks and Learning Systems · 2024-12 · 35 (12) · 18285-18299
DOI: 10.1109/tnnls.2023.3314031 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Transactions on Neural Networks and Learning Systems, 2024-12. Verfasst von Zhijie Zheng und 4 weiteren Autor:innen. Die Arbeit wurde bisher 38-mal zitiert (Crossref) und nennt 51 Literaturangaben. Gefördert von: National Natural Science Foundation of China, Major State Basic Research Development Program of China.
Abstract (Originalsprache)
For fine-grained human perception tasks such as pose estimation and activity recognition, radar-based sensors show advantages over optical cameras in low-visibility, privacy-aware, and wall-occlusive environments. Radar transmits radio frequency signals to irradiate the target of interest and store the target information in the echo signals. One common approach is to transform the echoes into radar images and extract the features with convolutional neural networks. This article introduces RadarFormer, the first method that introduces the self-attention (SA) mechanism to perform human perception tasks directly from radar echoes. It bypasses the imaging algorithm and realizes end-to-end signal processing. Specifically, we give constructive proof that processing radar echoes using the SA mechanism is at least as expressive as processing radar images using the convolutional layer. On this foundation, we design RadarFormer, which is a Transformer-like model to process radar signals. It benefits from the fast-/slow-time SA mechanism considering the physical characteristics of radar signals. RadarFormer extracts human representations from radar echoes and handles various downstream human perception tasks. The experimental results demonstrate that our method outperforms the state-of-the-art radar-based methods both in performance and computational cost and obtains accurate human perception results even in dark and occlusive environments.
Förderung
- National Natural Science Foundation of China
- Major State Basic Research Development Program of China
Zitieren
APA
Zheng, Z., Zhang, D., Liang, X., Liu, X., & Fang, G. (2024). RadarFormer: End-to-End Human Perception With Through-Wall Radar and Transformers. IEEE Transactions on Neural Networks and Learning Systems, 35(12), 18285-18299. https://doi.org/10.1109/tnnls.2023.3314031
Vancouver
Zheng Z, Zhang D, Liang X, Liu X, Fang G. RadarFormer: End-to-End Human Perception With Through-Wall Radar and Transformers. IEEE Transactions on Neural Networks and Learning Systems. 2024;35(12):18285-18299. doi:10.1109/tnnls.2023.3314031
BibTeX
@article{Zheng2024,
author = {Zheng, Zhijie and Zhang, Diankun and Liang, Xiao and Liu, Xiaojun and Fang, Guangyou},
title = {RadarFormer: End-to-End Human Perception With Through-Wall Radar and Transformers},
journal = {IEEE Transactions on Neural Networks and Learning Systems},
year = {2024},
volume = {35},
number = {12},
pages = {18285--18299},
doi = {10.1109/tnnls.2023.3314031}
}
EUVIMED