RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-wave Point Cloud Sequence
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (7) · 5791-5799
DOI: 10.1609/aaai.v40i7.37500 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zengyuan Lai und 8 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref).
Abstract (Originalsprache)
Millimeter-wave radar offers a privacy-preserving and environment-robust alternative to vision-based sensing, enabling human motion analysis in challenging conditions such as low light, occlusions, rain, or smoke. However, its sparse point clouds pose significant challenges for semantic understanding. We present RadarLLM, the first framework that leverages large language models (LLMs) for human motion understanding from radar signals. RadarLLM introduces two key innovations: (1) a motion-guided radar tokenizer based on our Aggregate VQ-VAE architecture, integrating deformable body templates and masked trajectory modeling to convert spatial-temporal radar sequences into compact semantic tokens; and (2) a radar-aware language model that establishes cross-modal alignment between radar and text in a shared embedding space. To overcome the scarcity of paired radar-text data, we generate a realistic radar-text dataset from motion-text datasets with a physics-aware synthesis pipeline. Extensive experiments on both synthetic and real-world benchmarks show that RadarLLM achieves state-of-the-art performance, enabling robust and interpretable motion understanding under privacy and visibility constraints, even in adverse environments.
Zitieren
APA
Lai, Z., Yang, J., Xia, S., Lin, L., Sun, L., Wang, R., Liu, J., Wu, Q., & Pei, L. (2026). RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-wave Point Cloud Sequence. Proceedings of the AAAI Conference on Artificial Intelligence, 40(7), 5791-5799. https://doi.org/10.1609/aaai.v40i7.37500
Vancouver
Lai Z, Yang J, Xia S, Lin L, Sun L, Wang R, et al. RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-wave Point Cloud Sequence. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(7):5791-5799. doi:10.1609/aaai.v40i7.37500
BibTeX
@article{Lai2026,
author = {Lai, Zengyuan and Yang, Jiarui and Xia, Songpengcheng and Lin, Lizhou and Sun, Lan and Wang, Renwen and Liu, Jianran and Wu, Qi and Pei, Ling},
title = {RadarLLM: Empowering Large Language Models to Understand Human Motion from Millimeter-wave Point Cloud Sequence},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {7},
pages = {5791--5799},
doi = {10.1609/aaai.v40i7.37500}
}
EUVIMED