MSET: Multimodal Semantic-Enhanced Real-World Beam Prediction via Temporal Modeling With Visual Foundation Models
IEEE Internet of Things Journal · 15.05.2026 · 13 (10) · 22130-22148
DOI: 10.1109/jiot.2026.3669716 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEEE Internet of Things Journal, erschienen am 15.05.2026. Verfasst von Feixiang Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: National Key Research and Development Program of China, Higher Education Discipline Innovation Project.
Abstract (Originalsprache)
While machine learning (ML) has been explored for beam prediction, many methods remain constrained by single-modality inputs, shallow temporal modeling, and limited robustness to interference and domain shift. We introduce multimodal semantic-enhanced real-world beam prediction via temporal modeling with visual foundation models (MSET), a multimodal framework that couples visual semantics with positional priors in a causal, lightweight design. A visual foundation model (VFM), instantiated as a Swin-Transformer, learns rich spatial and semantic cues from RGB images and segment anything model (SAM)-derived region masks, and a lightweight ResNet-18 student distills this knowledge for efficient inference. Short-horizon dynamics are captured by a causal temporal convolutional network (TCN) with an adaptive receptive field, whose volatility-driven depth gate expands context under motion spikes and contracts it in calm periods. On top of semantic-aware frame embeddings, a temporally aware cross-attention (TACA) aligns original and semantic-enhanced tokens, while a Mamba-conditioned GPS prior, implemented via a selective state-space model (SSM), issues a location-conditioned single query with positional bias to attend the fused tokens. We further extend inference to dense deployments with multiple proximate candidates and address target selection under ambiguity. Experiments on DeepSense 6G dataset show consistent Top- $k$ improvements across single/multitarget and day/night scenarios, indicating reduced sweep reliance and strong generalization under realistic V2I dynamics.
Förderung
- National Key Research and Development Program of China
- Higher Education Discipline Innovation Project
Zitieren
APA
Liu, F., Li, X., Gao, W., Xiong, J., Niu, G., & Chen, C. S. (2026). MSET: Multimodal Semantic-Enhanced Real-World Beam Prediction via Temporal Modeling With Visual Foundation Models. IEEE Internet of Things Journal, 13(10), 22130-22148. https://doi.org/10.1109/jiot.2026.3669716
Vancouver
Liu F, Li X, Gao W, Xiong J, Niu G, Chen CS. MSET: Multimodal Semantic-Enhanced Real-World Beam Prediction via Temporal Modeling With Visual Foundation Models. IEEE Internet of Things Journal. 2026;13(10):22130-22148. doi:10.1109/jiot.2026.3669716
BibTeX
@article{Liu2026,
author = {Liu, Feixiang and Li, Xiaohui and Gao, Wenhui and Xiong, Jiaqing and Niu, Guanchong and Chen, Chung Shue},
title = {MSET: Multimodal Semantic-Enhanced Real-World Beam Prediction via Temporal Modeling With Visual Foundation Models},
journal = {IEEE Internet of Things Journal},
year = {2026},
volume = {13},
number = {10},
pages = {22130--22148},
doi = {10.1109/jiot.2026.3669716}
}
EUVIMED