EUVIMEDEUVIMED entdecken
Crossref

MSET: Multimodal Semantic-Enhanced Real-World Beam Prediction via Temporal Modeling With Visual Foundation Models

Feixiang Liu, Xiaohui Li, Wenhui Gao, Jiaqing Xiong, Guanchong Niu, Chung Shue Chen

IEEE Internet of Things Journal · 15.05.2026 · 13 (10) · 22130-22148

DOI: 10.1109/jiot.2026.3669716 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Internet of Things Journal, erschienen am 15.05.2026. Verfasst von Feixiang Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 67 Literaturangaben. Gefördert von: National Key Research and Development Program of China, Higher Education Discipline Innovation Project.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

While machine learning (ML) has been explored for beam prediction, many methods remain constrained by single-modality inputs, shallow temporal modeling, and limited robustness to interference and domain shift. We introduce multimodal semantic-enhanced real-world beam prediction via temporal modeling with visual foundation models (MSET), a multimodal framework that couples visual semantics with positional priors in a causal, lightweight design. A visual foundation model (VFM), instantiated as a Swin-Transformer, learns rich spatial and semantic cues from RGB images and segment anything model (SAM)-derived region masks, and a lightweight ResNet-18 student distills this knowledge for efficient inference. Short-horizon dynamics are captured by a causal temporal convolutional network (TCN) with an adaptive receptive field, whose volatility-driven depth gate expands context under motion spikes and contracts it in calm periods. On top of semantic-aware frame embeddings, a temporally aware cross-attention (TACA) aligns original and semantic-enhanced tokens, while a Mamba-conditioned GPS prior, implemented via a selective state-space model (SSM), issues a location-conditioned single query with positional bias to attend the fused tokens. We further extend inference to dense deployments with multiple proximate candidates and address target selection under ambiguity. Experiments on DeepSense 6G dataset show consistent Top- $k$ improvements across single/multitarget and day/night scenarios, indicating reduced sweep reliance and strong generalization under realistic V2I dynamics.

Förderung

  • National Key Research and Development Program of China
  • Higher Education Discipline Innovation Project

Zitieren

APA

Liu, F., Li, X., Gao, W., Xiong, J., Niu, G., & Chen, C. S. (2026). MSET: Multimodal Semantic-Enhanced Real-World Beam Prediction via Temporal Modeling With Visual Foundation Models. IEEE Internet of Things Journal, 13(10), 22130-22148. https://doi.org/10.1109/jiot.2026.3669716

Vancouver

Liu F, Li X, Gao W, Xiong J, Niu G, Chen CS. MSET: Multimodal Semantic-Enhanced Real-World Beam Prediction via Temporal Modeling With Visual Foundation Models. IEEE Internet of Things Journal. 2026;13(10):22130-22148. doi:10.1109/jiot.2026.3669716

BibTeX
@article{Liu2026,
  author = {Liu, Feixiang and Li, Xiaohui and Gao, Wenhui and Xiong, Jiaqing and Niu, Guanchong and Chen, Chung Shue},
  title = {MSET: Multimodal Semantic-Enhanced Real-World Beam Prediction via Temporal Modeling With Visual Foundation Models},
  journal = {IEEE Internet of Things Journal},
  year = {2026},
  volume = {13},
  number = {10},
  pages = {22130--22148},
  doi = {10.1109/jiot.2026.3669716}
}
Kostenlos starten →