APUT: Large Language Models as Cross-Modal Consistency Reasoning Engines for Egocentric State Estimation
Journal of Frontiers in Technology and Engineering Integration · 03.06.2026 · 2 (4)
DOI: 10.66393/t13z9817 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Frontiers in Technology and Engineering Integration, online erschienen am 03.06.2026. Verfasst von Haochen Huang.
Abstract (Originalsprache)
Current multimodal large language models (LLMs) excel at passive representation alignment. However, in highly dynamic, partially observable environments (e.g., egocentric vision with severe occlusion and the “cocktail party” effect), standard models suffer from representational collapse. Unobserved targets are frequently dropped from working memory or their states are hallucinated. To address this, the Active-Perception Unified Transformer (APUT) is proposed. Crucially, APUT is not designed as an improved multimodal tracker, but rather as a reasoning-based state estimator. Identity tracking is treated as a dynamic constraint satisfaction problem (CSP). By modeling a structured state matrix through a dedicated encoder, the LLM operates over explicit tokenized world states. Implicit latent tracking collapses under contradictory cross-modal evidence because state representations are entangled, whereas explicit constraint graphs allow for discrete hypothesis elimination. Furthermore, a partial delta-update mechanism and a multiphase training pipeline, ranging from teacher-forced alignment to critic-guided hypothesis ranking, are introduced. Through this architecture, it is demonstrated how robust object permanence emerges from internal logical coherence under supervised structured learning, providing a novel paradigm for embodied state estimation.
Zitieren
APA
Huang, H. (2026). APUT: Large Language Models as Cross-Modal Consistency Reasoning Engines for Egocentric State Estimation. Journal of Frontiers in Technology and Engineering Integration, 2(4). https://doi.org/10.66393/t13z9817
Vancouver
Huang H. APUT: Large Language Models as Cross-Modal Consistency Reasoning Engines for Egocentric State Estimation. Journal of Frontiers in Technology and Engineering Integration. 2026;2(4). doi:10.66393/t13z9817
BibTeX
@article{Huang2026,
author = {Huang, Haochen},
title = {APUT: Large Language Models as Cross-Modal Consistency Reasoning Engines for Egocentric State Estimation},
journal = {Journal of Frontiers in Technology and Engineering Integration},
year = {2026},
volume = {2},
number = {4},
doi = {10.66393/t13z9817}
}
EUVIMED