Hi-Agent: Hierarchical vision-language agents for mobile device control
iFuture · 2026-07
DOI: 10.26599/if.2026.9710002 ↗Kurz gefasst
Fachartikel in der Zeitschrift iFuture, 2026-07. Verfasst von Zhe Wu und 12 weiteren Autor:innen.
Abstract (Originalsprache)
Building agents that autonomously operate mobile devices has attracted increasing attention. While Vision-Language Models (VLMs) show promise, most existing approaches rely on direct state-to-action mappings, which lack structured reason-ing and planning, and thus generalize poorly to novel tasks or unseen UI layouts. We introduce Hi-Agent, a trainable hierarchical vision-language agent for mobile control, featuring a high-level reasoning model and a low-level action model that are jointly optimized. For efficient training, we reformulate multi-step decision-making as a sequence of single-step subgoals and propose a foresight advantage function, which leverages execution feedback from the low-level model to guide high-level optimization. This design alleviates the path explosion issue encountered by Group Relative Policy Optimization (GRPO) in long-horizon tasks and enables stable, critic-free joint training. Hi-Agent achieves a new State-Of-The-Art (SOTA) 87.9% task success rate on the Android-in-the-Wild (AitW) bench-mark, significantly outperforming prior methods across three paradigms: prompt-based (AppAgent: 17.7%), supervised (Filtered BC: 54.5%), and reinforcement learning-based (DigiRL: 71.9%). It also demonstrates competitive zero-shot generalization on the ScreenSpot-v2 benchmark. On the more challenging Android-World benchmark, Hi-Agent also scales effectively with larger backbones, show-ing strong adaptability in high-complexity mobile control scenarios.
Zitieren
APA
Wu, Z., Lu, H., Xing, J., Zhang, C., Li, Y., Zhu, Y., Yang, Y., Jing, Y., Li, K., Shao, K., Hao, J., Wang, J., & Shi, Y. (2026). Hi-Agent: Hierarchical vision-language agents for mobile device control. iFuture. https://doi.org/10.26599/if.2026.9710002
Vancouver
Wu Z, Lu H, Xing J, Zhang C, Li Y, Zhu Y, et al. Hi-Agent: Hierarchical vision-language agents for mobile device control. iFuture. 2026. doi:10.26599/if.2026.9710002
BibTeX
@article{Wu2026,
author = {Wu, Zhe and Lu, Hongjin and Xing, Junliang and Zhang, Changhao and Li, Yuxuan and Zhu, Yin and Yang, Yuhao and Jing, Yuheng and Li, Kai and Shao, Kun and Hao, Jianye and Wang, Jun and Shi, Yuanchun},
title = {Hi-Agent: Hierarchical vision-language agents for mobile device control},
journal = {iFuture},
year = {2026},
doi = {10.26599/if.2026.9710002}
}
EUVIMED