EUVIMEDEUVIMED entdecken
Crossref

Hi-Agent: Hierarchical vision-language agents for mobile device control

Zhe Wu, Hongjin Lu, Junliang Xing, Changhao Zhang, Yuxuan Li, Yin Zhu, Yuhao Yang, Yuheng Jing, Kai Li, Kun Shao, Jianye Hao, Jun Wang und 1 weitere

iFuture · 2026-07

DOI: 10.26599/if.2026.9710002 ↗

Kurz gefasst

Fachartikel in der Zeitschrift iFuture, 2026-07. Verfasst von Zhe Wu und 12 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Building agents that autonomously operate mobile devices has attracted increasing attention. While Vision-Language Models (VLMs) show promise, most existing approaches rely on direct state-to-action mappings, which lack structured reason-ing and planning, and thus generalize poorly to novel tasks or unseen UI layouts. We introduce Hi-Agent, a trainable hierarchical vision-language agent for mobile control, featuring a high-level reasoning model and a low-level action model that are jointly optimized. For efficient training, we reformulate multi-step decision-making as a sequence of single-step subgoals and propose a foresight advantage function, which leverages execution feedback from the low-level model to guide high-level optimization. This design alleviates the path explosion issue encountered by Group Relative Policy Optimization (GRPO) in long-horizon tasks and enables stable, critic-free joint training. Hi-Agent achieves a new State-Of-The-Art (SOTA) 87.9% task success rate on the Android-in-the-Wild (AitW) bench-mark, significantly outperforming prior methods across three paradigms: prompt-based (AppAgent: 17.7%), supervised (Filtered BC: 54.5%), and reinforcement learning-based (DigiRL: 71.9%). It also demonstrates competitive zero-shot generalization on the ScreenSpot-v2 benchmark. On the more challenging Android-World benchmark, Hi-Agent also scales effectively with larger backbones, show-ing strong adaptability in high-complexity mobile control scenarios.

Zitieren

APA

Wu, Z., Lu, H., Xing, J., Zhang, C., Li, Y., Zhu, Y., Yang, Y., Jing, Y., Li, K., Shao, K., Hao, J., Wang, J., & Shi, Y. (2026). Hi-Agent: Hierarchical vision-language agents for mobile device control. iFuture. https://doi.org/10.26599/if.2026.9710002

Vancouver

Wu Z, Lu H, Xing J, Zhang C, Li Y, Zhu Y, et al. Hi-Agent: Hierarchical vision-language agents for mobile device control. iFuture. 2026. doi:10.26599/if.2026.9710002

BibTeX
@article{Wu2026,
  author = {Wu, Zhe and Lu, Hongjin and Xing, Junliang and Zhang, Changhao and Li, Yuxuan and Zhu, Yin and Yang, Yuhao and Jing, Yuheng and Li, Kai and Shao, Kun and Hao, Jianye and Wang, Jun and Shi, Yuanchun},
  title = {Hi-Agent: Hierarchical vision-language agents for mobile device control},
  journal = {iFuture},
  year = {2026},
  doi = {10.26599/if.2026.9710002}
}
Kostenlos starten →