EUVIMEDEUVIMED entdecken
Crossref

History-Aware Reasoning for GUI Agents

Ziwei Wang, Leyang Yang, Xiaoxuan Tang, Sheng Zhou, Dajun Chen, Wei Jiang, Yong Li

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (43) · 36448-36456

DOI: 10.1609/aaai.v40i43.40966 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Ziwei Wang und 6 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Advances in Multimodal Large Language Models have significantly enhanced Graphical User Interface (GUI) automation. Equipping GUI agents with reliable episodic reasoning capabilities is essential for bridging the gap between users’ concise task descriptions and the complexities of real-world execution. Current methods integrate Reinforcement Learning (RL) with System-2 Chain-of-Thought, yielding notable gains in reasoning enhancement. For long-horizon GUI tasks, historical interactions connect each screen to the goal-oriented episode chain, and effectively leveraging these clues is crucial for the current decision. However, existing native GUI agents exhibit weak short-term memory in their explicit reasoning, interpreting the chained interactions as discrete screen understanding, i.e., unawareness of the historical interactions within the episode. This history-agnostic reasoning challenges their performance in GUI automation. To alleviate this weakness, we propose a History-Aware Reasoning (HAR) framework, which encourages an agent to reflect on its own errors and acquire episodic reasoning knowledge from them via tailored strategies that enhance short-term memory in long-horizon interaction. The framework mainly comprises constructing a reflective learning scenario, synthesizing tailored correction guidelines, and designing a hybrid RL reward function. Using the HAR framework, we develop a native end-to-end model, HAR-GUI-3B, which alters the inherent reasoning mode from history-agnostic to history-aware, equipping the GUI agent with stable short-term memory and reliable perception of screen details. Comprehensive evaluations across a range of GUI-related benchmarks demonstrate the effectiveness and generalization of our method.

Zitieren

APA

Wang, Z., Yang, L., Tang, X., Zhou, S., Chen, D., Jiang, W., & Li, Y. (2026). History-Aware Reasoning for GUI Agents. Proceedings of the AAAI Conference on Artificial Intelligence, 40(43), 36448-36456. https://doi.org/10.1609/aaai.v40i43.40966

Vancouver

Wang Z, Yang L, Tang X, Zhou S, Chen D, Jiang W, et al. History-Aware Reasoning for GUI Agents. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(43):36448-36456. doi:10.1609/aaai.v40i43.40966

BibTeX
@article{Wang2026,
  author = {Wang, Ziwei and Yang, Leyang and Tang, Xiaoxuan and Zhou, Sheng and Chen, Dajun and Jiang, Wei and Li, Yong},
  title = {History-Aware Reasoning for GUI Agents},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {43},
  pages = {36448--36456},
  doi = {10.1609/aaai.v40i43.40966}
}
Kostenlos starten →