EUVIMEDEUVIMED entdecken
Crossref

RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Users

Suyu Ye, Haojun Shi, Darren Shih, Hyokun Yun, Tanya G. Roosta, Tianmin Shu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (40) · 34441-34449

DOI: 10.1609/aaai.v40i40.40742 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Suyu Ye und 5 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

To achieve successful assistance with long-horizon web-based tasks, AI agents must be able to sequentially follow real-world user instructions over a long period. Unlike existing web-based agent benchmarks, sequential instruction following in the real world poses significant challenges beyond performing a single, clearly defined task. For instance, real-world human instructions can be ambiguous, require different levels of AI assistance, and may evolve over time, reflecting changes in the user's mental state. To address this gap, we introduce RealWebAssist, a novel benchmark designed to evaluate sequential instruction-following in realistic scenarios involving long-horizon interactions with the web, visual GUI grounding, and understanding ambiguous real-world user instructions. RealWebAssist includes a dataset of sequential instructions collected from real-world human users. Each user instructs a web-based assistant to perform a series of tasks on multiple websites. A successful agent must reason about the true intent behind each instruction, keep track of the mental state of the user, understand user-specific routines, and ground the intended tasks to actions on the correct GUI elements. Our experimental results show that state-of-the-art models struggle to understand and ground user instructions, posing critical challenges in following real-world user instructions for long-horizon web assistance.

Zitieren

APA

Ye, S., Shi, H., Shih, D., Yun, H., Roosta, T. G., & Shu, T. (2026). RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Users. Proceedings of the AAAI Conference on Artificial Intelligence, 40(40), 34441-34449. https://doi.org/10.1609/aaai.v40i40.40742

Vancouver

Ye S, Shi H, Shih D, Yun H, Roosta TG, Shu T. RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Users. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(40):34441-34449. doi:10.1609/aaai.v40i40.40742

BibTeX
@article{Ye2026,
  author = {Ye, Suyu and Shi, Haojun and Shih, Darren and Yun, Hyokun and Roosta, Tanya G. and Shu, Tianmin},
  title = {RealWebAssist: A Benchmark for Long-Horizon Web Assistance with Real-World Users},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {40},
  pages = {34441--34449},
  doi = {10.1609/aaai.v40i40.40742}
}
Kostenlos starten →