AVA: Towards Autonomous Visualization Agents through Visual Perception‐Driven Decision‐Making
Computer Graphics Forum · 2024-06 · 43 (3)
DOI: 10.1111/cgf.15093 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computer Graphics Forum, online erschienen am 10.06.2024. Verfasst von S. Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 17-mal zitiert (Crossref) und nennt 41 Literaturangaben. Gefördert von: Lawrence Livermore National Laboratory, Laboratory Directed Research and Development.
Abstract (Originalsprache)
With recent advances in multi‐modal foundation models, the previously text‐only large language models (LLM) have evolved to incorporate visual input, opening up unprecedented opportunities for various applications in visualization. Compared to existing work on LLM‐based visualization works that generate and control visualization with textual input and output only, the proposed approach explores the utilization of the visual processing ability of multi‐modal LLMs to develop Autonomous Visualization Agents (AVAs) that can evaluate the generated visualization and iterate on the result to accomplish user‐defined objectives defined through natural language. We propose the first framework for the design of AVAs and present several usage scenarios intended to demonstrate the general applicability of the proposed paradigm. Our preliminary exploration and proof‐of‐concept agents suggest that this approach can be widely applicable whenever the choices of appropriate visualization parameters require the interpretation of previous visual output. Our study indicates that AVAs represent a general paradigm for designing intelligent visualization systems that can achieve high‐level visualization goals, which pave the way for developing expert‐level visualization agents in the future.
Förderung
- Lawrence Livermore National Laboratory
- Laboratory Directed Research and Development
Zitieren
APA
Liu, S., Miao, H., Li, Z., Olson, M., Pascucci, V., & Bremer, P. (2024). AVA: Towards Autonomous Visualization Agents through Visual Perception‐Driven Decision‐Making. Computer Graphics Forum, 43(3). https://doi.org/10.1111/cgf.15093
Vancouver
Liu S, Miao H, Li Z, Olson M, Pascucci V, Bremer P. AVA: Towards Autonomous Visualization Agents through Visual Perception‐Driven Decision‐Making. Computer Graphics Forum. 2024;43(3). doi:10.1111/cgf.15093
BibTeX
@article{Liu2024,
author = {Liu, S. and Miao, H. and Li, Z. and Olson, M. and Pascucci, V. and Bremer, P‐T.},
title = {AVA: Towards Autonomous Visualization Agents through Visual Perception‐Driven Decision‐Making},
journal = {Computer Graphics Forum},
year = {2024},
volume = {43},
number = {3},
doi = {10.1111/cgf.15093}
}
EUVIMED