Evaluating the Viability of Foundation Models in Virtual Agent Perception
Computer Animation and Virtual Worlds · 2026-05 · 37 (3)
DOI: 10.1002/cav.70153 ↗Kurz gefasst
Fachartikel in der Zeitschrift Computer Animation and Virtual Worlds, online erschienen am 05.06.2026. Verfasst von Rubens Halbig Montanha und 4 weiteren Autor:innen. Die Arbeit nennt 20 Literaturangaben. Gefördert von: Coordenação de Aperfeiçoamento de Pessoal de Nível Superior, Conselho Nacional de Desenvolvimento Científico e Tecnológico.
Abstract (Originalsprache)
This work presents a comparative study evaluating how recent foundation models, capable of handling various computer vision tasks, perform at simulating a virtual agent's perception and using that perception to navigate the environment and achieve its goals. To achieve this, we selected two recent foundation models well known in the literature and compared them with a recent procedural approach by replicating these models' evaluation methods. We employed these models for evacuating a burning building, aiming to identify either exit signs or people to follow in the environment to find an escape route. Our results show that out‐of‐the‐box foundation models were unable to outperform a specialized procedural method and performed significantly worse on the task, although they still exhibit strong adaptability to previously unknown contexts. However, among the two foundation models tested, we found that a VLM capable of image reasoning and detection performed significantly better than a self‐supervised generalized computer vision model.
Förderung
- Coordenação de Aperfeiçoamento de Pessoal de Nível Superior
- Conselho Nacional de Desenvolvimento Científico e Tecnológico
Zitieren
APA
Montanha, R. H., Passos, A. C. A., Melgaré, J. K., Araújo, V. F. D. A., & Musse, S. R. (2026). Evaluating the Viability of Foundation Models in Virtual Agent Perception. Computer Animation and Virtual Worlds, 37(3). https://doi.org/10.1002/cav.70153
Vancouver
Montanha RH, Passos ACA, Melgaré JK, Araújo VFDA, Musse SR. Evaluating the Viability of Foundation Models in Virtual Agent Perception. Computer Animation and Virtual Worlds. 2026;37(3). doi:10.1002/cav.70153
BibTeX
@article{Montanha2026,
author = {Montanha, Rubens Halbig and Passos, Ana Carolina Andrade and Melgaré, Júlia Kubiak and Araújo, Victor Flávio de Andrade and Musse, Soraia Raupp},
title = {Evaluating the Viability of Foundation Models in Virtual Agent Perception},
journal = {Computer Animation and Virtual Worlds},
year = {2026},
volume = {37},
number = {3},
doi = {10.1002/cav.70153}
}
EUVIMED