ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning
ACM Transactions on Graphics · 03.07.2026 · 45 (4) · 1-16
DOI: 10.1145/3811349 ↗Kurz gefasst
Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 03.07.2026. Verfasst von Haichen Gao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 68 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Providing haptic feedback in virtual reality (VR) remains challenging, with active haptic systems often being bulky and expensive, while passive haptics typically require pre-fabricated objects or extensive real-world setup. We present ProXeek, an LLM-based multi-agent system that opportunistically leverages real-world objects as haptic proxies for VR experiences. During VR development, the designers could annotate the virtual interactables with emphasized haptic characteristics using our node-based editing interface. On the user side, end users capture environmental snapshots via commodity VR headset, and the system employs LLM-based multimodal reasoning and multi-objective optimization to match virtual objects with viable physical proxies. Our technical evaluation demonstrates that ProXeek effectively identifies viable proxies, confirming the effectiveness of the algorithmic constraint design in our matching pipeline. Our user study further reveals that the ProXeek-selected physical objects could deliver significantly higher haptic fidelity and enhance presence compared to 3D-printed replicas and visual-only interaction.
Zitieren
APA
Gao, H., Hu, T., Li, Z., & Zhu, K. (2026). ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning. ACM Transactions on Graphics, 45(4), 1-16. https://doi.org/10.1145/3811349
Vancouver
Gao H, Hu T, Li Z, Zhu K. ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning. ACM Transactions on Graphics. 2026;45(4):1-16. doi:10.1145/3811349
BibTeX
@article{Gao2026,
author = {Gao, Haichen and Hu, Tianrui and Li, Zeshui and Zhu, Kening},
title = {ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning},
journal = {ACM Transactions on Graphics},
year = {2026},
volume = {45},
number = {4},
pages = {1--16},
doi = {10.1145/3811349}
}
EUVIMED