EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning

Haichen Gao, Tianrui Hu, Zeshui Li, Kening Zhu

ACM Transactions on Graphics · 03.07.2026 · 45 (4) · 1-16

DOI: 10.1145/3811349 ↗

Kurz gefasst

Fachartikel in der Zeitschrift ACM Transactions on Graphics, online erschienen am 03.07.2026. Verfasst von Haichen Gao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 68 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Providing haptic feedback in virtual reality (VR) remains challenging, with active haptic systems often being bulky and expensive, while passive haptics typically require pre-fabricated objects or extensive real-world setup. We present ProXeek, an LLM-based multi-agent system that opportunistically leverages real-world objects as haptic proxies for VR experiences. During VR development, the designers could annotate the virtual interactables with emphasized haptic characteristics using our node-based editing interface. On the user side, end users capture environmental snapshots via commodity VR headset, and the system employs LLM-based multimodal reasoning and multi-objective optimization to match virtual objects with viable physical proxies. Our technical evaluation demonstrates that ProXeek effectively identifies viable proxies, confirming the effectiveness of the algorithmic constraint design in our matching pipeline. Our user study further reveals that the ProXeek-selected physical objects could deliver significantly higher haptic fidelity and enhance presence compared to 3D-printed replicas and visual-only interaction.

Zitieren

APA

Gao, H., Hu, T., Li, Z., & Zhu, K. (2026). ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning. ACM Transactions on Graphics, 45(4), 1-16. https://doi.org/10.1145/3811349

Vancouver

Gao H, Hu T, Li Z, Zhu K. ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning. ACM Transactions on Graphics. 2026;45(4):1-16. doi:10.1145/3811349

BibTeX
@article{Gao2026,
  author = {Gao, Haichen and Hu, Tianrui and Li, Zeshui and Zhu, Kening},
  title = {ProXeek: Seeking and Leveraging Real-World Objects and Environments as Haptic Proxies for Virtual Reality through Multimodal Reasoning},
  journal = {ACM Transactions on Graphics},
  year = {2026},
  volume = {45},
  number = {4},
  pages = {1--16},
  doi = {10.1145/3811349}
}
Kostenlos starten →