Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques
Sensors · 13.06.2024 · 24 (12) · 3837
DOI: 10.3390/s24123837 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 13.06.2024. Verfasst von Pau Mora und 4 weiteren Autor:innen. Beteiligte Einrichtung: Research in Human-Centred Technology University Research Institute, Universitat Politècnica de València, 46022 Valencia, Spain. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Community's Horizon 2020 (FETPROACT-2018- 872 2020), Spanish Ministry of Universities, European Community’s Horizon 2020 (FETPROACT-2018-2020).
Abstract (Originalsprache)
Virtualization plays a critical role in enriching the user experience in Virtual Reality (VR) by offering heightened realism, increased immersion, safer navigation, and newly achievable levels of interaction and personalization, specifically in indoor environments. Traditionally, the creation of virtual content has fallen under one of two broad categories: manual methods crafted by graphic designers, which are labor-intensive and sometimes lack precision; traditional Computer Vision (CV) and Deep Learning (DL) frameworks that frequently result in semi-automatic and complex solutions, lacking a unified framework for both 3D reconstruction and scene understanding, often missing a fully interactive representation of the objects and neglecting their appearance. To address these diverse challenges and limitations, we introduce the Virtual Experience Toolkit (VET), an automated and user-friendly framework that utilizes DL and advanced CV techniques to efficiently and accurately virtualize real-world indoor scenarios. The key features of VET are the use of ScanNotate, a retrieval and alignment tool that enhances the precision and efficiency of its precursor, supported by upgrades such as a preprocessing step to make it fully automatic and a preselection of a reduced list of CAD to speed up the process, and the implementation in a user-friendly and fully automatic Unity3D application that guides the users through the whole pipeline and concludes in a fully interactive and customizable 3D scene. The efficacy of VET is demonstrated using a diversified dataset of virtualized 3D indoor scenarios, supplementing the ScanNet dataset.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Research in Human-Centred Technology University Research Institute, Universitat Politècnica de València, 46022 Valencia, Spain.
ORCID-Profile: Pau Mora, Eugenio Ivorra
Förderung
- European Community's Horizon 2020 (FETPROACT-2018- 872 2020)
- Spanish Ministry of Universities
- European Community’s Horizon 2020 (FETPROACT-2018-2020)
- University Teacher Training scholarship granted by the Spanish Ministry of Universities
Zitieren
APA
Mora, P., Garcia, C., Ivorra, E., Ortega, M., & Alcañiz, M. L. (2024). Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques. Sensors, 24(12), 3837. https://doi.org/10.3390/s24123837
Vancouver
Mora P, Garcia C, Ivorra E, Ortega M, Alcañiz ML. Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques. Sensors. 2024;24(12):3837. doi:10.3390/s24123837
BibTeX
@article{Mora2024,
author = {Mora, Pau and Garcia, Clara and Ivorra, Eugenio and Ortega, Mario and Alcañiz, Mariano L},
title = {Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques},
journal = {Sensors},
year = {2024},
volume = {24},
number = {12},
pages = {3837},
doi = {10.3390/s24123837}
}
EUVIMED