EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques

Pau Mora iD, Clara Garcia, Eugenio Ivorra iD, Mario Ortega, Mariano L Alcañiz

Sensors · 13.06.2024 · 24 (12) · 3837

DOI: 10.3390/s24123837 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 13.06.2024. Verfasst von Pau Mora und 4 weiteren Autor:innen. Beteiligte Einrichtung: Research in Human-Centred Technology University Research Institute, Universitat Politècnica de València, 46022 Valencia, Spain. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 57 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: European Community's Horizon 2020 (FETPROACT-2018- 872 2020), Spanish Ministry of Universities, European Community’s Horizon 2020 (FETPROACT-2018-2020).

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Virtualization plays a critical role in enriching the user experience in Virtual Reality (VR) by offering heightened realism, increased immersion, safer navigation, and newly achievable levels of interaction and personalization, specifically in indoor environments. Traditionally, the creation of virtual content has fallen under one of two broad categories: manual methods crafted by graphic designers, which are labor-intensive and sometimes lack precision; traditional Computer Vision (CV) and Deep Learning (DL) frameworks that frequently result in semi-automatic and complex solutions, lacking a unified framework for both 3D reconstruction and scene understanding, often missing a fully interactive representation of the objects and neglecting their appearance. To address these diverse challenges and limitations, we introduce the Virtual Experience Toolkit (VET), an automated and user-friendly framework that utilizes DL and advanced CV techniques to efficiently and accurately virtualize real-world indoor scenarios. The key features of VET are the use of ScanNotate, a retrieval and alignment tool that enhances the precision and efficiency of its precursor, supported by upgrades such as a preprocessing step to make it fully automatic and a preselection of a reduced list of CAD to speed up the process, and the implementation in a user-friendly and fully automatic Unity3D application that guides the users through the whole pipeline and concludes in a fully interactive and customizable 3D scene. The efficacy of VET is demonstrated using a diversified dataset of virtualized 3D indoor scenarios, supplementing the ScanNet dataset.

Schlagwörter

Stichwörter der Autor:innen

3D scene understandingScanNetindoor scenesscene reconstructionvirtual reality (VR)

Autor:innen und Institutionen

  • Research in Human-Centred Technology University Research Institute, Universitat Politècnica de València, 46022 Valencia, Spain.

ORCID-Profile: Pau Mora, Eugenio Ivorra

Förderung

  • European Community's Horizon 2020 (FETPROACT-2018- 872 2020)
  • Spanish Ministry of Universities
  • European Community’s Horizon 2020 (FETPROACT-2018-2020)
  • University Teacher Training scholarship granted by the Spanish Ministry of Universities

Zitieren

APA

Mora, P., Garcia, C., Ivorra, E., Ortega, M., & Alcañiz, M. L. (2024). Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques. Sensors, 24(12), 3837. https://doi.org/10.3390/s24123837

Vancouver

Mora P, Garcia C, Ivorra E, Ortega M, Alcañiz ML. Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques. Sensors. 2024;24(12):3837. doi:10.3390/s24123837

BibTeX
@article{Mora2024,
  author = {Mora, Pau and Garcia, Clara and Ivorra, Eugenio and Ortega, Mario and Alcañiz, Mariano L},
  title = {Virtual Experience Toolkit: An End-to-End Automated 3D Scene Virtualization Framework Implementing Computer Vision Techniques},
  journal = {Sensors},
  year = {2024},
  volume = {24},
  number = {12},
  pages = {3837},
  doi = {10.3390/s24123837}
}
Kostenlos starten →