EUVIMEDEUVIMED entdecken
Crossref

RASP: Robot Active Scene Perception With Joint Viewpoint Planning and Depth Completion in Cluttered Environments

Yizhe Liu, Tong Jia, Haiyu Zhang, Guowei Yang, Hao Wang, Dongyue Chen

IEEE Transactions on Automation Science and Engineering · 2025 · 22 · 20611-20623

DOI: 10.1109/tase.2025.3604455 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IEEE Transactions on Automation Science and Engineering, 2025. Verfasst von Yizhe Liu und 5 weiteren Autor:innen. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 41 Literaturangaben. Gefördert von: National Key Research and Development Program of China, National Natural Science Foundation of China, Major Program of National Natural Science Foundation of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Capturing dense visual perception in cluttered environments using depth sensors is crucial for downstream robotics tasks. However, occlusions between objects and unreliable depth data make it very challenging for robots to perceive comprehensive and accurate scene information. To address these issues, we propose a novel robot active scene perception method called RASP, which is composed of two parts. First, we introduce a temporal attention-based view planning algorithm, which actively plans the minimum feasible viewpoint sequence based on latent dependencies in all previous observations to maximize the information perception of the cluttered environments. Subsequently, for the unreliable depth data obtained by the depth sensor, especially caused by transparent and specular objects in the scene, we design a geometry-guided depth completion network that fully utilizes the 3D scene information during the progressive perception process. Specifically, multi-level scene geometric features are extracted and projected into the image space, combining with the image features to guide the depth completion step. These two parts are learned jointly to achieve consistent and accurate results. Extensive experiments demonstrate that our method outperforms state-of-the-art methods. Furthermore, we show that our method significantly improves the performance of downstream grasping tasks. Note to Practitioners—With the development of depth sensing technology, consumer-level depth cameras have been widely used in many robotics and automation tasks. However, in cluttered environments, due to the occlusions between objects, it is difficult for depth cameras to capture comprehensive scene information from a single perspective. In addition, depth sensors often generate unreliable depth data when facing transparent and specular objects. To address these challenges, this article proposes a robot active scene perception method that combines view planning and depth completion. Our view planning strategy actively explores the minimum feasible viewpoint sequence to maximize the acquisition of scene information, while the depth completion network accurately restores unreliable depth information. Experimental results show that our method performs well in cluttered environments containing transparent and specular objects. The proposed method provides high-quality perception input for robots to perform intelligent tasks in practice.

Förderung

  • National Key Research and Development Program of China
  • National Natural Science Foundation of China
  • Major Program of National Natural Science Foundation of China
  • Higher Education Discipline Innovation Project
  • Liaoning Provincial “Selecting the Best Candidates by Opening Competition Mechanism” Science and Technology Program

Zitieren

APA

Liu, Y., Jia, T., Zhang, H., Yang, G., Wang, H., & Chen, D. (2025). RASP: Robot Active Scene Perception With Joint Viewpoint Planning and Depth Completion in Cluttered Environments. IEEE Transactions on Automation Science and Engineering, 22, 20611-20623. https://doi.org/10.1109/tase.2025.3604455

Vancouver

Liu Y, Jia T, Zhang H, Yang G, Wang H, Chen D. RASP: Robot Active Scene Perception With Joint Viewpoint Planning and Depth Completion in Cluttered Environments. IEEE Transactions on Automation Science and Engineering. 2025;22:20611-20623. doi:10.1109/tase.2025.3604455

BibTeX
@article{Liu2025,
  author = {Liu, Yizhe and Jia, Tong and Zhang, Haiyu and Yang, Guowei and Wang, Hao and Chen, Dongyue},
  title = {RASP: Robot Active Scene Perception With Joint Viewpoint Planning and Depth Completion in Cluttered Environments},
  journal = {IEEE Transactions on Automation Science and Engineering},
  year = {2025},
  volume = {22},
  pages = {20611--20623},
  doi = {10.1109/tase.2025.3604455}
}
Kostenlos starten →