EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Move to See More: Approaching Object With Partial Occlusion Using Large Multimodal Model and Active Object Detection

Aoqi Wang, Guohui Tian, Yuhao Wang, Zhongyang Li

IET Cyber-Systems and Robotics · 2025-01 · 7 (1)

DOI: 10.1049/csy2.70008 ↗

Kurz gefasst

Fachartikel in der Zeitschrift IET Cyber-Systems and Robotics, online erschienen am 27.03.2025. Verfasst von Aoqi Wang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 30 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Taishan Scholar Project of Shandong Province, National Natural Science Foundation of China, National Key Research and Development Program of China.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Active object detection (AOD) is a crucial task in the field of robotics. A key challenge in household environments for AOD is that the target object is often undetectable due to partial occlusion, which leads to the failure of traditional methods. To address the occlusion problem, this paper first proposes a novel occlusion handling method based on the large multimodal model (LMM). The method utilises an LMM to detect and analyse input RGB images and generates adjustment actions to progressively eliminate occlusion. After the occlusion is handled, an improved AOD method based on a deep Q‐learning network (DQN) is used to complete the task. We introduce an attention mechanism to process image features, enabling the model to focus on critical regions of the input images. Additionally, a new reward function is proposed that comprehensively considers the bounding box of the target object and the robot's distance to the object, along with the actions performed by the robot. Experiments on the dataset and in real‐world scenarios validate the effectiveness of the proposed method in performing AOD tasks under partial occlusion.

Förderung

  • Taishan Scholar Project of Shandong Province
  • National Natural Science Foundation of China
  • National Key Research and Development Program of China

Zitieren

APA

Wang, A., Tian, G., Wang, Y., & Li, Z. (2025). Move to See More: Approaching Object With Partial Occlusion Using Large Multimodal Model and Active Object Detection. IET Cyber-Systems and Robotics, 7(1). https://doi.org/10.1049/csy2.70008

Vancouver

Wang A, Tian G, Wang Y, Li Z. Move to See More: Approaching Object With Partial Occlusion Using Large Multimodal Model and Active Object Detection. IET Cyber-Systems and Robotics. 2025;7(1). doi:10.1049/csy2.70008

BibTeX
@article{Wang2025,
  author = {Wang, Aoqi and Tian, Guohui and Wang, Yuhao and Li, Zhongyang},
  title = {Move to See More: Approaching Object With Partial Occlusion Using Large Multimodal Model and Active Object Detection},
  journal = {IET Cyber-Systems and Robotics},
  year = {2025},
  volume = {7},
  number = {1},
  doi = {10.1049/csy2.70008}
}
Kostenlos starten →