EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

GPTArm: An Autonomous Task Planning Manipulator Grasping System Based on Vision–Language Models

Jiaqi Zhang, Zinan Wang, Jiaxin Lai, Hongfei Wang

Machines · 19.03.2025 · 13 (3) · 247

DOI: 10.3390/machines13030247 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machines, online erschienen am 19.03.2025. Verfasst von Jiaqi Zhang und 3 weiteren Autor:innen. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The integration of vision–language models (VLMs) with robotic systems represents a transformative advancement in autonomous task planning and execution. However, traditional robotic arms relying on pre-programmed instructions exhibit limited adaptability in dynamic environments and face semantic gaps between perception and execution, hindering their ability to handle complex task demands. This paper introduces GPTArm, an environment-aware robotic arm system driven by GPT-4V, designed to overcome these challenges through hierarchical task decomposition, closed-loop error recovery, and multimodal interaction. The proposed robotic task processing framework (RTPF) integrates real-time visual perception, contextual reasoning, and autonomous strategy planning, enabling robotic arms to interpret natural language commands, decompose user-defined tasks into executable subtasks, and dynamically recover from errors. Experimental evaluations across ten manipulation tasks demonstrate GPTArm’s superior performance, achieving a success rate of up to 91.4% in standardized benchmarks and robust generalization to unseen objects. Leveraging GPT-4V’s reasoning and YOLOv10’s precise small-object localization, the system surpasses existing methods in accuracy and adaptability. Furthermore, GPTArm supports flexible natural language interaction via voice and text, significantly enhancing user experience in human–robot collaboration.

Zitieren

APA

Zhang, J., Wang, Z., Lai, J., & Wang, H. (2025). GPTArm: An Autonomous Task Planning Manipulator Grasping System Based on Vision–Language Models. Machines, 13(3), 247. https://doi.org/10.3390/machines13030247

Vancouver

Zhang J, Wang Z, Lai J, Wang H. GPTArm: An Autonomous Task Planning Manipulator Grasping System Based on Vision–Language Models. Machines. 2025;13(3):247. doi:10.3390/machines13030247

BibTeX
@article{Zhang2025,
  author = {Zhang, Jiaqi and Wang, Zinan and Lai, Jiaxin and Wang, Hongfei},
  title = {GPTArm: An Autonomous Task Planning Manipulator Grasping System Based on Vision–Language Models},
  journal = {Machines},
  year = {2025},
  volume = {13},
  number = {3},
  pages = {247},
  doi = {10.3390/machines13030247}
}
Kostenlos starten →