EUVIMEDEUVIMED entdecken
Crossref

Development of a robotic arm grasping system using generative artificial intelligence

Jin-Siang Shaw, Bo-Rui Zhu

Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science · 2026-07 · 240 (13) · 4755-4762

DOI: 10.1177/09544062261428249 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, online erschienen am 21.03.2026. Verfasst von Jin-Siang Shaw und 1 weiteren Autor:innen. Die Arbeit nennt 22 Literaturangaben. Gefördert von: National Science and Technology Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This study developed a robotic arm grasping system that belongs to the vision-language-action (VLA) model. The model is divided into three parts: language, vision, and action. GPT-3.5-Turbo served as the language model, and YOLOv7 was used for vision—to identify object positions. The system is capable of simulating factory object grasping tasks, accepting natural-language inputs with nonfixed word orders and translating them into system commands using the natural-language model. In addition, the Google speech-recognition technology was integrated with a graphical user interface to achieve diversified input methods. The system checks whether the corresponding objects exist in the gripping area through the vision model and converts the visual points into six parameters for the robotic arm, directing it to move to the expected position to complete the tasks. For the GPT natural-language model, the average matching percentage across all six languages was 97.18% after fine-tuning with both Chinese and English. In the experimental validation of the VLA model, the YOLOv7 vision model achieved an average image recognition accuracy of 95.02% for object detection. Using both voice and text inputs in Chinese, the GPT model achieved a recognition success rate of 91.67%, and the overall system’s object pick-up rate was 94%, based on 94 successful object grasps out of 100 intended targets. Owing to its adaptability to multilingual and diverse inputs, the proposed system can be applied to object grasping tasks in environments such as factories. The novelty of this work lies in its practical, system-level integration of a multilingual natural-language interface, real-time vision-based object detection, and robotic grasp execution within a modular vision–language–action (VLA) framework for industrial object manipulation.

Förderung

  • National Science and Technology Council

Zitieren

APA

Shaw, J., & Zhu, B. (2026). Development of a robotic arm grasping system using generative artificial intelligence. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science, 240(13), 4755-4762. https://doi.org/10.1177/09544062261428249

Vancouver

Shaw J, Zhu B. Development of a robotic arm grasping system using generative artificial intelligence. Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science. 2026;240(13):4755-4762. doi:10.1177/09544062261428249

BibTeX
@article{Shaw2026,
  author = {Shaw, Jin-Siang and Zhu, Bo-Rui},
  title = {Development of a robotic arm grasping system using generative artificial intelligence},
  journal = {Proceedings of the Institution of Mechanical Engineers, Part C: Journal of Mechanical Engineering Science},
  year = {2026},
  volume = {240},
  number = {13},
  pages = {4755--4762},
  doi = {10.1177/09544062261428249}
}
Kostenlos starten →