EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Two-Stage Generate-Then-Reason Vision–Language Framework for Task-Conditioned Grasp Pose Selection

Bo-Kai Peng, Shao-Kang Huang, Chen-Chien Hsu, Wei-Yen Wang

Electronics · 05.11.2026 · 15 (21) · 5137

DOI: 10.3390/electronics15215137 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Electronics, online erschienen am 05.11.2026. Verfasst von Bo-Kai Peng und 3 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 23 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Taiwan Normal University.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Task-conditioned robotic manipulation requires grasps that are geometrically plausible and compatible with downstream function. Although vision–language models (VLMs) have shown promise for semantic reasoning in robotic manipulation, VLM-selected grasps may not consistently satisfy both requirements. To address this issue, we propose a two-stage generate-then-reason framework for task-conditioned 6-DoF grasp selection. In Stage 1, a VLM localizes task-relevant object regions, followed by segmentation and GraspNet-based generation of plausible grasp candidates. In Stage 2, another VLM ranks grounded grasp candidates according to their compatibility with the specified downstream task. The evaluation in this journal includes a 60-trial full-system physical study and four controlled replay experiments that examine Stage 1 localization, Stage 2 cross-model ranking, repeated-inference reliability, and cross-stage candidate-pool sensitivity. The physical system achieved an overall task success rate of 63.3%. In the replay analysis, Gemini 3 Flash Preview achieved the highest Stage 1 localization accuracy of 95.0% and the lowest median Stage 1 latency of 11.09 s. For Stage 2 ranking, GPT-5.6 Sol achieved the highest Top-1 task-valid rate of 92.9% on identical frozen candidate pools and showed the strongest repeated-inference behavior among the evaluated critics. The cross-stage sensitivity experiment yielded task-valid selection rates ranging from 70.8% to 87.5% across six model combinations, indicating sensitivity of downstream selection to upstream candidate evidence within the evaluated conditions. These results indicate that physical task completion and stage-specific VLM behavior capture different aspects of system performance and should be evaluated separately when assessing VLM-based grasp-selection systems.

Förderung

  • National Taiwan Normal University

Zitieren

APA

Peng, B., Huang, S., Hsu, C., & Wang, W. (2026). A Two-Stage Generate-Then-Reason Vision–Language Framework for Task-Conditioned Grasp Pose Selection. Electronics, 15(21), 5137. https://doi.org/10.3390/electronics15215137

Vancouver

Peng B, Huang S, Hsu C, Wang W. A Two-Stage Generate-Then-Reason Vision–Language Framework for Task-Conditioned Grasp Pose Selection. Electronics. 2026;15(21):5137. doi:10.3390/electronics15215137

BibTeX
@article{Peng2026,
  author = {Peng, Bo-Kai and Huang, Shao-Kang and Hsu, Chen-Chien and Wang, Wei-Yen},
  title = {A Two-Stage Generate-Then-Reason Vision–Language Framework for Task-Conditioned Grasp Pose Selection},
  journal = {Electronics},
  year = {2026},
  volume = {15},
  number = {21},
  pages = {5137},
  doi = {10.3390/electronics15215137}
}
Kostenlos starten →