EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Active obstacle separation: Vision-language model (VLM) driven clearing decisions for robotic harvesting

Quan Zhao, Yuanyue Ge, Hui Zhao, Ya Xiong

Smart Agricultural Technology · 2026-08 · 14 · 102037

DOI: 10.1016/j.atech.2026.102037 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Smart Agricultural Technology, 2026-08. Verfasst von Quan Zhao und 3 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: National Natural Science Foundation of China, Beijing Academy of Agricultural and Forestry Sciences.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Severe occlusions frequently limit robotic fruit harvesting, while active obstacle separation methods offer solutions for reaching inaccessible targets. This work presents a VLM-centered perception-understanding-execution framework for active obstacle separation in strawberry harvesting. Synchronized RGB-D sensing produced a target-centric minimal scene graph that encoded spatial and occlusion relations into a short sequence of discrete descriptors explicitly designed as an interface for a VLM. The VLM analyzed this structured scene description and jointly mapped it to a normalized clearing record {action, direction, distance, force}, deciding not only whether and what to clear (push, drag, push-and-drag, or direct pick), but also how to clear via continuous parameters. An executability-aware decoder then aligned frames, standardized units, and synthesized task-space trajectories under minimal-clearance, collision, and safety constraints, turning generic VLM outputs into feasible motions. In physics-based simulations of greenhouse layouts, the framework achieved 89.4% clearing success on a mixed benchmark and up to 92.7% on single-action push scenes, with high run-to-run consistency. On the same mixed benchmark, clearing success increased from 60.0% with a rule-based scene-graph policy and 78.0% with an image-only VLM baseline to 89.2% with our framework, while achieving a mean path length of 0.55 m and an unsafe action rate of 4.8%. Under identical prompts and thresholds, the standardized scene-graph interface transferred across different VLM backends, and GPT-5 yielded the strongest performance among the evaluated models. This work illustrates a promising path toward leveraging VLMs for active obstacle clearing strategies in robotic harvesting.

Förderung

  • National Natural Science Foundation of China
  • Beijing Academy of Agricultural and Forestry Sciences

Zitieren

APA

Zhao, Q., Ge, Y., Zhao, H., & Xiong, Y. (2026). Active obstacle separation: Vision-language model (VLM) driven clearing decisions for robotic harvesting. Smart Agricultural Technology, 14, 102037. https://doi.org/10.1016/j.atech.2026.102037

Vancouver

Zhao Q, Ge Y, Zhao H, Xiong Y. Active obstacle separation: Vision-language model (VLM) driven clearing decisions for robotic harvesting. Smart Agricultural Technology. 2026;14:102037. doi:10.1016/j.atech.2026.102037

BibTeX
@article{Zhao2026,
  author = {Zhao, Quan and Ge, Yuanyue and Zhao, Hui and Xiong, Ya},
  title = {Active obstacle separation: Vision-language model (VLM) driven clearing decisions for robotic harvesting},
  journal = {Smart Agricultural Technology},
  year = {2026},
  volume = {14},
  pages = {102037},
  doi = {10.1016/j.atech.2026.102037}
}
Kostenlos starten →