EUVIMEDEUVIMED entdecken
Crossref

Segmentation, Reconstruction, and Grasp‐Pose Estimation Using an RGB‐D Camera for Robotic Apple Harvesting

Tantan Jin, Xiongzhe Han, Haetnim Jeong, Jie Guo, Zhao Zhang, Jianqiang Lu

Annals of the New York Academy of Sciences · 2026-11 · 1565 (1)

DOI: 10.1111/nyas.70447 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Annals of the New York Academy of Sciences, online erschienen am 10.11.2026. Verfasst von Tantan Jin und 5 weiteren Autor:innen. Beteiligte Einrichtung: Interdisciplinary Program in Smart Agriculture, College of Agriculture and Life Sciences, Kangwon National University, Chuncheon, Republic of Korea. Hauptthemen laut MEDLINE: Malus, Robotics, Imaging, Three-Dimensional, Fruit. Die Arbeit nennt 37 Literaturangaben. Gefördert von: New Faculty Startup Fund from Seoul National University, Agricultural Policy Support Technology Development Program of the Rural Development Administration, AX-Based Intelligent Agricultural Work Collaboration Industrialization Technology Development Program of the Ministry of Agriculture, Food and Rural Affairs.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate segmentation, three‐dimensional reconstruction, and grasp‐pose estimation of fruits and surrounding structures are essential for robotic harvesting in complex orchard environments. However, occlusions caused by branches, support wires, and irregular canopy structures remain major challenges for perception and manipulation. This study proposes SimAM‐SegFormer‐DeepLabV3+, a lightweight semantic segmentation model for real‐time recognition of apples, branches, and wires. The model integrates a SegFormer encoder into DeepLabV3+ and incorporates a simple attention module to enhance multiscale feature representation. The segmentation results are fused with RGB‐D data for semantic‐guided 3D reconstruction. Apples are reconstructed using RANSAC‐based sphere fitting, whereas branches and wires are reconstructed using PCA‐guided cylinder fitting. Based on the reconstructed scene, a grasp pose estimation strategy is developed to generate obstacle‐aware approach vectors while satisfying end‐effector constraints. Experimental results showed that SimAM‐SegFormer‐DeepLabV3+ achieved a mean IoU of 88.6% and a mean pixel accuracy of 95.1%. Reconstruction count accuracy reached 96.4%, 90.0%, and 88.2% for apples, branches, and wires, respectively. The mean dimensionless grasp‐direction error was below 15 in three directions. In grasping experiments, the system achieved success rates of 92% and 82% in simple and complex indoor scenes, respectively. This study provides a reference for vision‐based apple harvesting using agricultural robotic arms.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

MalusRoboticsImaging, Three-DimensionalFruitAlgorithms

Stichwörter der Autor:innen

apple harvestinggrasp‐pose estimationorchard systemssemantic segmentationthree‐dimensional reconstruction

Autor:innen und Institutionen

  • Interdisciplinary Program in Smart Agriculture, College of Agriculture and Life Sciences, Kangwon National University, Chuncheon, Republic of Korea.
  • Department of Smart Agricultural Systems Science, Seoul National University, Seoul, Republic of Korea.
  • Department of Biosystems Engineering, Seoul National University, Seoul, Republic of Korea.
  • Research Institute of Agriculture and Life Sciences, Seoul National University, Seoul, Republic of Korea.
  • Horticultural Research Division, Gangwon Agricultural Research & Extension Services, Chuncheon, Republic of Korea.
  • School of Mechanical and Electrical Engineering, Hainan University, Haikou, China.

Förderung

  • New Faculty Startup Fund from Seoul National University
  • Agricultural Policy Support Technology Development Program of the Rural Development Administration
  • AX-Based Intelligent Agricultural Work Collaboration Industrialization Technology Development Program of the Ministry of Agriculture, Food and Rural Affairs
  • Xiongzhe Han

Zitieren

APA

Jin, T., Han, X., Jeong, H., Guo, J., Zhang, Z., & Lu, J. (2026). Segmentation, Reconstruction, and Grasp‐Pose Estimation Using an RGB‐D Camera for Robotic Apple Harvesting. Annals of the New York Academy of Sciences, 1565(1). https://doi.org/10.1111/nyas.70447

Vancouver

Jin T, Han X, Jeong H, Guo J, Zhang Z, Lu J. Segmentation, Reconstruction, and Grasp‐Pose Estimation Using an RGB‐D Camera for Robotic Apple Harvesting. Annals of the New York Academy of Sciences. 2026;1565(1). doi:10.1111/nyas.70447

BibTeX
@article{Jin2026,
  author = {Jin, Tantan and Han, Xiongzhe and Jeong, Haetnim and Guo, Jie and Zhang, Zhao and Lu, Jianqiang},
  title = {Segmentation, Reconstruction, and Grasp‐Pose Estimation Using an RGB‐D Camera for Robotic Apple Harvesting},
  journal = {Annals of the New York Academy of Sciences},
  year = {2026},
  volume = {1565},
  number = {1},
  doi = {10.1111/nyas.70447}
}
Kostenlos starten →