EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

Obscured tree branches segmentation and 3D reconstruction using deep learning and geometrical constraints

Eugene Kok, Xing Wang, Chao Chen

Computers and Electronics in Agriculture · 2023-07 · 210 · 107884

DOI: 10.1016/j.compag.2023.107884 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computers and Electronics in Agriculture, 2023-07. Verfasst von Eugene Kok und 2 weiteren Autor:innen. Die Arbeit wurde bisher 40-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0). Gefördert von: Australian Research Council.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The shortage of agricultural labourers worldwide has left many groups unharvested and wasted, which motivates researchers worldwide to research fruit harvesting robots extensively. One of the major problems in fruit harvesting is to selectively avoid hard obstacles such as tree branches so that more optimal picking positions can be found and more fruits throughout the tree can be harvested. However, tree branches are often obscured in unstructured natural orchards and thus necessary branch reconstruction and recovery are required. The current branch reconstruction and recovery methods for harvesting robots focus on planar reconstruction with few occlusions while the existing 3D tree modelling methods are not optimised for harvesting purposes that require low computational cost and high localisation accuracy. This work presented a novel framework that reconstructs and recovers 3D obscured branches from planar images and depth maps captured by an RGB-D camera. The framework comprises three parts: branch segmentation using Unet++, branch reconstruction using Point2Skeleton and branch recovery using a novel obscured branch recovery (OBR) algorithm. Branch segmentation using Unet++ with InceptionV3 encoder shows the best overall result with IoU and F1-score of 0.6249 and 0.7692 respectively. OBR recovery algorithm achieves average reconstruction accuracy of 0.72. The mean error of the reconstructed total surface and obscured surface using OBR is 18.68 mm and 38.11 mm with a standard deviation of 14.3 mm and 32.64 mm. The result shows that this framework can effectively reconstruct spatial information of visible and obscured branches from a single view image which can potentially be utilised in harvesting robots.

Förderung

  • Australian Research Council

Zitieren

APA

Kok, E., Wang, X., & Chen, C. (2023). Obscured tree branches segmentation and 3D reconstruction using deep learning and geometrical constraints. Computers and Electronics in Agriculture, 210, 107884. https://doi.org/10.1016/j.compag.2023.107884

Vancouver

Kok E, Wang X, Chen C. Obscured tree branches segmentation and 3D reconstruction using deep learning and geometrical constraints. Computers and Electronics in Agriculture. 2023;210:107884. doi:10.1016/j.compag.2023.107884

BibTeX
@article{Kok2023,
  author = {Kok, Eugene and Wang, Xing and Chen, Chao},
  title = {Obscured tree branches segmentation and 3D reconstruction using deep learning and geometrical constraints},
  journal = {Computers and Electronics in Agriculture},
  year = {2023},
  volume = {210},
  pages = {107884},
  doi = {10.1016/j.compag.2023.107884}
}
Kostenlos starten →