EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

The apple detection method based on multimodal features

Xiaoyang Liu iD, Chongyang Hu, Xupeng Huang, Chenxin Sun, Rongjin Zhu, Cheng Wang, Yuxiang Zhang, Qian Shen, Hongbiao Zhou, Chengzhi Ruan

PLOS One · 2025

DOI: 10.1371/journal.pone.0334911 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Xiaoyang Liu und 9 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Automation, Huaiyin Institute of Technology, Huaian, China. Hauptthemen laut MEDLINE: Malus, Fruit, Image Processing, Computer-Assisted, Color. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Research Foundation of Higher Education Institutions in Jiangsu Province, the Open Project Program of the Key Laboratory for Agricultural Machinery Intelligent Control and Manufacturing of Fujian Education Institutions.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Accurate detection of apples and other fruits in complex environments remains a formidable challenge due to the intricate interplay of varying lighting conditions, occlusions, and background clutter. Traditional detection methods, which primarily rely on RGB images or incremental improvement of deep learning models, often fail to achieve satisfactory detection accuracy. To address this, an innovative method of apple detection is proposed to improve the detection performance through multimodal feature fusion rather than radical architectural modifications. The proposed method integrates four complementary modalities: RGB image, color and edge feature maps, depth feature map, and point clouds. Chromatic properties of fruits and geometric boundaries of fruit-tree structures are captured by color and edge feature maps extracted from RGB inputs, which are weighted and fused into a composite feature channel. The depth map and point clouds acquired via binocular active infrared stereo cameras provide additional spatial information. The depth feature image is used as a standalone feature channel. Given the significant modal discrepancies between point clouds and RGB data, a preprocessing pipeline is implemented: voxel sampling and local anomaly detection are first applied to denoise and fill holes in point clouds, followed by recalibrated mapping to ensure spatial alignment with RGB image. The XYZ coordinates of processed point clouds are then used as three distinct feature channels. Finally, the YOLOv5 input layer is redesigned to accept multi-channel feature inputs.Multimodal fusion enriches the feature representation accessible to YOLOv5, enhancing model robustness against lighting variations and background noise. Experimental results demonstrate that the proposed method achieves 95.8% precision (P), 96.0% recall (R), and 95.9% F1-score in complex scenarios. Compared to baseline methods using RGB-only and RGB+depth inputs, precision improvements of 7.4% and 6.3% are observed respectively.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

MalusFruitImage Processing, Computer-AssistedColorEnvironmentDeep LearningComputer SimulationAlgorithmsRemote Sensing TechnologyAgriculture

Autor:innen und Institutionen

  • Faculty of Automation, Huaiyin Institute of Technology, Huaian, China.
  • The Key Laboratory for Agricultural Machinery Intelligent Control and Manufacturing of Fujian Education Institutions, Wuyishan, China.
  • School of Mechanical and Electrical Engineering, Wuyi University, Wuyishan, China.

ORCID-Profile: Xiaoyang Liu

Förderung

  • National Natural Science Foundation of China
  • Natural Science Research Foundation of Higher Education Institutions in Jiangsu Province
  • the Open Project Program of the Key Laboratory for Agricultural Machinery Intelligent Control and Manufacturing of Fujian Education Institutions
  • Natural Science Foundation of Fujian Province

Zitieren

APA

Liu, X., Hu, C., Huang, X., Sun, C., Zhu, R., Wang, C., Zhang, Y., Shen, Q., Zhou, H., & Ruan, C. (2025). The apple detection method based on multimodal features. PLOS One. https://doi.org/10.1371/journal.pone.0334911

Vancouver

Liu X, Hu C, Huang X, Sun C, Zhu R, Wang C, et al. The apple detection method based on multimodal features. PLOS One. 2025. doi:10.1371/journal.pone.0334911

BibTeX
@article{Liu2025,
  author = {Liu, Xiaoyang and Hu, Chongyang and Huang, Xupeng and Sun, Chenxin and Zhu, Rongjin and Wang, Cheng and Zhang, Yuxiang and Shen, Qian and Zhou, Hongbiao and Ruan, Chengzhi},
  title = {The apple detection method based on multimodal features},
  journal = {PLOS One},
  year = {2025},
  doi = {10.1371/journal.pone.0334911}
}
Kostenlos starten →