The apple detection method based on multimodal features
PLOS One · 2025
DOI: 10.1371/journal.pone.0334911 ↗Kurz gefasst
Fachartikel in der Zeitschrift PLOS One, 2025. Verfasst von Xiaoyang Liu und 9 weiteren Autor:innen. Beteiligte Einrichtung: Faculty of Automation, Huaiyin Institute of Technology, Huaian, China. Hauptthemen laut MEDLINE: Malus, Fruit, Image Processing, Computer-Assisted, Color. Die Arbeit wurde bisher 4-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Natural Science Foundation of China, Natural Science Research Foundation of Higher Education Institutions in Jiangsu Province, the Open Project Program of the Key Laboratory for Agricultural Machinery Intelligent Control and Manufacturing of Fujian Education Institutions.
Abstract (Originalsprache)
Accurate detection of apples and other fruits in complex environments remains a formidable challenge due to the intricate interplay of varying lighting conditions, occlusions, and background clutter. Traditional detection methods, which primarily rely on RGB images or incremental improvement of deep learning models, often fail to achieve satisfactory detection accuracy. To address this, an innovative method of apple detection is proposed to improve the detection performance through multimodal feature fusion rather than radical architectural modifications. The proposed method integrates four complementary modalities: RGB image, color and edge feature maps, depth feature map, and point clouds. Chromatic properties of fruits and geometric boundaries of fruit-tree structures are captured by color and edge feature maps extracted from RGB inputs, which are weighted and fused into a composite feature channel. The depth map and point clouds acquired via binocular active infrared stereo cameras provide additional spatial information. The depth feature image is used as a standalone feature channel. Given the significant modal discrepancies between point clouds and RGB data, a preprocessing pipeline is implemented: voxel sampling and local anomaly detection are first applied to denoise and fill holes in point clouds, followed by recalibrated mapping to ensure spatial alignment with RGB image. The XYZ coordinates of processed point clouds are then used as three distinct feature channels. Finally, the YOLOv5 input layer is redesigned to accept multi-channel feature inputs.Multimodal fusion enriches the feature representation accessible to YOLOv5, enhancing model robustness against lighting variations and background noise. Experimental results demonstrate that the proposed method achieves 95.8% precision (P), 96.0% recall (R), and 95.9% F1-score in complex scenarios. Compared to baseline methods using RGB-only and RGB+depth inputs, precision improvements of 7.4% and 6.3% are observed respectively.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Faculty of Automation, Huaiyin Institute of Technology, Huaian, China.
- The Key Laboratory for Agricultural Machinery Intelligent Control and Manufacturing of Fujian Education Institutions, Wuyishan, China.
- School of Mechanical and Electrical Engineering, Wuyi University, Wuyishan, China.
ORCID-Profile: Xiaoyang Liu
Förderung
- National Natural Science Foundation of China
- Natural Science Research Foundation of Higher Education Institutions in Jiangsu Province
- the Open Project Program of the Key Laboratory for Agricultural Machinery Intelligent Control and Manufacturing of Fujian Education Institutions
- Natural Science Foundation of Fujian Province
Zitieren
APA
Liu, X., Hu, C., Huang, X., Sun, C., Zhu, R., Wang, C., Zhang, Y., Shen, Q., Zhou, H., & Ruan, C. (2025). The apple detection method based on multimodal features. PLOS One. https://doi.org/10.1371/journal.pone.0334911
Vancouver
Liu X, Hu C, Huang X, Sun C, Zhu R, Wang C, et al. The apple detection method based on multimodal features. PLOS One. 2025. doi:10.1371/journal.pone.0334911
BibTeX
@article{Liu2025,
author = {Liu, Xiaoyang and Hu, Chongyang and Huang, Xupeng and Sun, Chenxin and Zhu, Rongjin and Wang, Cheng and Zhang, Yuxiang and Shen, Qian and Zhou, Hongbiao and Ruan, Chengzhi},
title = {The apple detection method based on multimodal features},
journal = {PLOS One},
year = {2025},
doi = {10.1371/journal.pone.0334911}
}
EUVIMED