Integrating Heuristic Methods with Deep Reinforcement Learning for Online 3D Bin-Packing Optimization
Sensors · 20.08.2024 · 24 (16) · 5370
DOI: 10.3390/s24165370 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 20.08.2024. Verfasst von Ching-Chang Wong und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical and Computer Engineering, Tamkang University, New Taipei City 25137, Taiwan. Die Arbeit wurde bisher 14-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Science and Technology Council (NSTC) of Taiwan, R.O.C..
Abstract (Originalsprache)
This study proposes a method named Hybrid Heuristic Proximal Policy Optimization (HHPPO) to implement online 3D bin-packing tasks. Some heuristic algorithms for bin-packing and the Proximal Policy Optimization (PPO) algorithm of deep reinforcement learning are integrated to implement this method. In the heuristic algorithms for bin-packing, an extreme point priority sorting method is proposed to sort the generated extreme points according to their waste spaces to improve space utilization. In addition, a 3D grid representation of the space status of the container is used, and some partial support constraints are proposed to increase the possibilities for stacking objects and enhance overall space utilization. In the PPO algorithm, some heuristic algorithms are integrated, and the reward function and the action space of the policy network are designed so that the proposed method can effectively complete the online 3D bin-packing task. Some experimental results illustrate that the proposed method has good results in achieving online 3D bin-packing tasks in some simulation environments. In addition, an environment with image vision is constructed to show that the proposed method indeed enables an actual robot manipulator to successfully and effectively complete the bin-packing task in a real environment.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Electrical and Computer Engineering, Tamkang University, New Taipei City 25137, Taiwan.
ORCID-Profile: Ching-Chang Wong
Förderung
- National Science and Technology Council (NSTC) of Taiwan, R.O.C.
Zitieren
APA
Wong, C. C., Tsai, T. T., & Ou, C. K. (2024). Integrating Heuristic Methods with Deep Reinforcement Learning for Online 3D Bin-Packing Optimization. Sensors, 24(16), 5370. https://doi.org/10.3390/s24165370
Vancouver
Wong CC, Tsai TT, Ou CK. Integrating Heuristic Methods with Deep Reinforcement Learning for Online 3D Bin-Packing Optimization. Sensors. 2024;24(16):5370. doi:10.3390/s24165370
BibTeX
@article{Wong2024,
author = {Wong, Ching-Chang and Tsai, Tai-Ting and Ou, Can-Kun},
title = {Integrating Heuristic Methods with Deep Reinforcement Learning for Online 3D Bin-Packing Optimization},
journal = {Sensors},
year = {2024},
volume = {24},
number = {16},
pages = {5370},
doi = {10.3390/s24165370}
}
EUVIMED