A Multimodal Large Language Model Framework for Intelligent Perception and Decision-Making in Smart Manufacturing
Sensors · 13.05.2025 · 25 (10) · 3072
DOI: 10.3390/s25103072 ↗Kurz gefasst
Fachartikel in der Zeitschrift Sensors, online erschienen am 13.05.2025. Verfasst von Tianyu Wang und 3 weiteren Autor:innen. Beteiligte Einrichtung: State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Industrial Internet Innovation and Development Program Project.
Abstract (Originalsprache)
In modern manufacturing, making accurate and timely decisions requires the ability to effectively handle multiple types of data. This paper presents a multimodal system designed specifically for smart manufacturing applications. The system combines various data sources including images, sensor data, and production records, using advanced multimodal large language models. This approach addresses common limitations of traditional single-modal methods, such as isolated data analysis and poor integration between different data types. Key contributions include a unified method for representing different data types, dynamic semantic tokenization for better data processing, strong alignment strategies across modalities, and a practical two-stage training method involving initial large-scale pretraining and later fine-tuning for specific tasks. Additionally, a novel Transformer-based model is introduced for generating both images and text, significantly improving real-time decision-making capabilities. Experiments on relevant industrial datasets show that this method consistently performs better than current state-of-the-art approaches in tasks like image–text retrieval and visual question answering. The results demonstrate the effectiveness and versatility of the proposed methods, offering important insights and practical solutions to enhance intelligent manufacturing, predictive maintenance, and anomaly detection, thus supporting the development of more efficient and reliable industrial systems.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China.
- Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China.
- National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University, Shenyang 110819, China.
ORCID-Profile: Daqi Jiang, Dong Li
Förderung
- National Key Research and Development Program of China
- National Industrial Internet Innovation and Development Program Project
Zitieren
APA
Wang, T., Zhang, B., Jiang, D., & Li, D. (2025). A Multimodal Large Language Model Framework for Intelligent Perception and Decision-Making in Smart Manufacturing. Sensors, 25(10), 3072. https://doi.org/10.3390/s25103072
Vancouver
Wang T, Zhang B, Jiang D, Li D. A Multimodal Large Language Model Framework for Intelligent Perception and Decision-Making in Smart Manufacturing. Sensors. 2025;25(10):3072. doi:10.3390/s25103072
BibTeX
@article{Wang2025,
author = {Wang, Tianyu and Zhang, Bowen and Jiang, Daqi and Li, Dong},
title = {A Multimodal Large Language Model Framework for Intelligent Perception and Decision-Making in Smart Manufacturing},
journal = {Sensors},
year = {2025},
volume = {25},
number = {10},
pages = {3072},
doi = {10.3390/s25103072}
}
EUVIMED