EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

A Multimodal Large Language Model Framework for Intelligent Perception and Decision-Making in Smart Manufacturing

Tianyu Wang, Bowen Zhang, Daqi Jiang iD, Dong Li iD

Sensors · 13.05.2025 · 25 (10) · 3072

DOI: 10.3390/s25103072 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 13.05.2025. Verfasst von Tianyu Wang und 3 weiteren Autor:innen. Beteiligte Einrichtung: State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China. Die Arbeit wurde bisher 25-mal zitiert (Crossref) und nennt 52 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: National Key Research and Development Program of China, National Industrial Internet Innovation and Development Program Project.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

In modern manufacturing, making accurate and timely decisions requires the ability to effectively handle multiple types of data. This paper presents a multimodal system designed specifically for smart manufacturing applications. The system combines various data sources including images, sensor data, and production records, using advanced multimodal large language models. This approach addresses common limitations of traditional single-modal methods, such as isolated data analysis and poor integration between different data types. Key contributions include a unified method for representing different data types, dynamic semantic tokenization for better data processing, strong alignment strategies across modalities, and a practical two-stage training method involving initial large-scale pretraining and later fine-tuning for specific tasks. Additionally, a novel Transformer-based model is introduced for generating both images and text, significantly improving real-time decision-making capabilities. Experiments on relevant industrial datasets show that this method consistently performs better than current state-of-the-art approaches in tasks like image–text retrieval and visual question answering. The results demonstrate the effectiveness and versatility of the proposed methods, offering important insights and practical solutions to enhance intelligent manufacturing, predictive maintenance, and anomaly detection, thus supporting the development of more efficient and reliable industrial systems.

Schlagwörter

Stichwörter der Autor:innen

Transformer modeldecision-makingmultimodal large language modelsemantic tokenizationsmart manufacturing

Autor:innen und Institutionen

  • State Key Laboratory of Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China.
  • Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang 110016, China.
  • National Frontiers Science Center for Industrial Intelligence and Systems Optimization, Northeastern University, Shenyang 110819, China.

ORCID-Profile: Daqi Jiang, Dong Li

Förderung

  • National Key Research and Development Program of China
  • National Industrial Internet Innovation and Development Program Project

Zitieren

APA

Wang, T., Zhang, B., Jiang, D., & Li, D. (2025). A Multimodal Large Language Model Framework for Intelligent Perception and Decision-Making in Smart Manufacturing. Sensors, 25(10), 3072. https://doi.org/10.3390/s25103072

Vancouver

Wang T, Zhang B, Jiang D, Li D. A Multimodal Large Language Model Framework for Intelligent Perception and Decision-Making in Smart Manufacturing. Sensors. 2025;25(10):3072. doi:10.3390/s25103072

BibTeX
@article{Wang2025,
  author = {Wang, Tianyu and Zhang, Bowen and Jiang, Daqi and Li, Dong},
  title = {A Multimodal Large Language Model Framework for Intelligent Perception and Decision-Making in Smart Manufacturing},
  journal = {Sensors},
  year = {2025},
  volume = {25},
  number = {10},
  pages = {3072},
  doi = {10.3390/s25103072}
}
Kostenlos starten →