EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Multi-Hardware Benchmarking of Open-Source Large Language Models with Retrieval-Augmented Generation for Mitsubishi FX-Series PLC Instruction List Code Generation

Ming-Feng Yeh, Ching-Chuan Luo, Cheng-Lin Lu

Sensors · 05.06.2026 · 26 (11) · 3602

DOI: 10.3390/s26113602 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Sensors, online erschienen am 05.06.2026. Verfasst von Ming-Feng Yeh und 2 weiteren Autor:innen. Beteiligte Einrichtung: Department of Electrical Engineering, Lunghwa University of Science and Technology, Taoyuan 333326, Taiwan. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 28 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: the National Science and Technology Council (NSTC), National Science and Technology Council.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Smart manufacturing relies on programmable logic controllers (PLCs) that translate sensor inputs into actuator commands. Generating PLC programs in legacy textual languages such as Mitsubishi FX-series Instruction List (IL) remains an expert-only task, and IL’s deprecation in IEC 61131-3 Edition 3.0 leaves it under-represented in the corpora that train modern large language models (LLMs). We benchmark ten open-source LLMs (five vendors, 7B–122B parameters) in both LLM-only and Retrieval-Augmented Generation (RAG) configurations on a frozen 285-question dataset; the pipeline uses ChromaDB with all-MiniLM-L6-v2 embeddings and Maximal Marginal Relevance (MMR) retrieval (k=3, λ=0.5). To move beyond lexical similarity we introduce a three-tier static syntax checker (Lexical/Syntactic/Semantic) calibrated against a 93.3% ground-truth pass rate. RAG raises the syntactic pass rate by +6.7 to +61.1 percentage points across all ten models; the best configuration, qwen3.5:122b with RAG, reaches 95.8%, exceeding the ground-truth baseline. Two outliers (llama3.3:70b at +6.7 pp, gpt-oss:120b at +25.6 pp) are reported rather than excluded. The results indicate that for deprecated-but-deployed industrial languages a curated dialect corpus paired with a locally-hosted open-source LLM is more effective than scaling raw model size, supporting reproducible, on-premise industrial-monitoring and code-generation tooling for sustainable smart manufacturing.

Schlagwörter

Stichwörter der Autor:innen

Instruction ListMitsubishi FX-seriesindustrial monitoringlarge language modelon-premise AI for industrial automationprogrammable logic controllerretrieval-augmented generationsmart manufacturingstatic syntax verificationsustainable manufacturing

Autor:innen und Institutionen

  • Department of Electrical Engineering, Lunghwa University of Science and Technology, Taoyuan 333326, Taiwan.
  • Department of Electrical Engineering, Ming Chi University of Technology, New Taipei 243303, Taiwan.

Förderung

  • the National Science and Technology Council (NSTC)
  • National Science and Technology Council

Zitieren

APA

Yeh, M. F., Luo, C. C., & Lu, C. L. (2026). Multi-Hardware Benchmarking of Open-Source Large Language Models with Retrieval-Augmented Generation for Mitsubishi FX-Series PLC Instruction List Code Generation. Sensors, 26(11), 3602. https://doi.org/10.3390/s26113602

Vancouver

Yeh MF, Luo CC, Lu CL. Multi-Hardware Benchmarking of Open-Source Large Language Models with Retrieval-Augmented Generation for Mitsubishi FX-Series PLC Instruction List Code Generation. Sensors. 2026;26(11):3602. doi:10.3390/s26113602

BibTeX
@article{Yeh2026,
  author = {Yeh, Ming-Feng and Luo, Ching-Chuan and Lu, Cheng-Lin},
  title = {Multi-Hardware Benchmarking of Open-Source Large Language Models with Retrieval-Augmented Generation for Mitsubishi FX-Series PLC Instruction List Code Generation},
  journal = {Sensors},
  year = {2026},
  volume = {26},
  number = {11},
  pages = {3602},
  doi = {10.3390/s26113602}
}
Kostenlos starten →