Evaluating large language models for automated design structure matrix extraction from unstructured documents: an empirical study
Proceedings of the Design Society · 2026-08 · 6 · 2433-2442
DOI: 10.1017/pds.2026.10601 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the Design Society, online erschienen am 02.07.2026. Verfasst von Anubhab Majumder und 3 weiteren Autor:innen. Die Arbeit nennt 24 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).
Abstract (Originalsprache)
Design Structure Matrices (DSMs) capture dependencies between system entities and help analyze system complexity, but manually creating them from unstructured documents is time consuming. This work proposes an automated DSM extraction framework using LLMs and RAG with an explicit reasoning step before the LLM determines the presence of a dependency between two system entities. Using a hand-curated dataset, we evaluate three LLM models (GPT-4o-mini, GPT-3.5, and GPT-4o) across six performance metrics and cost.The findings show that reasoning length affects LLM’s DSM extraction performance.
Zitieren
APA
Majumder, A., Parasuram, S., Bhattacharya, K., & Chakrabarti, A. (2026). Evaluating large language models for automated design structure matrix extraction from unstructured documents: an empirical study. Proceedings of the Design Society, 6, 2433-2442. https://doi.org/10.1017/pds.2026.10601
Vancouver
Majumder A, Parasuram S, Bhattacharya K, Chakrabarti A. Evaluating large language models for automated design structure matrix extraction from unstructured documents: an empirical study. Proceedings of the Design Society. 2026;6:2433-2442. doi:10.1017/pds.2026.10601
BibTeX
@article{Majumder2026,
author = {Majumder, Anubhab and Parasuram, Sahana and Bhattacharya, Kausik and Chakrabarti, Amaresh},
title = {Evaluating large language models for automated design structure matrix extraction from unstructured documents: an empirical study},
journal = {Proceedings of the Design Society},
year = {2026},
volume = {6},
pages = {2433--2442},
doi = {10.1017/pds.2026.10601}
}
EUVIMED