EUVIMEDEUVIMED entdecken
Mixed-Methods-StudieOpen Access · CC BY-NC-ND 4.0

Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert‐Like Systems Engineering Artifacts and a Characterization of Failure Modes

Taylan G. Topcu, Mohammed Husain, Max Ofsa, Paul Wach

Systems Engineering · 2025-09 · 28 (5) · 583-604

DOI: 10.1002/sys.21810 ↗

Kurz gefasst

Mixed-Methods-Studie in der Zeitschrift Systems Engineering, online erschienen am 21.02.2025. Verfasst von Taylan G. Topcu und 3 weiteren Autor:innen. Die Arbeit wurde bisher 11-mal zitiert (Crossref) und nennt 122 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Multi‐purpose large language models (LLMs), a subset of generative artificial intelligence (AI), have recently made significant progress. While expectations for LLMs to assist systems engineering (SE) tasks are paramount; the interdisciplinary and complex nature of systems, along with the need to synthesize deep‐domain knowledge and operational context, raise questions regarding the efficacy of LLMs to generate SE artifacts, particularly given that they are trained using data that is broadly available on the internet. To that end, we present results from an empirical exploration, where a human expert‐generated SE artifact was taken as a benchmark, parsed, and fed into various LLMs through prompt engineering to generate segments of typical SE artifacts. This procedure was applied without any fine‐tuning or calibration to document baseline LLM performance. We then adopted a two‐fold mixed‐methods approach to compare AI generated artifacts against the benchmark. First, we quantitatively compare the artifacts using natural language processing algorithms and find that when prompted carefully, the state‐of‐the‐art algorithms cannot differentiate AI‐generated artifacts from the human‐expert benchmark. Second, we conduct a qualitative deep dive to investigate how they differ in terms of quality. We document that while the two‐material appear very similar, AI generated artifacts exhibit serious failure modes that could be difficult to detect. We characterize these as: premature requirements definition, unsubstantiated numerical estimates, and propensity to overspecify. We contend that this study tells a cautionary tale about why the SE community must be more cautious adopting AI suggested feedback, at least when generated by multi‐purpose LLMs.

Zitieren

APA

Topcu, T. G., Husain, M., Ofsa, M., & Wach, P. (2025). Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert‐Like Systems Engineering Artifacts and a Characterization of Failure Modes. Systems Engineering, 28(5), 583-604. https://doi.org/10.1002/sys.21810

Vancouver

Topcu TG, Husain M, Ofsa M, Wach P. Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert‐Like Systems Engineering Artifacts and a Characterization of Failure Modes. Systems Engineering. 2025;28(5):583-604. doi:10.1002/sys.21810

BibTeX
@article{Topcu2025,
  author = {Topcu, Taylan G. and Husain, Mohammed and Ofsa, Max and Wach, Paul},
  title = {Trust at Your Own Peril: A Mixed Methods Exploration of the Ability of Large Language Models to Generate Expert‐Like Systems Engineering Artifacts and a Characterization of Failure Modes},
  journal = {Systems Engineering},
  year = {2025},
  volume = {28},
  number = {5},
  pages = {583--604},
  doi = {10.1002/sys.21810}
}
Kostenlos starten →