From Output to Process: A Case Study of Reasoning Patterns in LLMs for AI Risk Scenario Generation
IEICE Transactions on Information and Systems · 01.05.2026 · E109.D (5) · 683-694
DOI: 10.1587/transinf.2025dkp0006 ↗Kurz gefasst
Fachartikel in der Zeitschrift IEICE Transactions on Information and Systems, erschienen am 01.05.2026. Verfasst von Arisa MOROZUMI und 1 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 43 Literaturangaben.
Abstract (Originalsprache)
Large Language Models (LLMs) are increasingly used for the critical task of generating AI risk scenarios, yet practitioners lack empirical guidance on model selection. This study addresses that gap through a case study benchmarking 23 LLMs against a real-world AI system to analyze their underlying reasoning patterns. We introduce a novel “Hit Rate” metric based on actual incidents to quantitatively measure performance. The results suggest significant, statistically-verified performance disparities among models and show that this gap is uncorrelated with superficial linguistic fluency. Instead, we indicate that the performance gap appears to be strongly linked to the model’s underlying reasoning pattern, which leaves an unmistakable qualitative signature on the final outputs. A “Systematic Top-Down” approach, which mirrors expert human analysis, consistently produces specific and actionable scenarios, while less structured methods yield generic or contextually flawed warnings. These findings serve as a strong caution against model-agnosticism, establishing that an LLM’s reasoning process—suggested by the specificity and actionability of its outputs—is a critical factor for its efficacy in safety-critical tasks.
Zitieren
APA
MOROZUMI, A., & HAYASHI, H. (2026). From Output to Process: A Case Study of Reasoning Patterns in LLMs for AI Risk Scenario Generation. IEICE Transactions on Information and Systems, E109.D(5), 683-694. https://doi.org/10.1587/transinf.2025dkp0006
Vancouver
MOROZUMI A, HAYASHI H. From Output to Process: A Case Study of Reasoning Patterns in LLMs for AI Risk Scenario Generation. IEICE Transactions on Information and Systems. 2026;E109.D(5):683-694. doi:10.1587/transinf.2025dkp0006
BibTeX
@article{MOROZUMI2026,
author = {MOROZUMI, Arisa and HAYASHI, Hisashi},
title = {From Output to Process: A Case Study of Reasoning Patterns in LLMs for AI Risk Scenario Generation},
journal = {IEICE Transactions on Information and Systems},
year = {2026},
volume = {E109.D},
number = {5},
pages = {683--694},
doi = {10.1587/transinf.2025dkp0006}
}
EUVIMED