Socio‐Technical Risk‐Informed AI‐Driven Automation Trustworthiness Evaluation (ST‐RATE): Part 1‐Conceptualization
Risk Analysis · 2026-11 · 46 (11)
DOI: 10.1111/risa.70367 ↗At a glance
Journal article in Risk Analysis, published online on 2026-10-07. Written by Muhammad Hammad Khalid and 3 co-authors. Institution: Socio-Technical Risk Analysis (SoTeRiA) Laboratory, Urbana, Illinois, USA. Cited 1 times so far (Crossref); 85 references. The full text is freely available (open access, CC BY 4.0). Funded by: U.S. Department of Energy (DOE) through the Nuclear Energy University Program (NEUP).
Abstract (original language)
The growing integration of artificial intelligence (AI)‐driven technologies into nuclear power plant (NPP) operations and maintenance holds significant potential to enhance safety, operational efficiency, and reliability. However, deploying such AI‐driven automation technologies in high‐consequence settings requires a rigorous, traceable, and risk‐informed evaluation of their trustworthiness. Although researchers across various domains have proposed definitions and metrics for AI‐driven automation trustworthiness, no agreed‐upon formal or quantitative method exists that is suitable for the highly regulated nuclear sector. To address this gap, this paper, the first in a two‐part series, presents three contributions: (I) a cross‐disciplinary literature review that categorizes existing definitions and evaluation methodologies for AI‐driven automation trustworthiness and identifies their inherent limitations with respect to applicability in the nuclear domain; (II) develops a new definition of AI‐driven automation trustworthiness as “the degree of confidence that the AI‐driven automation system will function as expected across its conditions of use” to address the limitations identified in Contribution I; and (III) building on the new definition, conceptualizes the Socio‐Technical Risk‐informed AI‐driven Automation Trustworthiness Evaluation (ST‐RATE) methodology, grounded in three foundational pillars: (a) risk‐informed performance expectations, (b) socio‐technical context across operational conditions, and (c) uncertainty‐based trustworthiness evaluation. While developed primarily for the nuclear domain, the definition and ST‐RATE methodology has the potential to be adapted for other safety‐critical domains with appropriate modifications. The ST‐RATE methodological framework is operationalized and demonstrated through an AI‐driven automated firewatch case study in a companion paper (Companion paper Part 2, 2025). ST‐RATE is the first‐of‐its‐kind methodology that enables risk‐informed and uncertainty‐based quantification of trustworthiness despite limited empirical data and without over‐reliance on subjective expert judgment, addressing the key limitations of existing approaches identified in the literature review.
Subjects
Author keywords
Authors and institutions
- Socio-Technical Risk Analysis (SoTeRiA) Laboratory, Urbana, Illinois, USA.
- Department of Nuclear, Plasma, and Radiological Engineering, University of Illinois at Urbana- Champaign, Urbana, Illinois, USA.
- Idaho National Laboratory (INL), Idaho Falls, Idaho, USA.
ORCID profiles: Muhammad Hammad Khalid
Funding
- U.S. Department of Energy (DOE) through the Nuclear Energy University Program (NEUP)
Cite
APA
Khalid, M. H., Bui, H., Rashdan, A. A., & Mohaghegh, Z. (2026). Socio‐Technical Risk‐Informed AI‐Driven Automation Trustworthiness Evaluation (ST‐RATE): Part 1‐Conceptualization. Risk Analysis, 46(11). https://doi.org/10.1111/risa.70367
Vancouver
Khalid MH, Bui H, Rashdan AA, Mohaghegh Z. Socio‐Technical Risk‐Informed AI‐Driven Automation Trustworthiness Evaluation (ST‐RATE): Part 1‐Conceptualization. Risk Analysis. 2026;46(11). doi:10.1111/risa.70367
BibTeX
@article{Khalid2026,
author = {Khalid, Muhammad Hammad and Bui, Ha and Rashdan, Ahmad Al and Mohaghegh, Zahra},
title = {Socio‐Technical Risk‐Informed AI‐Driven Automation Trustworthiness Evaluation (ST‐RATE): Part 1‐Conceptualization},
journal = {Risk Analysis},
year = {2026},
volume = {46},
number = {11},
doi = {10.1111/risa.70367}
}
EUVIMED