EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Socio‐Technical Risk‐Informed AI‐Driven Automation Trustworthiness Evaluation (ST‐RATE): Part 1‐Conceptualization

Muhammad Hammad Khalid iD, Ha Bui, Ahmad Al Rashdan, Zahra Mohaghegh

Risk Analysis · 2026-11 · 46 (11)

DOI: 10.1111/risa.70367 ↗

At a glance

Journal article in Risk Analysis, published online on 2026-10-07. Written by Muhammad Hammad Khalid and 3 co-authors. Institution: Socio-Technical Risk Analysis (SoTeRiA) Laboratory, Urbana, Illinois, USA. Cited 1 times so far (Crossref); 85 references. The full text is freely available (open access, CC BY 4.0). Funded by: U.S. Department of Energy (DOE) through the Nuclear Energy University Program (NEUP).

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

The growing integration of artificial intelligence (AI)‐driven technologies into nuclear power plant (NPP) operations and maintenance holds significant potential to enhance safety, operational efficiency, and reliability. However, deploying such AI‐driven automation technologies in high‐consequence settings requires a rigorous, traceable, and risk‐informed evaluation of their trustworthiness. Although researchers across various domains have proposed definitions and metrics for AI‐driven automation trustworthiness, no agreed‐upon formal or quantitative method exists that is suitable for the highly regulated nuclear sector. To address this gap, this paper, the first in a two‐part series, presents three contributions: (I) a cross‐disciplinary literature review that categorizes existing definitions and evaluation methodologies for AI‐driven automation trustworthiness and identifies their inherent limitations with respect to applicability in the nuclear domain; (II) develops a new definition of AI‐driven automation trustworthiness as “the degree of confidence that the AI‐driven automation system will function as expected across its conditions of use” to address the limitations identified in Contribution I; and (III) building on the new definition, conceptualizes the Socio‐Technical Risk‐informed AI‐driven Automation Trustworthiness Evaluation (ST‐RATE) methodology, grounded in three foundational pillars: (a) risk‐informed performance expectations, (b) socio‐technical context across operational conditions, and (c) uncertainty‐based trustworthiness evaluation. While developed primarily for the nuclear domain, the definition and ST‐RATE methodology has the potential to be adapted for other safety‐critical domains with appropriate modifications. The ST‐RATE methodological framework is operationalized and demonstrated through an AI‐driven automated firewatch case study in a companion paper (Companion paper Part 2, 2025). ST‐RATE is the first‐of‐its‐kind methodology that enables risk‐informed and uncertainty‐based quantification of trustworthiness despite limited empirical data and without over‐reliance on subjective expert judgment, addressing the key limitations of existing approaches identified in the literature review.

Subjects

Author keywords

AI‐driven automation trustworthinessartificial intelligence (AI)nuclear power plantsrisk‐informedsocio‐technicaluncertainty analysis

Authors and institutions

  • Socio-Technical Risk Analysis (SoTeRiA) Laboratory, Urbana, Illinois, USA.
  • Department of Nuclear, Plasma, and Radiological Engineering, University of Illinois at Urbana- Champaign, Urbana, Illinois, USA.
  • Idaho National Laboratory (INL), Idaho Falls, Idaho, USA.

ORCID profiles: Muhammad Hammad Khalid

Funding

  • U.S. Department of Energy (DOE) through the Nuclear Energy University Program (NEUP)

Cite

APA

Khalid, M. H., Bui, H., Rashdan, A. A., & Mohaghegh, Z. (2026). Socio‐Technical Risk‐Informed AI‐Driven Automation Trustworthiness Evaluation (ST‐RATE): Part 1‐Conceptualization. Risk Analysis, 46(11). https://doi.org/10.1111/risa.70367

Vancouver

Khalid MH, Bui H, Rashdan AA, Mohaghegh Z. Socio‐Technical Risk‐Informed AI‐Driven Automation Trustworthiness Evaluation (ST‐RATE): Part 1‐Conceptualization. Risk Analysis. 2026;46(11). doi:10.1111/risa.70367

BibTeX
@article{Khalid2026,
  author = {Khalid, Muhammad Hammad and Bui, Ha and Rashdan, Ahmad Al and Mohaghegh, Zahra},
  title = {Socio‐Technical Risk‐Informed AI‐Driven Automation Trustworthiness Evaluation (ST‐RATE): Part 1‐Conceptualization},
  journal = {Risk Analysis},
  year = {2026},
  volume = {46},
  number = {11},
  doi = {10.1111/risa.70367}
}
Start for free →