Evaluation of AI Performance
THE JOURNAL OF JAPAN SOCIETY FOR CLINICAL ANESTHESIA · 15.11.2025 · 45 (6) · 577-585
DOI: 10.2199/jjsca.45.577 ↗Kurz gefasst
Fachartikel in der Zeitschrift THE JOURNAL OF JAPAN SOCIETY FOR CLINICAL ANESTHESIA, erschienen am 15.11.2025. Verfasst von Manabu YOSHIMURA.
Abstract (Originalsprache)
This paper begins by clarifying the definition of artificial intelligence(AI), noting the prevalent perception that AI is a powerful technology capable of accomplishing virtually any task. In the United States and China, model development has reached a saturation point, prompting a shift from mere construction of AI predictive models to more in-depth discussions of their performance and evaluation. In contrast, many studies in Japan continue to focus only on internally validated models. In contrast, external validation is now the norm in international settings, accompanied by the increasing use of randomized controlled trials(RCTs)to determine whether such models truly improve clinical outcomes. Drawing on the TRIPOD-AI and CONSORT-AI statements, we provide a detailed explanation of these evolving standards. Furthermore, although the area under the receiver operating characteristic curve(AUROC)is frequently used to evaluate predictive model performance, recent findings suggest that relying solely on this metric is inadequate. Overemphasis on AUROC or accuracy can lead to “alarm fatigue”, ultimately exhausting healthcare personnel. Despite the proliferation of AI predictive models, few have brought about meaningful changes in clinical practice, underscoring the need to critically examine what truly constitutes a “useful” AI predictive model.
Zitieren
APA
YOSHIMURA, M. (2025). Evaluation of AI Performance. THE JOURNAL OF JAPAN SOCIETY FOR CLINICAL ANESTHESIA, 45(6), 577-585. https://doi.org/10.2199/jjsca.45.577
Vancouver
YOSHIMURA M. Evaluation of AI Performance. THE JOURNAL OF JAPAN SOCIETY FOR CLINICAL ANESTHESIA. 2025;45(6):577-585. doi:10.2199/jjsca.45.577
BibTeX
@article{YOSHIMURA2025,
author = {YOSHIMURA, Manabu},
title = {Evaluation of AI Performance},
journal = {THE JOURNAL OF JAPAN SOCIETY FOR CLINICAL ANESTHESIA},
year = {2025},
volume = {45},
number = {6},
pages = {577--585},
doi = {10.2199/jjsca.45.577}
}
EUVIMED