EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Beyond Accuracy: A Critical Review of Evaluation, Clinical Validation, and Safety Frameworks for Agentic AI in Medicine

James C. L. Chow, Kay Li

Information · 01.11.2026 · 17 (11) · 1082

DOI: 10.3390/info17111082 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Information, online erschienen am 01.11.2026. Verfasst von James C. L. Chow und 1 weiteren Autor:innen. Die Arbeit nennt 105 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Canadian Institutes of Health Research Planning and Dissemination Grant—Institute Community Support, Natural Sciences and Engineering Research Council of Canada.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Agentic artificial intelligence (AI) extends conventional predictive and generative AI by enabling systems to pursue goals through reasoning, planning, information retrieval, memory, tool use, and interaction with clinical workflows. These capabilities create new opportunities for medicine but also a distinct evaluation problem: the reliability of an agent cannot be inferred from the accuracy of its underlying model or final answer alone. Errors may arise during information retrieval, tool use, state updates, inter-agent communication, or autonomous action and may propagate across a clinical workflow. This critical review examines approaches for evaluating and clinically validating agentic AI in medicine. Existing frameworks, including TRIPOD+AI, CONSORT-AI, SPIRIT-AI, DECIDE-AI, STARD-AI, FUTURE-AI, and Good Machine Learning Practice, provide important foundations but were not primarily designed for systems with persistent state, repeated actions, tool use, and multi-agent interaction. Emerging agent benchmarks increasingly address interactive tasks, tool use, trajectory reliability, and adversarial testing. We propose a five-level evaluation framework spanning model, agent, trajectory, system, and clinical outcome, together with a clinical validation ladder progressing from component testing and simulation to retrospective evaluation, prospective silent deployment, supervised clinical use, bounded action, and post-deployment surveillance. Key gaps include standardized trajectory metrics, evaluation of correlated multi-agent errors, autonomy-aware validation, human–agent performance, and lifecycle monitoring. Radiotherapy is used as a high-stakes clinical exemplar. We argue that agentic AI should be evaluated as a dynamic sociotechnical system and that clinical evidence must extend beyond accuracy toward behavioral reliability, safety, and demonstrated clinical benefit.

Förderung

  • Canadian Institutes of Health Research Planning and Dissemination Grant—Institute Community Support
  • Natural Sciences and Engineering Research Council of Canada

Zitieren

APA

Chow, J. C. L., & Li, K. (2026). Beyond Accuracy: A Critical Review of Evaluation, Clinical Validation, and Safety Frameworks for Agentic AI in Medicine. Information, 17(11), 1082. https://doi.org/10.3390/info17111082

Vancouver

Chow JCL, Li K. Beyond Accuracy: A Critical Review of Evaluation, Clinical Validation, and Safety Frameworks for Agentic AI in Medicine. Information. 2026;17(11):1082. doi:10.3390/info17111082

BibTeX
@article{Chow2026,
  author = {Chow, James C. L. and Li, Kay},
  title = {Beyond Accuracy: A Critical Review of Evaluation, Clinical Validation, and Safety Frameworks for Agentic AI in Medicine},
  journal = {Information},
  year = {2026},
  volume = {17},
  number = {11},
  pages = {1082},
  doi = {10.3390/info17111082}
}
Kostenlos starten →