EUVIMEDОткрийте EUVIMED
Open access · CC BY 4.0

Generative AI for Hospital Cybersecurity: A Framework for Evaluating Large Language Models for Planning, Threat Detection, and Incident Response

Ayman Diyab, Ahmad Diyab, Ishaan Dhillon

Machine Learning and Knowledge Extraction · 2026-08-11 · 8 (8) · 237

DOI: 10.3390/make8080237 ↗

At a glance

Journal article in Machine Learning and Knowledge Extraction, published online on 2026-08-11. Written by Ayman Diyab and 2 co-authors. 48 references. The full text is freely available (open access, CC BY 4.0).

РезюмеБиблиографски данниCiteRelated studies

Abstract (original language)

The increasing digitization of healthcare records and growing reliance on interconnected systems have increased hospitals’ exposure to cyber threats, including ransomware, brute-force attacks, and Structured Query Language (SQL) injection. Traditional cybersecurity approaches alone are often insufficient to address these threats, prompting growing interest in artificial intelligence (AI)-based decision-support tools. This paper evaluates the potential of ChatGPT for hospital cybersecurity and incident response while introducing a structured qualitative framework for evaluating Large Language Model (LLM)-generated cybersecurity recommendations in healthcare. Through three progressively designed experiments and a ransomware case study, we evaluate ChatGPT’s role in developing a hospital cybersecurity plan, detecting brute-force login attempts, responding to an SQL injection attack, and managing a ransomware incident. Responses are assessed using five evaluation dimensions: specificity, completeness, technical correctness, feasibility, and alignment with the National Institute of Standards and Technology (NIST) Cybersecurity Framework (CSF), including both explicit mapping and function coverage. The results demonstrate that ChatGPT provides structured, context-aware guidance that aligns well with NIST CSF 2.0 and addresses governance and third-party risks. However, the recommendations also exhibit limitations, including limited operational depth, assumptions about technology and regulatory environments, lack of prioritization for resource-constrained settings, and limited consideration of implementation costs. Overall, the proposed evaluation framework provides a systematic approach for assessing LLM-generated cybersecurity guidance, while the findings indicate that ChatGPT can serve as a valuable decision-support tool that should complement, rather than replace, qualified cybersecurity professionals.

Cite

APA

Diyab, A., Diyab, A., & Dhillon, I. (2026). Generative AI for Hospital Cybersecurity: A Framework for Evaluating Large Language Models for Planning, Threat Detection, and Incident Response. Machine Learning and Knowledge Extraction, 8(8), 237. https://doi.org/10.3390/make8080237

Vancouver

Diyab A, Diyab A, Dhillon I. Generative AI for Hospital Cybersecurity: A Framework for Evaluating Large Language Models for Planning, Threat Detection, and Incident Response. Machine Learning and Knowledge Extraction. 2026;8(8):237. doi:10.3390/make8080237

BibTeX
@article{Diyab2026,
  author = {Diyab, Ayman and Diyab, Ahmad and Dhillon, Ishaan},
  title = {Generative AI for Hospital Cybersecurity: A Framework for Evaluating Large Language Models for Planning, Threat Detection, and Incident Response},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2026},
  volume = {8},
  number = {8},
  pages = {237},
  doi = {10.3390/make8080237}
}
Start for free →