SPOT THE BOT: A comparative quality assessment of AI-generated written instructions for lung ultrasound training
The Ultrasound Journal · 10.04.2026 · 18 (1) · 18495
DOI: 10.5826/tuj.2026.18495 ↗Kurz gefasst
Fachartikel in der Zeitschrift The Ultrasound Journal, online erschienen am 10.04.2026. Verfasst von Adrian Wong und 6 weiteren Autor:innen. Beteiligte Einrichtung: Dept of Intensive Care Medicine, Ng Teng Fong General Hospital, Singapore; Faculty of Medicine, Universiti Malaya, Malaysia. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
The rapid proliferation of artificial intelligence (AI) in medical education has outpaced the development of quality assurance methods for AI-generated content. This study provides the first systematic evaluation of AI-generated instructional materials for lung ultrasound (LUS) training.
Methods
The ATLAS study employed a cross-sectional, multi-rater evaluation design comparing six instruction sources (five AI systems and human-generated content) across ten LUS content sessions. Expert evaluators (n=39) assessed materials using five standardized domains: Medical Accuracy, Evidence Completeness, Clarity, Practical Utility, and Pedagogical Quality. Statistical analysis included Kruskal-Wallis tests and pairwise comparisons with Bonferroni correction.
Results
Significant differences existed between instruction sources (H = 92.582, p < 0.001). Manus AI achieved the highest overall rating (4.55±0.83) and significantly outperformed human instructions in Medical Accuracy (p = 0.0002) and Evidence Completeness (p < 0.001). Gemini AI (3.94±0.97) performed statistically equivalent to human instructions (4.23±1.00). ChatGPT (2.62±1.35) and Meta (1.53±1.02) performed significantly worse than human instructions (p < 0.001). Clarity emerged as the most discriminating criterion with the widest performance range (1.53-4.54).
Conclusions
Certain AI systems can generate high-quality LUS instructional materials that match or exceed human-generated content. However, significant quality variations across AI systems emphasize the critical importance of systematic evaluation before implementation. These findings support cautious but optimistic integration of high-performing AI systems into medical education with appropriate quality assurance measures.
Autor:innen und Institutionen
- Dept of Intensive Care Medicine, Ng Teng Fong General Hospital, Singapore; Faculty of Medicine, Universiti Malaya, Malaysia.
- Dept of Emergency Medicine, Hospital Kuala Lumpur, Malaysia.
- Plain Anaesthesia, United Kingdom.
- Dept of Intensive Care Medicine, Ng Teng Fong General Hospital, Singapore.
- Dept of Emergency Medicine, Faculty of Medicine, Universiti Teknologi MARA, Kuala Lumpur, Malaysia.
- Cardiac Anesthesia and Intensive Care, Ente Ospedaliero Cantonale (EOC), Istituto Cardiocentro Ticino, Università della Svizzera Italiana (USI), Lugano, Switzerland.
ORCID-Profile: Adrian Wong
Zitieren
APA
Wong, A., Roslan, N. L., Ho, S., Tan, R. A., Noor, J., Via, G., & Corradi, F. (2026). SPOT THE BOT: A comparative quality assessment of AI-generated written instructions for lung ultrasound training. The Ultrasound Journal, 18(1), 18495. https://doi.org/10.5826/tuj.2026.18495
Vancouver
Wong A, Roslan NL, Ho S, Tan RA, Noor J, Via G, et al. SPOT THE BOT: A comparative quality assessment of AI-generated written instructions for lung ultrasound training. The Ultrasound Journal. 2026;18(1):18495. doi:10.5826/tuj.2026.18495
BibTeX
@article{Wong2026,
author = {Wong, Adrian and Roslan, Nurul Liana and Ho, Serene and Tan, Rou An and Noor, Julina and Via, Gabriele and Corradi, Francesco},
title = {SPOT THE BOT: A comparative quality assessment of AI-generated written instructions for lung ultrasound training},
journal = {The Ultrasound Journal},
year = {2026},
volume = {18},
number = {1},
pages = {18495},
doi = {10.5826/tuj.2026.18495}
}
EUVIMED