EUVIMEDEUVIMED entdecken
Open Access

SPOT THE BOT: A comparative quality assessment of AI-generated written instructions for lung ultrasound training

Adrian Wong iD, Nurul Liana Roslan, Serene Ho, Rou An Tan, Julina Noor, Gabriele Via, Francesco Corradi

The Ultrasound Journal · 10.04.2026 · 18 (1) · 18495

DOI: 10.5826/tuj.2026.18495 ↗

Kurz gefasst

Fachartikel in der Zeitschrift The Ultrasound Journal, online erschienen am 10.04.2026. Verfasst von Adrian Wong und 6 weiteren Autor:innen. Beteiligte Einrichtung: Dept of Intensive Care Medicine, Ng Teng Fong General Hospital, Singapore; Faculty of Medicine, Universiti Malaya, Malaysia. Die Arbeit nennt 25 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

The rapid proliferation of artificial intelligence (AI) in medical education has outpaced the development of quality assurance methods for AI-generated content. This study provides the first systematic evaluation of AI-generated instructional materials for lung ultrasound (LUS) training.

Methods

The ATLAS study employed a cross-sectional, multi-rater evaluation design comparing six instruction sources (five AI systems and human-generated content) across ten LUS content sessions. Expert evaluators (n=39) assessed materials using five standardized domains: Medical Accuracy, Evidence Completeness, Clarity, Practical Utility, and Pedagogical Quality. Statistical analysis included Kruskal-Wallis tests and pairwise comparisons with Bonferroni correction.

Results

Significant differences existed between instruction sources (H = 92.582, p < 0.001). Manus AI achieved the highest overall rating (4.55±0.83) and significantly outperformed human instructions in Medical Accuracy (p = 0.0002) and Evidence Completeness (p < 0.001). Gemini AI (3.94±0.97) performed statistically equivalent to human instructions (4.23±1.00). ChatGPT (2.62±1.35) and Meta (1.53±1.02) performed significantly worse than human instructions (p < 0.001). Clarity emerged as the most discriminating criterion with the widest performance range (1.53-4.54).

Conclusions

Certain AI systems can generate high-quality LUS instructional materials that match or exceed human-generated content. However, significant quality variations across AI systems emphasize the critical importance of systematic evaluation before implementation. These findings support cautious but optimistic integration of high-performing AI systems into medical education with appropriate quality assurance measures.

Autor:innen und Institutionen

  • Dept of Intensive Care Medicine, Ng Teng Fong General Hospital, Singapore; Faculty of Medicine, Universiti Malaya, Malaysia.
  • Dept of Emergency Medicine, Hospital Kuala Lumpur, Malaysia.
  • Plain Anaesthesia, United Kingdom.
  • Dept of Intensive Care Medicine, Ng Teng Fong General Hospital, Singapore.
  • Dept of Emergency Medicine, Faculty of Medicine, Universiti Teknologi MARA, Kuala Lumpur, Malaysia.
  • Cardiac Anesthesia and Intensive Care, Ente Ospedaliero Cantonale (EOC), Istituto Cardiocentro Ticino, Università della Svizzera Italiana (USI), Lugano, Switzerland.

ORCID-Profile: Adrian Wong

Zitieren

APA

Wong, A., Roslan, N. L., Ho, S., Tan, R. A., Noor, J., Via, G., & Corradi, F. (2026). SPOT THE BOT: A comparative quality assessment of AI-generated written instructions for lung ultrasound training. The Ultrasound Journal, 18(1), 18495. https://doi.org/10.5826/tuj.2026.18495

Vancouver

Wong A, Roslan NL, Ho S, Tan RA, Noor J, Via G, et al. SPOT THE BOT: A comparative quality assessment of AI-generated written instructions for lung ultrasound training. The Ultrasound Journal. 2026;18(1):18495. doi:10.5826/tuj.2026.18495

BibTeX
@article{Wong2026,
  author = {Wong, Adrian and Roslan, Nurul Liana and Ho, Serene and Tan, Rou An and Noor, Julina and Via, Gabriele and Corradi, Francesco},
  title = {SPOT THE BOT: A comparative quality assessment of AI-generated written instructions for lung ultrasound training},
  journal = {The Ultrasound Journal},
  year = {2026},
  volume = {18},
  number = {1},
  pages = {18495},
  doi = {10.5826/tuj.2026.18495}
}
Kostenlos starten →