EUVIMEDEUVIMED entdecken
Querschnittstudie

Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study

Hassan Tarek Hakam iD, Nicolas Vuillerme iD, Felix Muehlensiepen iD, Nikolai Ramadanov iD, Jonathan Lettner iD, Marko Ostojic iD, Roland Becker iD, Robert Prill iD

Journal of Medical Internet Research · 13.11.2026 · 28 · e91095

DOI: 10.2196/91095 ↗

Kurz gefasst

Querschnittstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 13.11.2026. Verfasst von Hassan Tarek Hakam und 7 weiteren Autor:innen. Beteiligte Einrichtung: Center of Orthopaedics and Trauma Surgery, Universitätsklinikum Brandenburg an der Havel, Brandenburg an der Havel, Brandenburg, Germany. Hauptthemen laut MEDLINE: Physicians, Emergency Medicine, Abstracting and Indexing, Artificial Intelligence. Die Arbeit nennt 114 Literaturangaben.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are increasingly used in academic writing and can generate linguistically polished but factually incorrect scientific content. Such misinformation may distort evidence appraisal and, if translated into clinical practice, compromise patient care.

Objective

This study aimed to estimate physicians’ likelihood of misperceiving false information embedded in LLM-generated abstracts as true and reporting an intent to use that information in future practice. We also explored which features were associated with perceived credibility and intended clinical action.

Methods

We conducted a cross-sectional, vignette-based survey of physicians working in emergency medicine and trauma surgery across 4 hospitals in Germany. Each participant assessed 4 abstracts generated using GPT-5 Thinking mode, of which 2 contained intentionally false claims supported by plausible explanations and 2 contained accurate claims. The primary outcome was the proportion of physicians reporting an intent to use or act on recommendations contained in abstracts with false claims. Secondary outcomes included perceived accuracy, rejection of accurate claims, and ratings of precision, structure, and grammar.

Results

Twenty physicians provided 80 ratings of 4 abstracts. Overall, physicians reported that they would act on 68% (54/80) of abstracts. Using abstract truth status as the reference, the decision to act had a sensitivity of 78% (31/40), specificity of 43% (17/40), positive predictive value of 57% (31/54), and negative predictive value of 65% (17/26). The positive likelihood ratio was 1.35 (95% CI 0.98-1.85), the negative likelihood ratio was 0.53 (95% CI 0.27-1.04), and the diagnostic odds ratio was 2.55 (95% CI 0.96-6.73). True and false abstracts received similar ratings for perceived accuracy (mean 3.15, SD 0.89 vs mean 2.88, SD 1.04; P=.21), precision (mean 3.15, SD 0.89 vs mean 2.93, SD 1.00; P=.29), structure (mean 3.23, SD 0.77 vs mean 3.20, SD 0.88; P=.89), and grammar (mean 3.40, SD 0.59 vs mean 3.43, SD 0.68; P=.86). Abstracts that physicians intended to act on were rated substantially higher for perceived accuracy, precision, and structure than those they would not act on (all P

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

PhysiciansEmergency MedicineAbstracting and IndexingArtificial IntelligenceIntentionTraumatologyCross-Sectional StudiesHumansLarge Language ModelsGermanyFemaleMale

Stichwörter der Autor:innen

clinical decision-makingcontent analysisemergency medicinelarge language modelsmisinformationsurvey

Autor:innen und Institutionen

  • Center of Orthopaedics and Trauma Surgery, Universitätsklinikum Brandenburg an der Havel, Brandenburg an der Havel, Brandenburg, Germany.
  • Laboratoire d'Informatique de Grenoble (LIG), SANGRIA, National Center for Scientific Research (CNRS), Grenoble Institute of Technology, Université Grenoble Alpes, Grenoble, France.
  • Faculty of Health Sciences, Medizinische Hochschule Brandenburg Theodor Fontane, Brandenburg an der Havel, Brandenburg, Germany.
  • Institut Universitaire de France, Paris, Île-de-France, France.
  • Center for Health Services Research, Medizinische Hochschule Brandenburg Theodor Fontane, Rüdersdorf bei Berlin, Brandenburg, Germany.
  • Corporate Member of Freie Universität Berlin and Humboldt-Universität zu Berlin, Charité - Universitätsmedizin Berlin, Berlin, Berlin, Germany.

ORCID-Profile: Hassan Tarek Hakam, Nicolas Vuillerme, Felix Muehlensiepen, Nikolai Ramadanov, Jonathan Lettner, Marko Ostojic, Roland Becker, Robert Prill

Zitieren

APA

Hakam, H. T., Vuillerme, N., Muehlensiepen, F., Ramadanov, N., Lettner, J., Ostojic, M., Becker, R., & Prill, R. (2026). Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study. Journal of Medical Internet Research, 28, e91095. https://doi.org/10.2196/91095

Vancouver

Hakam HT, Vuillerme N, Muehlensiepen F, Ramadanov N, Lettner J, Ostojic M, et al. Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study. Journal of Medical Internet Research. 2026;28:e91095. doi:10.2196/91095

BibTeX
@article{Hakam2026,
  author = {Hakam, Hassan Tarek and Vuillerme, Nicolas and Muehlensiepen, Felix and Ramadanov, Nikolai and Lettner, Jonathan and Ostojic, Marko and Becker, Roland and Prill, Robert},
  title = {Hallucinated Evidence and Real-World Risk—Physicians’ Intent to Act on AI-Generated Abstracts in Emergency Medicine and Trauma Surgery: Cross-Sectional, Vignette-Based Diagnostic Accuracy Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e91095},
  doi = {10.2196/91095}
}
Kostenlos starten →