EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

No significant differences between large language model prompting strategies and human experts in summarisation of ‘the formal EU–US meniscus rehabilitation consensus: An ESSKA‐AOSSM‐AASPT Initiative’

Keine signifikanten Unterschiede zwischen Prompting-Strategien großer Sprachmodelle und menschlichen Expert:innen bei der Zusammenfassung der formellen EU–US-Knöchelrehabilitations-Consensus: eine ESSKA‐AOSSM‐AASPT-Initiative

Jakob Lindskog iD, Felix Öttl iD, Yinan Yu iD, Mark V Paterno iD, Airelle Giordano iD, Alexandre Rambaud iD, Robert Prill iD, Aleksandra Królikowska iD, Kristian Samuelsson iD, Eric Hamrin Senorski iD

Journal of Experimental Orthopaedics · 2026-10 · 13 (4)

DOI: 10.1002/jeo2.70964 ↗

Auf einen Blick

In einer explorativen Vergleichsstudie unterschieden sich ChatGPT, Claude, Gemini und von Expert:innen verfasste Zusammenfassungen nicht signifikant in der Bewertung. Die Ratings waren insgesamt hoch.

Fragestellung

Unterscheiden sich Prompting-Strategien großer Sprachmodelle und von Menschen verfasste Ausgaben bei der Zusammenfassung und Anwendung des EU–US-Knöchelrehabilitations-Consensus?

Methode

Explorative Vergleichsstudie zu zwei Teilen des Konsensus; drei Bedingungen: Single-Pass, sequenzielles Chain-of-Thought und menschliche Expert:innen. Bewertung durch 3 verblindete Physiotherapeut:innen; Friedman- und Wilcoxon-Tests; Gwet's AC2.

Ergebnisse

Keine statistisch signifikanten Unterschiede zwischen LLM-Prompting und Expert:innenausgaben für Zusammenfassungen, klinische Anwendung und Fehlinterpretationsresistenz. Interrater-Übereinstimmung: Gwet's AC2 0,79–0,99.

Schlussfolgerung der Autor:innen

Zwischen ChatGPT, Claude, Gemini und Expert:innentexten wurden keine signifikanten Unterschiede beobachtet; die Autor:innen sehen LLMs als mögliche Ergänzung für die Zusammenfassung und Interpretation von Empfehlungen.

Kernaussagen

  • Keine signifikanten Unterschiede in den Reviewer-Ratings
  • Hohe Ratings für Zusammenfassungen und klinische Szenarien
  • Interrater-Übereinstimmung von Gwet's AC2 0,79–0,99

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Experimental Orthopaedics, online erschienen am 28.10.2026. Verfasst von Jakob Lindskog und 9 weiteren Autor:innen. Beteiligte Einrichtung: Unit of Physiotherapy, Department of Health and Rehabilitation Institute of Neuroscience and Physiology, Sahlgrenska Academy, University of Gothenbur…. Die Arbeit nennt 31 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Purpose

This study aimed to evaluate the interpretive and applicative performance of three conditions applied to the formal EU‐US, ESSKA‐AOSSM‐AASPT consensus on meniscus rehabilitation: (1) single‐pass prompting, (2) sequential chain‐of‐thought prompting and (3) human expert‐written output.

Methods

This exploratory comparative study evaluated three conditions applied to two parts of the formal EU‐US ESSKA‐AOSSM‐AASPT meniscus rehabilitation consensus: (1) single‐pass prompting, (2) sequential chain‐of‐thought prompting using ChatGPT (GPT‐5.1), Claude (Sonnet 4.5), and Gemini (3.0), and (3) human expert‐written output from three collaborating human experts. For each part, three tasks were assessed: generation of summaries, application of summaries to three predefined clinical scenarios, and resistance to three predefined misinterpretation queries. Three blinded senior sports physiotherapists evaluated summary quality using five QUEST categories and rated the clinical application and misinterpretation resistance responses. Median ratings were compared using Friedman and post‐hoc Wilcoxon signed‐rank tests. Gwet's AC2 coefficient with quadratic weights was used to assess inter‐rater agreement for each QUEST category and for the clinical application and misinterpretation resistance items.

Results

Across consensus parts, no statistically significant differences were identified between large language model prompting strategies and human expert outputs for QUEST categories, clinical application, and misinterpretation resistance queries. Sequential chain‐of‐thought prompting received higher numerical reviewer ratings than single‐pass prompting for ChatGPT and Gemini, particularly for quality of information and trust and confidence, although differences were not statistically significant. Inter‐rater agreement ranged from substantial to almost perfect (Gwet's AC2 = 0.79–0.99).

Conclusion

No statistically significant differences in reviewer ratings were observed between ChatGPT, Claude, Gemini and human expert output, with generally high ratings for summaries and responses to clinical scenarios. Large language models may therefore represent a potential adjunct for summarisation and interpretation of sports medicine consensus recommendations to inform rehabilitation decision‐making. Level of Evidence Level V.

Schlagwörter

Stichwörter der Autor:innen

LLMartificial intelligenceclinical decision supportmachine learningprompt engineeringrehabilitation

Autor:innen und Institutionen

  • Unit of Physiotherapy, Department of Health and Rehabilitation Institute of Neuroscience and Physiology, Sahlgrenska Academy, University of Gothenburg Gothenburg Sweden.
  • Sahlgrenska Sports Medicine Center, Sahlgrenska Academy Gothenburg Sweden.
  • Department of Orthopedic Surgery Balgrist University Hospital, University of Zürich Zurich Switzerland.
  • Department of Computer Science and Engineering Chalmers University of Technology and University of Gothenburg Gothenburg Sweden.
  • Division of Occupational Therapy and Physical Therapy Cincinnati Children's Hospital and University of Cincinnati College of Medicine Cincinnati Ohio USA.
  • Department of Physical Therapy University of Delaware Newark Delaware USA.

ORCID-Profile: Jakob Lindskog, Felix Öttl, Yinan Yu, Mark V Paterno, Airelle Giordano, Alexandre Rambaud, Robert Prill, Aleksandra Królikowska, Kristian Samuelsson, Eric Hamrin Senorski

Zitieren

APA

Lindskog, J., Öttl, F., Yu, Y., Paterno, M. V., Giordano, A., Rambaud, A., Prill, R., Królikowska, A., Samuelsson, K., & Hamrin Senorski, E. (2026). No significant differences between large language model prompting strategies and human experts in summarisation of ‘the formal EU–US meniscus rehabilitation consensus: An ESSKA‐AOSSM‐AASPT Initiative’. Journal of Experimental Orthopaedics, 13(4). https://doi.org/10.1002/jeo2.70964

Vancouver

Lindskog J, Öttl F, Yu Y, Paterno MV, Giordano A, Rambaud A, et al. No significant differences between large language model prompting strategies and human experts in summarisation of ‘the formal EU–US meniscus rehabilitation consensus: An ESSKA‐AOSSM‐AASPT Initiative’. Journal of Experimental Orthopaedics. 2026;13(4). doi:10.1002/jeo2.70964

BibTeX
@article{Lindskog2026,
  author = {Lindskog, Jakob and Öttl, Felix and Yu, Yinan and Paterno, Mark V and Giordano, Airelle and Rambaud, Alexandre and Prill, Robert and Królikowska, Aleksandra and Samuelsson, Kristian and Hamrin Senorski, Eric},
  title = {No significant differences between large language model prompting strategies and human experts in summarisation of ‘the formal EU–US meniscus rehabilitation consensus: An ESSKA‐AOSSM‐AASPT Initiative’},
  journal = {Journal of Experimental Orthopaedics},
  year = {2026},
  volume = {13},
  number = {4},
  doi = {10.1002/jeo2.70964}
}
Kostenlos starten →