EUVIMEDEUVIMED entdecken
LeitlinieDelphi-StudieOpen Access

From Divergent Guidelines to Delphi-Ready Items: Tutorial for Large Language Model–Assisted Guideline Harmonization Using Penile Cancer as a Worked Example

Von divergenten Leitlinien zu Delphi-fähigen Items: Tutorial zur LLM-gestützten Harmonisierung von Leitlinien am Beispiel Peniskarzinom

Maximilian Haas, Emily Rinderknecht, Matthias May, Maximilian Burger, Christian Gilfrich, Anne Herrmann, Sabine Brookman-May, Christoph Alexander Eckl, Udo Kruschwitz, Samy Ateia, Gregor Donabauer

Journal of Medical Internet Research · 20.10.2026 · 28 · e96886-e96886

DOI: 10.2196/96886 ↗

Auf einen Blick

Das Tutorial zeigt einen reproduzierbaren Workflow, um divergente Leitlinienempfehlungen und Literatur-Lücken in Delphi-fähige Kandidatenitems zu überführen. Im Beispiel Peniskarzinom entstanden 62 begutachtete Items.

Fragestellung

Wie lässt sich ein reproduzierbarer Workflow zur Harmonisierung divergenter klinischer Leitlinien und Literatur-Lücken mit LLMs entwickeln und demonstrieren?

Methode

Tutorial mit Worked Example Peniskarzinom; manuelle Leitlinienauswahl, PubMed-Suche, 3 LLMs, vordefinierte Prompts, multimodale Extraktion, semantische Harmonisierung, Stance-Klassifikation und externe Expertenbewertung.

Ergebnisse

Aus 5 Leitlinien wurden 157 Empfehlungen extrahiert und zu 68 harmonisierten Items verdichtet. Davon waren 15 einheitlich unterstützt; parallel entstanden 95 White-Area-Items, 9 davon echte Lücken. Ergebnis waren 62 Delphi-fähige Items.

Schlussfolgerung der Autor:innen

Die Methode bietet laut Autor:innen einen skalierbaren, transparenten und übertragbaren Ansatz zur Identifikation und Strukturierung von Leitlinieninkonsistenzen und Evidenzlücken.

Kernaussagen

  • 157 Empfehlungen aus 5 Leitlinien wurden zu 68 Items harmonisiert
  • 15 Items waren einheitlich unterstützt, 53 divergent/unklar
  • 62 human-reviewed Delphi-ready candidate items entstanden

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

Kurz gefasst

Leitlinie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 20.10.2026. Verfasst von Maximilian Haas und 10 weiteren Autor:innen. Beteiligte Einrichtung: Department for Urology, University Hospital Regensburg, Caritas Krankenhaus St. Josef, Landshuter Str. 65, Regensburg, Bavaria, 93053, Germany, 49 94…. Hauptthemen laut MEDLINE: Delphi Technique, Penile Neoplasms, Practice Guidelines as Topic. Die Arbeit nennt 101 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

Auf einen BlickAbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Clinical practice guidelines are central instruments of evidence-based care; yet, in many domains, particularly rare or complex diseases, multiple guidelines coexist and frequently offer conflicting advice. Existing strategies for resolving such divergences are often labor-intensive, opaque, and insufficiently reproducible.

Objective

This tutorial aimed to develop and demonstrate a reproducible workflow for transforming divergent clinical guideline recommendations and literature-derived white areas into human-reviewed Delphi-ready candidate items.

Methods

Following a step-by-step tutorial structure, we demonstrate the workflow using penile cancer as a worked example. The process comprises manual selection of clinical guidelines, a systematic PubMed search for literature-derived candidate issues potentially unaddressed by the selected guidelines, selection of 3 complementary large language models (LLMs), and execution of predefined prompts, multimodal recommendation extraction, semantic harmonization, and item-level guideline stance classification. The evaluative process combines a 3-LLM ensemble consistency check with an independent pre-Delphi appraisal by 4 penile cancer experts using a predefined 5-point suitability scale and acceptance threshold.

Results

In the worked example in the field of penile cancer as a high-stakes use case, the workflow extracted 157 clinical recommendations from 5 guidelines and consolidated them into 68 harmonized guideline-derived items. Stance classification identified 15 uniformly supported items and 53 divergent, ambiguous, or incompletely addressed items. In parallel, literature processing generated 95 harmonized white-area items, of which 9 were classified as true white-area items not addressed by the included guidelines. The final methodological output was a set of 62 human-reviewed Delphi-ready candidate items for subsequent clinical consensus voting.

Conclusions

This methodology provides a scalable, transparent, and transferable approach for identifying and structuring interguideline inconsistencies and evidence gaps. By combining LLM-assisted analysis with independent external expert appraisal, it supports the reproducible transformation of heterogeneous guideline content into Delphi-ready candidate items.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Delphi TechniquePenile NeoplasmsPractice Guidelines as TopicMaleHumansLarge Language Models

Stichwörter der Autor:innen

Delphi methodartificial intelligenceclinical practice guidelinesconsensus methodologyevidence harmonizationlarge language modelsrare diseasesreproducibilitywhite area analysis

Autor:innen und Institutionen

  • Department for Urology, University Hospital Regensburg, Caritas Krankenhaus St. Josef, Landshuter Str. 65, Regensburg, Bavaria, 93053, Germany, 49 941782 ext 3506.
  • Department of Urology, University Hospital Augsburg, Augsburg, Bavaria, Germany.
  • Department of Urology, Barmherzige Brüder Klinikum St. Elisabeth in Straubing, Straubing, Bavaria, Germany.
  • Department of Epidemiology and Preventive Medicine, Medical Sociology, University of Regensburg, Regensburg, Bavaria, Germany.
  • Department of Urology, Ludwig-Maximilians-Universität München, Munich, Bavaria, Germany.
  • Faculty of Information Science, University of Regensburg, Regensburg, Germany.

Zitieren

APA

Haas, M., Rinderknecht, E., May, M., Burger, M., Gilfrich, C., Herrmann, A., Brookman-May, S., Eckl, C. A., Kruschwitz, U., Ateia, S., & Donabauer, G. (2026). From Divergent Guidelines to Delphi-Ready Items: Tutorial for Large Language Model–Assisted Guideline Harmonization Using Penile Cancer as a Worked Example. Journal of Medical Internet Research, 28, e96886-e96886. https://doi.org/10.2196/96886

Vancouver

Haas M, Rinderknecht E, May M, Burger M, Gilfrich C, Herrmann A, et al. From Divergent Guidelines to Delphi-Ready Items: Tutorial for Large Language Model–Assisted Guideline Harmonization Using Penile Cancer as a Worked Example. Journal of Medical Internet Research. 2026;28:e96886-e96886. doi:10.2196/96886

BibTeX
@article{Haas2026,
  author = {Haas, Maximilian and Rinderknecht, Emily and May, Matthias and Burger, Maximilian and Gilfrich, Christian and Herrmann, Anne and Brookman-May, Sabine and Eckl, Christoph Alexander and Kruschwitz, Udo and Ateia, Samy and Donabauer, Gregor},
  title = {From Divergent Guidelines to Delphi-Ready Items: Tutorial for Large Language Model–Assisted Guideline Harmonization Using Penile Cancer as a Worked Example},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e96886--e96886},
  doi = {10.2196/96886}
}
Kostenlos starten →