From Divergent Guidelines to Delphi-Ready Items: Tutorial for Large Language Model–Assisted Guideline Harmonization Using Penile Cancer as a Worked Example
Von divergenten Leitlinien zu Delphi-fähigen Items: Tutorial zur LLM-gestützten Harmonisierung von Leitlinien am Beispiel Peniskarzinom
Journal of Medical Internet Research · 20.10.2026 · 28 · e96886-e96886
DOI: 10.2196/96886 ↗Auf einen Blick
Das Tutorial zeigt einen reproduzierbaren Workflow, um divergente Leitlinienempfehlungen und Literatur-Lücken in Delphi-fähige Kandidatenitems zu überführen. Im Beispiel Peniskarzinom entstanden 62 begutachtete Items.
Fragestellung
Wie lässt sich ein reproduzierbarer Workflow zur Harmonisierung divergenter klinischer Leitlinien und Literatur-Lücken mit LLMs entwickeln und demonstrieren?
Methode
Tutorial mit Worked Example Peniskarzinom; manuelle Leitlinienauswahl, PubMed-Suche, 3 LLMs, vordefinierte Prompts, multimodale Extraktion, semantische Harmonisierung, Stance-Klassifikation und externe Expertenbewertung.
Ergebnisse
Aus 5 Leitlinien wurden 157 Empfehlungen extrahiert und zu 68 harmonisierten Items verdichtet. Davon waren 15 einheitlich unterstützt; parallel entstanden 95 White-Area-Items, 9 davon echte Lücken. Ergebnis waren 62 Delphi-fähige Items.
Schlussfolgerung der Autor:innen
Die Methode bietet laut Autor:innen einen skalierbaren, transparenten und übertragbaren Ansatz zur Identifikation und Strukturierung von Leitlinieninkonsistenzen und Evidenzlücken.
Kernaussagen
- 157 Empfehlungen aus 5 Leitlinien wurden zu 68 Items harmonisiert
- 15 Items waren einheitlich unterstützt, 53 divergent/unklar
- 62 human-reviewed Delphi-ready candidate items entstanden
Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.
Kurz gefasst
Leitlinie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 20.10.2026. Verfasst von Maximilian Haas und 10 weiteren Autor:innen. Beteiligte Einrichtung: Department for Urology, University Hospital Regensburg, Caritas Krankenhaus St. Josef, Landshuter Str. 65, Regensburg, Bavaria, 93053, Germany, 49 94…. Hauptthemen laut MEDLINE: Delphi Technique, Penile Neoplasms, Practice Guidelines as Topic. Die Arbeit nennt 101 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Clinical practice guidelines are central instruments of evidence-based care; yet, in many domains, particularly rare or complex diseases, multiple guidelines coexist and frequently offer conflicting advice. Existing strategies for resolving such divergences are often labor-intensive, opaque, and insufficiently reproducible.
Objective
This tutorial aimed to develop and demonstrate a reproducible workflow for transforming divergent clinical guideline recommendations and literature-derived white areas into human-reviewed Delphi-ready candidate items.
Methods
Following a step-by-step tutorial structure, we demonstrate the workflow using penile cancer as a worked example. The process comprises manual selection of clinical guidelines, a systematic PubMed search for literature-derived candidate issues potentially unaddressed by the selected guidelines, selection of 3 complementary large language models (LLMs), and execution of predefined prompts, multimodal recommendation extraction, semantic harmonization, and item-level guideline stance classification. The evaluative process combines a 3-LLM ensemble consistency check with an independent pre-Delphi appraisal by 4 penile cancer experts using a predefined 5-point suitability scale and acceptance threshold.
Results
In the worked example in the field of penile cancer as a high-stakes use case, the workflow extracted 157 clinical recommendations from 5 guidelines and consolidated them into 68 harmonized guideline-derived items. Stance classification identified 15 uniformly supported items and 53 divergent, ambiguous, or incompletely addressed items. In parallel, literature processing generated 95 harmonized white-area items, of which 9 were classified as true white-area items not addressed by the included guidelines. The final methodological output was a set of 62 human-reviewed Delphi-ready candidate items for subsequent clinical consensus voting.
Conclusions
This methodology provides a scalable, transparent, and transferable approach for identifying and structuring interguideline inconsistencies and evidence gaps. By combining LLM-assisted analysis with independent external expert appraisal, it supports the reproducible transformation of heterogeneous guideline content into Delphi-ready candidate items.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department for Urology, University Hospital Regensburg, Caritas Krankenhaus St. Josef, Landshuter Str. 65, Regensburg, Bavaria, 93053, Germany, 49 941782 ext 3506.
- Department of Urology, University Hospital Augsburg, Augsburg, Bavaria, Germany.
- Department of Urology, Barmherzige Brüder Klinikum St. Elisabeth in Straubing, Straubing, Bavaria, Germany.
- Department of Epidemiology and Preventive Medicine, Medical Sociology, University of Regensburg, Regensburg, Bavaria, Germany.
- Department of Urology, Ludwig-Maximilians-Universität München, Munich, Bavaria, Germany.
- Faculty of Information Science, University of Regensburg, Regensburg, Germany.
Zitieren
APA
Haas, M., Rinderknecht, E., May, M., Burger, M., Gilfrich, C., Herrmann, A., Brookman-May, S., Eckl, C. A., Kruschwitz, U., Ateia, S., & Donabauer, G. (2026). From Divergent Guidelines to Delphi-Ready Items: Tutorial for Large Language Model–Assisted Guideline Harmonization Using Penile Cancer as a Worked Example. Journal of Medical Internet Research, 28, e96886-e96886. https://doi.org/10.2196/96886
Vancouver
Haas M, Rinderknecht E, May M, Burger M, Gilfrich C, Herrmann A, et al. From Divergent Guidelines to Delphi-Ready Items: Tutorial for Large Language Model–Assisted Guideline Harmonization Using Penile Cancer as a Worked Example. Journal of Medical Internet Research. 2026;28:e96886-e96886. doi:10.2196/96886
BibTeX
@article{Haas2026,
author = {Haas, Maximilian and Rinderknecht, Emily and May, Matthias and Burger, Maximilian and Gilfrich, Christian and Herrmann, Anne and Brookman-May, Sabine and Eckl, Christoph Alexander and Kruschwitz, Udo and Ateia, Samy and Donabauer, Gregor},
title = {From Divergent Guidelines to Delphi-Ready Items: Tutorial for Large Language Model–Assisted Guideline Harmonization Using Penile Cancer as a Worked Example},
journal = {Journal of Medical Internet Research},
year = {2026},
volume = {28},
pages = {e96886--e96886},
doi = {10.2196/96886}
}
EUVIMED