EUVIMEDOpdag EUVIMED
Open access

ChatGPT Responses in Hard Flaccid Syndrome: Exploratory Evaluation

ChatGPT-Antworten beim Hard Flaccid Syndrome: explorative Bewertung

Maher Abdessater, Ramy Touma Sawaya, Oussama Nasrallah, Samer Traboulsi, Bassel Bachir

JMIR Formative Research · 2026-11-10 · 10 · e93849-e93849

DOI: 10.2196/93849 ↗

Auf einen Blick

ChatGPT-Antworten zu 16 HFS-Fragen zeigten stabile Lesbarkeit und eine hohe kurzfristige Übereinstimmung. Die Genauigkeit war jedoch je nach klinischem Bereich unterschiedlich.

Fragestellung

Wie genau, lesbar und kurzfristig reproduzierbar sind ChatGPT-Antworten auf patientengenerierte Fragen zum Hard Flaccid Syndrome (HFS)?

Methode

Explorative Bewertung von 16 Fragen aus 4 Domänen; Bewertung der Genauigkeit durch 3 Urolog:innen, Lesbarkeit mit dem Gunning Fog Index, Wiederholung nach 48 Stunden.

Ergebnisse

Lesbarkeit: Gunning Fog Index 15,19 (SD 1,68) vs. 14,59 (SD 1,66; P =.33). T1: 43,8 % korrekt und vollständig, 50 % korrekt aber unzureichend, 6,3 % gemischt. T2: 62,5 % korrekt und vollständig. 87,5 % gleich klassifiziert.

Schlussfolgerung der Autor:innen

ChatGPT zeigte stabile Lesbarkeit und hohe kurzfristige Konsistenz bei HFS-Fragen. Die Genauigkeit blieb variabel; die Antworten können über die verfügbare Evidenz hinausgehen.

Kernaussagen

  • Gunning Fog Index: 15,19 vs. 14,59; P =.33
  • T1: 43,8 % korrekt und vollständig; T2: 62,5 %
  • 87,5 % der Antworten hatten identische Endklassifikationen

Deutsche Zusammenfassung: KI-gestützt aus dem Original-Abstract erstellt; alle Zahlen wurden automatisch mit dem Original abgeglichen. Maßgeblich ist der Original-Abstract.

At a glance

Journal article in JMIR Formative Research, published online on 2026-11-10. Written by Maher Abdessater and 4 co-authors. Institution: American University of Beirut Medical Center, P.O. Box 11-0236, Beirut, Beyrouth, 1107 2020, Lebanon, 961 01350000 ext 5913. Major MEDLINE topics: Generative Artificial Intelligence, Genital Diseases, Male. 22 references. The full text is freely available in PubMed Central.

Auf einen BlickAbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Hard flaccid syndrome (HFS) is a rare and poorly described andrological condition characterized by a persistent semirigid penis in the flaccid state and frequently accompanied by erectile dysfunction, sensory disturbances, urinary symptoms, and psychosocial distress. Due to the absence of standardized diagnostic and therapeutic guidelines, patients increasingly seek health information from AI-based tools such as ChatGPT. However, the reliability of such tools in rare diseases remains uncertain.

Objective

This study aimed to perform an exploratory evaluation of the accuracy, readability, and short-term reproducibility of ChatGPT responses to patient-generated questions regarding HFS.

Methods

Sixteen clinically relevant questions spanning 4 domains (epidemiology, treatment, diagnostics, and symptoms and case presentation) were developed using authentic patient-derived queries and previously published AI evaluations in rare diseases. Questions were submitted to the ChatGPT web interface (OpenAI) in separate chat sessions and repeated after 48 hours to assess short-term reproducibility. Three urologists with expertise in male sexual health independently evaluated response accuracy using a validated 4-point scale. Readability was assessed using the Gunning Fog Index, whereas interrater reliability was evaluated using intraclass correlation coefficients (ICCs).

Results

Overall readability remained stable between sessions, with mean Gunning Fog Index scores of 15.19 (SD 1.68) at baseline and 14.59 (SD 1.66) after 48 hours ( P =.33), corresponding to an upper high school to early college reading level. No significant differences in readability were observed across domains at either time point (T1 [November 27, 2025]: P =.76; T2 [December 1, 2025]: P =.68). Accuracy differed across clinical domains. At T1, a total of 43.8% (7/16) of the responses were classified as correct and comprehensive, 50% (8/16) were classified as correct but inadequate, and 6.3% (1/16) were classified as having mixed correct and incorrect information; no responses were classified as completely incorrect. At T2, performance improved, with 62.5% (10/16) of responses classified as correct and comprehensive and 37.5% (6/16) classified as correct but inadequate. Interrater reliability demonstrated fair agreement at the individual reviewer level and moderate agreement when ratings were averaged (T1 single-measure ICC=0.36, 95% CI 0.06‐0.67; T1 average-measure ICC=0.63, 95% CI 0.17‐0.86; T2 single-measure ICC=0.33, 95% CI 0.04‐0.64; T2 average-measure ICC=0.59, 95% CI 0.10‐0.84). Short-term reproducibility was substantial, with 87.5% (14/16) of responses receiving identical final accuracy classifications between sessions.

Conclusions

ChatGPT demonstrated stable readability and high short-term consistency when addressing HFS-related questions. However, accuracy remained variable across clinical domains, with responses occasionally extending beyond the available evidence base in this poorly characterized condition. These findings suggest that AI tools may serve as accessible informational resources but should complement rather than replace expert clinical evaluation and evidence-based guidance, particularly in rare diseases for which clinical consensus remains limited.

Subjects

MeSH terms (MEDLINE)

Generative Artificial IntelligenceGenital Diseases, MaleHumansMaleComprehensionLarge Language ModelsReproducibility of ResultsSurveys and Questionnaires

Author keywords

AIChatGPTartificial intelligencehard flaccid syndromemedical information accuracyrare andrological disorders

Authors and institutions

  • American University of Beirut Medical Center, P.O. Box 11-0236, Beirut, Beyrouth, 1107 2020, Lebanon, 961 01350000 ext 5913.
  • University Medical Center Rizk Hospital, Beirut, Beyrouth, Lebanon.

Cite

APA

Abdessater, M., Touma Sawaya, R., Nasrallah, O., Traboulsi, S., & Bachir, B. (2026). ChatGPT Responses in Hard Flaccid Syndrome: Exploratory Evaluation. JMIR Formative Research, 10, e93849-e93849. https://doi.org/10.2196/93849

Vancouver

Abdessater M, Touma Sawaya R, Nasrallah O, Traboulsi S, Bachir B. ChatGPT Responses in Hard Flaccid Syndrome: Exploratory Evaluation. JMIR Formative Research. 2026;10:e93849-e93849. doi:10.2196/93849

BibTeX
@article{Abdessater2026,
  author = {Abdessater, Maher and Touma Sawaya, Ramy and Nasrallah, Oussama and Traboulsi, Samer and Bachir, Bassel},
  title = {ChatGPT Responses in Hard Flaccid Syndrome: Exploratory Evaluation},
  journal = {JMIR Formative Research},
  year = {2026},
  volume = {10},
  pages = {e93849--e93849},
  doi = {10.2196/93849}
}
Start for free →