EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Prompting Strategies for Readability Alignment Between User-Generated Ophthalmology Questions and Large Language Model Answers: Comparative Evaluation Study

Mário César Bulla, Daniel Lavinsky, Alexandre Antonio Marques Rosa, Pedro Carlos Carricondo

JMIR Formative Research · 05.11.2026 · 10 · e97408-e97408

DOI: 10.2196/97408 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift JMIR Formative Research, online erschienen am 05.11.2026. Verfasst von Mário César Bulla und 3 weiteren Autor:innen. Beteiligte Einrichtung: Faculdade de Medicina, Universidade de São Paulo, São Paulo, Brazil, 55 51-98118-5252. Hauptthemen laut MEDLINE: Comprehension, Ophthalmology, Health Literacy. Die Arbeit nennt 63 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Accessible health communication is essential for effective patient education, yet most medical information remains written at a level above the general population’s recommended reading level. Large language models (LLMs) can simplify complex medical information, but it remains unclear whether responses can be automatically adapted to match the linguistic complexity of user-generated questions seeking medical information.

Objective

To evaluate whether the LLM-generated responses to user-generated ophthalmology questions align with their readability and to compare different prompting strategies for readability control across multiple LLM architectures, including their impact on response quality as well as computational cost and efficiency.

Methods

A total of 400 user-generated ophthalmology questions collected from public YouTube comments were categorized into 3 readability tiers based on the Flesch Reading Ease Score. A controlled comparative evaluation of responses generated by 4 LLMs (GPT 5.2, Gemini 3, Claude Opus 4.5, and OSS-20b) was performed under 4 prompting conditions: Baseline (no readability control), Formula-Informed Readability Prompt, Semantic Readability Inference, and External Calculator–Guided Prompt. Agreement between question and response readability was assessed using quadratic-weighted Cohen kappa. Response quality was evaluated using BERTScore and LLM-as-a-judge assessments. Computational efficiency was evaluated through token usage, estimated cost per response, and latency.

Results

The external calculator–guided strategy produced the strongest alignment in readability across models, achieving the highest agreement (Gemini 3: κ=0.94; GPT 5.2: κ=0.90), while maintaining favorable computational efficiency. Formula-informed prompting showed intermediate performance, whereas semantic inference demonstrated poor agreement due to unreliable readability estimation. High-performing models (GPT 5.2 and Gemini 3) maintained very high factual accuracy (>98%) across strategies, indicating that readability control did not substantially degrade correctness. In contrast, the open-source OSS-20b model showed lower accuracy (83.75%‐90%) and greater sensitivity to readability constraints. Semantic similarity remained high across models (Bidirectional Encoder Representations from Transformers Score [BERTScore] F 1 >0.87). Longer questions were associated with slightly lower accuracy, particularly in lower-performing configurations.

Conclusions

Aligning LLM responses to the readability of user-generated questions is feasible and can be reliably achieved using externally guided readability signals. Hybrid pipelines that combine traditional readability metrics with LLM-generated content may therefore represent a practical approach for scalable, readability-adaptive health communication systems.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

ComprehensionOphthalmologyHealth LiteracyLarge Language ModelsHumans

Stichwörter der Autor:innen

artificial intelligencehealth communicationhealth literacynatural language processingophthalmologyreadability

Autor:innen und Institutionen

  • Faculdade de Medicina, Universidade de São Paulo, São Paulo, Brazil, 55 51-98118-5252.
  • Clínica Bulla, Rua São Joaquim 611, Suite 1203, São Leopoldo, 93010-190, Brazil, 55 51-98118-5252.
  • Faculdade de Medicina, Universidade Federal do Rio Grande do Sul, Porto Alegre, Rio Grande do Sul, Brazil.
  • Faculdade de Medicina, Universidade Federal do Pará, Belém, Pará, Brazil.

Zitieren

APA

Bulla, M. C., Lavinsky, D., Rosa, A. A. M., & Carricondo, P. C. (2026). Prompting Strategies for Readability Alignment Between User-Generated Ophthalmology Questions and Large Language Model Answers: Comparative Evaluation Study. JMIR Formative Research, 10, e97408-e97408. https://doi.org/10.2196/97408

Vancouver

Bulla MC, Lavinsky D, Rosa AAM, Carricondo PC. Prompting Strategies for Readability Alignment Between User-Generated Ophthalmology Questions and Large Language Model Answers: Comparative Evaluation Study. JMIR Formative Research. 2026;10:e97408-e97408. doi:10.2196/97408

BibTeX
@article{Bulla2026,
  author = {Bulla, Mário César and Lavinsky, Daniel and Rosa, Alexandre Antonio Marques and Carricondo, Pedro Carlos},
  title = {Prompting Strategies for Readability Alignment Between User-Generated Ophthalmology Questions and Large Language Model Answers: Comparative Evaluation Study},
  journal = {JMIR Formative Research},
  year = {2026},
  volume = {10},
  pages = {e97408--e97408},
  doi = {10.2196/97408}
}
Kostenlos starten →