EUVIMEDOtkrijte EUVIMED
ValidierungsstudieOpen access

Large Language Model–Generated Patient Instructions for Prescriptions in Primary Health Care: Preclinical Algorithm Validation

Zilma Silveira Nogueira Reis iD, Elisa Tuler Albergaria iD, Adriana Silvina Pagano iD, Eura Martins Lage iD, Flávia Ribeiro de Oliveira iD, Cristiane Dos Santos Dias iD, Juliana Almeida Oliveira iD, Gláucia Miranda Varella Pereira iD, Isaias Jose Ramos de Oliveira iD, Érico Franco Mineiro iD, Igor Carvalho Lima Oliveira iD, Davi Dos Reis de Jesus iD and 5 more

Journal of Medical Internet Research · 2026-05-26 · 28 · e84444

DOI: 10.2196/84444 ↗

At a glance

Validierungsstudie in Journal of Medical Internet Research, published online on 2026-05-26. Written by Zilma Silveira Nogueira Reis and 16 co-authors. Institution: Health Informatics Center, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil. Major MEDLINE topics: Primary Health Care, Algorithms, Electronic Prescribing. Cited 4 times so far (Crossref); 31 references. The full text is freely available in PubMed Central.

SažetakSubjectsBibliografski podaciCiteRelated studies

Abstract (original language)

Background

The application of generative artificial intelligence to simplify medication use instructions has the potential to enhance people’s health by improving treatment adherence.

Objective

We evaluated the performance of large language models (LLMs) in generating medication usage instructions to complement prescriptions in primary health care.

Methods

This randomized, blinded experimental preclinical study used prescription-inducing scenarios, assigned to 62 health care professionals, to validate instructions generated by LLMs during electronic prescriptions. The instructions were generated by ChatGPT-4.0 (OpenAI), Llama3.1-8B (Meta), and Llama3.1-8B-RAG (Meta) using retrieval-augmented generation based on patient information leaflets. Performance metrics assessed adequacy, completeness, clarity, language simplification, usefulness, and errors in the generated instructions, with scores to analyze overall and individual metrics.

Results

The 3 models yielded high overall scores for producing qualified instructions (ChatGPT-4.0: median 88.4, IQR 22.8; Llama3.1-8B: median 66.5, IQR 50.9; Llama3.1-8B-RAG: median 79.9, IQR 34.4; Kruskal-Wallis test P=.003). Llama3.1-8B-RAG received evaluations with similar overall scores to ChatGPT-4.0 (post hoc test, P=.05) and similar to Llama3.1-8B (post hoc test, P=.44). ChatGPT-4.0 outperformed Llama3.1-8B (Bonferroni test, P

Subjects

MeSH terms (MEDLINE)

Primary Health CareAlgorithmsElectronic PrescribingLarge Language ModelsHumansGenerative Artificial Intelligence

Author keywords

digital healthdrug prescriptionsgenerative artificial intelligencelarge language modelsprimary health care

Authors and institutions

  • Health Informatics Center, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil.
  • Computer Science Department, Universidade Federal de São João Del Rei, São João Del Rei, Brazil.
  • Arts Faculty, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil.
  • Department of Physiotherapy, School of Physical Education, Physiotherapy and Occupational Therapy, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil.
  • School of Architecture, Universidade Federal de Minas Gerais, Belo Horizonte, Brazil.
  • National Secretary of Primary Care of the Brazilian Ministry of Health, Brasilia, Brazil.

ORCID profiles: Zilma Silveira Nogueira Reis, Elisa Tuler Albergaria, Adriana Silvina Pagano, Eura Martins Lage, Flávia Ribeiro de Oliveira, Cristiane Dos Santos Dias, Juliana Almeida Oliveira, Gláucia Miranda Varella Pereira, Isaias Jose Ramos de Oliveira, Érico Franco Mineiro, Igor Carvalho Lima Oliveira, Davi Dos Reis de Jesus, Antônio Pereira de Souza Júnior, Igor de Carvalho Gomes, Rodrigo André Cuevas Gaete, Ricardo Cruz-Correia, Leonardo Rocha

Cite

APA

Silveira Nogueira Reis, Z., Tuler Albergaria, E., Silvina Pagano, A., Martins Lage, E., Ribeiro de Oliveira, F., Dos Santos Dias, C., Almeida Oliveira, J., Miranda Varella Pereira, G., Jose Ramos de Oliveira, I., Franco Mineiro, É., Carvalho Lima Oliveira, I., Dos Reis de Jesus, D., Pereira de Souza Júnior, A., de Carvalho Gomes, I., André Cuevas Gaete, R., Cruz-Correia, R., & Rocha, L. (2026). Large Language Model–Generated Patient Instructions for Prescriptions in Primary Health Care: Preclinical Algorithm Validation. Journal of Medical Internet Research, 28, e84444. https://doi.org/10.2196/84444

Vancouver

Silveira Nogueira Reis Z, Tuler Albergaria E, Silvina Pagano A, Martins Lage E, Ribeiro de Oliveira F, Dos Santos Dias C, et al. Large Language Model–Generated Patient Instructions for Prescriptions in Primary Health Care: Preclinical Algorithm Validation. Journal of Medical Internet Research. 2026;28:e84444. doi:10.2196/84444

BibTeX
@article{SilveiraNogueiraReis2026,
  author = {Silveira Nogueira Reis, Zilma and Tuler Albergaria, Elisa and Silvina Pagano, Adriana and Martins Lage, Eura and Ribeiro de Oliveira, Flávia and Dos Santos Dias, Cristiane and Almeida Oliveira, Juliana and Miranda Varella Pereira, Gláucia and Jose Ramos de Oliveira, Isaias and Franco Mineiro, Érico and Carvalho Lima Oliveira, Igor and Dos Reis de Jesus, Davi and Pereira de Souza Júnior, Antônio and de Carvalho Gomes, Igor and André Cuevas Gaete, Rodrigo and Cruz-Correia, Ricardo and Rocha, Leonardo},
  title = {Large Language Model–Generated Patient Instructions for Prescriptions in Primary Health Care: Preclinical Algorithm Validation},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e84444},
  doi = {10.2196/84444}
}
Start for free →