EUVIMEDScopri EUVIMED
Open access

Interdisciplinary Development and Fine-Tuning of CARDIO, a Large Language Model for Cardiovascular Health Education in HIV Care: Tutorial

Ryan Rullo iD, Ali Maatouk iD, Tinglin Huang iD, Jialin Chen iD, Weikang Qiu iD, Giselle O'Connor iD, Julie Womack iD, Tatiana Sadak iD, Christine Rodriguez iD, Pedro Carneiro iD, Tania de Jesus Espinosa iD, Ami Marshall iD and 2 more

Journal of Medical Internet Research · 2025

DOI: 10.2196/77053 ↗

At a glance

Journal article in Journal of Medical Internet Research, 2025. Written by Ryan Rullo and 13 co-authors. Institution: School of Nursing, Yale University, Orange, CT, United States. Major MEDLINE topics: HIV Infections, Cardiovascular Diseases, Patient Education as Topic, Language. Cited 1 times so far (Crossref); 80 references. The full text is freely available in PubMed Central.

AbstractSubjectsDati bibliograficiCiteRelated studies

Abstract (original language)

Background

The integration of artificial intelligence in health care presents a significant opportunity to revolutionize patient care. In the United States, an estimated 129 million people have at least 1 chronic illness, with 42% having 2 or more. Despite being largely preventable, the prevalence of chronic illness is expected to rise and impose significant economic burdens and financial toxicity on health care consumers.

Objective

We leveraged an interdisciplinary team encompassing nursing, public health, and computer science to optimize health through prevention education for cardiovascular and metabolic comorbidities in persons living with HIV. In this tutorial, we describe the iterative, data-based development and evaluation of an intersectionality-informed large language model designed to support patient teaching in this population.

Methods

First, we curated data by scraping publicly available, authoritative, evidence-based sources to capture a comprehensive dataset, supplemented by publicly available HIV forum content. Second, we benchmarked candidate large language models and generated a fine-tuning dataset using GPT-4 through multiturn question and answer conversations, using standardized metrics to assess baseline model performance. Third, we iteratively refined the selected model via low-rank adaptation and reinforcement learning, integrating quantitative metrics with qualitative expert evaluations.

Results

Pre-existing large language models (LLMs) demonstrated poor n-gram agreement, dissonance from model answers (accuracy 4.16, readability 4.63, and professionalism 4.58), and difficult readability (Kincaid 8.54 and Jargon 4.44). After prompt adjustments and fine-tuning, preliminary results demonstrate the potential of a customized Llama-based LLM to provide personalized, culturally salient patient education.

Conclusions

We present a data-based, step-by-step tutorial for interdisciplinary development of CARDIO, a specialized LLM, for cardiovascular health education in HIV care. Through comprehensive data curation and scraping, systematic benchmarking, and a dual-stage fine-tuning pipeline, CARDIO’s performance improved markedly (accuracy 5.0, readability 4.98, professionalism 4.98, Kincaid 7.17, and Jargon 2.92). Although patient pilot testing remains forthcoming, our results demonstrate that targeted data curation, rigorous benchmarking, and iterative fine-tuning have provided a robust evaluation of the model’s potential. By building an LLM tailored to cardiovascular health promotion and patient education, this work lays the foundation for innovative artificial intelligence–driven strategies to manage comorbid conditions in people living with HIV.

Subjects

MeSH terms (MEDLINE)

HIV InfectionsCardiovascular DiseasesPatient Education as TopicLanguageHumansArtificial IntelligenceLarge Language Models

Author keywords

HIVartificial intelligencecardiovascular healthcardiovascular riskcomputer sciencedischarge educationhealth promotioninterdisciplinary researchlarge language modelsnursingpatient education

Authors and institutions

  • School of Nursing, Yale University, Orange, CT, United States.
  • Department of Computer Science, School of Engineering and Applied Science, Yale University, New Haven, CT, United States.
  • VA Connecticut Healthcare System, West Haven, CT, United States.
  • Department of Social and Behavioral Sciences, School of Public Health, Yale University, New Haven, CT, United States.

ORCID profiles: Ryan Rullo, Ali Maatouk, Tinglin Huang, Jialin Chen, Weikang Qiu, Giselle O'Connor, Julie Womack, Tatiana Sadak, Christine Rodriguez, Pedro Carneiro, Tania de Jesus Espinosa, Ami Marshall, Rex Ying, S Raquel Ramos

Cite

APA

Rullo, R., Maatouk, A., Huang, T., Chen, J., Qiu, W., O'Connor, G., Womack, J., Sadak, T., Rodriguez, C., Carneiro, P., de Jesus Espinosa, T., Marshall, A., Ying, R., & Ramos, S. R. (2025). Interdisciplinary Development and Fine-Tuning of CARDIO, a Large Language Model for Cardiovascular Health Education in HIV Care: Tutorial. Journal of Medical Internet Research. https://doi.org/10.2196/77053

Vancouver

Rullo R, Maatouk A, Huang T, Chen J, Qiu W, O'Connor G, et al. Interdisciplinary Development and Fine-Tuning of CARDIO, a Large Language Model for Cardiovascular Health Education in HIV Care: Tutorial. Journal of Medical Internet Research. 2025. doi:10.2196/77053

BibTeX
@article{Rullo2025,
  author = {Rullo, Ryan and Maatouk, Ali and Huang, Tinglin and Chen, Jialin and Qiu, Weikang and O'Connor, Giselle and Womack, Julie and Sadak, Tatiana and Rodriguez, Christine and Carneiro, Pedro and de Jesus Espinosa, Tania and Marshall, Ami and Ying, Rex and Ramos, S Raquel},
  title = {Interdisciplinary Development and Fine-Tuning of CARDIO, a Large Language Model for Cardiovascular Health Education in HIV Care: Tutorial},
  journal = {Journal of Medical Internet Research},
  year = {2025},
  doi = {10.2196/77053}
}
Start for free →