A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation
JMIR Research Protocols · 2026-10-01 · 15 · e79966-e79966
DOI: 10.2196/79966 ↗At a glance
Studienprotokoll in JMIR Research Protocols, published online on 2026-10-01. Written by Fatima Sayed and 3 co-authors. Institution: Department of Software and Information Systems, University of North Carolina at Charlotte, Charlotte, NC, United States. Major MEDLINE topics: Pre-Exposure Prophylaxis, HIV Infections. 121 references. The full text is freely available in PubMed Central.
Abstract (original language)
Background
Chatbots have the potential to reduce barriers to preexposure prophylaxis (PrEP), including lack of awareness, misconceptions, and stigma, by providing anonymous and continuous support. However, in the context of PrEP, chatbots are still nascent; they lack personalized informational expertise, peer experiential expertise, and human-like emotional support to facilitate future PrEP uptake. These personalized and relatable forms of support are crucial for increasing engagement, influencing health decisions, and fostering resilience and well-being.
Objective
In this paper, we describe the iterative development and evaluation plans of a retrieval-augmented generation (RAG) chatbot for providing personalized information, peer experiential expertise, and human-like emotional support to PrEP candidates.
Methods
We used an iterative design process consisting of 2 phases: prototype conceptualization and iterative chatbot development. In the conceptualization phase, we identified real-world PrEP needs and designed a functional dialogue flow diagram for PrEP support. Chatbot development included developing 2 components: a query preprocessor and a RAG module. The preprocessor uses the Segment Any Text (SAT) tool (developed by Markus Frohmann, Igor Sterner, Ivan Vulić, Benjamin Minixhofer, and Markus Schedl) for query segmentation and a Gemma 2 fine-tuned support classifier to identify informational, emotional, and contextual data from real-world queries. To implement the RAG module, we used Sentence-Bidirectional Encoder Representations from Transformers (SBERT) embeddings with cosine similarity, and performed topic matching to identify topically relevant documents based on the query topic to support document retrieval. Extensive prompt engineering was used to guide the large language model (LLM), Gemini-2.0-flash, in generating tailored responses. We conducted 10 rounds of internal evaluations to assess and improve the chatbot responses based on 10 criteria: clarity, accuracy, actionability, relevancy, information detail, tailored information, comprehensiveness, language suitability, tone, and empathy. Finally, we conducted a blinded comparative study and used a linear mixed model to validate the chatbot against a general LLM and real-world user responses.
Results
We developed a RAG chatbot and iteratively refined it based on the internal evaluation feedback. Prompt engineering is essential in guiding the LLM to generate responses tailored to information, experiential, and emotional user needs. We found that prompt effectiveness varied with task complexity; this was likely due to LLM sensitivity to the structure of prompts and to linguistic variability. Prompt decomposition and segmenting prompt instructions helped improve comprehensiveness and relevancy for complex and long queries. The linear mixed model analysis revealed that while both the general LLM and RAG were preferred over user responses, the general LLM maintained a superior edge over the RAG chatbot across most criteria, with the exceptions of empathy and tone.
Conclusions
Our RAG chatbot leverages social media data to provide personalized information, peer experiences, and human-like emotional support; these elements are essential for addressing PrEP misconceptions and promoting self-efficacy. Further analysis incorporating expert and user feedback will be conducted to help validate and improve the chatbot’s potential.
Subjects
MeSH terms (MEDLINE)
Author keywords
Authors and institutions
- Department of Software and Information Systems, University of North Carolina at Charlotte, Charlotte, NC, United States.
- School of Health Information Science, University of Victoria, PO Box 1700 STN CSC, Victoria, BC, V8W 2Y2, Canada, +1-250-721-8575.
- Department of Epidemiology, Emory University, Atlanta, GA, United States.
Cite
APA
Sayed, F., Park, A., Sullivan, P. S., & Ge, Y. (2026). A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation. JMIR Research Protocols, 15, e79966-e79966. https://doi.org/10.2196/79966
Vancouver
Sayed F, Park A, Sullivan PS, Ge Y. A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation. JMIR Research Protocols. 2026;15:e79966-e79966. doi:10.2196/79966
BibTeX
@article{Sayed2026,
author = {Sayed, Fatima and Park, Albert and Sullivan, Patrick Sean and Ge, Yaorong},
title = {A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation},
journal = {JMIR Research Protocols},
year = {2026},
volume = {15},
pages = {e79966--e79966},
doi = {10.2196/79966}
}
EUVIMED