A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation
JMIR Research Protocols · 01.10.2026 · 15 · e79966-e79966
DOI: 10.2196/79966 ↗Kurz gefasst
Studienprotokoll in der Zeitschrift JMIR Research Protocols, online erschienen am 01.10.2026. Verfasst von Fatima Sayed und 3 weiteren Autor:innen. Beteiligte Einrichtung: Department of Software and Information Systems, University of North Carolina at Charlotte, Charlotte, NC, United States. Hauptthemen laut MEDLINE: Pre-Exposure Prophylaxis, HIV Infections. Die Arbeit nennt 121 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.
Abstract (Originalsprache)
Background
Chatbots have the potential to reduce barriers to preexposure prophylaxis (PrEP), including lack of awareness, misconceptions, and stigma, by providing anonymous and continuous support. However, in the context of PrEP, chatbots are still nascent; they lack personalized informational expertise, peer experiential expertise, and human-like emotional support to facilitate future PrEP uptake. These personalized and relatable forms of support are crucial for increasing engagement, influencing health decisions, and fostering resilience and well-being.
Objective
In this paper, we describe the iterative development and evaluation plans of a retrieval-augmented generation (RAG) chatbot for providing personalized information, peer experiential expertise, and human-like emotional support to PrEP candidates.
Methods
We used an iterative design process consisting of 2 phases: prototype conceptualization and iterative chatbot development. In the conceptualization phase, we identified real-world PrEP needs and designed a functional dialogue flow diagram for PrEP support. Chatbot development included developing 2 components: a query preprocessor and a RAG module. The preprocessor uses the Segment Any Text (SAT) tool (developed by Markus Frohmann, Igor Sterner, Ivan Vulić, Benjamin Minixhofer, and Markus Schedl) for query segmentation and a Gemma 2 fine-tuned support classifier to identify informational, emotional, and contextual data from real-world queries. To implement the RAG module, we used Sentence-Bidirectional Encoder Representations from Transformers (SBERT) embeddings with cosine similarity, and performed topic matching to identify topically relevant documents based on the query topic to support document retrieval. Extensive prompt engineering was used to guide the large language model (LLM), Gemini-2.0-flash, in generating tailored responses. We conducted 10 rounds of internal evaluations to assess and improve the chatbot responses based on 10 criteria: clarity, accuracy, actionability, relevancy, information detail, tailored information, comprehensiveness, language suitability, tone, and empathy. Finally, we conducted a blinded comparative study and used a linear mixed model to validate the chatbot against a general LLM and real-world user responses.
Results
We developed a RAG chatbot and iteratively refined it based on the internal evaluation feedback. Prompt engineering is essential in guiding the LLM to generate responses tailored to information, experiential, and emotional user needs. We found that prompt effectiveness varied with task complexity; this was likely due to LLM sensitivity to the structure of prompts and to linguistic variability. Prompt decomposition and segmenting prompt instructions helped improve comprehensiveness and relevancy for complex and long queries. The linear mixed model analysis revealed that while both the general LLM and RAG were preferred over user responses, the general LLM maintained a superior edge over the RAG chatbot across most criteria, with the exceptions of empathy and tone.
Conclusions
Our RAG chatbot leverages social media data to provide personalized information, peer experiences, and human-like emotional support; these elements are essential for addressing PrEP misconceptions and promoting self-efficacy. Further analysis incorporating expert and user feedback will be conducted to help validate and improve the chatbot’s potential.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Department of Software and Information Systems, University of North Carolina at Charlotte, Charlotte, NC, United States.
- School of Health Information Science, University of Victoria, PO Box 1700 STN CSC, Victoria, BC, V8W 2Y2, Canada, +1-250-721-8575.
- Department of Epidemiology, Emory University, Atlanta, GA, United States.
Zitieren
APA
Sayed, F., Park, A., Sullivan, P. S., & Ge, Y. (2026). A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation. JMIR Research Protocols, 15, e79966-e79966. https://doi.org/10.2196/79966
Vancouver
Sayed F, Park A, Sullivan PS, Ge Y. A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation. JMIR Research Protocols. 2026;15:e79966-e79966. doi:10.2196/79966
BibTeX
@article{Sayed2026,
author = {Sayed, Fatima and Park, Albert and Sullivan, Patrick Sean and Ge, Yaorong},
title = {A Conversational Agent for Providing Personalized Preexposure Prophylaxis (PrEP) Support: Protocol for Chatbot Implementation and Evaluation},
journal = {JMIR Research Protocols},
year = {2026},
volume = {15},
pages = {e79966--e79966},
doi = {10.2196/79966}
}
EUVIMED