EUVIMEDOpdag EUVIMED
Open access

AI-Powered Drug Classification and Indication Mapping for Pharmacoepidemiologic Studies: Prompt Development and Validation

Benjamin Ogorek iD, Thomas Rhoads iD, Eric Finkelman iD, Isaac R Rodriguez-Chavez iD

JMIR AI · 2025-06-12 · 4 · e65481

DOI: 10.2196/65481 ↗

At a glance

Journal article in JMIR AI, published online on 2025-06-12. Written by Benjamin Ogorek and 3 co-authors. Institution: Spencer Health Solutions, Inc, Morrisville, NC, United States. Cited 5 times so far (Crossref); 51 references. The full text is freely available in PubMed Central.

AbstractSubjectsBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Background

Pharmacoepidemiologic studies, which promote rational drug use and improve health outcomes, often require Anatomical Therapeutic Chemical Classification System (ATC) drug classification within real-world data (RWD) sources. Existing classification tools are expensive, brittle, or have restrictive terms of service, and lack context that may inform classification itself.

Objective

This study sought to establish large language models (LLMs) as an assisting technology in the drug classification task. This included developing artificial intelligence prompts that reason about drugs using RWD and showing that the resulting accuracy, efficiency, and effectiveness are favorable to alternative methods.

Methods

A prompt was constructed to classify aspirin as either an analgesic or antithrombotic and evaluated within 12,294 anonymized daily dose strings from a polychronic population residing in the United States and Canada. The patients used a smart medication dispenser called “spencer” and consented to the use of their data for research. The LLM prompt requested that the best and next-best second-level ATC code be returned, and grading was performed on a 3-point scale. After success in a pilot sample of 20, an inference sample of 200 was taken without replacement. Finite population inference was carried out on the proportion of outputs receiving 1 of the top 2 grades. As a benchmark, Google’s Programmable Search Engine was used to query the drug name plus “ATC code” followed by regex-based extraction of ATC codes. All imperfect results were reviewed.

Results

The population consisted of 12,294 daily dose strings from 86.26% (2908/3371) patients residing in Canada and 13.73% (463/3371) residing in the United States. A prompt using the chain-of-thought reasoning was able to distinguish between aspirin’s analgesic versus antithrombotic therapeutic uses and performed well in the pilot sample. In the inferential sample, 87.5% (175/200) were graded as perfect, 5% (10/200) had a minor issue, and 7.5% (15/200) had a major issue. The estimate of the proportion of at least mostly correct classification was 92.5% (185/200, 80% CI 90.1%-94.9%). For the search-based algorithm, 82.5% (165/200) were deemed acceptable. The chain-of-thought reasoning was most helpful with supplements (eg, folic acid) when high doses indicated antianemic preparations. The problem formulation of daily dose inputs and multiple ATC outputs was sometimes incompatible with the drug (eg, pregabalin, calcitriol, and methotrexate).

Conclusions

GPT-4o offers cost-effective drug classification from RWD without violating any terms of service. Using a chain-of-thought prompting technique, GPT-4o can reason about drug dosages that affect the class. The wide accessibility of LLMs gives every research team the ability to classify drugs at scale, a key prerequisite of pharmacoepidemiologic research.

Subjects

Author keywords

AIATCAnatomical Therapeutic ChemicalLLMsNLPartificial intelligencedrug classificationgenerative language modellarge language modelsnatural language processingsmart hubspencer device

Authors and institutions

  • Spencer Health Solutions, Inc, Morrisville, NC, United States.
  • 4Biosolutions Consulting, Rockville, MD, United States.

ORCID profiles: Benjamin Ogorek, Thomas Rhoads, Eric Finkelman, Isaac R Rodriguez-Chavez

Cite

APA

Ogorek, B., Rhoads, T., Finkelman, E., & Rodriguez-Chavez, I. R. (2025). AI-Powered Drug Classification and Indication Mapping for Pharmacoepidemiologic Studies: Prompt Development and Validation. JMIR AI, 4, e65481. https://doi.org/10.2196/65481

Vancouver

Ogorek B, Rhoads T, Finkelman E, Rodriguez-Chavez IR. AI-Powered Drug Classification and Indication Mapping for Pharmacoepidemiologic Studies: Prompt Development and Validation. JMIR AI. 2025;4:e65481. doi:10.2196/65481

BibTeX
@article{Ogorek2025,
  author = {Ogorek, Benjamin and Rhoads, Thomas and Finkelman, Eric and Rodriguez-Chavez, Isaac R},
  title = {AI-Powered Drug Classification and Indication Mapping for Pharmacoepidemiologic Studies: Prompt Development and Validation},
  journal = {JMIR AI},
  year = {2025},
  volume = {4},
  pages = {e65481},
  doi = {10.2196/65481}
}
Start for free →