EUVIMEDEUVIMED entdecken
VergleichsstudieOpen Access

Assessing Eligibility for Anticancer Drug Health Insurance Reimbursement Using Large Language Models: Benchmark Development and Comparative Study

Junhyuk Seo, Taerim Kim, Ju-Hyun Kim

Journal of Medical Internet Research · 15.06.2026 · 28 · e95877-e95877

DOI: 10.2196/95877 ↗

Kurz gefasst

Vergleichsstudie in der Zeitschrift Journal of Medical Internet Research, online erschienen am 15.06.2026. Verfasst von Junhyuk Seo und 2 weiteren Autor:innen. Beteiligte Einrichtung: Healthcare Research Institute, ETOILE Inc, Seoul, Republic of Korea. Hauptthemen laut MEDLINE: Antineoplastic Agents, Benchmarking, Eligibility Determination, Insurance, Health, Reimbursement. Die Arbeit wurde bisher 1-mal zitiert (Crossref) und nennt 31 Literaturangaben. Der Volltext ist frei in PubMed Central verfügbar.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Administrative costs in the health care system are driven in part by complex insurance eligibility determinations. Large language models (LLMs) are increasingly used for health insurance–related queries, yet their reliability for structured logical reasoning over coverage criteria has not been systematically evaluated.

Objective

This study aimed to develop a benchmark for anticancer drug reimbursement eligibility determination and evaluate whether LLMs can reliably perform eligibility verification.

Methods

We constructed a benchmark based on South Korea’s National Health Insurance reimbursement guidelines for 3 gynecologic cancers (cervical, uterine, and ovarian), using a tristate adjudication framework (eligible, ineligible, and undeterminable). Three gynecologic oncology experts and a utilization review nurse validated the benchmark. Six LLMs from 3 providers (Anthropic, Google, and OpenAI) were evaluated using the official guideline document as input. Each case was evaluated 3 times per model, with final predictions determined by majority vote, and performance was compared across the 3 outcome classes.

Results

The benchmark comprises 74 anticancer regimens with 222 cases. Overall verification accuracy ranged from 77.9% to 88.7% across the 6 models. Eligible and ineligible cases were classified with high recall (86.5%‐98.6%), but undeterminable cases showed a marked decline across all models (44.6%‐70.3%). Performance varied by cancer type, with uterine cancer showing the lowest undeterminable recall (16.7%), corresponding to the highest guideline complexity. Undeterminable cases were predominantly misclassified as eligible rather than ineligible. The tristate framework enabled logic-based error analysis of 235 incorrect predictions, revealing information gap-filling as the dominant failure pattern (n=196, 83.4%), followed by criterion misapplication (n=20, 8.5%) and false uncertainty (n=19, 8.1%). Subtype analysis indicated that information gap-filling errors were concentrated at hierarchical elements of the guideline. Sensitivity analyses showed that converting the guideline document to structured text degraded performance, while web search–enabled condition (0%‐3.2% tool invocation across models) and structure-guided prompting did not produce significant changes from baseline.

Conclusions

In this benchmark, LLMs classified clearly eligible and ineligible cases with relatively high recall but showed limited reliability on undeterminable cases. The dominant error pattern was information gap-filling, in which models inferred eligibility rather than withholding judgment. These findings indicate that LLMs, in their current form, should be deployed as supervised decision-support tools rather than as independent adjudicators in reimbursement review.

Schlagwörter

MeSH-Schlagwörter (MEDLINE)

Antineoplastic AgentsBenchmarkingEligibility DeterminationInsurance, Health, ReimbursementLarge Language ModelsFemaleHumansRepublic of Korea

Stichwörter der Autor:innen

National Health Insuranceanticancer drugsbenchmarkclinical decision supporteligibility verificationgynecologic cancerhealth insurancelarge language modelsnatural language processingreimbursement

Substanzen

Antineoplastic Agents

Autor:innen und Institutionen

  • Healthcare Research Institute, ETOILE Inc, Seoul, Republic of Korea.
  • Department of Digital Health, Samsung Advanced Institute of Health Sciences and Technology (SAIHST), Sungkyunkwan University, Seoul, Republic of Korea.
  • Department of Emergency Medicine, Sungkyunkwan University School of Medicine, Samsung Medical Center, Seoul, Republic of Korea.
  • Department of Obstetrics and Gynecology, University of Ulsan College of Medicine, Asan Medical Center, 88, Olympic-ro 43-gil, Songpa-gu, Seoul, 05505, Republic of Korea, 82 10-9668-6227.

Zitieren

APA

Seo, J., Kim, T., & Kim, J. H. (2026). Assessing Eligibility for Anticancer Drug Health Insurance Reimbursement Using Large Language Models: Benchmark Development and Comparative Study. Journal of Medical Internet Research, 28, e95877-e95877. https://doi.org/10.2196/95877

Vancouver

Seo J, Kim T, Kim JH. Assessing Eligibility for Anticancer Drug Health Insurance Reimbursement Using Large Language Models: Benchmark Development and Comparative Study. Journal of Medical Internet Research. 2026;28:e95877-e95877. doi:10.2196/95877

BibTeX
@article{Seo2026,
  author = {Seo, Junhyuk and Kim, Taerim and Kim, Ju-Hyun},
  title = {Assessing Eligibility for Anticancer Drug Health Insurance Reimbursement Using Large Language Models: Benchmark Development and Comparative Study},
  journal = {Journal of Medical Internet Research},
  year = {2026},
  volume = {28},
  pages = {e95877--e95877},
  doi = {10.2196/95877}
}
Kostenlos starten →