EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Artificial Intelligence to Automate Health Economic Modelling: A Case Study to Evaluate the Potential Application of Large Language Models

Tim Reason, William Rawlinson, Julia Langham, Andy Gimblett, Bill Malcolm, Sven Klijn

PharmacoEconomics - Open · 2024-03 · 8 (2) · 191-203

DOI: 10.1007/s41669-024-00477-8 ↗

Kurz gefasst

Fachartikel in der Zeitschrift PharmacoEconomics - Open, online erschienen am 10.02.2024. Verfasst von Tim Reason und 5 weiteren Autor:innen. Beteiligte Einrichtung: Estima Scientific, Mediaworks, 191 Wood Ln, London, W12 7FP, UK. tim.reason@estima-sci.com. Die Arbeit wurde bisher 45-mal zitiert (Crossref) und nennt 40 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0). Gefördert von: Bristol-Myers Squibb.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

BACKGROUND: Current generation large language models (LLMs) such as Generative Pre-Trained Transformer 4 (GPT-4) have achieved human-level performance on many tasks including the generation of computer code based on textual input. This study aimed to assess whether GPT-4 could be used to automatically programme two published health economic analyses. METHODS: The two analyses were partitioned survival models evaluating interventions in non-small cell lung cancer (NSCLC) and renal cell carcinoma (RCC). We developed prompts which instructed GPT-4 to programme the NSCLC and RCC models in R, and which provided descriptions of each model's methods, assumptions and parameter values. The results of the generated scripts were compared to the published values from the original, human-programmed models. The models were replicated 15 times to capture variability in GPT-4's output. RESULTS: GPT-4 fully replicated the NSCLC model with high accuracy: 100% (15/15) of the artificial intelligence (AI)-generated NSCLC models were error-free or contained a single minor error, and 93% (14/15) were completely error-free. GPT-4 closely replicated the RCC model, although human intervention was required to simplify an element of the model design (one of the model's fifteen input calculations) because it used too many sequential steps to be implemented in a single prompt. With this simplification, 87% (13/15) of the AI-generated RCC models were error-free or contained a single minor error, and 60% (9/15) were completely error-free. Error-free model scripts replicated the published incremental cost-effectiveness ratios to within 1%. CONCLUSION: This study provides a promising indication that GPT-4 can have practical applications in the automation of health economic model construction. Potential benefits include accelerated model development timelines and reduced costs of development. Further research is necessary to explore the generalisability of LLM-based automation across a larger sample of models.

Autor:innen und Institutionen

  • Estima Scientific, Mediaworks, 191 Wood Ln, London, W12 7FP, UK. tim.reason@estima-sci.com.
  • Estima Scientific, Mediaworks, 191 Wood Ln, London, W12 7FP, UK.
  • Bristol Myers Squibb, Uxbridge, UK.
  • Bristol Myers Squibb, Princeton, NJ, USA.

Förderung

  • Bristol-Myers Squibb

Zitieren

APA

Reason, T., Rawlinson, W., Langham, J., Gimblett, A., Malcolm, B., & Klijn, S. (2024). Artificial Intelligence to Automate Health Economic Modelling: A Case Study to Evaluate the Potential Application of Large Language Models. PharmacoEconomics - Open, 8(2), 191-203. https://doi.org/10.1007/s41669-024-00477-8

Vancouver

Reason T, Rawlinson W, Langham J, Gimblett A, Malcolm B, Klijn S. Artificial Intelligence to Automate Health Economic Modelling: A Case Study to Evaluate the Potential Application of Large Language Models. PharmacoEconomics - Open. 2024;8(2):191-203. doi:10.1007/s41669-024-00477-8

BibTeX
@article{Reason2024,
  author = {Reason, Tim and Rawlinson, William and Langham, Julia and Gimblett, Andy and Malcolm, Bill and Klijn, Sven},
  title = {Artificial Intelligence to Automate Health Economic Modelling: A Case Study to Evaluate the Potential Application of Large Language Models},
  journal = {PharmacoEconomics - Open},
  year = {2024},
  volume = {8},
  number = {2},
  pages = {191--203},
  doi = {10.1007/s41669-024-00477-8}
}
Kostenlos starten →