Evaluating ChatGPT text mining of clinical records for companion animal obesity monitoring
Veterinary Record · 03.02.2024 · 194 (3)
DOI: 10.1002/vetr.3669 ↗Kurz gefasst
Fachartikel in der Zeitschrift Veterinary Record, online erschienen am 06.12.2023. Verfasst von Ivo S Fins und 6 weiteren Autor:innen. Beteiligte Einrichtung: Small Animal Veterinary Surveillance Network, Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Liverpool, UK. Hauptthemen laut MEDLINE: Pets, Data Mining. Die Arbeit wurde bisher 12-mal zitiert (Crossref) und nennt 19 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Biotechnology and Biological Sciences Research Council, Dogs Trust, Petplan Charitable Trust.
Abstract (Originalsprache)
Background
Veterinary clinical narratives remain a largely untapped resource for addressing complex diseases. Here we compare the ability of a large language model (ChatGPT) and a previously developed regular expression (RegexT) to identify overweight body condition scores (BCS) in veterinary narratives pertaining to companion animals.
Methods
BCS values were extracted from 4415 anonymised clinical narratives using either RegexT or by appending the narrative to a prompt sent to ChatGPT, prompting the model to return the BCS information. Data were manually reviewed for comparison.
Results
The precision of RegexT was higher (100%, 95% confidence interval [CI] 94.81%–100%) than that of ChatGPT (89.3%, 95% CI 82.75%–93.64%). However, the recall of ChatGPT (100%, 95% CI 96.18%–100%) was considerably higher than that of RegexT (72.6%, 95% CI 63.92%–79.94%).
Limitations
Prior anonymisation and subtle prompt engineering are needed to improve ChatGPT output.
Conclusions
Large language models create diverse opportunities and, while complex, present an intuitive interface to information. However, they require careful implementation to avoid unpredictable errors.
Schlagwörter
MeSH-Schlagwörter (MEDLINE)
Autor:innen und Institutionen
- Small Animal Veterinary Surveillance Network, Institute of Infection, Veterinary and Ecological Sciences, University of Liverpool, Liverpool, UK.
- Department of Computer Science, Durham University, Durham, UK.
- Institute for Animal Health and Food Safety, University of Las Palmas de Gran Canaria, Las Palmas, Gran Canaria, Spain.
ORCID-Profile: Ivo S Fins, Heather Davies
Förderung
- Biotechnology and Biological Sciences Research Council
- Dogs Trust
- Petplan Charitable Trust
- British Small Animal Veterinary Association
Zitieren
APA
Fins, I. S., Davies, H., Farrell, S., Torres, J. R., Pinchbeck, G., Radford, A. D., & Noble, P. J. (2024). Evaluating ChatGPT text mining of clinical records for companion animal obesity monitoring. Veterinary Record, 194(3). https://doi.org/10.1002/vetr.3669
Vancouver
Fins IS, Davies H, Farrell S, Torres JR, Pinchbeck G, Radford AD, et al. Evaluating ChatGPT text mining of clinical records for companion animal obesity monitoring. Veterinary Record. 2024;194(3). doi:10.1002/vetr.3669
BibTeX
@article{Fins2024,
author = {Fins, Ivo S and Davies, Heather and Farrell, Sean and Torres, Jose R and Pinchbeck, Gina and Radford, Alan D and Noble, Peter-John},
title = {Evaluating ChatGPT text mining of clinical records for companion animal obesity monitoring},
journal = {Veterinary Record},
year = {2024},
volume = {194},
number = {3},
doi = {10.1002/vetr.3669}
}
EUVIMED