EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare

Lovedeep Gondara, Jonathan Simkin, Graham Sayle, Shebnum Devji, Gregory Arbour, Raymond Ng

Machine Learning and Knowledge Extraction · 17.10.2025 · 7 (4) · 121

DOI: 10.3390/make7040121 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Machine Learning and Knowledge Extraction, online erschienen am 17.10.2025. Verfasst von Lovedeep Gondara und 5 weiteren Autor:innen. Die Arbeit wurde bisher 10-mal zitiert (Crossref) und nennt 15 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Objectives

To guide language model (LM) selection by comparing finetuning vs. zero-shot use, generic pretraining vs. domain-adjacent vs. further domain-specific pretraining, and bidirectional language models (BiLMs) such as BERT vs. unidirectional LMs (LLMs) for clinical classification. Materials and Methods: We evaluated BiLMs (RoBERTa, PathologyBERT, Gatortron) and LLM (Mistral nemo instruct 12B) on three British Columbia Cancer Registry (BCCR) pathology classification tasks varying in difficulty/data size. We assessed zero-shot vs. finetuned BiLMs, zero-shot LLM, and further BCCR-specific pretraining using macro-average F1 scores.

Results

Finetuned BiLMs outperformed zero-shot BiLMs and zero-shot LLM. The zero-shot LLM outperformed zero-shot BiLMs but was consistently outperformed by finetuned BiLMs. Domain-adjacent BiLMs generally outperformed generic BiLMs after finetuning. Further domain-specific pretraining boosted complex/low-data task performance, with otherwise modest gains.

Conclusions

For specialized classification, finetuning BiLMs is crucial, often surpassing zero-shot LLMs. Domain-adjacent pretrained models are recommended. Further domain-specific pretraining provides significant performance boosts, especially for complex/low-data scenarios. BiLMs remain relevant, offering strong performance/resource balance for targeted clinical tasks.

Zitieren

APA

Gondara, L., Simkin, J., Sayle, G., Devji, S., Arbour, G., & Ng, R. (2025). Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare. Machine Learning and Knowledge Extraction, 7(4), 121. https://doi.org/10.3390/make7040121

Vancouver

Gondara L, Simkin J, Sayle G, Devji S, Arbour G, Ng R. Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare. Machine Learning and Knowledge Extraction. 2025;7(4):121. doi:10.3390/make7040121

BibTeX
@article{Gondara2025,
  author = {Gondara, Lovedeep and Simkin, Jonathan and Sayle, Graham and Devji, Shebnum and Arbour, Gregory and Ng, Raymond},
  title = {Small or Large? Zero-Shot or Finetuned? Guiding Language Model Choice for Specialized Applications in Healthcare},
  journal = {Machine Learning and Knowledge Extraction},
  year = {2025},
  volume = {7},
  number = {4},
  pages = {121},
  doi = {10.3390/make7040121}
}
Kostenlos starten →