EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC 4.0

Efficiency Analysis of Fine-Tuning Compact Language Models for Biomedical Information Retrieval Systems

Martony Demes da Silva

Revista de Informática Teórica e Aplicada · 10.08.2026 · 33 (4) · 21-31

DOI: 10.22456/2175-2745.153249 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Revista de Informática Teórica e Aplicada, online erschienen am 10.08.2026. Verfasst von Martony Demes da Silva. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The rise of semantic search systems and Retrieval-Augmented Generation (RAG) architectures in the biomedical domain poses critical challenges regarding computational costs and data sovereignty. This paper investigates the effectiveness of specializing ultra-compact language models for clinical embedding generation, utilizing the EmbeddingGemma 300M as the base model. By applying Low-Rank Adaptation (LoRA) techniques and low-level optimizations via the Unsloth library, the study demonstrates the feasibility of performing fine-tuning on entry-level hardware (NVIDIA Tesla T4). Experimental results show a 79.5% reduction in Multiple Negatives Ranking Loss (MNRL) in only 44.7 minutes, consuming just 6.56 GB of VRAM. The achieved throughput of 3,809 tokens/s and the residual operational cost (~$0.26) validate the proposal as a sustainable and private alternative. The results demonstrate competitive performance, approaching the effectiveness of significantly larger models in specific biomedical retrieval tasks with a fraction of the computational cost, thus promoting technological democratization for healthcare institutions with limited resources.

Zitieren

APA

Silva, M. D. D. (2026). Efficiency Analysis of Fine-Tuning Compact Language Models for Biomedical Information Retrieval Systems. Revista de Informática Teórica e Aplicada, 33(4), 21-31. https://doi.org/10.22456/2175-2745.153249

Vancouver

Silva MDD. Efficiency Analysis of Fine-Tuning Compact Language Models for Biomedical Information Retrieval Systems. Revista de Informática Teórica e Aplicada. 2026;33(4):21-31. doi:10.22456/2175-2745.153249

BibTeX
@article{Silva2026,
  author = {Silva, Martony Demes da},
  title = {Efficiency Analysis of Fine-Tuning Compact Language Models for Biomedical Information Retrieval Systems},
  journal = {Revista de Informática Teórica e Aplicada},
  year = {2026},
  volume = {33},
  number = {4},
  pages = {21--31},
  doi = {10.22456/2175-2745.153249}
}
Kostenlos starten →