Efficiency Analysis of Fine-Tuning Compact Language Models for Biomedical Information Retrieval Systems
Revista de Informática Teórica e Aplicada · 10.08.2026 · 33 (4) · 21-31
DOI: 10.22456/2175-2745.153249 ↗Kurz gefasst
Fachartikel in der Zeitschrift Revista de Informática Teórica e Aplicada, online erschienen am 10.08.2026. Verfasst von Martony Demes da Silva. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC 4.0).
Abstract (Originalsprache)
The rise of semantic search systems and Retrieval-Augmented Generation (RAG) architectures in the biomedical domain poses critical challenges regarding computational costs and data sovereignty. This paper investigates the effectiveness of specializing ultra-compact language models for clinical embedding generation, utilizing the EmbeddingGemma 300M as the base model. By applying Low-Rank Adaptation (LoRA) techniques and low-level optimizations via the Unsloth library, the study demonstrates the feasibility of performing fine-tuning on entry-level hardware (NVIDIA Tesla T4). Experimental results show a 79.5% reduction in Multiple Negatives Ranking Loss (MNRL) in only 44.7 minutes, consuming just 6.56 GB of VRAM. The achieved throughput of 3,809 tokens/s and the residual operational cost (~$0.26) validate the proposal as a sustainable and private alternative. The results demonstrate competitive performance, approaching the effectiveness of significantly larger models in specific biomedical retrieval tasks with a fraction of the computational cost, thus promoting technological democratization for healthcare institutions with limited resources.
Zitieren
APA
Silva, M. D. D. (2026). Efficiency Analysis of Fine-Tuning Compact Language Models for Biomedical Information Retrieval Systems. Revista de Informática Teórica e Aplicada, 33(4), 21-31. https://doi.org/10.22456/2175-2745.153249
Vancouver
Silva MDD. Efficiency Analysis of Fine-Tuning Compact Language Models for Biomedical Information Retrieval Systems. Revista de Informática Teórica e Aplicada. 2026;33(4):21-31. doi:10.22456/2175-2745.153249
BibTeX
@article{Silva2026,
author = {Silva, Martony Demes da},
title = {Efficiency Analysis of Fine-Tuning Compact Language Models for Biomedical Information Retrieval Systems},
journal = {Revista de Informática Teórica e Aplicada},
year = {2026},
volume = {33},
number = {4},
pages = {21--31},
doi = {10.22456/2175-2745.153249}
}
EUVIMED