Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings
Bioengineering · 25.03.2026 · 13 (4) · 378
DOI: 10.3390/bioengineering13040378 ↗Kurz gefasst
Fachartikel in der Zeitschrift Bioengineering, online erschienen am 25.03.2026. Verfasst von Nora Y Sun und 8 weiteren Autor:innen. Beteiligte Einrichtung: Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA. Die Arbeit nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Schmidt Sciences, Dalio Philanthropies, Gerstner Philanthropies.
Abstract (Originalsprache)
Background
Large language models (LLMs) are being used by surgeons for education and reference yet concerns about hallucinations and reliability limit safe adoption. Retrieval-augmented generation (RAG) can offer a potential solution by grounding responses in a high-quality external database (e.g., medical textbooks) to enhance accuracy. However, performance tradeoffs across different RAG configurations—many of which exponentially increase computational cost—remain poorly characterized.
Methods
In total, 120 lightweight, open-source RAG configurations were evaluated across 40 plastic surgery-focused question-answering tasks (20 single-hop, 20 multi-hop), spanning multiple subspecialties (4800 total evaluations). Configurations varied by base LLM (Phi-3-mini-128k-instruct vs. BioMistral-7B), embedding model, database size, chunk size, and query hop type. Performance was assessed using semantic similarity (Ragas) to physician-validated reference answers. Performance was analyzed using linear mixed-effects regression with query as a random effect and fixed and interaction effects selected via likelihood testing and AIC.
Results
High performance was achievable using lightweight, open-source models. While BioMistral-7B had high mean sematic similarity under specific configurations (mean semantic similarity up to 0.786), Phi-3-mini-128k-instruct demonstrated more consistent performance across query complexity. Larger database sizes significantly improved semantic similarity, with the largest gain at intermediate sizes (e.g., size 5: +0.043, p = 0.001). Embedding choice had a strong effect, with bge-large-en-v1.5 improving performance (p = 0.0016) and Bio_ClinicalBERT markedly reducing it (p < 0.001). Multi-hop queries substantially reduced performance (p < 0.001), though this effect was attenuated for Phi-3-mini-128k-instruct via a strong model × hop-type interaction (p < 0.001).
Conclusions
RAG systems for plastic surgery do not require large proprietary models, as performance depends on configuration choices and interaction effects rather than isolated components. With advancements, predictive modeling may enable resource-efficient, safe deployment of clinical RAG systems.
Schlagwörter
Stichwörter der Autor:innen
Autor:innen und Institutionen
- Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA.
- Department of Statistics, Harvard University, Cambridge, MA 02138, USA.
- Department of Data Science, University of California, Berkeley, CA 94720, USA.
- Department of Radiology AI IT, Mayo Clinic, Rochester, MN 55905, USA.
- Center for Digital Health, Mayo Clinic, Rochester, MN 55905, USA.
- Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL 32224, USA.
ORCID-Profile: Nora Y Sun, Syed Ali Haider, Bernardo Collaco, Nadia G Wood, Antonio Jorge Forte
Förderung
- Schmidt Sciences
- Dalio Philanthropies
- Gerstner Philanthropies
Zitieren
APA
Sun, N. Y., Genovese, A., Prabha, S., Gomez-Cabello, C. A., Haider, S. A., Collaco, B., Pan, T., Wood, N. G., & Forte, A. J. (2026). Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings. Bioengineering, 13(4), 378. https://doi.org/10.3390/bioengineering13040378
Vancouver
Sun NY, Genovese A, Prabha S, Gomez-Cabello CA, Haider SA, Collaco B, et al. Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings. Bioengineering. 2026;13(4):378. doi:10.3390/bioengineering13040378
BibTeX
@article{Sun2026,
author = {Sun, Nora Y and Genovese, Ariana and Prabha, Srinivasagam and Gomez-Cabello, Cesar A and Haider, Syed Ali and Collaco, Bernardo and Pan, Theophilus and Wood, Nadia G and Forte, Antonio Jorge},
title = {Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings},
journal = {Bioengineering},
year = {2026},
volume = {13},
number = {4},
pages = {378},
doi = {10.3390/bioengineering13040378}
}
EUVIMED