EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings

Nora Y Sun iD, Ariana Genovese, Srinivasagam Prabha, Cesar A Gomez-Cabello, Syed Ali Haider iD, Bernardo Collaco iD, Theophilus Pan, Nadia G Wood iD, Antonio Jorge Forte iD

Bioengineering · 25.03.2026 · 13 (4) · 378

DOI: 10.3390/bioengineering13040378 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Bioengineering, online erschienen am 25.03.2026. Verfasst von Nora Y Sun und 8 weiteren Autor:innen. Beteiligte Einrichtung: Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA. Die Arbeit nennt 48 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Schmidt Sciences, Dalio Philanthropies, Gerstner Philanthropies.

AbstractSchlagwörterBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Background

Large language models (LLMs) are being used by surgeons for education and reference yet concerns about hallucinations and reliability limit safe adoption. Retrieval-augmented generation (RAG) can offer a potential solution by grounding responses in a high-quality external database (e.g., medical textbooks) to enhance accuracy. However, performance tradeoffs across different RAG configurations—many of which exponentially increase computational cost—remain poorly characterized.

Methods

In total, 120 lightweight, open-source RAG configurations were evaluated across 40 plastic surgery-focused question-answering tasks (20 single-hop, 20 multi-hop), spanning multiple subspecialties (4800 total evaluations). Configurations varied by base LLM (Phi-3-mini-128k-instruct vs. BioMistral-7B), embedding model, database size, chunk size, and query hop type. Performance was assessed using semantic similarity (Ragas) to physician-validated reference answers. Performance was analyzed using linear mixed-effects regression with query as a random effect and fixed and interaction effects selected via likelihood testing and AIC.

Results

High performance was achievable using lightweight, open-source models. While BioMistral-7B had high mean sematic similarity under specific configurations (mean semantic similarity up to 0.786), Phi-3-mini-128k-instruct demonstrated more consistent performance across query complexity. Larger database sizes significantly improved semantic similarity, with the largest gain at intermediate sizes (e.g., size 5: +0.043, p = 0.001). Embedding choice had a strong effect, with bge-large-en-v1.5 improving performance (p = 0.0016) and Bio_ClinicalBERT markedly reducing it (p < 0.001). Multi-hop queries substantially reduced performance (p < 0.001), though this effect was attenuated for Phi-3-mini-128k-instruct via a strong model × hop-type interaction (p < 0.001).

Conclusions

RAG systems for plastic surgery do not require large proprietary models, as performance depends on configuration choices and interaction effects rather than isolated components. With advancements, predictive modeling may enable resource-efficient, safe deployment of clinical RAG systems.

Schlagwörter

Stichwörter der Autor:innen

artificial intelligencelarge language modelsplastic surgeryretrieval-augmented generation

Autor:innen und Institutionen

  • Division of Plastic Surgery, Mayo Clinic, Jacksonville, FL 32224, USA.
  • Department of Statistics, Harvard University, Cambridge, MA 02138, USA.
  • Department of Data Science, University of California, Berkeley, CA 94720, USA.
  • Department of Radiology AI IT, Mayo Clinic, Rochester, MN 55905, USA.
  • Center for Digital Health, Mayo Clinic, Rochester, MN 55905, USA.
  • Department of Artificial Intelligence and Informatics, Mayo Clinic, Jacksonville, FL 32224, USA.

ORCID-Profile: Nora Y Sun, Syed Ali Haider, Bernardo Collaco, Nadia G Wood, Antonio Jorge Forte

Förderung

  • Schmidt Sciences
  • Dalio Philanthropies
  • Gerstner Philanthropies

Zitieren

APA

Sun, N. Y., Genovese, A., Prabha, S., Gomez-Cabello, C. A., Haider, S. A., Collaco, B., Pan, T., Wood, N. G., & Forte, A. J. (2026). Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings. Bioengineering, 13(4), 378. https://doi.org/10.3390/bioengineering13040378

Vancouver

Sun NY, Genovese A, Prabha S, Gomez-Cabello CA, Haider SA, Collaco B, et al. Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings. Bioengineering. 2026;13(4):378. doi:10.3390/bioengineering13040378

BibTeX
@article{Sun2026,
  author = {Sun, Nora Y and Genovese, Ariana and Prabha, Srinivasagam and Gomez-Cabello, Cesar A and Haider, Syed Ali and Collaco, Bernardo and Pan, Theophilus and Wood, Nadia G and Forte, Antonio Jorge},
  title = {Performance Modeling of Lightweight Retrieval-Augmented Large Language Models for Low-Resource Plastic Surgery Settings},
  journal = {Bioengineering},
  year = {2026},
  volume = {13},
  number = {4},
  pages = {378},
  doi = {10.3390/bioengineering13040378}
}
Kostenlos starten →