BENCHMARK DE MODELOS DE LINGUAGEM DE CÓDIGO ABERTO PARA TEXT-TO-SQL EM DADOS ONCOLÓGICOS BRASILEIROS
REMUNOM · 26.06.2026 · 13 (14) · 1-67
DOI: 10.66104/zvzdrv85 ↗Kurz gefasst
Fachartikel in der Zeitschrift REMUNOM, online erschienen am 26.06.2026. Verfasst von Fabrício de Carvalho Mota und 2 weiteren Autor:innen. Die Arbeit nennt 336 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
This work presents an experimental benchmark to evaluate open-source language models in the Text-to-SQL task, using questions in Brazilian Portuguese and public data on oncology care in Brazil. The study compared Qwen2.5-Coder 7B, DeepSeek-Coder 6.7B, and Llama 3, executed locally through Ollama, across four experimental scenarios defined by the presence or absence of the database schema in the prompt and by the presence or absence of linguistic variations in the questions. The evaluation was based on the Execution Accuracy (EX) metric, complemented by an analysis of error categories and generation and execution latencies. The results showed that Qwen2.5-Coder 7B achieved the best overall performance, with an EX of 80.0%, as well as the lowest average generation latency. Providing the database schema was the most influential factor, increasing the average model accuracy from 53.9% to 93.3%. The linguistic variations used did not degrade the average model performance, suggesting robustness only to the controlled superficial noise adopted in the experiment. The findings indicate that open-source language models can be applied to SQL query generation in controlled scenarios, although their performance strongly depends on the structural context provided and on the quality of the question formulation.
Zitieren
APA
Mota, F. D. C., Silva, W. D. V. R. D., & Soares, J. D. N. (2026). BENCHMARK DE MODELOS DE LINGUAGEM DE CÓDIGO ABERTO PARA TEXT-TO-SQL EM DADOS ONCOLÓGICOS BRASILEIROS. REMUNOM, 13(14), 1-67. https://doi.org/10.66104/zvzdrv85
Vancouver
Mota FDC, Silva WDVRD, Soares JDN. BENCHMARK DE MODELOS DE LINGUAGEM DE CÓDIGO ABERTO PARA TEXT-TO-SQL EM DADOS ONCOLÓGICOS BRASILEIROS. REMUNOM. 2026;13(14):1-67. doi:10.66104/zvzdrv85
BibTeX
@article{Mota2026,
author = {Mota, Fabrício de Carvalho and Silva, Wanderson de Vasconcelos Rodrigues da and Soares, Jeferson do Nascimento},
title = {BENCHMARK DE MODELOS DE LINGUAGEM DE CÓDIGO ABERTO PARA TEXT-TO-SQL EM DADOS ONCOLÓGICOS BRASILEIROS},
journal = {REMUNOM},
year = {2026},
volume = {13},
number = {14},
pages = {1--67},
doi = {10.66104/zvzdrv85}
}
EUVIMED