EUVIMEDOpdag EUVIMED
Open access · CC BY 4.0

Investigating the Refactoring Capabilities of Small Open-Weight Language Models

Tamás Márton, Balázs Szalontai, Balázs Pintér, Tibor Gregorics

Software · 2026-04-29 · 5 (2) · 19

DOI: 10.3390/software5020019 ↗

At a glance

Journal article in Software, published online on 2026-04-29. Written by Tamás Márton and 3 co-authors. 25 references. The full text is freely available (open access, CC BY 4.0). Funded by: KÖP-25 University Excellence Scholarship Program of the Ministry for Culture and Innovation from the source of the National Research, Development and Innovatio….

AbstractBibliografiske oplysningerCiteRelated studies

Abstract (original language)

Refactoring is essential for developing maintainable software. Using Large Language Models in software engineering is widespread, but compared to well-established domains such as code generation, reliable refactoring is still relatively underexplored. In this paper, we perform a broad analysis on the refactoring capabilities of small open-weight language models (SLMs) by evaluating 12 models on 3453 Python programs. Our study focuses on the two defining aspects of refactoring: behavior preservation and code quality improvement. We evaluate these properties using unit tests and various code metrics. Across models ranging from 0.5B to 8 B parameters, most models improve code quality. Larger models are more reliable, as they preserve behavior more consistently. Reasoning models often make more significant changes while refactoring. Allowing models to generate reasoning traces improves performance, but only for models larger than 4B. For smaller models, reasoning in fact reduces refactoring reliability. The difficulty of the underlying task affects refactoring performance, with more complex tasks associated with higher failure rates. Our results indicate that current open SLMs can support refactoring tasks, especially larger ones with reasoning capabilities, but they are best used with human oversight.

Funding

  • KÖP-25 University Excellence Scholarship Program of the Ministry for Culture and Innovation from the source of the National Research, Development and Innovatio…

Cite

APA

Márton, T., Szalontai, B., Pintér, B., & Gregorics, T. (2026). Investigating the Refactoring Capabilities of Small Open-Weight Language Models. Software, 5(2), 19. https://doi.org/10.3390/software5020019

Vancouver

Márton T, Szalontai B, Pintér B, Gregorics T. Investigating the Refactoring Capabilities of Small Open-Weight Language Models. Software. 2026;5(2):19. doi:10.3390/software5020019

BibTeX
@article{Mrton2026,
  author = {Márton, Tamás and Szalontai, Balázs and Pintér, Balázs and Gregorics, Tibor},
  title = {Investigating the Refactoring Capabilities of Small Open-Weight Language Models},
  journal = {Software},
  year = {2026},
  volume = {5},
  number = {2},
  pages = {19},
  doi = {10.3390/software5020019}
}
Start for free →