EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Evaluating Reranking Strategies for Portuguese Information Retrieval: Fine-Tuning, LLMs, and Sociocultural Aspects

Renato Okabayashi Miyaji, Pedro Luiz Pizzigatti Corrêa

Journal of the Brazilian Computer Society · 05.05.2026 · 32 (1) · 1207-1220

DOI: 10.5753/jbcs.2026.5659 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of the Brazilian Computer Society, online erschienen am 05.05.2026. Verfasst von Renato Okabayashi Miyaji und 1 weiteren Autor:innen. Die Arbeit nennt 27 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Reranking plays a crucial role in improving Information Retrieval (IR) performance, particularly in low-resource languages, such as Portuguese. In this study, we evaluate different reranking strategies for Portuguese IR, comparing multilingual and Portuguese-specific models, as well as not-so-large language models and large language models (LLMs). We assess the performance of BM25 combined with ptT5 fine-tuned on multilingual and Brazilian Portuguese datasets, alongside multilingual state-of-the-art rerankers (BGE m3) and LLM as rerankers RankGPT (GPT-4) and Sabiá 3, a Portuguese-specific LLM. Additionally, we introduce a novel dynamic In-Context Learning (DICL) prompting strategy to enhance LLM performance. Experiments conducted on the Quati and Pirá 2.0 datasets show that fine-tuning on native Brazilian Portuguese data significantly improves retrieval effectiveness by up to 5 p.p. in nDCG compared to using translated multilingual datasets. Two fine-tuning approaches were tested: a binary classification strategy with ‘true’ and ‘false’ tokens and a relevance score-based training, both outperforming models fine-tuned on translated multilingual data. RankGPT achieved the best overall results, yet Sabiá 3 demonstrated competitive performance, particularly on queries related to sociocultural aspects. The DICL strategy further improved the results of both LLMs, significantly boosting their MRR@10. These findings highlight the importance of language-specific training and suggest that not-so-large language models can be viable alternatives for reranking tasks in Portuguese IR.

Zitieren

APA

Miyaji, R. O., & Corrêa, P. L. P. (2026). Evaluating Reranking Strategies for Portuguese Information Retrieval: Fine-Tuning, LLMs, and Sociocultural Aspects. Journal of the Brazilian Computer Society, 32(1), 1207-1220. https://doi.org/10.5753/jbcs.2026.5659

Vancouver

Miyaji RO, Corrêa PLP. Evaluating Reranking Strategies for Portuguese Information Retrieval: Fine-Tuning, LLMs, and Sociocultural Aspects. Journal of the Brazilian Computer Society. 2026;32(1):1207-1220. doi:10.5753/jbcs.2026.5659

BibTeX
@article{Miyaji2026,
  author = {Miyaji, Renato Okabayashi and Corrêa, Pedro Luiz Pizzigatti},
  title = {Evaluating Reranking Strategies for Portuguese Information Retrieval: Fine-Tuning, LLMs, and Sociocultural Aspects},
  journal = {Journal of the Brazilian Computer Society},
  year = {2026},
  volume = {32},
  number = {1},
  pages = {1207--1220},
  doi = {10.5753/jbcs.2026.5659}
}
Kostenlos starten →