EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

Efficiency-Aware Group Size Optimization for GRPO via Multi-Fidelity Bayesian Optimization

Taehyeon Kim, Kyung-Taek Lee

AI · 23.06.2026 · 7 (7) · 234

DOI: 10.3390/ai7070234 ↗

Kurz gefasst

Fachartikel in der Zeitschrift AI, online erschienen am 23.06.2026. Verfasst von Taehyeon Kim und 1 weiteren Autor:innen. Die Arbeit nennt 21 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: Korea government.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Group Relative Policy Optimization (GRPO) streamlines the alignment of Large Language Models (LLMs) and Vision–Language Models (VLMs) by eliminating the Critic model. However, its efficiency heavily depends on the group size, G. While a larger G improves reward estimation and stabilizes the Advantage, Ai, it drastically increases VRAM usage and reduces throughput. Standard heuristics like a fixed G of 64 create significant bottlenecks in resource-constrained settings. This paper introduces an Efficiency-Aware optimization framework utilizing Multi-fidelity Bayesian Optimization and Hyperband (BOHB) to dynamically identify the optimal group size, G∗. The method uses a multi-objective function that balances reward accuracy, Ai variance, and hardware utilization, applying z-score normalization. By employing Successive Halving to quickly evaluate candidates at low fidelity, the framework reduces search costs by up to 74% compared with random search. Tested across text-only LLMs (Qwen2.5-7B/1.5B) and multimodal VLMs (Qwen2.5-VL-3B), the framework demonstrates that the discovered G∗ saves up to 72.5% in VRAM compared with the baseline of 64, while maintaining reward accuracy within 5.8%. Sensitivity analyses on hyperparameters like λ, α, and β confirm the framework’s robustness. Rather than treating group size as a mere engineering heuristic, this study establishes a principled methodological advance by formalizing the trade-off between statistical estimation stability and hardware constraints into a unified optimization framework for resource-efficient RLHF.

Förderung

  • Korea government

Zitieren

APA

Kim, T., & Lee, K. (2026). Efficiency-Aware Group Size Optimization for GRPO via Multi-Fidelity Bayesian Optimization. AI, 7(7), 234. https://doi.org/10.3390/ai7070234

Vancouver

Kim T, Lee K. Efficiency-Aware Group Size Optimization for GRPO via Multi-Fidelity Bayesian Optimization. AI. 2026;7(7):234. doi:10.3390/ai7070234

BibTeX
@article{Kim2026,
  author = {Kim, Taehyeon and Lee, Kyung-Taek},
  title = {Efficiency-Aware Group Size Optimization for GRPO via Multi-Fidelity Bayesian Optimization},
  journal = {AI},
  year = {2026},
  volume = {7},
  number = {7},
  pages = {234},
  doi = {10.3390/ai7070234}
}
Kostenlos starten →