EUVIMEDEUVIMED entdecken
Crossref

Difficulty Is Not Enough: Curriculum Learning for LLMs Fine-tuning Must Consider Utility

Zishang Jiang, Jinyi Han, Tingyun Li, Xinyi Wang, Sihang Jiang, Xiaojun Meng, Jiansheng Wei, Jiaqing Liang, Yanghua Xiao

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 31365-31373

DOI: 10.1609/aaai.v40i37.40400 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Zishang Jiang und 8 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Fine-tuning plays an essential role in improving the performance of large language models (LLMs) on specific tasks. A central challenge lies in designing data-efficient strategy to achieve better fine-tuning performance. Curriculum learning, which organizes data from easy to hard, has become a widely adopted technique in LLMs training. However, existing methods for curriculum learning focus only on the difficulty of samples, while neglecting their contribution to improving model performance, making them vulnerable when applied to fine-tuning LLMs. To address this, we propose Difficulty-Utility Curriculum Learning (DUCL), a curriculum learning framework that jointly considers difficulty and utility. DUCL introduces a novel scoring method, Difficulty-Utility Evaluation (DUE), and a soft scheduling strategy called Window Ordering, which together promote efficient and effective fine-tuning. Our method not only improves convergence and final performance with negligible computational overhead, but is also broadly applicable across a wide range of tasks, making it a practical and scalable solution for LLMs fine-tuning.

Zitieren

APA

Jiang, Z., Han, J., Li, T., Wang, X., Jiang, S., Meng, X., Wei, J., Liang, J., & Xiao, Y. (2026). Difficulty Is Not Enough: Curriculum Learning for LLMs Fine-tuning Must Consider Utility. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 31365-31373. https://doi.org/10.1609/aaai.v40i37.40400

Vancouver

Jiang Z, Han J, Li T, Wang X, Jiang S, Meng X, et al. Difficulty Is Not Enough: Curriculum Learning for LLMs Fine-tuning Must Consider Utility. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):31365-31373. doi:10.1609/aaai.v40i37.40400

BibTeX
@article{Jiang2026,
  author = {Jiang, Zishang and Han, Jinyi and Li, Tingyun and Wang, Xinyi and Jiang, Sihang and Meng, Xiaojun and Wei, Jiansheng and Liang, Jiaqing and Xiao, Yanghua},
  title = {Difficulty Is Not Enough: Curriculum Learning for LLMs Fine-tuning Must Consider Utility},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {37},
  pages = {31365--31373},
  doi = {10.1609/aaai.v40i37.40400}
}
Kostenlos starten →