EUVIMEDEUVIMED entdecken
Open Access · CC BY-NC-ND 4.0

LLM-based NLG Evaluation: Current Status and Challenges

Mingqi Gao, Xinyu Hu, Xunjian Yin, Jie Ruan, Xiao Pu, Xiaojun Wan

Computational Linguistics · 24.06.2025 · 51 (2) · 661-687

DOI: 10.1162/coli_a_00561 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Computational Linguistics, online erschienen am 24.06.2025. Verfasst von Mingqi Gao und 5 weiteren Autor:innen. Die Arbeit wurde bisher 95-mal zitiert (Crossref) und nennt 129 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY-NC-ND 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Evaluating natural language generation (NLG) is a vital but challenging problem in natural language processing. Traditional evaluation metrics mainly capturing content (e.g., n-gram) overlap between system outputs and references are far from satisfactory, and large language models (LLMs) such as ChatGPT have demonstrated great potential in NLG evaluation in recent years. Various automatic evaluation methods based on LLMs have been proposed, including metrics derived from LLMs, prompting LLMs, fine-tuning LLMs, and human–LLM collaborative evaluation. In this survey, we first give a taxonomy of LLM-based NLG evaluation methods, and discuss their pros and cons, respectively. Lastly, we discuss several open problems in this area and point out future research directions.

Zitieren

APA

Gao, M., Hu, X., Yin, X., Ruan, J., Pu, X., & Wan, X. (2025). LLM-based NLG Evaluation: Current Status and Challenges. Computational Linguistics, 51(2), 661-687. https://doi.org/10.1162/coli_a_00561

Vancouver

Gao M, Hu X, Yin X, Ruan J, Pu X, Wan X. LLM-based NLG Evaluation: Current Status and Challenges. Computational Linguistics. 2025;51(2):661-687. doi:10.1162/coli_a_00561

BibTeX
@article{Gao2025,
  author = {Gao, Mingqi and Hu, Xinyu and Yin, Xunjian and Ruan, Jie and Pu, Xiao and Wan, Xiaojun},
  title = {LLM-based NLG Evaluation: Current Status and Challenges},
  journal = {Computational Linguistics},
  year = {2025},
  volume = {51},
  number = {2},
  pages = {661--687},
  doi = {10.1162/coli_a_00561}
}
Kostenlos starten →