EUVIMEDEUVIMED entdecken
Crossref

An Information Fusion Based Decision‐Making Framework for Generative Performance Assessment of Large Language Models

Yongqing Huang, Wei Jiang, Yunling Ye

Quality and Reliability Engineering International · 2026-12 · 42 (8) · 4548-4560

DOI: 10.1002/qre.70358 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Quality and Reliability Engineering International, online erschienen am 17.08.2026. Verfasst von Yongqing Huang und 2 weiteren Autor:innen. Die Arbeit nennt 47 Literaturangaben. Gefördert von: Fundamental Research Funds for the Central Universities.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

A subjective‐objective information ‐ based decision ‐ making fusion framework is proposed to assess the generative performance of large language models (LLMs) by combining objective evidence and expert‐based subjective judgement. The Analytic Hierarchy Process (AHP) is first used to quantify the objective weights of sub‐indicators and compute the objective performance based on the standardised benchmarks. The Best‐Worst Method is then used to quantify the subjective weights of sub‐indicators based on domain experts. After that, an iterative solution method is introduced to align the divergence domain expert opinion, update the subjective weights, and compute the subjective performance of LLMs. Finally, the performance of LLM can be quantified by the weighted sum of subjective and objective performance. Comparative and sensitivity analyses are carried out to validate the feasibility and superiority of the proposed method. Overall, the proposed method can deal with the potential subjective uncertainty in expert opinion and supporting final decision‐making.

Förderung

  • Fundamental Research Funds for the Central Universities

Zitieren

APA

Huang, Y., Jiang, W., & Ye, Y. (2026). An Information Fusion Based Decision‐Making Framework for Generative Performance Assessment of Large Language Models. Quality and Reliability Engineering International, 42(8), 4548-4560. https://doi.org/10.1002/qre.70358

Vancouver

Huang Y, Jiang W, Ye Y. An Information Fusion Based Decision‐Making Framework for Generative Performance Assessment of Large Language Models. Quality and Reliability Engineering International. 2026;42(8):4548-4560. doi:10.1002/qre.70358

BibTeX
@article{Huang2026,
  author = {Huang, Yongqing and Jiang, Wei and Ye, Yunling},
  title = {An Information Fusion Based Decision‐Making Framework for Generative Performance Assessment of Large Language Models},
  journal = {Quality and Reliability Engineering International},
  year = {2026},
  volume = {42},
  number = {8},
  pages = {4548--4560},
  doi = {10.1002/qre.70358}
}
Kostenlos starten →