An Information Fusion Based Decision‐Making Framework for Generative Performance Assessment of Large Language Models
Quality and Reliability Engineering International · 2026-12 · 42 (8) · 4548-4560
DOI: 10.1002/qre.70358 ↗Kurz gefasst
Fachartikel in der Zeitschrift Quality and Reliability Engineering International, online erschienen am 17.08.2026. Verfasst von Yongqing Huang und 2 weiteren Autor:innen. Die Arbeit nennt 47 Literaturangaben. Gefördert von: Fundamental Research Funds for the Central Universities.
Abstract (Originalsprache)
A subjective‐objective information ‐ based decision ‐ making fusion framework is proposed to assess the generative performance of large language models (LLMs) by combining objective evidence and expert‐based subjective judgement. The Analytic Hierarchy Process (AHP) is first used to quantify the objective weights of sub‐indicators and compute the objective performance based on the standardised benchmarks. The Best‐Worst Method is then used to quantify the subjective weights of sub‐indicators based on domain experts. After that, an iterative solution method is introduced to align the divergence domain expert opinion, update the subjective weights, and compute the subjective performance of LLMs. Finally, the performance of LLM can be quantified by the weighted sum of subjective and objective performance. Comparative and sensitivity analyses are carried out to validate the feasibility and superiority of the proposed method. Overall, the proposed method can deal with the potential subjective uncertainty in expert opinion and supporting final decision‐making.
Förderung
- Fundamental Research Funds for the Central Universities
Zitieren
APA
Huang, Y., Jiang, W., & Ye, Y. (2026). An Information Fusion Based Decision‐Making Framework for Generative Performance Assessment of Large Language Models. Quality and Reliability Engineering International, 42(8), 4548-4560. https://doi.org/10.1002/qre.70358
Vancouver
Huang Y, Jiang W, Ye Y. An Information Fusion Based Decision‐Making Framework for Generative Performance Assessment of Large Language Models. Quality and Reliability Engineering International. 2026;42(8):4548-4560. doi:10.1002/qre.70358
BibTeX
@article{Huang2026,
author = {Huang, Yongqing and Jiang, Wei and Ye, Yunling},
title = {An Information Fusion Based Decision‐Making Framework for Generative Performance Assessment of Large Language Models},
journal = {Quality and Reliability Engineering International},
year = {2026},
volume = {42},
number = {8},
pages = {4548--4560},
doi = {10.1002/qre.70358}
}
EUVIMED