ELM-Bench: A Multidimensional Methodological Framework for Large Language Model Evaluation in Electricity Markets
Energies · 25.07.2025 · 18 (15) · 3982
DOI: 10.3390/en18153982 ↗Kurz gefasst
Fachartikel in der Zeitschrift Energies, online erschienen am 25.07.2025. Verfasst von Hang Fan und 6 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: State Grid LiaoNing Electric Power Supply Co. Ltd., Electric Power Research Insitute.
Abstract (Originalsprache)
The large language model (LLM) has significant potential for application in the field of electricity markets, but there are shortcomings in professional evaluation methods for LLM: single task, limited dataset coverage, and lack of depth. To this end, this article proposes the ELM-Bench framework for evaluating the LLM of the Chinese electricity market, which evaluates the model from 3 dimensions of understanding, generation, and safety through 7 tasks (such as common-sense Q&A and terminology explanations) with 2841 samples. At the same time, a specialized domain model QwenGOLD was fine-tuned based on the general LLM. The evaluation results show that the top-level general model performs well in general tasks due to high-quality pre-training, while QwenGOLD performs better in tasks such as prediction and decision-making in professional fields, verifying the effectiveness of domain fine-tuning. The study also found that fine-tuning has limited improvement on LLM’s basic abilities, but its score in professional prediction tasks is second only to Deepseek-V3, indicating that some general LLMs can handle domain data well without professional training. This can provide a basis for model selection in different scenarios, balancing performance and training costs.
Förderung
- State Grid LiaoNing Electric Power Supply Co. Ltd., Electric Power Research Insitute
Zitieren
APA
Fan, H., Ji, S., Yuan, P., Zhao, Q., Wang, S., Tan, X., & Duan, Y. (2025). ELM-Bench: A Multidimensional Methodological Framework for Large Language Model Evaluation in Electricity Markets. Energies, 18(15), 3982. https://doi.org/10.3390/en18153982
Vancouver
Fan H, Ji S, Yuan P, Zhao Q, Wang S, Tan X, et al. ELM-Bench: A Multidimensional Methodological Framework for Large Language Model Evaluation in Electricity Markets. Energies. 2025;18(15):3982. doi:10.3390/en18153982
BibTeX
@article{Fan2025,
author = {Fan, Hang and Ji, Shijie and Yuan, Peng and Zhao, Qingsong and Wang, Shuaikang and Tan, Xiaowei and Duan, Yunjie},
title = {ELM-Bench: A Multidimensional Methodological Framework for Large Language Model Evaluation in Electricity Markets},
journal = {Energies},
year = {2025},
volume = {18},
number = {15},
pages = {3982},
doi = {10.3390/en18153982}
}
EUVIMED