EUVIMEDEUVIMED entdecken
Open Access · CC BY 4.0

ELM-Bench: A Multidimensional Methodological Framework for Large Language Model Evaluation in Electricity Markets

Hang Fan, Shijie Ji, Peng Yuan, Qingsong Zhao, Shuaikang Wang, Xiaowei Tan, Yunjie Duan

Energies · 25.07.2025 · 18 (15) · 3982

DOI: 10.3390/en18153982 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Energies, online erschienen am 25.07.2025. Verfasst von Hang Fan und 6 weiteren Autor:innen. Die Arbeit wurde bisher 2-mal zitiert (Crossref) und nennt 32 Literaturangaben. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0). Gefördert von: State Grid LiaoNing Electric Power Supply Co. Ltd., Electric Power Research Insitute.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

The large language model (LLM) has significant potential for application in the field of electricity markets, but there are shortcomings in professional evaluation methods for LLM: single task, limited dataset coverage, and lack of depth. To this end, this article proposes the ELM-Bench framework for evaluating the LLM of the Chinese electricity market, which evaluates the model from 3 dimensions of understanding, generation, and safety through 7 tasks (such as common-sense Q&A and terminology explanations) with 2841 samples. At the same time, a specialized domain model QwenGOLD was fine-tuned based on the general LLM. The evaluation results show that the top-level general model performs well in general tasks due to high-quality pre-training, while QwenGOLD performs better in tasks such as prediction and decision-making in professional fields, verifying the effectiveness of domain fine-tuning. The study also found that fine-tuning has limited improvement on LLM’s basic abilities, but its score in professional prediction tasks is second only to Deepseek-V3, indicating that some general LLMs can handle domain data well without professional training. This can provide a basis for model selection in different scenarios, balancing performance and training costs.

Förderung

  • State Grid LiaoNing Electric Power Supply Co. Ltd., Electric Power Research Insitute

Zitieren

APA

Fan, H., Ji, S., Yuan, P., Zhao, Q., Wang, S., Tan, X., & Duan, Y. (2025). ELM-Bench: A Multidimensional Methodological Framework for Large Language Model Evaluation in Electricity Markets. Energies, 18(15), 3982. https://doi.org/10.3390/en18153982

Vancouver

Fan H, Ji S, Yuan P, Zhao Q, Wang S, Tan X, et al. ELM-Bench: A Multidimensional Methodological Framework for Large Language Model Evaluation in Electricity Markets. Energies. 2025;18(15):3982. doi:10.3390/en18153982

BibTeX
@article{Fan2025,
  author = {Fan, Hang and Ji, Shijie and Yuan, Peng and Zhao, Qingsong and Wang, Shuaikang and Tan, Xiaowei and Duan, Yunjie},
  title = {ELM-Bench: A Multidimensional Methodological Framework for Large Language Model Evaluation in Electricity Markets},
  journal = {Energies},
  year = {2025},
  volume = {18},
  number = {15},
  pages = {3982},
  doi = {10.3390/en18153982}
}
Kostenlos starten →