Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic Empirical Study
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (41) · 35239-35247
DOI: 10.1609/aaai.v40i41.40831 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Yuqi Zhu und 9 weiteren Autor:innen.
Abstract (Originalsprache)
Large Language Models (LLMs) hold promise in automating data analysis tasks, yet open-source models face significant limitations in these kinds of reasoning-intensive scenarios. In this work, we investigate strategies to enhance the data analysis capabilities of open-source LLMs. By curating a seed dataset of diverse, realistic scenarios, we evaluate models across three dimensions: data understanding, code generation, and strategic planning. Our analysis reveals three key findings: (1) Strategic planning quality serves as the primary determinant of model performance; (2) Interaction design and task complexity significantly influence reasoning capabilities; (3) Data quality demonstrates a greater impact than diversity in achieving optimal performance. We leverage these insights to develop a data synthesis methodology, demonstrating significant improvements in open-source LLMs' analytical reasoning capabilities.
Zitieren
APA
Zhu, Y., Zhong, Y., Zhang, J., Zhang, Z., Qiao, S., Luo, Y., Du, L., Zheng, D., Zhang, N., & Chen, H. (2026). Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic Empirical Study. Proceedings of the AAAI Conference on Artificial Intelligence, 40(41), 35239-35247. https://doi.org/10.1609/aaai.v40i41.40831
Vancouver
Zhu Y, Zhong Y, Zhang J, Zhang Z, Qiao S, Luo Y, et al. Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic Empirical Study. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(41):35239-35247. doi:10.1609/aaai.v40i41.40831
BibTeX
@article{Zhu2026,
author = {Zhu, Yuqi and Zhong, Yi and Zhang, Jintian and Zhang, Ziheng and Qiao, Shuofei and Luo, Yujie and Du, Lun and Zheng, Da and Zhang, Ningyu and Chen, Huajun},
title = {Why Do Open-Source LLMs Struggle with Data Analysis? A Systematic Empirical Study},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {41},
pages = {35239--35247},
doi = {10.1609/aaai.v40i41.40831}
}
EUVIMED