TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws
Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 31068-31076
DOI: 10.1609/aaai.v40i37.40367 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Nan Hu und 10 weiteren Autor:innen.
Abstract (Originalsprache)
Recent studies have explored the capabilities of large language models (LLMs) in solving knowledge-intensive mathematical reasoning problems. However, existing benchmarks predominantly involve static theorems that LLMs have encountered during pretraining, failing to assess dynamic knowledge integration. In this work, we introduce TaxReasoning, a novel benchmark designed to evaluate LLMs’ abilities in real-world tax calculation scenarios. These tasks require not only mathematical reasoning and numerical computation, but also the extraction and application of complex, frequently updated tax regulations. Through extensive experiments with state-of-the-art LLMs using diverse prompting strategies and knowledge augmentation techniques, we uncover substantial limitations in their ability to handle dynamic, knowledge-intensive questions—primarily due to missing domain-specific knowledge and ineffective retrieval. Even the best-performing models fall significantly short of human-level performance. Our analysis points to key avenues for improvement, including enhancing LLMs' reasoning capabilities, developing more effective knowledge summarization techniques, and improving retrieval strategies. TaxReasoning offers a critical testbed for advancing LLMs in dynamic knowledge-intensive domains.
Zitieren
APA
Hu, N., Wu, Y., Li, J., Hu, H., Qi, G., Zhai, S., Chen, Y., Wu, T., Wu, T., Chen, J., & Pan, J. Z. (2026). TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 31068-31076. https://doi.org/10.1609/aaai.v40i37.40367
Vancouver
Hu N, Wu Y, Li J, Hu H, Qi G, Zhai S, et al. TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):31068-31076. doi:10.1609/aaai.v40i37.40367
BibTeX
@article{Hu2026,
author = {Hu, Nan and Wu, Yike and Li, Jiaye and Hu, HuiKang and Qi, Guilin and Zhai, Songlin and Chen, Yongrui and Wu, Tianxing and Wu, Tongtong and Chen, Jiaoyan and Pan, Jeff Z.},
title = {TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws},
journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
year = {2026},
volume = {40},
number = {37},
pages = {31068--31076},
doi = {10.1609/aaai.v40i37.40367}
}
EUVIMED