EUVIMEDEUVIMED entdecken
Crossref

TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws

Nan Hu, Yike Wu, Jiaye Li, HuiKang Hu, Guilin Qi, Songlin Zhai, Yongrui Chen, Tianxing Wu, Tongtong Wu, Jiaoyan Chen, Jeff Z. Pan

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (37) · 31068-31076

DOI: 10.1609/aaai.v40i37.40367 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Nan Hu und 10 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Recent studies have explored the capabilities of large language models (LLMs) in solving knowledge-intensive mathematical reasoning problems. However, existing benchmarks predominantly involve static theorems that LLMs have encountered during pretraining, failing to assess dynamic knowledge integration. In this work, we introduce TaxReasoning, a novel benchmark designed to evaluate LLMs’ abilities in real-world tax calculation scenarios. These tasks require not only mathematical reasoning and numerical computation, but also the extraction and application of complex, frequently updated tax regulations. Through extensive experiments with state-of-the-art LLMs using diverse prompting strategies and knowledge augmentation techniques, we uncover substantial limitations in their ability to handle dynamic, knowledge-intensive questions—primarily due to missing domain-specific knowledge and ineffective retrieval. Even the best-performing models fall significantly short of human-level performance. Our analysis points to key avenues for improvement, including enhancing LLMs' reasoning capabilities, developing more effective knowledge summarization techniques, and improving retrieval strategies. TaxReasoning offers a critical testbed for advancing LLMs in dynamic knowledge-intensive domains.

Zitieren

APA

Hu, N., Wu, Y., Li, J., Hu, H., Qi, G., Zhai, S., Chen, Y., Wu, T., Wu, T., Chen, J., & Pan, J. Z. (2026). TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws. Proceedings of the AAAI Conference on Artificial Intelligence, 40(37), 31068-31076. https://doi.org/10.1609/aaai.v40i37.40367

Vancouver

Hu N, Wu Y, Li J, Hu H, Qi G, Zhai S, et al. TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(37):31068-31076. doi:10.1609/aaai.v40i37.40367

BibTeX
@article{Hu2026,
  author = {Hu, Nan and Wu, Yike and Li, Jiaye and Hu, HuiKang and Qi, Guilin and Zhai, Songlin and Chen, Yongrui and Wu, Tianxing and Wu, Tongtong and Chen, Jiaoyan and Pan, Jeff Z.},
  title = {TaxReasoning: Benchmarking Knowledge-Intensive Mathematical Reasoning with Evolving Tax Laws},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {37},
  pages = {31068--31076},
  doi = {10.1609/aaai.v40i37.40367}
}
Kostenlos starten →