EUVIMEDEUVIMED entdecken
Open Access · CC BY-SA 4.0

Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data

Zeyi Li, Kai Zhang, Annie Wong

Journal of Technology Informatics and Engineering · 17.06.2026 · 5 (2) · 75-90

DOI: 10.51903/jtie.v5i2.541 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Journal of Technology Informatics and Engineering, online erschienen am 17.06.2026. Verfasst von Zeyi Li und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

This paper presents a compact reproducible benchmark for evaluating numerical-reasoning guardrails in a quant research assistant. The revised experiment uses a fixed 2026 source snapshot derived from the SEC 2026 Q1 Financial Statement Data Sets and FRED CSV series for VIXCLS, DGS10, DGS3MO and T10Y3M. The benchmark contains 460 tasks: 300 SEC financial-ratio tasks over 50 issuer-period records, 120 FRED VIX and Treasury-rate change tasks, and 40 macro-regime classification tasks. Each answer is evaluated by five programmatic guardrails: numeric consistency, unit correctness, time-window correctness, formula correctness and citation/source consistency. Four controlled response profiles are tested: Naive-RAG, Calculator-Only, Prompted-Checklist and Guarded-Quant. These profiles are deterministic failure-mode controls rather than performance claims about any particular deployed LLM. The empirical results show that arithmetic alone is not sufficient for financial safety: Calculator-Only reaches 79.78% numeric accuracy but only 0.43% all-guardrails pass rate because source, unit, formula and window fields often fail. Guarded-Quant achieves an 88.48% all-guardrails pass rate, 97.17% numeric accuracy, 100.00% unit pass rate, 96.30% window pass rate, 98.26% formula pass rate and 96.30% citation pass rate. The findings support a modest claim: a compact benchmark can make numerical audit failures visible, but it should not be read as evidence of broad quant-assistant reliability without broader data, live model outputs and operational stress tests.

Zitieren

APA

Li, Z., Zhang, K., & Wong, A. (2026). Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data. Journal of Technology Informatics and Engineering, 5(2), 75-90. https://doi.org/10.51903/jtie.v5i2.541

Vancouver

Li Z, Zhang K, Wong A. Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data. Journal of Technology Informatics and Engineering. 2026;5(2):75-90. doi:10.51903/jtie.v5i2.541

BibTeX
@article{Li2026,
  author = {Li, Zeyi and Zhang, Kai and Wong, Annie},
  title = {Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data},
  journal = {Journal of Technology Informatics and Engineering},
  year = {2026},
  volume = {5},
  number = {2},
  pages = {75--90},
  doi = {10.51903/jtie.v5i2.541}
}
Kostenlos starten →