Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data
Journal of Technology Informatics and Engineering · 17.06.2026 · 5 (2) · 75-90
DOI: 10.51903/jtie.v5i2.541 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Technology Informatics and Engineering, online erschienen am 17.06.2026. Verfasst von Zeyi Li und 2 weiteren Autor:innen. Der Volltext ist frei zugänglich (Open Access, CC BY-SA 4.0).
Abstract (Originalsprache)
This paper presents a compact reproducible benchmark for evaluating numerical-reasoning guardrails in a quant research assistant. The revised experiment uses a fixed 2026 source snapshot derived from the SEC 2026 Q1 Financial Statement Data Sets and FRED CSV series for VIXCLS, DGS10, DGS3MO and T10Y3M. The benchmark contains 460 tasks: 300 SEC financial-ratio tasks over 50 issuer-period records, 120 FRED VIX and Treasury-rate change tasks, and 40 macro-regime classification tasks. Each answer is evaluated by five programmatic guardrails: numeric consistency, unit correctness, time-window correctness, formula correctness and citation/source consistency. Four controlled response profiles are tested: Naive-RAG, Calculator-Only, Prompted-Checklist and Guarded-Quant. These profiles are deterministic failure-mode controls rather than performance claims about any particular deployed LLM. The empirical results show that arithmetic alone is not sufficient for financial safety: Calculator-Only reaches 79.78% numeric accuracy but only 0.43% all-guardrails pass rate because source, unit, formula and window fields often fail. Guarded-Quant achieves an 88.48% all-guardrails pass rate, 97.17% numeric accuracy, 100.00% unit pass rate, 96.30% window pass rate, 98.26% formula pass rate and 96.30% citation pass rate. The findings support a modest claim: a compact benchmark can make numerical audit failures visible, but it should not be read as evidence of broad quant-assistant reliability without broader data, live model outputs and operational stress tests.
Zitieren
APA
Li, Z., Zhang, K., & Wong, A. (2026). Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data. Journal of Technology Informatics and Engineering, 5(2), 75-90. https://doi.org/10.51903/jtie.v5i2.541
Vancouver
Li Z, Zhang K, Wong A. Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data. Journal of Technology Informatics and Engineering. 2026;5(2):75-90. doi:10.51903/jtie.v5i2.541
BibTeX
@article{Li2026,
author = {Li, Zeyi and Zhang, Kai and Wong, Annie},
title = {Numerical-Reasoning Guardrails for a Quant Research Assistant: A Compact Reproducible Benchmark Using SEC and FRED Data},
journal = {Journal of Technology Informatics and Engineering},
year = {2026},
volume = {5},
number = {2},
pages = {75--90},
doi = {10.51903/jtie.v5i2.541}
}
EUVIMED