Evidence-Calibrated Financial Language Models for Macro-Policy Stance Classification and Decision Cards
Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023 · 30.07.2026 · 10 (01) · 1-23
DOI: 10.60087/jaigs.v10i01.488 ↗Kurz gefasst
Fachartikel in der Zeitschrift Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023, online erschienen am 30.07.2026. Verfasst von Hugo Yan. Der Volltext ist frei zugänglich (Open Access, CC BY 4.0).
Abstract (Originalsprache)
Central-bank language often conveys policy direction through qualifying clauses rather than isolated sentiment terms. This study evaluates hawkish, dovish, and neutral stance classification on all 496 FinBen-FOMC excerpts using leakage-controlled five-fold stratified-group cross-validation. Seven systems span a class-prior baseline, an n-gram language model, word-and-character TF-IDF, frozen FinBERT transfer scores, frozen MiniLM sentence embeddings, and two fused models. Four-fold inner cross-validation fits temperature and isotonic calibrators without access to outer-test labels. Evaluation combines macro-F1, Matthews correlation coefficient, expected calibration error, Brier score, confusion analysis, clustered bootstrap intervals, selective risk, and contrastive lexical evidence. TFIDF-LR achieved the highest pooled point estimates for macro-F1 (0.493) and MCC (0.246), followed closely by EvidenceFusion at 0.489 and 0.233. None of the macro-F1 differences between TFIDF-LR and another learned model was significant after Holm correction. Temperature scaling sharply reduced overconfidence in the n-gram and embedding systems; TFIDF+MiniLM attained the lowest temperature-scaled ECE (0.014), while TFIDF-LR achieved a Brier score of 0.587. Isotonic calibration lowered probability loss further for several models but reduced directional recall. Removing TFIDF-LR’s selected evidence terms reduced predicted-class confidence by 0.170 and changed 69.4% of labels, whereas matched random removal changed confidence by −0.019. At a 0.70 acceptance threshold, TFIDF-LR covered 6.45% of records with 21.9% risk, including a high-confidence error driven by tightening vocabulary despite explicit negation. Evidence-calibrated decision cards are therefore most appropriate for conservative analyst triage rather than autonomous policy interpretation.
Zitieren
APA
Yan, H. (2026). Evidence-Calibrated Financial Language Models for Macro-Policy Stance Classification and Decision Cards. Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023, 10(01), 1-23. https://doi.org/10.60087/jaigs.v10i01.488
Vancouver
Yan H. Evidence-Calibrated Financial Language Models for Macro-Policy Stance Classification and Decision Cards. Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023. 2026;10(01):1-23. doi:10.60087/jaigs.v10i01.488
BibTeX
@article{Yan2026,
author = {Yan, Hugo},
title = {Evidence-Calibrated Financial Language Models for Macro-Policy Stance Classification and Decision Cards},
journal = {Journal of Artificial Intelligence General science (JAIGS) ISSN:3006-4023},
year = {2026},
volume = {10},
number = {01},
pages = {1--23},
doi = {10.60087/jaigs.v10i01.488}
}
EUVIMED