Fatigue-Resistant Post-Training: Mitigating Cognitive Fatigue in Autoregressive Transformers via FI-Guided Optimization
Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 289-297
DOI: 10.1609/aaaiss.v10i1.43418 ↗Kurz gefasst
Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Ritvik Garimella und 4 weiteren Autor:innen.
Abstract (Originalsprache)
Autoregressive language models frequently degrade during long-horizon generation producing repetitive text, loosing instruction adherence, and exhibiting unstable entropy. We formalize this as Cognitive Fatigue and introduce the Fatigue Index (FI), an online diagnostic aggregating attention decay, entropy miscalibration, and representational drift. We study whether resistance to this degradation can be induced before generation through FI-guided post-training. FI is used as an auxiliary signal rather than a standalone objective, since directly optimizing an imperfect diagnostic may encourage proxy optimization without improving generation quality. We apply FI conditioning across supervised fine-tuning (SFT), and reinforcement learning (FI-GRPO, FI-DAPO) via LoRA adapters, pairing every FI-conditioned configuration against a FI-ablated twin. We evaluate six untrained open models from three architectural families and two parameter scales per family, then conduct post-training experiments on the three small-scale checkpoints across three task tiers. GRPO on Granite-3.1-2B shows accuracy gains alongside delayed fatigue onset and reduced repetition, demonstrating feasibility. However, improvements frequently decouple from task accuracy. These results indicate that FI can provide useful training signal in specific settings, but must be evaluated jointly with task quality rather than treated as a universally reliable objective.
Zitieren
APA
Garimella, R., Marwah, R., Jain, A., Bansal, K., & Sheth, A. (2026). Fatigue-Resistant Post-Training: Mitigating Cognitive Fatigue in Autoregressive Transformers via FI-Guided Optimization. Proceedings of the AAAI Symposium Series, 10(1), 289-297. https://doi.org/10.1609/aaaiss.v10i1.43418
Vancouver
Garimella R, Marwah R, Jain A, Bansal K, Sheth A. Fatigue-Resistant Post-Training: Mitigating Cognitive Fatigue in Autoregressive Transformers via FI-Guided Optimization. Proceedings of the AAAI Symposium Series. 2026;10(1):289-297. doi:10.1609/aaaiss.v10i1.43418
BibTeX
@article{Garimella2026,
author = {Garimella, Ritvik and Marwah, Riju and Jain, Atishay and Bansal, Khusham and Sheth, Amit},
title = {Fatigue-Resistant Post-Training: Mitigating Cognitive Fatigue in Autoregressive Transformers via FI-Guided Optimization},
journal = {Proceedings of the AAAI Symposium Series},
year = {2026},
volume = {10},
number = {1},
pages = {289--297},
doi = {10.1609/aaaiss.v10i1.43418}
}
EUVIMED