EUVIMEDEUVIMED entdecken
Crossref

Fatigue-Resistant Post-Training: Mitigating Cognitive Fatigue in Autoregressive Transformers via FI-Guided Optimization

Ritvik Garimella, Riju Marwah, Atishay Jain, Khusham Bansal, Amit Sheth

Proceedings of the AAAI Symposium Series · 06.11.2026 · 10 (1) · 289-297

DOI: 10.1609/aaaiss.v10i1.43418 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Symposium Series, online erschienen am 06.11.2026. Verfasst von Ritvik Garimella und 4 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Autoregressive language models frequently degrade during long-horizon generation producing repetitive text, loosing instruction adherence, and exhibiting unstable entropy. We formalize this as Cognitive Fatigue and introduce the Fatigue Index (FI), an online diagnostic aggregating attention decay, entropy miscalibration, and representational drift. We study whether resistance to this degradation can be induced before generation through FI-guided post-training. FI is used as an auxiliary signal rather than a standalone objective, since directly optimizing an imperfect diagnostic may encourage proxy optimization without improving generation quality. We apply FI conditioning across supervised fine-tuning (SFT), and reinforcement learning (FI-GRPO, FI-DAPO) via LoRA adapters, pairing every FI-conditioned configuration against a FI-ablated twin. We evaluate six untrained open models from three architectural families and two parameter scales per family, then conduct post-training experiments on the three small-scale checkpoints across three task tiers. GRPO on Granite-3.1-2B shows accuracy gains alongside delayed fatigue onset and reduced repetition, demonstrating feasibility. However, improvements frequently decouple from task accuracy. These results indicate that FI can provide useful training signal in specific settings, but must be evaluated jointly with task quality rather than treated as a universally reliable objective.

Zitieren

APA

Garimella, R., Marwah, R., Jain, A., Bansal, K., & Sheth, A. (2026). Fatigue-Resistant Post-Training: Mitigating Cognitive Fatigue in Autoregressive Transformers via FI-Guided Optimization. Proceedings of the AAAI Symposium Series, 10(1), 289-297. https://doi.org/10.1609/aaaiss.v10i1.43418

Vancouver

Garimella R, Marwah R, Jain A, Bansal K, Sheth A. Fatigue-Resistant Post-Training: Mitigating Cognitive Fatigue in Autoregressive Transformers via FI-Guided Optimization. Proceedings of the AAAI Symposium Series. 2026;10(1):289-297. doi:10.1609/aaaiss.v10i1.43418

BibTeX
@article{Garimella2026,
  author = {Garimella, Ritvik and Marwah, Riju and Jain, Atishay and Bansal, Khusham and Sheth, Amit},
  title = {Fatigue-Resistant Post-Training: Mitigating Cognitive Fatigue in Autoregressive Transformers via FI-Guided Optimization},
  journal = {Proceedings of the AAAI Symposium Series},
  year = {2026},
  volume = {10},
  number = {1},
  pages = {289--297},
  doi = {10.1609/aaaiss.v10i1.43418}
}
Kostenlos starten →