EUVIMEDEUVIMED entdecken
Crossref

Quiet Feature Learning in Algorithmic Tasks

Prudhviraj Naidu, Zixian Wang, Leon Bergen, Ramamohan Paturi

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (44) · 37756-37764

DOI: 10.1609/aaai.v40i44.41111 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Prudhviraj Naidu und 3 weiteren Autor:innen.

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

We train Transformer-based language models on ten foundational algorithmic tasks and observe pronounced phase transitions in their loss curves that deviate from established power-law scaling trends. Over large ranges of compute, the validation loss barely improves, then abruptly decreases. Probing the models’ internal representations reveals that quiet features are learned prior to any decrease in task loss. These quiet features represent intermediate algorithmic computations that do not by themselves improve the output loss. Ablation experiments demonstrate that individual quiet features are causally necessary for task performance. Our results demonstrate that substantial representational progress can remain hidden beneath an apparently flat loss curve, challenging the prevailing use of cross‑entropy as a proxy for learning and motivating richer diagnostics for monitoring model training.

Zitieren

APA

Naidu, P., Wang, Z., Bergen, L., & Paturi, R. (2026). Quiet Feature Learning in Algorithmic Tasks. Proceedings of the AAAI Conference on Artificial Intelligence, 40(44), 37756-37764. https://doi.org/10.1609/aaai.v40i44.41111

Vancouver

Naidu P, Wang Z, Bergen L, Paturi R. Quiet Feature Learning in Algorithmic Tasks. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(44):37756-37764. doi:10.1609/aaai.v40i44.41111

BibTeX
@article{Naidu2026,
  author = {Naidu, Prudhviraj and Wang, Zixian and Bergen, Leon and Paturi, Ramamohan},
  title = {Quiet Feature Learning in Algorithmic Tasks},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {44},
  pages = {37756--37764},
  doi = {10.1609/aaai.v40i44.41111}
}
Kostenlos starten →