EUVIMEDEUVIMED entdecken
Crossref

Activations as Features: Probing LLMs for Generalizable Essay Scoring Representations

Jinwei Chi, Ke Wang, Yu Chen, Xuanye Lin, Qiang Xu

Proceedings of the AAAI Conference on Artificial Intelligence · 14.03.2026 · 40 (36) · 30395-30403

DOI: 10.1609/aaai.v40i36.40292 ↗

Kurz gefasst

Fachartikel in der Zeitschrift Proceedings of the AAAI Conference on Artificial Intelligence, online erschienen am 14.03.2026. Verfasst von Jinwei Chi und 4 weiteren Autor:innen. Die Arbeit wurde bisher 1-mal zitiert (Crossref).

AbstractBibliografische AngabenZitierenVerwandte Studien

Abstract (Originalsprache)

Automated essay scoring (AES) is a challenging task in cross-prompt settings due to the diversity of scoring criteria. While previous studies have focused on the output of large language models (LLMs) to improve scoring accuracy, we believe activations from intermediate layers may also provide valuable information. To explore this possibility, we evaluated the discriminative power of LLMs’ activations in cross-prompt essay scoring task. Specifically, we used activation to fit probes and further analyzed the effects of different models and input content of LLMs on this discriminative power. By computing the directions of essays across various trait dimensions under different prompts, we analyzed the variation in evaluation perspectives of large language models concerning essay types and traits. Results show that the activations possess strong discriminative power in evaluating essay quality and that LLMs can adapt their evaluation perspectives to different traits and essay types, effectively handling the diversity of scoring criteria in cross-prompt settings.

Zitieren

APA

Chi, J., Wang, K., Chen, Y., Lin, X., & Xu, Q. (2026). Activations as Features: Probing LLMs for Generalizable Essay Scoring Representations. Proceedings of the AAAI Conference on Artificial Intelligence, 40(36), 30395-30403. https://doi.org/10.1609/aaai.v40i36.40292

Vancouver

Chi J, Wang K, Chen Y, Lin X, Xu Q. Activations as Features: Probing LLMs for Generalizable Essay Scoring Representations. Proceedings of the AAAI Conference on Artificial Intelligence. 2026;40(36):30395-30403. doi:10.1609/aaai.v40i36.40292

BibTeX
@article{Chi2026,
  author = {Chi, Jinwei and Wang, Ke and Chen, Yu and Lin, Xuanye and Xu, Qiang},
  title = {Activations as Features: Probing LLMs for Generalizable Essay Scoring Representations},
  journal = {Proceedings of the AAAI Conference on Artificial Intelligence},
  year = {2026},
  volume = {40},
  number = {36},
  pages = {30395--30403},
  doi = {10.1609/aaai.v40i36.40292}
}
Kostenlos starten →